Post-Mortem

It has been I-don’t-even-know-how-long (that is: an eternity) since I last offered any of my apps for purchase outside of the App Stores.
GimmeSomeTune was donationware (handled by PayPal), but there was no licensing, so that doesn’t count.
For a software-bundle (I can’t remember the name, and I don’t believe it’s around anymore), I created a non-updatable version of ScreenFloat 1.x that customers received, yet also without licenses, so that doesn’t count either.
I think the only app I ever really sold properly and had a license scheme for and handled sales myself was flickery, which I sold from my website through PayPal before the Mac App Store arrived.
But once the Mac App Store hit, I transitioned all my apps to it pretty much right away.
It’s just so convenient: no license creation, no license verification (apart from receipt validation, but that has become more convenient recently), easy updating, no handling of payments, invoices, refunds, and the potential of getting featured to lots and lots of users.

Why do it, then?

Besides all that, it was high time I set up a way to also sell my apps outside of the Mac App Store.
Without a licensing system for my apps, I’ve been unable to participate in software-bundles and/or collections. Lots of companies and corporations cannot purchase apps from the Mac App Store due to policies. Individuals who want to purchase my apps for work are unable to do so because of those same policies.
And while I am a strong proponent of the Mac App Store, I also believe in giving people a choice. Customers should have the choice of purchasing my apps on the App Store, or directly* from me. *: I don’t handle any of the payment and invoicing myself, that’s done by my Merchant of Record, Paddle.
Also, it certainly cannot hurt to have an alternate route to sell my apps.
What if the Mac App Store goes belly-up? Not likely, but still, I’d go belly-up along with it.
What if Apple doesn’t want my apps anymore? It could happen (it has been close before), and then I’d be screwed.

All that is just a long-winded way of saying: it makes sense to have some redundancy. A second way of selling my apps. One I have a bit more control over.

Now, I did slack off on this for a long time. I’ve been getting requests to participate in bundles for years (!) and still couldn’t bring myself to set it up. I don’t know why. I guess I thought my time was better spent working on my apps instead of backend stuff.
But then there were all these recent inquiries from companies, and users affected by policies of companies, who cannot purchase apps from the Mac App Store. For some reason, there’s been an uptick there, and requests have become more and more frequent.
So I hunkered down and finally started looking into all the things that needed to get done so I could sell my apps outside of the Mac App Store as well:

  • Decide on a Merchant of Record
    (crucial for the “selling” part)
  • Come up with and implement a license key scheme
    (crucial for legitimating purchases of my app)
  • Freshen up my PHP, HTML and JavaScript “skills” and integrate my backend with the merchant’s
    (crucial for everything)
  • Implement a way for the app to update itself
    (pretty much standard nowadays)
  • In that vein, implement a safer way to download files than just a plain file-straight-from-server download
    (so the app can verify the update is legit)
Getting a Merchant of Record

A “Merchant of Record” is a company that handles payments, invoicing, refunds, taxes, etc for indie software developers and other businesses.
There are actually quite a few to choose from: FastSpring, PayPal, Paddle, Stripe, and Gumroad, just to name a few.

  • PayPal I didn’t want to use because I remember its API from back in the day and it just gave me headaches. I’m actually amazed I pulled it off for flickery way back when. Also, and I might be wrong about this, but, I believe it still doesn’t handle taxes and stuff for you, and I just won’t do that myself anymore.
  • Gumroad seemed to me like more of a hobby thing? Anyway, I couldn’t see myself selling software through them.
  • FastSpring is a popular choice, loved for its extensive feature set.
  • Stripe seemed out of my league. Like, it’s a fancy masquerade ball, and I’m in the dark corner wearing sweats, eventually getting asked to leave because I make people “uncomfortable”.
Put the Paddle to the Metal

So I went with Paddle. I don’t feel frowned upon here in my sweats. And in my calculations, it seemed a bit less expensive than FastSpring. Plus, I’ve heard good things about it. The API is well-documented* and their support seemed… supportive.
Why’s there an ominous asterisk next to “well-documented”? I’m glad you asked!
There’s Paddle Classic, and as of recently, there’s the new Paddle Billing, with a completely different API.
Guess what this idiot (read: me) did. Yup, I spent 3 days looking into and partially implementing Paddle Classic, only to find out it’s no longer available for new signups. Fun!
Hey, Paddle, I have a suggestion: Instead of that tiny, friendly light-blue indicator at the top of the Paddle Classic API documentation page that you can easily overlook and even dismiss, why not make it a big, red, bold-letter banner? That would have saved me tons of time.
Instead, I sat there wondering why none of my test-calls from the Paddle dashboard worked. (Side note: Debugging remote PHP scripts is a freaking pain. Especially the way I do it – which might be the wrong way.)
But that’s alright. Every story needs its ups and downs. Why not begin with a down outright? Who knows, it could be all uphill from here!

For selling through Paddle, you need their approval. They take a look at your website and make sure everything’s on the up-and-up regarding your Privacy Policy, Terms of Service, payment flow, checkout and whatnot.
The entire process was fairly straight-forward. It might have helped that I’m already selling my apps through the Mac App Store, so they could see that I’m serious about my endeavors. For my Terms of Service I looked into the websites of other indies selling software through Paddle, which I thought made sense, since they’re already approved by them.
It took about three weeks with a bit of back and forth to get the approval, but that was no time lost, since I used it to start implementing the correct API on the backend.

Scheming up a License Key

A vital part of an app sold outside of the Mac App Store is requiring the user to have a license key in order to keep using it past its trial limitations. If you don’t have that, anyone could just use your app without paying for it. That’s fine for freeware or donationware and the like, but for an app you want to sell, it’s counterproductive.

Obviously, you want to be the only one able to create these license keys. Back in the day, for flickery, I had a very traditional format: a 32-odd-character dash-separated string, like FLKRY-ABA1-ABA2-ABA3…
It was just a salted MD5 hash of a certain order of transformed and salted md5 hashes of the customer’s name and email. In the app, the user would enter their name, email and the license key and flickery would re-create the license key itself from the supplied info and see if it matches up with the one the user supplied. If it did, the app was registered, if not, then not.
In order to forge a key, one would have to know the salts, the transformations and the order I used to create a valid key. Someone with a lot of time on their hands could eventually figure it out and create a key generator. Or hack the flickery binary and figure it out that way.
The salts were actually more pepper than salt, as they were hopefully secret. As a side note, I don’t think the license generation was ever cracked. The app itself was, though.

I’ve decided to go a different way this time around. The license now is a cryptographically signed payload (signed using a private key), consisting of the user’s data, app information and transactional data. All the app does is use the corresponding public key to verify the signature. If it’s valid, it can proceed with further activation steps. If not, something’s wrong with the license. As long as the private key actually stays private, there’s no feasible way to forge a license at this point in time.

I also wanted to make sure a license can only be active on a limited number of Macs or user accounts at a time, and have a way of blocklisting license keys infringing on my Terms of Service.
This is also where license types come into play.
There’s the “personal license” for individuals, which can be used to activate the app a limited amount of times (on any Mac or user account);
and the “commercial license”, which can be activated on a single Mac, but used with an unlimited amount of accounts on that Mac – a “seat” license.
With managing and limiting the activations comes the need to unregister individual copies of an app, or reset the entire license, so you can free up a slot to activate the app on a different Mac or user account. More work on the backend!

Consequently, in order to activate a copy of my app with a license key, an internet connection is required. But you also need an internet connection to download the app in the first place, so I don’t see a downside there. And there’s no always-online requirement. Only once in a while, the app will demand a connection to make sure the license and its activation are still good.

What was important to me about all this though is that none of the user’s information, not even the license key, is stored on my server. Yes, Paddle stores user data, because they have to. But I didn’t see the need to have user data be present on my server as well. It holds only the information it absolutely requires to activate, deactivate, reset, re-issue and refund licenses.

Most of this is handled on the backend, which took most of my time to implement and test. Even though I have been using PHP (with MySQL for database integration) on and off over the years, I’m by no means “fluent”, so I repeatedly had to consult different guides to figure out the best practices, and how to code safely, for everyone involved. JavaScript I’d never really used before (but had to for the Checkout pages), and HTML with CSS, well… my checkout pages are as pretty as they’re going to get, to put it bluntly.
But I learned a lot. I like doing server/website stuff for some reason. Maybe because it’s not my area of expertise, and I get to understand the workings of the internet a little bit better and learn so much. It just has a certain allure I can’t explain.

Next Time

In the next part, I’ll go over the integration with Paddle, and my backend implementation. I hope you’ll join me!

Read more

Yoink v3.6.5 re-introduces its clipboard history feature and the accompanying widget.
Here are a few details about its implementation for macOS Big Sur and newer.

The No-Button-Action Conundrum

Widgets on macOS Big Sur and up (and iOS, for that matter) can’t really react to a user’s click on them by themselves, so you cannot run any logic from a user’s input.
A click onto a widget will always bring you back to its containing app, where you can then run logic accordingly.
On iOS, that’s obvious, as it’ll open up the owning app and put it front and center.
On macOS, that can be a little more subtle, because macOS allows apps to run in the background (like Yoink does, mostly).

When you click on an item in Yoink’s widget, it’ll tell Yoink to run the logic to copy the clicked item (or pin it, or add it to Yoink, or reveal it in the browser, depending on the modifier key you pressed during the click).

But there’s a problem with that in the case of Yoink:

The Foreground Conundrum

No matter where you click on a widget (either the background of it, or a SwiftUI Link() object), it will bring its containing/owning app to the foreground.
For many apps, that will be fine. But Yoink is an app that runs in the background and only very rarely needs to actually become the active, keyboard-input-accepting app.

So I thought, perhaps I can have another application inside Yoink’s app bundle, which is LSBackgroundOnly, with a custom URL scheme that would be called from the widget using a SwiftUI Link() object. An app that has LSBackgroundOnly set to YES in its Info.plist cannot present any user interface, and cannot become “key”, even if the system tells it to.
But there’s another roadblock – even though only the secondary app inside Yoink’s app bundle should be able to open the custom URL scheme, the widget will send that Link() to its containing/owning app only, no matter the url scheme. In this case, the main Yoink app.

The point of the widget is to quickly re-copy something and then paste it somewhere right away.
Say you're editing text in TextEdit, then open the widget, click on it to re-copy something and then press command-v to paste it into TextEdit only to hear a beep because the app isn't active anymore (because Yoink is). So you have to click into the TextEdit document to make it active, and only then can you paste. That can (and will) become annoying very quickly.

In order to have Yoink never become the active app from a click on its widget, I’d have to move the widget plugin/appex bundle out of the Yoink app bundle’s PlugIns folder, into the secondary target one’s.
Only that way would the widget attempt to make the secondary target(and thus, not Yoink) frontmost – and fail because of LSBackgroundOnly, leaving the currently frontmost app active – and send the custom URL scheme link to it. The secondary target would then forward the link to the main Yoink app bundle.

Tada, it works. But that lead to yet another problem. (When did “it just works” turn into “it just won’t”? And while I’m at it, what’s with those widgets? They feel quite… neutered to me compared to what they were able to do before.)

The Widget Recognition Conundrum

So far, I managed to have Yoink not become active when an item in the widget is clicked.
But now, with the Yoink app containing the secondary target containing the widget, macOS was unwilling to recognize and show the widget in Notification Centre.
Interestingly, a double-click onto the secondary target in Finder would make it show right away.
Infuriatingly, launching the secondary target quietly from within Yoink at launch won’t – and I had a lot of different approaches:
– Launching the secondary app directly with NSWorkspace’s -launchApplicationAtURL:…
– Launching the secondary app via its URL scheme
– Having Finder open the secondary app
– Running an NSTask open -a <secondaryapp> /path/to/somefile
– Registering the app using LSRegisterURL

None of it worked. I figured, the problem was that it was Yoink launching the secondary app, not the “system” or “user”, like it was the case when double-clicking it in Finder.

Which made me think of login item helper apps.
Inside the macOS app sandbox, an app cannot set itself as a login item. It needs to have another “helper” app inside its Library/LoginItems folder which it designates as a login item, and that app will then in turn launch the containing app. Nuts, but there it is.

The point is, the system launches those login item helper apps, not the containing app.

Heureka. In Yoink, at launch, I set my secondary app, which contains the widget bundle, as a login item with SMLoginItemSetEnabled, causing the system to launch it. Now, finally, Notification Centre recognizes and shows Yoink’s widget.

What a needless journey.

Read more

[Note: This is a guest blog post written by Thomas Zoechling (@weichsel on twitter), a macOS developer based in Austria, about his Mac app Claquette]

Claquette Mac App Icon

Animated GIF is not a good video format.
Actually it even isn’t a proper video format because it lacks random access and audio support. Nonetheless animated GIFs experienced a rennaisance in recent years.
My theory is, that the format’s success doesn’t stem from the features it has, but from the ones it lacks:

  • No random access: Defaults to autoplay – No playback interaction needed
  • No sound: Guarantees silence – Always “Safe for Work”
  • No support for 32bit colors: Moderate file sizes and a characteristic look

Given those constraints, GIFs are a great way to communicate simple concepts or interactions.
Want to showcase an animation or an application feature? Instead of a static screenshot with paragraphs of text, a 3 second GIF works equally well and also draws more attention.

In my daily life as software developer, I often need a quick way to capture graphical bugs. Those clips can be easily shared with colleagues or included with bug reports.
Sometimes it’s also simpler to attach a GIF to customer support requests instead of explaining how a certain feature works.
To cover my own needs, I wrote a small macOS application that allows me to record the screen and export the result as animated GIF. The app uses a custom recording codec and also covers the rest of the GIF creation workflow like crop, trim and file size optimization.

You can download Claquette from the Mac App Store. For detailed product information, please visit our website.

Development

When I started to implement the Animated GIF feature for Claquette, I began with a naïve approach.
Armed with several years of experience in writing Objective-C and some knowledge about video APIs on the Mac, I wrote a small prototype.
That first version just read a video file frame by frame and sent the resulting frames to ImageIO. ImageIO is a system framework that supports reading and writing several file formats.
It also comes with a basic Animated GIF encoder and so I decided to skip any third party libraries and just use the built-in mechanisms of macOS.
I was able to come up with a working prototype in a single afternoon. The program was just a simple command line utility, but it was able to turn an arbitrary video file into an animated GIF.

There was just one problem… Or actually there were several of them: Due to the inner workings of ImageIO, the program used vast amounts of memory. Also, the encoding took very long and the files it created were huge. On top of all that, the resulting GIFs looked horrible.
So while it only took me one Sunday afternoon to create a working prototype, it took me several months to fix the above issues. Especially the large file size and the bad visual appearance of the resulting GIFs required a lot of research.

Getting the most out of a 30 year old file format

The original GIF specification (GIF87a) was written in 1987 – almost 30 years ago. Animation features were added in GIF89a, which is still the most recent version of the format.
So how is it possible that a file format designed for low resolution screens and 256 colors is still in use today?
It turns out that the GIF specification contains some sections that open room for exploitation. Additionally, the visual nature of GIFs allows for optimizations to trick human color perception.

The format is simple and has the following basic structure:

  1. Header
  2. Logical Screen Descriptor
  3. Global Color Table
  4. Graphic Control Extension (Frame #1)
    – Delay
    – Disposal Method
    – Local Color Table
    – Image Descriptor
    – Image Data
  5. Graphic Control Extensions (Frame #2)
  6. Graphic Control Extension (Frame #3)
  7. … (1 GCE for each animation frame)
  8. Trailer

Header and Trailer are basically just magic numbers that mark the start and the end of the file. The Logical Screen Descriptor contains some general image information like width, height and background color. The Global Color Table is a simple list of colors that may contain a maximum of 256 entries.
Main image information is stored in one or more Graphic Control Extension blocks.

Color Table Generation

The color table sections of the format specification are a good starting point to optimize the visual quality of an animated GIF.
Both palettes are restricted by the format’s infamous 256 color limit. When reducing an image that uses 32bit (16.777.216 colors, 256 alpha values) to 8bit (255 colors, 1 alpha bit) it becomes really important which colors you leave out. The process of reducing large palettes to small ones is called Color Quantization. Choosing a good quantization algorithm is crucial when aiming for visually similar images with a reduced palette.
Naïve quantization implementations are based on occurrence, where seldom used colors are left out in the final image. More elaborate algorithms use techniques like dimensional clustering or tree partitioning.

When developing for Apple devices there are several libraries that provide color quantization functionality. macOS and iOS even have basic color quantization algorithms built-in. Apple’s implementation is part of the ImageIO framework’s CGImageDestination API.

The following sample images were created using different quantization techniques. They illustrate the quality impact of the used algorithm on the final image.

CGImageDestination Quantization

The first image shows the output of CGImageDestination. The resulting colors are noticeably off. Apple’s encoder also messes up the transparency color in the GIF palette, which leads to holes in some areas of the image (e.g. the titlebar).

FFmpeg Quantization

The open source library FFmpeg also includes a color quantizer. FFmpeg produces way better results than CGImageDestination. The colors are more vibrant and the transparency color is set correctly.

Claquette Quantization

The color quantization library used by Claquette also outputs a good choice of colors. Additionally the app performs color matching to avoid color shifts and to guarantee correct gamma values.

Frame Difference Calculation

Another important factor during GIF generation is efficient frame difference calculation.
The disposal mode in the Graphic Control Extension allows an encoder to specify how the context is set up before the next frame gets rendered.
GIF89a defines 4 disposal modes:

  • Unspecified: Replaces the existing canvas with the full contents of the current frame.
  • Do not Dispose: Leaves the existing canvas as-is and composites the current (sub)frame over it.
  • Restore to Background: Sets a defined background color and draws the current frame. Areas outside of the subsection in the Image Descriptor shine through.
  • Restore to Previous: Fully restores the canvas to the last frame that did not specify a disposal method.

The Image Descriptor section can be used to define a sub-image which does not provide pixel data for a full frame. Instead it contains coordinates and pixel data for a subsection of the full image.
By using frame differences and sub-images with a proper disposal mode, redundant image data can be avoided. Depending on the nature of the input video, this can greatly reduce the file size of the final GIF.
Modern video codecs like H.264 use techniques like macro blocks and motion compensation. Those methods introduce small errors that propagate from frame to frame. Propagated errors show up as noise when calculating frame diffs and eventually lead to unnecessary large files.
Claquette uses a custom lossless codec, that only stores regions that actually changed. This guarantees exact frame diffs.

The following images show the difference between frame #1 and frame #2 of a screen recording. The only effective change between those frames is a change in the mouse cursor location. An exact diff should therefore only contain an offsetted cursor image.

FFmpeg Diff

The above diff image was created between 2 frames of an H.264 encoded movie. The visible noise is a result of intra-frame encoding errors.

Claquette Diff

The second image was created by Claquette’s image diff algorithm. It only contains the mouse cursor – The only image element that actually changed between frame #1 and #2.

Finishing Touches

After implementing technical details like encoding and optimization, there were still some features missing. Claquette needed an editing UI to handle the whole GIF creation workflow.
As I wanted to keep the app lightweight and simple to use, I decided to add only a small set of editing capabilities: Trim and Crop.
Claquette uses AVFoundation, and therefore I was able to use the AVPlayer class, which comes with a built-in trim tool.
Crop functionality was a bit harder to implement. AVFoundation doesn’t provide any UI component to display crop handles so I had to implement my own.
Besides the standard drag & move interface, my implementation also provides some unique features. It offers alignment guides with haptic feedback and the possibility to enter crop coordinates.

You can see the final implementation of the crop utility in the animated GIF below:

ClaquetteCrop

Launch

To launch the animated GIF feature, I prepared a press kit and wrote mails to review sites that mostly cover Mac software.
Additionally I submitted the app to Product Hunt and informed Apple’s App Store Marketing team.
I can really recommend to launch on Product Hunt: Claquette received over 400 upvotes and finished in the top 3 on launch day. The site also hosts a friendly community, that makes extensive use of the Q&A section below each hunted product.
I was also lucky to get some mentions from high profile Twitter users and good App Store reviews.
Two weeks after launch, Apple suddenly moved Claquette into the “New and Noteworthy” feature section on the Mac App Store front page. Of course this also lead to a noticeable spike in the sales charts.
Overall I was very pleased with the release and the reception of Claquette 1.5.

—
Thomas Zoechling is an independent software developer living in Vienna, Austria.
Besides working on his own products, he is also doing contract work. Currently he is helping IdeasOnCanvas to develop the excellent mind mapping software MindNode (macOS, iOS).

Read more