Leopard Code-Signing and the Application Firewall

Wednesday, 2007-11-07; 23:34:00

[UPDATE: For more info on Leopard's firewall, see this post.]

Many users of Mac OS X may be in for a bit of confusion when they update to Leopard and access the firewall settings. It's been completely revamped in Leopard, and the firewall exposed to the GUI in Leopard is completely different from the one exposed to the GUI in Tiger and earlier. The reason? Leopard's firewall is an application firewall, not a port firewall.

In Tiger (as well as Jaguar and Panther, IIRC), you accessed the firewall through the Sharing preference pane. It looked something like this:

Tiger Firewall GUI

This firewall was based off ipfw, a standard UNIX port firewall. ipfw works by blocking or allowing access to specific ports -- if a port was not in the exception list in System Preferences in Tiger, incoming connections on that port would be discarded and not allowed through, regardless of its destination. So, for example, if you were trying to host a networked game of D2X-XL, which communicates over port 28342, ipfw would block any clients that are attempting to connect to your server unless you specifically allowed incoming connections on port 28342 through.

Apple's exposure of ipfw to the GUI was pretty simplistic, and you could always access the manual for ipfw via the Terminal, and even set up more complicated rules.

Leopard's firewall, in contrast, is exposed to the GUI in the Security preference pane, and it looks like this:

Leopard Firewall GUI

It works on a per-application basis. That means that instead of configuring the firewall to allow connections through port 28342, you configure the firewall to allow connections to the D2X-XL application. Using Leopard's application firewall, D2X-XL is not only granted access to incoming connections on port 28342, but on any port. However, other applications still cannot access incoming connections, even over port 28342.

In Leopard, when you launch an application that wants access to incoming connections, Leopard throws up a dialog box and asks if you want to allow or deny access to incoming connections to that app. This dialog box does not come up for every application, only those requesting access to incoming connections. It looks like this:

Leopard Firewall Prompt

(Basically, it says, "Do you want the application 'Skype' to accept incoming connections?" with "Don't Allow" and "Always Allow" buttons, with "Always Allow" being the default.)

In practice, an application firewall is much more useful. Opening up a specific port pre-Leopard meant that you opened it up to any application, so it's possible that an application you didn't know about would be able to accept connections from the outside without your knowledge through that port you opened.

Typically, applications use only one port to accept incoming connections, and most applications that require incoming connections each use different ports, but that wasn't guaranteed. (For example, D2X-XL is the only app I know of that communicates over port 28342.) With Leopard, you have finer-grained control over what applications get access to incoming connections on your Mac, because you can selectively block access based on its destination. It's also easier to understand: when you look at the Firewall tab, you don't see a list of ports, you see a list of applications.

Note that I've been very careful to talk specifically about incoming connections and about what Apple's exposed to the GUI. ipfw has always been able to control outgoing connections as well, but Apple's GUI didn't allow you to customize that. Neither does Leopard's GUI: you can only control what comes in to your computer. Also, ipfw still exists in Leopard and is in fact still functional: rules created with ipfw supercede any rules defined by the new application firewall. Apple simply no longer exposes ipfw to the GUI. (Anybody know if existing ipfw rules are discarded when doing an upgrade installation of Leopard?)

Also note the distinction between incoming connections as opposed to incoming data. Going back to our example on D2X-XL, even if you don't allow access to port 28342 (pre-Leopard) or don't allow access to incoming connections from the D2X-XL application (Leopard), you can still connect to other game servers and still play network games. This is because incoming data that's in response to an outgoing connection is allowed through.

That's also why Apple doesn't expose restrictions on outgoing connections through the GUI. When you go to a website, you don't want to have to specifically allow your web browser to place outgoing connections. That would just be annoying, and for the most part, completely unnecessary. You're already telling the computer to connect to a website, so telling it to allow you to connect to the website before connecting to it is redundant.

But there is one case which could be a cause for alarm. What if you initially allow access to incoming connections for a specific app, and then that app changes due to some malicious activity? You definitely don't want that app still having access to incoming connections. How do you control this?

Via code-signing. There are two benefits to the this new feature in Leopard.

First of all, have you ever noticed in Tiger that when you download an updated version of an application, Mac OS X sometimes threw up a dialog box notifying you that the application had changed? Well, it was securing you against unauthorized access to your keychain. In a very similar situation, you don't want apps to be modified maliciously and then have access to incoming connections via the firewall simply because you allowed access before.

Tiger Keychain Prompt

Code-signing eliminates this hassle to the user. (When I worked at the elementary school, we constantly had problems with the aforementioned dialog box, because it would come up after every Mac OS X update. The AFP client would automatically connect to the server on login, automatically accessing the keychain, so changes to the AppleFileServer application would trigger this dialog. If the kids clicked the "Don't Change" button, then the server wouldn't mount because it couldn't access the keychain to get the login and password.)

Apple has explicitly code-signed all of its applications, and third-parties can supposedly do this as well. When explicitly code-signed applications get updated, Mac OS X makes sure that the updated application has the same signature, and if so, it knows that it came from the same source as the previous application so it doesn't have to worry about allowing access to keychain items (or incoming connections). Thus the "are you sure you want to change your Keychain items" dialog won't show up in Leopard anymore, unless there's been a malicious change in the app, or you're running apps that are not explicitly code-signed.

(This is also where Apple got into a bit of flak over Leopard's firewall. Even if you set the firewall to block all incoming connections for any app, Leopard still allows incoming connections to root user processes and explicitly code-signed processes. This is not mentioned at all in the preference pane, so it looks like Leopard's firewall is not working correctly. Apple does, however, document this in its knowledge base article on the new application firewall.)

The second benefit comes from implicit code-signing. When you run an app that is not explicitly code-signed, and you allow access to incoming connections via the firewall, Leopard implicitly code-signs the app automatically.

Now, say an implicitly code-signed app gets modified. Again, this throws up red flags to Mac OS X. If the app was specifically modified by the user (i.e.: the user dragged a new version of the app from a disk image to the Applications folder and replaces the old version), the user will simply be prompted to allow access to the firewall again. If the app was not specifically modified by the user, Mac OS X simply stops the app from opening at all.

And that's where my latest annoyance came in. I noticed that Skype would simply fail to launch, and copying the same version of the app from the pristine disk image to the Applications folder would allow it to launch again. That meant that Skype had to have been writing something to it's own application bundle, because that was the only variable that changed between the time that Skype wasn't able to launch and the time that it was.

And I found I had to do this every. single. time. I relaunched Skype. Because Skype was self-modifying it's own app bundle, Leopard noticed the change, saw that it wasn't a change initiated by the user, and simply stopped it from launching. Copying the app from the disk image again reset the implicit code-signing, and Skype was able to launch and I again allowed it access to incoming connections through the firewall.

The first lesson to take away from this is for programmers: never, ever write to your own application bundle. This has always been bad form on Mac OS X: the app bundle should never be changed, and any configuration or preference files should be stored in the appropriate subfolder of the user's Library folder. But now there's an added incentive to not write to your own bundle: it may cause your app not to launch. I'm actually happy about this side-effect, because I get particularly annoyed by apps that write to their own bundle. (And here one was doing it right under my own nose!)

Skype is not the only casualty of Leopard's code-signing feature. World of Warcraft also apparently writes to its own bundle, so it too is not launching with the only recourse being the reinstallation of WoW. (I suspect, however, that you can simply duplicate the app bundle after initially installing it, and then on each launch, simply delete the modified app bundle and replace it with a copy of the clean bundle, instead of having to go through the whole reinstallation process each time.) Any apps that accept incoming connections and also write to their own bundle will need to be changed before they'll consistently launch on Leopard.

If you're running Leopard and one of your apps isn't launching, it's worth knowing how to see if it's failing the implicit code-signing check: fire up the Console, and click on "All Messages" under the "LOG DATABASE QUERIES" section in the Log List. (Cue several teeth-gnashing comments regarding the ALL CAPS subsection headings. Question: why hasn't anybody complained about my ALL CAPS WEBLOG ENTRY TITLES?) Now try to launch the application. If you see the line "Check 1 failed. Can't run [app name]", then that app is failing the implicit code-signing check. Copy the app from the disk image again, and you should be able to run it again.

The second lesson is for users: a popular type of app modification is to go and mess around with the app bundle contents, particularly if you want to "skin" an app. You're still able to do that, but take note: if you do modify the contents of an app's bundle and it wants access to incoming connections, you'll need to re-authorize it when you finally relaunch the app. (Just take a look at this Leopard-specific hint at Mac OS X Hints: if you want to add RAW support for certain cameras to Mac OS X before Apple does, you previously simply just had to change a plist file pre-Leopard. With Leopard, you also have to change another file that includes the sha1 digest for that plist file: the sha1 digest is a code that pretty much guarantees that the contents of that file have not been modified.)

I'm not quite sure how Mac OS X knows how to distinguish between a user modification -- resulting in simply a re-prompt for firewall access -- and a non-user modification -- resulting in the denial of the app from launching. Does it have to do with whether the app was modified from the Finder or from a different app? That's an exercise for an ambitious or knowledgeable reader.

In any case, hopefully this provides a useful overview and troubleshooting tips with Leopard's firewall and code-signing feature. If you're a Mac developer or if you're just interested in code-signing, check out Apple's Code Signing Guide. It delves into detail about the consequences of code-signing (and not code-signing) and the specifics about how to actually code-sign applications.

Technological Supernova   Tips   Older   Newer   Post a Comment