Today, dear reader, I have a story of intrigue, of frustration, and of redemption. Well, it’s really just about hacking up my iTunes local backups to fix a backup/restore problem I was having with my iPhone. If you’re looking for steps to fix the problem, skip to the bottom. If you’re interested in hearing what I tried that didn’t work and why I was stuck in this situation, though, keep reading.

After updating to iOS 6.1 on my iPhone 5, I started having problems backing up my iPhone to my iMac. I didn’t notice at first, because lately I’ve been using my iMac as a second monitor for my MacBook Air rather than an actual iMac.

By “problems backing up my iPhone”, I pretty much mean exactly that. My iPhone worked normally, but every time I would plug it into my iMac and attempt a backup, I would get a generic dialog helpfully explaining with verbiage similar to: “iTunes could not back up the iPhone because an error occurred”. This would happen regardless of whether the backup was via a full sync or merely by pressing the “Back Up Now” button in iTunes.

Further details on the specific problem I was having were found in the Console:

AppleMobileBackup[5219]: ERROR: Backup message response: 1 Error copying keychain data (MBErrorDomain/1)
AppleMobileBackup[5219]: WARNING: No MobileSync error code defined for error code: 1
AppleMobileBackup[5219]: ERROR: Backup error: -1

Alright, I guess that’s somewhat helpful in pointing out the problem, but not the solution. Since it was related to the keychain, I figured I’d clear out all my network passwords on my iPhone (via Settings.app, go to [General —> Restore —> Restore Network Settings]). I also tried removing both my password lock and my restrictions password. None of this had any effect: my iPhone would still fail to backup with the same keychain error. I also tried backing up to a different Mac, and that didn’t work either.

A post on Apple’s discussions site claims iCloud backups proceed successfully in this situation, even though local backups don’t. I tried doing this, but my iPhone estimated that the backup would take 9 hours, so I cancelled it. I think iCloud backups include keychain items, so I’m not sure why local backups would fail but iCloud backups would work.

In any case, I was annoyed, so I proceeded with a complete restore of my iPhone. I’ve got a fix that doesn’t involve restoring your device, but if you are planning to restore your device at this point, let me give you an important recommendation: back your device up to iCloud first, if it works. Please, please do this.

I found out the hard way that when your backups aren’t working due to this error, restoring your backups won’t work either! Arrgh. If iCloud backups work, then you can probably successfully restore your device from them. Unfortunately, I’ve never backed up to iCloud even once, so I couldn’t restore from there. And I couldn’t restore from my local backups either. I would get similar generic error dialogs, such as this one:

And in the Console, I’d get similar log lines:

AppleMobileBackup[5494]: ERROR: Restore message response: 1 Error restoring keychain: 1 (MBErrorDomain/1)
AppleMobileBackup[5494]: WARNING: No MobileSync error code defined for error code: 1
AppleMobileBackup[5494]: ERROR: Restore error: -1

Worse, this seems to be a latent error that was triggered by my update to iOS 6.1. Using Time Machine, I pulled iPhone backups from early January, and they had the same restore problem. Arrrrgh. Without an iCloud backup, and without any working local backups, I was stuck with an iPhone that didn’t have any of my data. I also have hundreds of apps on my iPhone (I know, I know), so reorganizing them would’ve been a real pain. I could restore from my iPad, which is set up similarly, but has different data and doesn’t restore home folder organization. (By the way, I updated my iPad to iOS 6.1 as well, and it hasn’t had these problems.)

The internets weren’t helpful in solving this problem, but I did find this page which helpfully explains the structure of iTunes backups. In particular, there is a single file which contains all the keychain resources: in ~/Library/Application Support/MobileSync/Backup/, you should see folders named with the UDID of your devices. Inside each backup, there is a file named 51a4616e576dd33cd2abadfea874eb8ff246bf0e. This is the file corresponding to your device’s keychain backup.

With that information, I tried deleting just that one file, and then restoring the backup. iTunes changed its tune and claimed the backup was damaged, and refused to restore it. I tried replacing it with a different file, or replacing it with a zero-length file, and the same thing happened.

That afore-linked page also explains the format of the Manifest.mbdb file that contains all the information for which file corresponds to what data, so it was time to go futzing around with that. Here, ultimately, are the steps that fixed my local backup.

1. Connect your device to your Mac. Open iTunes, go to the Device Summary page, click where it says “Serial Number”, and iTunes should show you your device’s UDID number. Note this down.

2. In the Finder go to ~/Library/Application Support/MobileSync/Backup/.

3. Find the folder named with the UDID from step one, and duplicate it. Move the duplicate to a safe location: you want to have a pristine copy that you can always revert back to (even though it doesn’t actually work to restore). Better is having Time Machine enabled, so you have progressive copies of it, but you can’t enable that after-the-fact. Even if you have Time Machine backups, though, duplicate this folder just in case!

4. Now you need a local backup of a different device. If you have a second iOS device, connect it to your Mac and perform a backup. (Make sure that if you encrypt your backups, you’ve used the same password for this backup as you do for your problematic backup.) Note down the UDID of this device using the method in step one, too. If you have an older backup that you can successfully restore with, you can use that as well, if you want to fix your newer backup. (You can also connect a friend’s device to your computer and back it up, or have them send you just the two files you’ll need from their backup — more on that later — but this method changes the keychain data in your backup, so you’ll end up with your friend’s passwords on your device. They probably don’t want that.)

5. If you had a second device, skip this step. If you don’t have a second device, you’ll need to completely restore your device and perform a new backup. Here’s where you try an iCloud backup, if you haven’t before! Before you erase and restore, though, rename the folder ~/Library/Application Support/MobileSync/Backup/your-problematic-device-UDID/ to “your-problematic-device-UDID.bad”. Once you have a good backup, the new backup will have the same UDID — rename the folder to “UDID.good”, and rename the problematic backup back to the original UDID.

6. Now, you should have two backups: one is your problematic backup that you are trying to fix, and the second is the good backup with valid keychain data. You created a backup of your problematic backup, right? OK, here we go. From your good backup, copy the file named “51a4616e576dd33cd2abadfea874eb8ff246bf0e” into the folder of your problematic backup. To be clear, you copy the file located here:

   good-device-backup-UDID/51a4616e576dd33cd2abadfea874eb8ff246bf0e

   to the same location in your problematic backup:

   your-problematic-device-UDID/51a4616e576dd33cd2abadfea874eb8ff246bf0e

   When the Finder alerts you that there’s already a file in that location with that name, replace it.

7. OK, you’re going to need to do some hex editing. Go download Hex Fiend, a very competent, fast, free, and open source hex editor.

8. In Hex Fiend, open the “Manifest.mbdb” from both your good and your problematic backup. Make sure you know which is which!

9. In the Manifest.mbdb file of your good backup, search for the word “keychain”.

10. The webpage describing the iTunes backup format lists the possible fields for every record that this file contains. You should see the string “KeychainDomain”. Highlight it, and you’ll notice that Hex Fiend’s status bar shows you’ve selected 14 bytes of data.

    

11. If you highlight the character immediately before the string “KeychainData”, and set the data inspector to “Signed Int” or “Unsigned int”, it’ll say “14”. (If you don’t see the data inspector at the bottom of the window, you may need to enable it from the Views menu.) Note that this is the length of the string “KeychainData”.

    

    You should also see a string that says “keychain-backup.plist”. I tried changing that to make iTunes backup/restore a non-existent file, but it just gave me a different error.

    So, what we’re going to do is change the SHA 1 hash field of the keychain data in your problematic backup with the SHA 1 hash of the keychain data in your good backup. Since we already changed the actual keychain data itself in step 5, this should satisfy iTunes in verifying that the files contents are known and good.

12. Somewhere after the string “keychain-backup.plist” there should be a blank character that has a hex value of 14. Highlight it, and the data inspector should give you a value of 20. Not coincidentally, SHA 1 hashes are 160 bits in length, or 20 bytes!

    

13. Starting with the character immediately following the blank character, select 20 bytes worth of data. Copy it.

    

14. Now, repeat steps 8 through 12 with the Manifest.mbdb file from your bad backup, but instead of copying the SHA 1 hash, you’ll paste the SHA 1 hash that you just copied from your good Manifest.mbdb file.

15. Save. Now restore your device from this backup. It should work!

I suspect that a similar technique will allow you to replace other arbitrary data in your backups.