The trouble with creating bootable drives and discs
I harbor no nostalgia for Mac OS 9. Aside from a couple of old games, I never even think about using OS 9 anymore. With one exception...I long for OS 9's simplicity in creating bootable drives and discs.
How simple was it? For hard drives, it was as simple as dragging a System Folder from one drive to another. Actually, it was even simpler than that. You could usually get by with just copying the System and Finder files that resided inside the System Folder. The rest of the System Folder was desirable but optional. An Installer utility? It was there but not required. Invisible files? None to worry about. Permissions problems? Non-existent.
To create a bootable optical disc (or, similarly, a bootable Zip or floppy disk) was a bit more involved, but not by much.
So how does all of this compare to the situation with Mac OS X today? Here's how:
Bootable hard drives. Creating a bootable hard drive in Mac OS X is not difficult, but is still a significant undertaking. The most common method is to boot from a Mac OS X Install disc, run the Installer utility and install a new copy of Mac OS X. The other common alternative is to create a bootable (mirrored or cloned) copy of an existing bootable drive using backup software (such as Retrospect, DataBackup or SuperDuper!). In either case, you need to work with specialized software. Forget about simply copying a System and Finder file from the Finder. You need to copy tens of thousands of files from the source to the destination. A bootable hard drive requires all the essential files in the /System/Library and /Library folders as well as the invisible Unix directories. Even if you copied all of these files from the Finder, you would not have a bootable drive. You'd still have to worry about additional essential invisible files (most critically the mach.sym and mach_kernel files at the root level of the drive), the "volume bootability characteristics" (as can be set via the Unix bless command), and the proper permissions settings (including the critical "Ignore ownership on this volume" setting in a drive's Info window). Plus, you need a functioning admin user account. In theory, assuming you knew what to do, you could set all of this up yourself. But few users try. In almost all cases, Mac OS X users are dependent on utility software to create bootable drives.
Intel vs. PowerPC. The arrival of Intel Macs has added yet more wrinkles. A drive that boots on PowerPC Macs will not boot on Intel Macs and vice versa. This is partly because each platform uses its own version of Mac OS X. An Apple-supported universal version of Mac OS X does not yet exist (although it's supposedly coming in Leopard). Further complicating matters is that the drive format required for each platform is different. As Apple states: "A PowerPC-based Mac can only install Mac OS X on a disk with the Apple Partition Scheme; An Intel-based Mac can only install Mac OS X on a disk with the GUID Partition Scheme" (see this Apple Knowledge Base article for more details). All of this can lead to failures when trying to install Mac OS X or when trying to startup from a supposedly bootable drive, often without meaningful error messages to explain the cause (see this TidBITS article for some examples).
Even making a successful clone of a bootable drive can get a bit tricky (see this article for a look at just how tricky it can get to restore from a bootable backup). Your success in booting from an external drive will also vary with different hardware. For example, Intel Macs can boot from USB 2.0 drives; PowerPC Macs cannot (at least not without some low level fiddling, as covered in this Mac OS X Hints posting).
If you are determined, despite all of the above, to create a dual-platform "universal" bootable drive, it can be done. But it will take some effort. You'll likely need the guidance of tutorials, such as the one here at MacFixIt or other similar postings. If your eyes start to glaze over as you read these articles, don't be surprised.
Looking back, it's hard to believe how far we have strayed from the simplicity of Mac OS 9. Some may say this is the price we have to pay for the modern advantages of Mac OS X. I am not sure. Regardless, I am surprised at how passively Mac users have accepted all of this without complaining. Yes, the situation will improve with the release of Leopard. It will also be less of a concern as the number of active PowerPC Macs declines. But if you are holding out hope that the simplicity of OS 9 may someday reincarnate in Mac OS X, don't hold your breath. It's not gonna happen.
Bootable discs and flash drives. Matters only get worse when trying to create bootable optical discs (or flash drives). For starters, Mac OS 9 bootable discs had two desirable features largely absent in Mac OS X:
The discs contained a Finder. This meant that, when booting from optical discs, you went to a real Desktop and your internal hard drive mounted as a separate volume. From here, you could easily access files on the drive, launch applications, or even modify documents.?
The discs were easy to customize. You could put (and run) whatever applications you wanted on the disc, including third-party utilities. This was ideal for making your own Emergency Utilities disc.
Unlike with hard drives, you can't simply use the Mac OS X Install disc to install the needed software on a CD or DVD. If you can even get Installer to correctly target on an optical disc, it won't work for at least two reasons: the discs are typically not large enough to accommodate the required software and/or optical discs require special boot software that is not provided via the Installer utility.
Still, if all you want is a backup copy of a bootable disc, it is easy enough to do. It's especially easy if you have disc-duplicating software (such as Toast Titanium) and two optical drives. Otherwise, you can use Disk Utility. This method requires the intermediate step of creating a disk image file of the DVD, which you then use to burn to a blank DVD. Even here, if you have a recent Install DVD, you may get stuck at the starting gate: the DVD may be a dual-layer disc while your SuperDrive may only burn single-layer discs.
But what if you seek the Holy Grail here? That is, what if you would like to create your own custom bootable disc, such as an Emergency Disk with your own selection of utilities on it? And on top of that, what if you want it to be a universal disk, capable of booting on both Intel and PowerPC platforms? This turns out to be an almost insurmountable hassle.
As to the custom bootable disc part, you can get most of the way there easily enough, before you trip just before the finish line:
Use Disk Utility to create a read-write disk image of a Mac OS X Install disc. Mount the image. Assuming there is enough room, add your desired utilities to the image. If you need more room, you can delete non-essential files from the image. Then burn the image.
The result: a bootable disc with your custom utilities on it. It all seems fine except for one little problem: you can't access the software you added. There is no Finder available when booting from an optical disc. Instead, the disc defaults to launching one and only one utility on startup. The Mac OS X Install disc, for example, defaults to launching the Installer utility. This means that, while your custom software is on the disc, you have no direct way to access and launch any of the software. What to do?
The quickest, although not the most user-friendly, solution is to take advantage of the fact that you can launch Terminal from the Installer's Utilities menu (available when you boot from a Mac OS X Install disc). Had you booted from a hard drive, you could use the Unix open command to launch applications from Terminal. But Apple did not include this command on the Mac OS X Install disc. Happily, there is an alternative command that works: Type a period (.) followed by the path to the executable application. For example, after booting from a Mac OS X Install disc, here's how to launch the Console application in the Utilities folder on your internal hard drive:
1. Type: cd /Volumes/"{name of your hard drive boot volume}"/Applications/Utilities?This moves you to the Utilities directory on your hard drive, simplifying the next and critical step.?
2. Type: ./{name of application}.app/Contents/MacOS/{name of application}?Thus, to launch Console, you would type: ?./Console.app/Contents/MacOS/Console
[When you type the last command, you're actually launching the executable binary inside Console.app (which is actually a folder). While you can simply double-click Console.app to execute the binary in the Mac OS X Finder, you need to directly call the binary when using this Terminal procedure]
If you created a custom disc, with your own added software, you could use the same method to locate and launch any of the programs on the disc.
Unfortunately, your success here will be limited by two factors:
Some software requires writing to the startup disc in order to function. As you cannot write to a CD or DVD, this can be a deal-breaker. Apple provides a potential solution to this problem: it creates a RAM disk as part of the startup process. Files can be written here instead of to the disc itself. However, this does not appear to be accessible for applications launched by the user after startup is over.?
Some applications require accessing additional software in order to function. This may include software in the Frameworks and PrivateFrameworks folder of the /System/Library. It may also include software that the program expects to find (or create) in the Library folder of a user's account. This software is often not found in the subset of Mac OS X files included on a boot CD/DVD.
The overall result is that many programs will fail to launch from a boot disc. Even if they do launch successfully, they may not function properly (menu commands may not work, for example). In the case of the Console application, it launches fine but there are no logs on the startup disc to examine.
If you prefer to avoid Terminal, there are other solutions, but they require working with third party software:
Clone X 2. Although Clone X is known primarily as backup software, a new feature was added to it in 2.0.1: the ability to create a custom bootable CD. It does this by using the files on your current startup drive as the source. This should guarantee that the disc will work successfully on your Mac. It should result in greater success in running applications from the disc than via the Terminal route.
When you first setup to burn the CD, you are additionally given the chance to select what applications you want to run. These are copied to the CD. The resulting bootable CD launches an application launcher utility (included as part of Clone X) on startup. Whatever software you selected appears in the launcher. Cool. I tried it out and it worked perfectly.
TechTool Protege. Micromat's TechTool Protege is a bootable FireWire flash drive. Similar to how Clone X's bootable CD works, Protege goes directly to an application launcher on startup. Again, you can select in advance the utilities you want to be accessible from the launcher. Micromat's TechTool Pro and DiskStudio software are included by default.
BootCD. BootCD appears to be a near perfect solution. It creates a bootable CD that, amazingly, even manages to squeeze in a functioning Finder. Unfortunately, the current version of BootCD does not work with Tiger and, from the looks of things, I suspect it will never be updated to work in Tiger or any future OS X version. If you are still using Panther, however, it's worth a look.
None of these solutions creates a single universal dual-platform disc. A disc created by Clone X will boot whatever platform was used to create the disc, Intel or PowerPC, but not both. TechTool Protege comes with software that can be used to create either an Intel or PowerPC version of the drive, but you cannot create a single version that works on both platforms. Most people predict that Leopard will resolve this issue, allowing for universal boot discs. However, while we're waiting, a few developers have come up with solutions that already work in Tiger:
TechTool Pro. Micromat's TechTool Pro ships on a single DVD that contains two separately mounting partitions. One partition contains PowerPC software and the other Intel software. When you boot from the disc, your Mac automatically locates the bootable partition for that platform and boots from it, launching the TechTool Pro application.
Drive Genius. The people at Prosoft managed to perform an even cooler trick with their Drive Genius bootable disc. It is a CD (not a DVD) and it contains a single universal bootable system on it, successfully starting up on either a PowerPC or an Intel Mac. After booting, it launches the Drive Genius software. I suspect that even the folks at Apple could learn a thing or two from this disc.
Apple's position. Speaking of Apple, a question may be occurring to you by now: Given that it's obviously possible to create a customizable boot disc, and even make it dual-platform, why doesn't Apple provide the tools to allow end users to do this? Why the almost complete absence of even a mention of this subject in Apple's Knowledge Base documents?
I have asked these questions to several informed sources (outside of Apple) and have still not gotten a clear answer. The all seem to agree that it has something to do with Apple wanting to prevent piracy of Mac OS X or otherwise protect its copyright. I point out that there were no similar restrictions in Mac OS 9, and there did not seem to be any great ill effect. I go on to point out that the availability of BootCD (for Jaguar and Panther) did not lead to any problems that I was aware. So what's the big concern exactly? Can't Apple at least provide a way for end-users to do what can now be done using Clone X, so they could create their own custom emergency discs without needing third-party software? This would not conflict with the need for legal restrictions on those making a commercial product. At this point, the usual reply is a shrug and a desire to change the subject.
But what about the developers, such as Micromat and Prosoft, that produce bootable discs? How do they do it? Don't they get some help from Apple? Yes, Apple does give them support. But you would have to jump through several hoops before you could get it. First, you need to be an dues-paying Apple Developer. Second, you need to get a license to use Apple's boot disc technology, plus pay an annual fee to keep the license. Although there is an Apple Developer page that provides information regarding software licensing, it makes no mention of licensing bootable disc technology. You have to know to ask. And, if you do ask, Apple will tell you that, before they even consider giving you a license, you need to show them your proposed software in order to demonstrate why it requires a bootable disc. Even after all of that, you don't get the tools to put a Finder on a disc (as could be done with BootCD). You are restricted to the more limiting technology of having a single alternate utility launch at startup.
To be continued. Clearly (and sadly), Apple does not want to assist end users (or even most developers) in creating custom bootable optical discs. They certainly weren't going to provide any help to me. If I wanted to learn more about how all of this works, I would have to look elsewhere. So I did. Within some limits, I eventually learned how to create my own custom bootable CDs and DVDs, without using any third-party software. I'll tell you all about it in the conclusion of this topic, coming in next month's column.
Part 2: The trouble with creating bootable drives and discs
In the above, I described the hurdles that Mac OS X puts in your way when you try to create a bootable disc, especially a custom bootable CD, DVD or flash drive. I offered various solutions but omitted coverage of one critical category: a custom bootable disc that you can make yourself (that is, without having to purchase any specialized third-party software) and from which you can launch your preferred selection of utilities. That's the entire subject of this month's column.
Create a custom bootable CD or DVD
Here are the step-by-step "do-it-yourself" instructions for creating your own custom bootable Emergency Toolkit disc:
Step 1: Get a third-party bootable disc.
Perhaps this reminds you of the old joke: "How do you make ten million dollars in less than a year? Step 1: Start with 9 million dollars." Yes, I admit it. This appears to break my promise that you won't need to purchase any additional software. My escape clause is that you don't need to buy software that is just for this purpose. That is, many people already own the required bootable disc, because they wanted a repair utility, such as DiskWarrior or TechTool Pro, that exceeds what Apple's Disk Utility can do. If you are one of these people, you are good to go. If not, you'll need to get a disc. There is a possible alternative that I mention later; but having the disc is the preferred way to go.
By the way, the reason you need a third-party disc, rather than simply using a Mac OS X Install disc, is that third-party discs include two files that are critical for launching an application other than the Installer on startup. These files are not on Apple's Install discs. There may be ways to circumvent this (such as by directly modifying the file that calls the Installer utility), but I did not explore this route.
Also, though it may seem obvious, make sure that the third-party bootable disc you are using meets requirements to actually boot your system. Some discs are PowerPC-only and can't boot Intel-based and vice versa. Also, some discs don't have software new enough to boot the latest Macs (there is a way around this limitation in some cases, which we'll discuss later).
Step 2: Use Disk Utility to make a read/write image of the bootable disc (see last month's column, if you need help here).
Step 3: Mount the disk image and go to its /Applications folder. Delete any applications located there that you don't want on your final disc.
Note: There may be invisible items in this folder. To delete them, use any utility that allows you to temporarily make invisible items visible (such as the freeware InVisibles) and drag the unwanted files to the Trash.
Step 4: Copy the utilities that you want on the disc to the Applications folder. Include QuickerPicker as one of your utilities.
You are of course limited by what can fit within the size limits of the CD or DVD that you intend to burn. Also, some utilities may not run -- under any circumstances -- when you boot from a optical disc. This issue resulted in the first obstacle on my road to success. An essential application was an application launcher. This is the program that would actually open at startup and thereby provide access to all the other applications on the disc. I downloaded about a half-dozen potential candidates (including Path Finder and QuickSilver) and tested each one out. All failed except one. That is, when I burned and tried to boot from the disc, the system would crash at the point of trying to load the launcher application (the general causes of such failures are covered in my previous column). The lone exception was the aforementioned freeware utility called QuickerPicker. I may have missed others that would work as well, but I stopped my search at this point.
Essentially, QuickerPicker auto-generates a list of all the applications in the startup disc's /Applications folder and allows you to launch any of them. Obstacle solved!
Step 5: Modify the disc so that it will launch QuickerPicker at startup. This is the key step of the entire process. To do this, locate the icon for the etc folder (at the root level of the disc) and open it. Inside, you will find a file named rc.launchapp. Using any text editor (Mac OS X's TextEdit will do), open the file.
The exact contents of the file will vary depending upon which original bootable disc you chose. However, they all include a line that contains the absolute path to the executable file of the utility that launches at startup. The line may consist of just the path name or it may start with something like MYMACHOAPPBINARY = followed by the path name in quotes. In either case, this is the line you need to change. Change whatever path is listed to the absolute path for QuickerPicker's executable file:
/Applications/QuickerPicker.app/Contents/MacOS/QuickerPicker
Save the modified rc.launchapp file.
Note: There are most likely only three lines that are critical for the rc.launchapp file to work. This means, you could create this file yourself from scratch, rather than copy it from another disc. Here's how:
1. Launch TextEdit. From the Format menu, select Make Plain Text.
2. Enter the following three lines:?#! /bin/sh?/Applications/QuickerPicker.app/Contents/MacOS/QuickerPicker?/sbin/reboot
3. Save the file as rc.launchapp. A .txt extension will likely be added to the name. From the Finder, select Get Info for the file and delete the .txt extension from the Name & Extension field.
4. Make the file executable. You can do this by using the chmod command in Terminal (I'm omitting the details here) or via any of several shareware and freeware utilities. My preferred choice is XRay, even though it is not free. Use XRay to open the rc.launchapp file, then go to the Permissions section and enable the Execute bit for Owner, Group, and World.
5. Eliminate any Type and Creator codes for the file. This can also be done with XRay (as well as with Terminal or with any of several freeware/shareware alternatives). To use XRay, go to the Type, Creator & Extension section and clear out any Type or Creator entries there.
6. Save the rc.launchapp file. At this point, the icon for the file should change to the same UNIX Executable icon as the rc.launchapp files on existing bootable discs. The file is now ready to replace the rc.launchapp file on your custom disc.
Step 6: Launch Disk Utility, select the disk image from the column on the left and click the Burn icon in the toolbar. Burn the disc.
Step 7: Boot from the newly burned disc. The easiest way to do this is to leave the disc in the drive, select to restart and hold down the C key as soon as you hear the startup tone. If all went well, QuickerPicker should appear within a few minutes.
Create a custom bootable flash drive
While doing research for this article, so many DVDs wound up as coasters (not to mention the time wasted waiting for each DVD to burn) that I sought out a cheaper and faster alternative. The solution turned out to be my USB flash drive. On Intel-based Macs, you can boot from these drives.
The actual procedure is almost the same as working with a CD or DVD. The only difference is in the last two steps, which are as follows:
Step 6: Launch Disk Utility and select the Restore tab. Locate the name of the modified .dmg file in the list on the left of the window. Drag it to the Source box. For the Destination, drag the listing of USB flash drive volume. If there is anything on the drive, enable the Erase Destination checkbox. Click to Restore.
Note: The size of the dmg file may be larger than the capacity of your flash drive. For example, I had a 1GB drive but the DVD image was the standard 4.7GB. That's okay as long as the total space used on the disk image is less than the size of your drive. This is because only the used content should get copied. For example, in my case, I got the total used space down to about 950MB. For reasons I never figured out, Disk Utility would sometimes attempt to copy the entire 4.7GB of the disk image and thus fail. But if I tried again, it would usually succeed.
Note: If your flash drive is not already formatted as a Macintosh volume, reformat it before doing the Restore. To do so, select the drive from the list in Disk Utility, click the Erase tab, select "Mac OS Extended (Journaled)" as the Volume Format and click the Erase button. Interestingly, it was not necessary to partition the drive as a GUID Partition Table (via the Options in the Partition tab); it worked fine using the Apple Partition Map even though Apple recommends using GUID for booting on an Intel Mac.
Step 7: With the flash drive mounted in a USB port, restart your Intel Mac. Hold down the Option key at startup. From the screen that appears, click to select the flash drive as the startup device. QuickerPicker should soon appear.
As an added bonus, which was key to facilitating my experimenting, working with a flash drive means you can make changes to the contents of the drive without the hassle of having to reburn a DVD each time!
Choose your Mac OS X version
One last obstacle emerged during my testing ‚after I received my new Core 2 Duo MacBook Pro. These latest models will only boot from Mac OS X 10.4.8 or later. Unfortunately, all the third-party bootable discs I currently own use Mac OS X 10.4.78 or earlier. This meant that the above procedures would not succeed on my new machine.
Not willing to give up in defeat, I found a solution. The procedure can apply to any similar situation where you want to use an OS version other than the one on the third-party bootable disc. Here's what to do:
1. Start with a Mac OS X Install disc that contains the OS X version you want to use. Make a read-write image of the disc. In my case, I used the Mac OS X Install 1 DVD that came with my MacBook Pro.
2. Delete non-essential contents from the disk image. In my case, I wanted to get the total used space to be less than 1GB (so it would fit on my 1GB flash drive). This was accomplished by deleting all the items in the /Applications and /System/Installation/Packages folders (including invisible items).
3. Add the desired applications, including QuickerPicker, to the /Applications folder (again, making sure the total used space stays below the size of your destination drive).
4. From any Mac OS X 10.5 third-party bootable disc (even one that contains an older version of OS X!), locate the two essential files in the etc folder: the aforementioned rc.launchapp and a file called rc.cdrom.postWS.
5. Copy these two files to the etc folder of the disk image. Modify the rc.launchapp file to point to QuickerPicker, as described above.
6. Use Disk Utility's Restore command to copy the modified disk image to the flash drive, again as described above.
When I did this, the resulting flash drive booted successfully from Mac OS X 10.5.8 and launched QuickerPicker. You could presumably use a similar procedure to create a bootable DVD. Unfortunately, this was less than a complete success. The biggest problem (and it is indeed a big one!) was that I could not get any keyboard input to be accepted by any application (except QuickerPicker itself). Only mouse input worked. The reason remains a mystery for now, although I suspect that using the Install Disc as the starting point is not a good idea after all. I am still working on this.
Finishing up
You now know how to create a custom disc or USB flash drive that can boot and launch your own desired set of utilities ‚within limits. Yes, you could accomplish a similar feat using a utility such as Clone X 2 (as described in my previous column). But now you know how to do it yourself with greater flexibility and probably less cost. At the very least, you hopefully know more about how these bootable discs work than you did before.
In brief, to create a custom bootable disc or flash drive, all you need is (1) any bootable disc (even a Mac OS X Install disc, in some cases) and (2) the rc.launchapp and rc.cdrom.postWS files from a third-party bootable disc. The rc.launchapp file is so simple, you can create the needed modified version yourself (as I described above). Even the rc.cdrom.postWS file is only a few dozen lines of code. You could similarly make your own version of this file. Still, these files are copyrighted by Apple ‚which means that Apple would likely frown on any public distribution of these files (self-made or copied) or discs made using them. So restrict the results of your handiwork here to just yourself!
Finally, I don't assume that my experimenting has necessarily uncovered the best ways to accomplish the desired goals. If you know of alternative better solutions, I would like to hear about it. You can send me an email message or post a comment.
In accordance with Title 17 U.S.C. Section 107, this material is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. DesignKeyWest has no affiliation whatsoever with the originator of this article nor is DesignKeyWest endorsed or sponsored by the originator.
