I discovered that using a GPT partition scheme meant that my stick wouldn’t work with older systems, which is a shame because I had a partition for using like a regular USB stick which is useful for transferring files around.

As it happens you can have disks that are both GPT and MBR, as detailed at http://www.rodsbooks.com/gdisk/hybrid.html (note that that guide is written by Rod Smith, the maintainer of rEFInd mentioned is my last post on this, and also gdisk, which I used for making hybrid MBRs before I realised it was also one of the things iPartition could do).

Another thought I had was: what about these news PCs that will be using UEFI (an updated and more standardised EFI)? I have some theories about being able to put the right file (say /efi/boot/bootx64.efi, which is a standard fallback location) in a partition by itself, with bootx64.efi being a GRUB EFI application made to use my existing HFS+ partition for it’s modules. Unfortunately I haven’t got any computers to test this on.

Sounds useful. Should be a lot easier to do than it is. This has been an ongoing project of incremental improvements and sudden inspirations, I don’t think I’ll be able to document it all.

OK start from basics. To be bootable you need something to boot. GRUB is a bootloader that will work on PC and Mac. OK good, how to make a computer boot it. Using the age old method of BIOS booting. Put a little bit of machine code on the first block (512 bytes) of the disk, BIOS computers will run this, and it should do things like find any more machine code it needs and list and boot operating systems on the computer. GRUB needs to be installed to work on BIOS computers in sort of 3 parts. That first bit of boot code, which finds more boot code on the disk at a static offset which needs to be preprogrammed in the first part. Because there will be some sort of partitioning scheme like MBR that takes up the rest of the disk, the scheme will have to leave some space somewhere for the 2nd part of the machine code. Once GRUB has got all that together it should have enough functionality to access a volume/partition on the disk and load the rest of its modules and settings from the filesystem.

So I made an MBR partition scheme with enough enough free space on my new USB stick and set about installing grub on it. The grub distribution running on my Debian virtual machine was designed to tune the grub installation on the computer it was running on. I extracted elements from it to create a more generic set of grub installation scripts. And reading the manuals and doing some tests I managed to get an image of my USB stick to boot in a VM. Unfortunately I don’t have a spare computer lying around the test the actual USB stick.

GRUB has a nice bit of functionality which can make CD ISO9660 images (.iso) into loopbacked filesystems, and it is good at booting linux with just a kernel file and an initial RAM disk file. So any linux distribution which can be passed boot parameters to make itself aware of this and do a similar loopback mount of the iso (like SystemRescueCD and Debian install discs) can be booted like this without having to make separate partitions for them, which is nice.

A Windows install would be nice for fixing (read reinstalling) Windows stuff too. The one I have a license for is Windows 7 Professional 64 bit. I don’t think I’ll be able to do this with just an iso image. GRUB is not so good at booting windows, it relies on chainloading: being forwarded an address with more machine code to run to boot the operating system. I copied all the files on the windows install CD to a partition on the memory stick so most of the stuff it needs should be there. As far as I can tell there’s nothing on my Windows 7 install CD image to make the boot code for arbitrary partitions on a USB stick. Luckily I discovered ms-sys, which has recently added being able to to write what it calls an NTFS boot record, which I hoped I would be able to bootstrap the files on. Like the boot code at the start of a disk before the MBR partition table this boot record sits at the start of a partition, before the NTFS filesystem structure. Chainloading this boot record worked to my great surprise, I hadn’t found anyone else on the internet using this method.

So I had a USB stick that would theoretically work on a BIOS PC. Good start. Now lets’ looking into Macs. Macs don’t use BIOS. Bad start. They work with a sort custom version of an outdated EFI (Extensible Firmware Interface, designed to replace BIOS) spec. Firstly it required a GPT partition scheme to the disk. OK I can do that. Then there are two ways to specify what to boot, either by setting flags and details in the computer’s NVRAM, which require a system running which can do that (like the “bless” command on OS X) so is not ideal for making a repair disk, or with headers in Apple’s HFS+ filesystem. There are no other ways. So GRUB will have to use EFI, on a GPT partitioned disk with a HFS+ filesystem, very different to the BIOS, MBR, FAT32 setup I have.

Let’s see about reconciling the two. BIOS systems can work with GPT, there’s still the 1 block gap at the start of the disk. The GPT filesystem can then set aside some free space for the rest of GRUB. I can keep my old FAT32 partition with grub BIOS stuff in it. As long as there is a HFS+ partition with the right headers in it pointing to an EFI application, a Mac will be able to boot it. Grub can come in an EFI version, but it doesn’t do BIOS so I’ll need both versions on the stick. The Mac version will need its own modules, but should share the config file containing a list of operating systems and how to boot them with the BIOS version, GRUB config files have the “configfile” directive for this and grub can traverse different partitions and filesystem formats easily, as long as you include the right modules.

Around this time I learned that the Mac app I’ve had for a while, iPartition, was much more capable than I thought originally. It excels in nondestructive moving and resizing of partitions and changing the partition scheme and can give complete control over where on the disk partitions go. It could even easily create the BIOS boot partition GRUB needed on GPT disks. It gave me functionality I was expecting to need a collection of obscure command line tools for. I recommend it for anyone who needs to do more than basic disk partitioning.

At this point I had a USB stick that would boot in my bios based virtual machine, and would boot on my actual Mac. I consider that a success. I still can’t get Windows 7 install to work with EFI. You can’t chainload BIOS boot code from EFI. Another bootloader, rEFInd claims to be able to get into the Mac’s (and some other manufacturers’) CSM (Compatibility Support Module), but that may be cheating, and using two bootloaders would make things more complex. Windows 7 does support booting from EFI, and I’ve managed to make a little progress with the EFI app “bootmgfw.efi” on the install disk, but either it’s just doesn’t have the routines for the situation it’s in, or maybe I can change the BCD to make it work.