The most common problems that novices run into with Linux installation relate to boot loaders, partitions and the boot process. After seeing people in chat rooms get into a horrible mess due to the inconsistent terminology and prevalent misunderstandings about how partitioning works, I've tried to write a definitive explanation.
Of necessity, I've taken a historical approach. This whole area has become a mess because of the need to maintain backward compatibility with old hardware and OS's while overcoming their limitations.
Early Days - DOS 2 and the PC/XT
When IBM introduced the XT, it shipped with a 10 MB hard drive (actually the controller could handle 5 MB, 10 MB, 15 MB and 26 MB drives by setting links). While 10 MB sounds tiny by today's standards, it was massive compared to the prevalent 360 KB floppy systems. In addition, DOS did not dominate the market to the extent that it later did - some users were running UCSD p-System, CP/M-86, Venix and other alternative operating systems. IBM therefore had to provide a way to allow multiple different operating systems to share a single hard drive.
This was done by placing a partition table on the outermost cylinder of the hard drive. The partition table is actually a small data structure which follows 446 bytes of boot loader code in the first sector of the hard drive. It contains four 16-byte entries, each of which contains the following information:
Extended Partitions - DOS 3.3
DOS 3.0 to 3.2 uses the same basic scheme. The only difference is that the FAT entries in DOS 3 were increased in size to 16 bits, thus allowing the maximum drive size to be increased - although because internally, DOS tracked sectors using 16-bit values, the maximum drive size was only 32 MB (65536 sectors x 512 bytes per sector).
This posed a problem for manufacturers, as larger hard drives were starting to appear. With the PS/2 Model 80, introduced in April 1987, IBM was planning to support drives up to 110 MB in size, which DOS could not support directly.
Table - Partition Type numbers
So a change to the partitioning scheme was introduced. DOS would now support two different partition types. The original DOS partition would be renamed a "Primary DOS Partition", while a new "Extended DOS Partition" would be introduced. An Extended DOS Partition does not contain a conventional boot sector; rather it contains yet another "Master Boot Record". This "MBR" is slightly different from the one on the outer cylinder, though - it cannot contain boot loader code, and it can only contain two entries: one for a conventional DOS partition, and one for the remainder of this extended DOS partition, which is itself an extended partition, and so on, recursively. Each of these DOS partitions within the extended DOS partition is called a logical drive by DOS. An Extended DOS Partition, notice, is just another type of primary partition, so that in practice, a large drive would have two entries in the MBR partition table: one for the Primary DOS Partition and one for the Extended DOS Partition, which would in turn contain all the other logical DOS drives.
Sometimes the chain or linked list of MBRs that makes up an extended partition can be corrupted and broken; this is rare but it has happened to one of my colleagues. This is where the 0xAA55 magic number at the end of the MBR comes in handy - partition repair tools like gpart are able to use any remaining information in the primary MBR that points to the beginning of the extended partition, and then search around for sectors that end with the magic number. With a little luck - hopefully nothing else on the disk is using the magic number - it will be able to reconstruct the linked list and bring partitions back to life.
Since each MBR within an extended partition only contains one entry for a real partition, there's no more need to worry about it being different from other partitions in the same MBR. This means that an Extended DOS Partition can contain multiple DOS drives, and it was not uncommon for computers of that era to have multiple DOS drives - C:, D:, E:, F: and so on, each being at most 32 MB.
The next stage in the evolution was modifying the internals of DOS to support 24-bit and then 32-bit sector numbers. This allowed drives up to 2 GB in size, although because the FAT still had 16-bit entries, allocation of space was done in clusters which were up to 64 KB in size - very inefficient when you have lots of small files (e.g. on the old bulletin board systems). The final stage was the introduction of 32-bit entries in the FAT, which occured with Windows 95 and allowed for large drive sizes with smaller clusters for reasonably efficient allocation. This is generally referred to as the VFAT or FAT32 format, which is still found today on some Windows systems.
The Windows NT family also supports NTFS (NT FIle System), which has never had the size restrictions of the FAT scheme, but has had a few updates in order to deal with increasingly large drives - but these are well outside the scope of this article. Perhaps the most important point to make here about NTFS is that Linux has never had particulary good support for NTFS - there has been support for NTFS read access in the kernel for some time (although Red Hat somewhat annoyingly disable it by default) and write support is still questionable - it can reliably write within an existing file on NTFS, but cannot create new files or extend the size of existing files. For this reason, many distributions have NTFS support, but with write support disabled. However, NTFS support has been improving rapidly and may be usable by the time you read this - check the /usr/src/linux/Documentation/filesystems/ntfs.txt file in your kernel's source code for details.
Users who wish to dual-boot between Windows 2000/XP and Linux, and share data between them are advised to either format their C: drive as VFAT or create a separate VFAT partition which can be used specifically for shared files. Or you can always do what I do - run both Windows and Linux simultaneously under VMWare, with Linux sharing home directories via Samba. Expensive, and needs a lot of RAM, though.
Linux Terminology
Linux uses slightly different terminology:
A typical PC has two IDE channels, each of which can have a master device and a slave device attached. Linux refers to these in the following way:
SCSI drives are named /dev/sda, /dev/sdb and so on.
The four entries in the MBR's partition table refer to the first four partitions, which (on the typical hard drive) Linux names /dev/hda1, /dev/hda2, /dev/hda3 and /dev/hda4. Logical partitions (within an extended partition) are then identified as /dev/hda5, /dev/hda6, and so on. Notice that if a drive is configured with one primary partition and three logical partitions, then it will probably look something like this:
Note that /dev/hda3 and /dev/hda4 do not exist at all - they correspond to the unused entries in the MBR partition table - and while /dev/hda2 does exist, it is not referred to directly by Linux in normal usage; it is simply the extended partition which contains the three logical partitions.
Linux also relaxes another rule: You can have multiple Linux partition entries in the MBR partition table, which makes things a lot easier.
Filesystems
Now it's important to introduce another term: filesystems. A partition is just a cylindrical area (volume, really) of the disk that is reserved. That area has to have some kind of format and directory structure within it, so that an operating system can access files and directories. This format is called a filesystem. We've already discussed two filesystems used by DOS/Windows: FAT and NTFS. Linux has its own filesystems, and in typical open-source style, there are many to choose from. Most Linux distributions will, by default, use the ext2 or ext3 filesystems, but there are many others to choose from, each with their own advantages and disadvantages: ResiserFS, JFS, XFS and many others.
Generally, a filesystem is the same size as the partition it is in, but this is not always the case. It's important to realise that a filesystem can be smaller than the partition it occupies and that generally, any resizing operation (and these can be done on the fly with some Linux filesystems) generally consists of either first shrinking the filesystem and then shrinking its partition, or first enlarging the partition and then enlarging the filesystem inside it
Resizing Windows Partitions Before Installing Linux
Linux is an operating system, and is intended to reside in its own partitions on a disk drive. It's not an application that installs and runs under Windows. The biggest barrier that novices face when installing Linux is the fact that manufacturers usually deliver computers with the hard drive completely occupied by a single Windows "C: drive" partition. Linux can't install into that, no matter how much free space you have inside it.
The answer is to shrink the Windows partition, creating unallocated space on the disk, so that the Linux installer can then create its partitions normally (I'm assuming you don't want to simply blow away Windows, which is the easiest approach, but want to keep a dual-boot system while you explore Linux). There are several tools that can shrink (and otherwise manipulate) the commonest partitions.
Partition Magic ( http://www.powerquest.com/partitionmagic/ ), originally from PowerQuest but now owned by Symantec, is a commercial program that lets you create, resize and convert between the commonest partition types: FAT, FAT32, NTFS and the Linux ext2/ext3 filesystems. Although it can create ext3 filesystems, I would recommend that you not use this function, but instead rely on PM only to shrink any existing Windows partition and let the Linux installer create partitions for itself. (Similarly, it's best to ignore PM's Boot Magic and instead let the Linux installer set up GRUB or LILO).
There are several free & open source solutions to the partition resizing problem, too. If your system is running Windows 98 or Me, then the C: drive will be in FAT32 format. You can defragment the drive under Windows and then use a program called fips ( http://www.igd.fhg.de/~aschaefe/fips/ ) to split the partition in two. The first partition will contain all your Windows programs and data, while the second contains only empty space and can now be deleted, leaving space for your Linux installation. Many Linux distributions provide the fips program on their CD's, usually in a directory called dosutils or similar.
For Windows XP, 2000 and NT systems where the C: drive is NTFS, the best free option is a utility called ntfsresize (http://mlf.linux.rulez.org/mlf/ezaz/ntfsresize.html). But this runs under Linux, which you haven't installed yet - how to resolve this chicken-and-egg conundrum? And again, there are several answers.
Many recent Linux distributions have integrated ntfsresize into their installation scripts, often with a graphical front end to make life easy. Examples include Mandrake 10.0 and later, SuSE 9.1, Conectiva 10 and others. With these distributions, the installation process incorporates resizing of NTFS partitions, making installation a snap. Warning: the early versions of distributions using the Linux 2.6 kernel would sometimes render Windows XP unbootable (see sidebar "Problems with Linux 2.6 Kernel") but the latest versions should have these problems ironed out.
Alternatively, you can use any of several so-called "live CD" Linux distributions that boot and run off a CD-ROM and which incorporate ntfsresize, such as recent versions of Knoppix (http://www.knopper.net/knoppix/index-en.html)or Mepis (http://www.mepis.org/). Finally, there are several mini-distributions that are designed specifically for system repair, such as SystemRescueCD (http://www.sysresccd.org/) and RIP (Recovery Is Possible) (http://www.tux.org/pub/people/kent-robotti/looplinux/rip/). RIP, in particular, is small enough to fit (and boot from) two floppy disks, a CD-ROM or a USB nerd-stick, which is particularly useful.
Resizing an NTFS partition with SystemRescueCD is as easy as booting from the CD, entering the command "run_qtparted" at the command prompt, selecting the correct mouse type, then - with the mouse - selecting the correct drive, then right-clicking on the partition and choosing "Resize...". After entering the desired size, select File -> Commit and that's it - all done!
Boot Loaders
Technically, it is possible to simply put an operating system in the first few sectors of a floppy or hard disk and have the computer simply load it into memory and execute it - in fact, with Linux kernels up to and including 2.4, you can easily copy a kernel file to /dev/fd0 in order to create a bootable floppy disk. However, in most situations, a boot loader is used; the term boot comes from bootstrap, describing the way in which the computer executes a sequence of increasingly large programs, each loading the next, until the operating system us up and running (like "lifting oneself up by one's bootstraps"). The boot loader adds functionality, such as the ability to choose between alternative operating systems, alter the boot configuration or perform debugging of the boot process.
The DOS Boot Sequence
The boot sequence for DOS and Windows is quite standardized:
2. The MBR runs, consults the partition table to find the active partition, and loads the first sector from it.
3. This now runs, loads the DOS BIOS and BDOS files, and DOS starts running.
The Windows (NT, 2K, XP) Boot Sequence
Windows NT has a more sophisticated boot loader which does offer the opportunity to boot other operating systems. This is configured by the (normally hidden) file C:\BOOT.INI. It is possible to use the Windows NT boot loader to chain to a Linux boot loader - see the article "Using the NT Boot Loader to Boot Linux". The boot sequence is similar to that of DOS:
3. This now runs, loads the ntldr file and executes it.
4. ntldr reads the boot.ini file and displays a menu which may allow other operating systems to boot
5. If Windows is selected, it runs NTDETECT.EXE, collects hardware information and passes this to the kernel via the registry, then loads Ntoskrnl.exe and hal.dll, along with any required device drivers, and then turns control over to the kernel
Listing - Example Windows 2000 BOOT.INI file which can chain to a GRUB boot sector
[boot loader] timeout=30 default=multi(0)disk(0)rdisk(0)partition(1)\WINNT [operating systems] multi(0)disk(0)rdisk(0)partition(1)\WINNT="Microsoft Windows 2000 Professional" /fastdetect c:\boot.lnx="Red Hat 9 - Freya2"
LILO
LILO (http://lilo.go.dyndns.org/) is the LInux LOader, and is one of the earliest boot loaders for Linux. LILO works by replacing the DOS/Windows code in the MBR with its own code. The problem that LILO faces is that it is a small program that does not understand filesystems and files - it cannot load a file like /boot/vmlinuz. All it knows about are sectors, basically - and given a list of sector numbers, it can load what it finds there. The list of sector numbers is stored in a map file (/boot/map) - but of course, LILO can't access that as a file, so it has to know what sectors it occupies.
All of this is sorted out by the lilo command, which is run whenever required - e.g. after kernel updates and changes, new operating systems added, and so on. When you run the lilo command, it reads the /etc/lilo.conf file and creates the map file - for each Linux kernel and related initrd file it stores their sector locations into the map file, along with the kernel command line and various options. Finally, it also works out the sector locations of the map file itself, and writes that into the MBR along with the boot loader stage one code.
Its bootstrap sequence works like this:
2. The LILO code in the MBR loads a stage two loader which in turn produces the LILO: prompt and waits for the user to select a kernel image. The user can press Tab to see a list of available images, and select one by typing its label
3. The stage two loader loads the corresponding kernel or chains to the selected partition's boot loader. If loading a kernel, LILO can pass command line arguments, specify the root filesystem (and whether it will be mounted ro), specify an initrd file, etc.
4. The kernel starts running, then loads init, which in turn starts running the startup scripts
Listing - A typical lilo.conf file, which can boot two different versions of Linux as well as the original Windows
boot=/dev/hda map=/boot/map install=/boot/boot.b message=/boot/lilo.msg prompt timeout=100 image=/boot/vmlinuz-2.6.9-vidalinux_r1 label = gentoo root = /dev/hdb2 initrd = /boot/initrd-2.6.9-vidalinux_r1.img read-only image=/boot/vmlinuz label = Linux root = /dev/hda6 read-only other=/dev/hda1 label=Windows table=/dev/hda
LILO normally loads and displays the prompt "LILO:". But cunningly, each letter of that prompt is produced by a different stage of the boot process, which is useful in diagnosing problems.
Many of these problems can be fixed by rebooting from rescue/repair media and then re-running the lilo command. In the case of drive geometry problems, my experience has been that these can often be fixed by adding the "linear" option in /etc/lilo.conf.
LILO fell out of fashion a few years ago, due to its inability to deal with drives that had more than 1024 cylinders (this is why distributions like Red Hat created a separate partition for the /boot filesystem - by keeping that within the first 1024 cylinders one could guarantee that no part of a boot loader or kernel file would fall outside LILO's 1024-cylinder restrictions). LILO has subsequently been updated to deal with large drives, but most distributions now use a newer boot loader:
GRUB
GRUB (http://www.gnu.org/software/grub/grub.en.html) is the GNU GRand Unified Bootloader. The big philosophical difference between GRUB and LILO is that GRUB contains a mini filesystem driver and is therefore able to access files by name, rather than sector locations. This elimnates the need to run a mapping command (like lilo) each time a file is added or moved. GRUB also includes facilities for editing the boot parameters at boot time, or even constructing completely ad-hoc boot configurations.
The GRUB boot sequence is a bit more complex:
2. The Stage 1 boot loader uses an internal cylinder/head/sector map to load the GRUB stage 1.5 loader, which contains the mini filesystem driver. From this point on, GRUB is able to access files by pathname inside the supported filesystems
3. GRUB now loads the stage 2 driver (/boot/grub/stage2). This reads /boot/grub/grub.conf and displays the boot menu
4. The user selects an OS, and stage 2 loads the desired kernel, passing command line arguments, etc. or chains to the desired partition boot sector to load some other OS.
GRUB is configured by /boot/grub/grub.conf (or /boot/grub/menu.lst). This is a text file that sets up some default parameters and then has a section for each bootable partition or OS
Listing - A sample GRUB.CONF from a dual-boot Red Hat 9 and Windows 2000 system.
#boot=/dev/hda3 default=0 timeout=10 splashimage=(hd0,2)/grub/splash.xpm.gz title Red Hat Linux (2.4.20-31.9) (runlevel 3) root (hd0,2) kernel /vmlinuz-2.4.20-31.9 ro root=/dev/Volume00/LogVol00 3 initrd /initrd-2.4.20-31.9.img title Red Hat Linux (2.4.20-31.9) root (hd0,2) kernel /vmlinuz-2.4.20-31.9 ro root=/dev/Volume00/LogVol00 initrd /initrd-2.4.20-31.9.img title Win2K rootnoverify (hd0,0) chainloader +1 You'll notice in this listing that each OS has a title line, followed by a root directive. This specifies the root filesystem to be used by the GRUB loader, not the root filesystem for Linux after it starts. Here, the /boot filesystem is /dev/hda3, which in GRUB's zero-based indexing scheme is hard disk zero, partition two, or (hd0,2).
The kernel directive specifies the location of the kernel file, relative to the GRUB root - for example, /vmlinux which is in the root directory of the /boot filesystem. In this example, the first and second sections are very similar - they only vary in the command-line arguments for the kernel: the first one appends a three on the end of the line so that the system boots to runlevel 3 (no GUI).
Where to install the Boot Loader?
A common problem for first-time installers is where to put the boot loader. Typically, Linux installation programs will suggest putting the GRUB or LILO boot loader in the MBR - but hesitant novices tend to view the Master Boot Record with awe and will often override this and install it into the Linux root or boot partitions instead. There are two problems with this:
How to Uninstall the Boot Loader?
Normally, you don't uninstall an operating system - you just overwrite it with something else. However, there's a gotcha here - the GRUB boot loader, in particular, loads its stage 1.5 and 2 loaders from files inside /boot. If you delete the partition that contains /boot, then GRUB cannot load its stage 1.5 loader to display the boot menu and cannot boot Windows, let alone Linux.
The safest thing to do is to replace the default DOS/Windows boot loader code in the MBR before you start deleting partitions. The simplest way to do this is to boot from a DOS floppy disk or Windows recovery/repair disk and run the command
fdisk /mbr Alternatively, you can boot from a Windows XP Setup CD-ROM, select 'R' to repair the installation, enter the Administrator password and at the Recovery Console command prompt, give the command:
fixmbr and press the 'Y' key to proceed.
How to Reinstall a Broken GRUB Boot Loader
The technique required varies with the distribution, but basically, the idea is to boot off some other media such as a rescue disk, mount the required partitions (usually those for /, /boot and /usr) and then use the grub-install command to place the GRUB boot sector into the right place, with the right CHS values for the stage 1.5 loader. For example, Red Hat and Fedora installation CD's can be used in a recovery mode, like this:
Boot from the installation CD. At the syslinux boot prompt, give the command "linux rescue". This will load the installation kernel, but will not perform an installation - instead, it will attempt to recover the system. The first few questions ask about language and keyboard - just answer as required, or with the defaults, until you get to the question that asks whether you want to mount the hard disk partitions under /mnt/sysimage - answer yes to this, and mount them read/write, rather than read-only.
Once the system has mounted the partitions, you will be given a command prompt. Give the following commands:
chroot /mnt/sysimage grub-install /dev/hda This will install the GRUB boot sector into the MBR, and you should now shut down, remove the CD-ROM and reboot, whereupon you should be presented with the GRUB boot menu and will be able to select Linux (or Windows, in a dual-boot system).
Problems with Linux 2.6 Kernel
The way that the Linux kernel reports drive geometry (the number of cylinders, heads and sectors per track) was changed between the Linux 2.4 and 2.6 kernels, and this can give rise to some problems with utilities such as GNU parted, ntfsresize, GRUB and others.
Users of Fedora Core 2 struck this problem when GRUB wrote out a new MBR with updated drive geometry, rendering Windows XP unbootable. One way to fix the problem is to reboot using a recovery CD that uses the 2.4 kernel and give the command:
sfdisk -d /dev/hda | sfdisk --no-reread -H255 /dev/hda This will rewrite the MBR with updated values that make sense to Windows, which usually expects to see a drive geometry with 255 heads and 63 sectors per track.. However, on some systems (e.g. the IBM Thinkpad I'm writing this on) the Windows MBR was written with a drive geometry of 240 heads, so you might have to try that command with the -H240 option instead. A better approach would be to use the sfdisk -l or fdisk -l command to list the drive geometry and write it down before performing any disk trickery with a Linux 2.6 kernel. Remember, many Linux installation CD's produce a shell prompt on a spare virtual terminal during installation, or can be booted into a rescue mode which will allow this command to be executed.
For more information on this problem, see https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=115980 and http://portal.suse.com/sdb/en/2004/05/fhassel_windows_not_booting91.html
References and Further Reading
The master boot record and partition table layout is detailed in /usr/include/linux/genhd.h
The ntfsresize FAQ Page: http://mlf.linux.rulez.org/mlf/ezaz/ntfsresize.html
Red Hat's documentation on using FIPS to split a FAT partition: http://www.redhat.com/docs/manuals/linux/RHL-8.0-Manual/install-guide/s1-x86-dualboot-fips.html
Repairing Damaged MBRs and Boot Sectors in x86-based Computers (from the Windows XP Resource Kit): http://www.microsoft.com/resources/documentation/Windows/XP/all/reskit/en-us/Default.asp?url=/resources/documentation/Windows/XP/all/reskit/en-us/prkd_tro_evwd.asp
Tools and Resources
GNU Parted is a program for creating, deleting, resizing, checking and copying partitions and the filesystems inside them. It primarily operates on the commonest Linux filesystems and FAT32. See http://www.gnu.org/software/parted/
There are two graphical front ends that simplify the use of Parted and can also integrate external tools such as ntfsresize:
QtParted is superficially very similar to Partition Magic; since it uses the Qt libraries for graphics, it's most likely to appeal to KDE users, although it can be statically linked for standalone use. See http://qtparted.sourceforge.net/
GParted is the Gnome Partition Editor, and uses gtkmm for its graphics toolkit. See http://gparted.sourceforge.net/
Gpart is a tool that attempts to repair damaged partition tables by scanning the disk looking for the signatures that indicate the locations of accidentally-deleted or otherwise lost partitions. See http://www.stud.uni-hannover.de/user/76201/gpart/
Download of the Month - SystemRescueCD
This has gone straight into my CD wallet for future use, replacing a number of other tools that were on various CD's and floppies. SystemRescueCD is a live CD distribution based on Gentoo Linux (with kernel 2.4.27, avoiding the drive geometry problems of 2.6) which includes all of the utilities discussed here and a few more: GNU Parted, QtParted, ntfsresize, partimage, dump & restore, sfdisk, dar (disk archiver - useful for backing up to CD-R[W]), and various filesystem maintenance utilities, including support for the IBM JFS and SGI's XFS. It also has support for the EVMS and LVM logical volume management systems, providing commands for managing physical volumes, volume groups and logical volumes.
But wait! There's more ClamAV (the manual documents how to update the virus definitions in a RAM drive), Midnight Commander, various editors (vim, nano, QTinyEditor), support for network filesystems like Samba, NFS and LUFS, and SSH (both client and server).
The CD has multiple boot images for different graphical resolutions, or to bypass hardware detection or network configuration, etc. And apart from the Linux rescue images, there are other images on there as well: the ntpass disk for resetting NT/2K/XP admin passwords (one of my favourite tools!), memtest86+ memory tester, aida hardware diagnostics, freedos (ideal for those occasional old network cards that use a DOS utility to reconfigure them, but it also includes a FAT partition resizer), Ranish Partition Manager, Darik's Boot & Nuke (for aggressively erasing hard drives)
There's an excellent little 45-page manual which contains lots of usage examples and discusses advanced techniques such as burning personalized or updated versions of the CD (which only requires the tools on the CD itself, very neat) as well as burning recovery DVD's that can include the bootable system plus over 4 GB of additional data. Finally, the CD includes the Capture-NTFS utilities, which are able to give it full NTFS read and write support by "capturing" the NTFS drivers off a Windows XP system.
With this CD slipped into your laptop bag somewhere, you can be pretty sure that no matter how you manage to screw up your system, you'll be able to repair it.
Download from: http://www.sysresccd.org/
Price: Free
Page last updated: 04/Jul/2006 Back to Home Copyright © 1987-2010 Les Bell and Associates Pty Ltd. All rights reserved. webmaster@lesbell.com.au