A typical Linux system has a lot of files, of various kinds, organized in a directory tree structure. To someone making the transition from the DOS/Windows world, the profusion of directories can be intimidating, to the extent that deciding where to install program files can be a major puzzle. There's also the problem of how to partition the hard drive when installing Linux - should everything just go in one large partition, and if not, what goes where? This article should answer those questions. An important factor in deciding how to partition the hard drive(s) is the way in which UNIX systems start up, and the related concept of runlevels. Most of the time, the system will be in runlevel 3 (multi-user with network connectivity and network daemons running) or runlevel 5 (the same, plus graphical login and X server running). When the system boots up, it initially mounts the root filesystem in read-only mode and then reads the /etc/inittab file, which tells it which runlevel it should now switch to (see the id: entry). It typically now runs the /etc/rc.d/rc.sysinit script, and then runs the startup scripts for the target runlevel, n, from the /etc/rc.d/rcn.d directory. As part of this process, it will remount the root filesystem read-write and mount any additional filesystems. Just where it does this varies from distribution to distribution, but reading your distro's startup scripts will reveal all. The important point is that before this point, only the root filesystem will be mounted - and even it might only be mounted read-only. Other filesystems - which might include /home, /usr and /var - will not be accessible, and so cannot be used during the boot process. If anything goes wrong during the boot process, and you get to a shell prompt to fix it, those filesystems will not be immediately available to you. Most systems support a single-user maintenance mode - runlevel 1 - in which only console logons are supported (i.e. other users cannot log in via external terminals or network sessions) and in which filesystems other than the root filesystem are not mounted. Single-user maintenance mode is used when performing major system administration tasks, such as creating or removing filesystems - something that would be dangerous if users were running programs with open files on the system. Top-level directories /bin, /sbin, /lib The /bin directory contains binary program files which are used by the system during startup, but which also may be required by users once the system is fully up and running. Examples include /bin/bash, cp, grep, more, pwd, touch and many others. By contrast, /sbin contains programs which are required by the system startup scripts but will not normally be run by users, other than root - in other words, system administration utilities. Examples include modprobe, ifconfig, syslogd, shutdown, route, sysctl, etc. The /lib directory contains shared object libraries (analogous to Windows DLL's) that are required at boot time. In particular, kernel modules will be found here, under /lib/modules. /usr/bin, /usr/sbin, /usr/lib, /usr/share/man, /usr/share/doc The first three directories are analogous to the directories previously mentioned, except that their contents are only required once the system is up and running - since at boot time, /usr may not be mounted. System documentation, in the form of man pages, can be found under /usr/share/man, while other documentation is under /usr/share/doc /usr/local/bin, /usr/local/sbin, /usr/local/lib, etc. The contents of the /bin, /sbin, /lib directories and their mates under /usr are usually installed there as part of the system installation process. On RPM-based distributions like Red Hat, SuSE, etc. the programs installed from RPM packages will wind up there, with documentation under /usr/share/{man,doc}. However, the typical Linux system will also have some programs which were installed by building them from source code tarballs. In that case, their files are placed under /usr/local. Hint: Placing /usr/local on its own filesystem will save it from being reformatted when you re-install. /boot When the system boots, it faces a chicken-and-egg situation: accessing the kernel and related files will require an understanding of Linux filesystem layouts - but the kernel and filesystem drivers have to be loaded off those very filesystems that the machine BIOS does not understand. /etc The /etc directory tree contains configuration files for the system, such as the inittab and fstab files which are used at boot time to set the target runlevel and to specify mount points for additional filesystems. As a result, /etc cannot be a separate filesystem - instead it must be a directory within the root filesystem. /home Users' home directories can usually be found under /home, with one exception: the root account's home directory is usually /root, which is a directory on the root filesystem. This is because, if root's home directory was on /home, an unmounted filesystem when the system boots into single-user maintenance mode, the root account would not be able to log in correctly. /home is quite often a separate filesystem, for a variety of reasons, listed below. /var As it runs, the typical system produces a number of files which need to be preserved, even through a system reboot. Examples include log files, print job spool files, mail spool files and the like. These are almost always placed in the /var filesystem. In addition, applications that share data for multiple users typically place their files under /var. For example, the anonymous FTP directories are often under /var/ftp, while the Apache web server files are commonly placed under /var/http. Database applications like MySQL also commonly place their data files under /var. /tmp This is the location of temporary files produced by programs - such as compilers - as they run. In general, these are short-term temporary files, of use to a program only while it is running, and so the contents of /tmp are not expected to be preserved through a reboot. In fact, the startup scripts of some systems /opt This is the intended location of major third-party application subsystems, such as Star Office, Open Office, Lotus Domino, IBM DB2, etc. Other stuff lost+found You will find a lost+found directory in the top-level directory of every filesystem. When fsck is run on an unmounted filesystem, any recovered data will be placed in files and placed in the lost+found directory. Hint: The existence of a lost+found directory is a good indication that the directory that contains it is a mount point for a filesystem. /dev The /dev subdirectory contains special files which link to the various input/output devices on the system. Because access to these is required at boot time, this is a subdirectory of the root filesystem How to partition? There are many reasons for formatting a system into separate filesystems: To avoid overwriting when upgrading/reinstalling Typically, Linux install programs offer the chance to partition the drive(s) and to specify mount points for the various partitions. You might notice when installing over an existing Linux setup, that after doing this, the "Format:" option on some partitions defaults to yes, and on others, to no. Typically, the root (/, including /etc, /root, /bin, etc.) and /usr filesystems will be reformatted; others will be left intact. In particular, if /home is an existing filesystem, it should probably not be formatted, so that users' files are preserved. Always be careful to check whether partitions will be reforatted by Linux installers. Subdirectories that you might want to consider making a separate filesystem to avoid loss of their contents during upgrades include /home, /usr/local, and /usr/src. Performance On a heavily loaded machine - with multiple users, or multiple network servers, performing multiple tasks at once - a single drive will have its read-write head seeking crazily back and forth, between program files, data files and the swap partition. Bear in mind that using multiple partitions/filesystems will tend to make this worse, by separating programs and the data files they operate on. However, this effect can be ameliorated by planning the partition layout to keep the most frequently-accessed partitions close together. However, on heavily-loaded systems, a useful performance boost can be achieved by using multiple hard drives. With a separate drives for the swap, /var, /tmp and /home, the heads on each can remain over the most-frequently accessed files for long periods, minimising head movement and maximising performance. In addition, particularly with SCSI drives, reads and writes can be performed in an overlapping fashion, further boosting performance. Take care not to distribute vital directories across multiple drives, however. The more drives you have, the greater the probability that one of them will drive, and if every drive has a directory on it that is required at startup time, the more likely it is that you'll have to perform low-level repairs. Backup The dump command is designed to perform full and incremental backups of a filesystem. Its operation is partially controlled by the fs_freq column of the /etc/fstab file, which indicates which filesystems need to be dumped, together with the /etc/dumpdates file which records the dates on which the various levels of dump were performed on different filesystems. The dump command only operates on entire filesystems, and cannot work on subdirectories. Therefore directory trees which need to be individually backed up are candidates for creation as filesystems in their own right. The most obvious candidate for this treatment is /home, but /var should also be considered. Different Filesystem Formats The standard filesystem for Linux systems is ext2, the second extended filesystem. However, many distributions provide support for other filesystems, such as ReiserFS, xfs, jfs and of course, ext3. Each has its own combination of benefits and costs, and the savvy administrator will use them judiciously. For example, ReiserFS's use of B+trees for its directories makes it a good choice for situations where there may be thousands of files in any single directory, such as news servers. Quotas The Linux kernel supports imposition of quotas on disk utilization on a per-user basis. This has to be set up on a per-filesystem basis - since the obvious candidate for this treatment is /home, this is just another reason why /home should be in a separate partition. Space management Running out of space is something that happens to us all from time to time. At that point, one easy solution is to add a second (or third, or . . ) hard drive, and move the filesystem under pressure to it. See the sidebar for a suggested technique. Security Sometimes, security can be improved by making a particular subdirectory into a filesystem in its own right. Two examples: First, /usr can be mounted using the ro option. This makes it a little more difficult for an attacker to upload files into it or otherwise tamper with it. Secondly, if you allow anonymous FTP uploads - never a good idea, but I suppose you have your reasons - then at least make /var/ftp/pub/incoming into a separate filesystem, to stop an attacker from filling up some other filesystem by uploading the contents of /dev/zero into it. Putting all these factors together, one can arrive at a range of different possible arrangements. I typically wind up with a simple arrangement on my notebook computers: a separate /home filesystem to avoid losing work when upgrading, with everything else in the root filesystem. However, my desktop and server setups are more complex, with multiple hard drives, separate /usr, /var, /tmp and /opt filesystems And of course, for the high-end setup or advanced user, there are advanced techniques such as RAID and LVM (Logical Volume Management). But that's another article. . .
Moving Filesystems
Sometimes the growth of a filesystem - particularly /home - means that it is necessary to find it a new home; in other words, add another physical disk and relocate the filesystem to its new home where there is room to grow.
Here is the procedure for adding another drive, with a single partition which will become the new /home filesystem (I'm assuming fdisk has already been used to partition it):
As root:
/dev/hdb1 /home ext3 defaults 1 2
should do it.
Sidebar:
Removing Old Files
Over a period of time, old and unneeded files will accumulate on your systems, such as
* Temporary files such as editor swap files (end in .swp)
* Core dump files (if ulimit is non-zero)