Unix/Linux Disk Partitioning Guide

One of the most critical decisions when installing Unix or Linux is how to best make use of limited disk space.  Even with today's large hard disks servers with hundreds of gigabytes of pictures, video, sound, database, and other data are common.  Multi-user machines may require protection such as read-only partitions, disk quotas, etc.

There are many reasons for partitioning a large hard disk into several smaller partitions.  (Note that in this document I often say partition when technically I mean file-system within a partition or slice.)  For a home user with a single small-ish disk (today's large disks are tomorrow's small ones) a single Linux partition (plus swap and maybe one for Windows for a dual-booted system) can be a reasonable choice.  But partitions provide some safety so I always would create several partitions.

Traditional Unix systems uses a similar concept called slices each of which may hold a filesystem just like other OSes put filesystems into partitions.  Until recently there was very little to gain by not using a standard layout.  However modern systems now support many per-filesystem features including security and robustness related mount options such as preventing SUID (or even executables) on data-only filesystems.  Today there is little to be gained by limiting the number of filesystems you create to 5 or so merely to satisfy a historical limit.  Using logical volume management you can create as many filesystems as makes sense in your situation.

Standard Unix systems are traditionally limited to 8 slices per disk, some of which have pre-defined uses.  (In the Unix world the terms partition and slice are unfortunately used inconsistently and sometimes interchangeably.)  So standard Unix and Solaris disk layouts for many years worked around that limit by using best practices that didn't need more.  However modern Unix systems (including Solaris 10) allows for a form of logical volume management, either with ZFS pools or by using Solaris Volume Management (SVM).  So you shouldn't be afraid to define additional filesystems if they do make sense.

Of course just because you can partition a disk doesn't mean you should!  The more partitions you create the more there is to manage.  If you guess wrong on the space required you may have to later grow a partition (not the big deal it used to be).  So you shouldn't make extra partitions unless you consider the extra work they create to be worth the additional protections they provide.

Here are some reasons to create extra partitions.  If you choose not to follow some vendor-recommended standard disk layout, see which of these apply to your situation to decide which partitions to create:

• If the root partition runs out of space the system will crash.  If some non-root partition runs out of space the system will remain up and the SA can login and fix things.  Thus the directories such as /home, /tmp, and /var that users can easily fill with downloads, email, etc., are prime candidates for extra partitions.  So are any other directories that might grow (directories for FTP uploads, database files, etc.).
• If the partition containing the log files runs out of space the system won't be able to write any log messages that could help an SA to determine what went wrong.  On the other hand some rapidly recurring error (or attack) can cause log files to grow very large, very quickly filling the partition containing them.  So the directory containing log files, /var/log, is a good choice for a separate partition.
• Parts of the file system may need to be mirrored or otherwise duplicated.  Many software tools exists that can do this but they work on whole partitions (such as "ghost").  (Yes there are other tools without that restriction but they are less common.)  Examples include a web site or the directory that holds a database's files.
• Parts of the system may be shared (using Samba or NFS).  While modern systems allow the sharing of part of a partition (that is, some directory and all its contents) security can be difficult to enforce.  It usually works better to share whole partitions.
• Disk quotas are assigned per user per partition.  So to setup and control disk quotas you must plan your partitions.  (For example student accounts on the YborStudent student server have different sized quotas for students for each of /home, /var, and /tmp.)
• Disk partitions can be mounted as read-only.  On a production web server for example, most of the system is static and could be mounted as read-only.  (Even log files can be created on a separate log server.)  Some parts of the system must be read/write to function but there are great security benefits to mounting as much as possible as read-only.  (Another example is /usr/share).  Besides read-only there are other mount options that provide additional security: nodev, nosuid, noexec, acl, and others.  Additional mount options can increase performance such as noatime.
• Many standard backup tools work on a per-filesystem basis (e.g., *dump).  Backups that span two or more tapes (or other media) can be problematic.  For one thing if you only have a single tape drive you can't automate the backups as a human must be there to change tapes.  It is worth considering making enough partitions so that each one is small enough to fit onto a single backup tape.
• Even journaling filesystems occasionally need to be checked with fsck.  If your disk is large it might take many minutes or longer to run fsck which will run automatically every so often.  (It has been estimated that a full, maximum sized ext4 filesystem would take 119 years to run fsck at today's speeds!)  By partitioning a large drive into small partitions the SA can stagger when such checks get done so only a small part of the disk is (reasonably quickly) scanned at any one time.  This greatly reduced the time needed to reboot after a system crash.
• The outer cylinders of a disk spin faster than the inside ones.  There can be a measurable performance difference.  You can put the most heavily used partitions on the faster cylinders and the least used filesystems on the slower ones.  For a home user the root partition and home are most heavily used, for a database/web/mail server, the partition holding the data is.  Swap may heavily used or not used at all, so it may belong on either the outside or inside cylinders of a disk.  (In the middle of a disk is also recommended when swapping heavily, so the heads don't have as far to travel.  Of course it works better to put the swap partition on a separate disk.)
• Older motherboards have a restriction on where on a disk a bootable partition can be located (and how large it can be).  This is because older BIOSes use ten bits to hold the starting cylinder number of any bootable partition, which means it must be located within the first 1,024 cylinders or you can't boot from it.  While not a problem with modern hardware it was common for older systems with large hard disks to have a small /boot partition to hold the few files needed to boot the system, located near the beginning of the disk.  This would include the kernel itself and the bootloader configuration file (e.g. grub.conf) amongst others.  Many standard Linux distributions create /boot partitions just in case of older hardware.  Some Unix distributions create a /kernel or /standalone partition for similar reasons.
• The boot partition must be readable by the system BIOS.  If the bootable partition is also the root partition then the whole root partition must be readable by BIOS.  This means you are restricted as to the type of filesystem you use; no LVM for example.  Instead a small boot partition is used (typically < 100 MB).  Then the root partition need not be readable by BIOS but by the kernel.

Some reasons not to partition a disk include:

• The directories /bin, /lib, and /etc should never be separate partitions!  At boot time only / is mounted initially.  The init program needs to access files in /etc and the bootup scripts need access to commands in /bin, which may depend on files in /lib.  Kernel modules required to complete the boot process are also kept in /lib.
• A few commands in /bin require DLLs (.so or shared object files) to work.  These DLLs are often found in /lib but many are kept in /usr/lib.  So some commands won't work at boot time if /usr is a separate partition.  In /bin you may find (on older systems) some static¹ versions of the more critical commands (the exact set and location varies with your flavor of Unix).
• Currently Solaris contains many restrictions on what can be a separate partition.  Some of these restrictions include inability to boot, install, patch, or use live upgrade in some cases.  You must check very carefully that your new partitioning scheme won't break the system.
• For production server systems there are best practices that have proven successful for most common scenarios.  You need a strong reason to use a non-standard disk layout for any common situation: web server, email server, etc.  Be certain the security, management, and performance benefits outweigh the extra work a non-standard disk layout creates.  (Imagine yourself relaxing on the warm sands of a tropical island when your vacation is interrupted by the temp system administrator who can't figure out your server.)

A problem with having many partitions is that you can run out of space in one partition while another has excess capacity.  When the partition plan fails you may have to create a new plan, backup all existing data in archives, re-format the disk, and restore all the data.  Obviously this should be avoided if at all possible.  (However, newer partitioning tools such as a bootable—or live—CD for gparted makes it much easier and less painful to modify a disk layout.)  One commonly used technique to handle this situation is to use symbolic links rather than re-partition.  Suppose for example you need to install a nifty word processor application in /opt/nifty, only the /opt partition (which might be the root partition) doesn't have enough free space.  How annoying!  But if /var partition does have extra space you can create a symlink to use it:

# mkdir /var/nifty; ln -s /var/nifty /opt/nifty

The problem with this approach is that too many such symlinks (sometimes referred to as a symlink farm) can make maintenance difficult.  Quotas, backups, log files, etc., can all be affected.  So adding such symlinks should be considered a "hack" and not a substitute for proper planning.

Modern Unix systems have virtual partitions called storage volumes or logical volumes or LVM.  (This idea was first widely deployed by Novell Netware.)  A volume should be thought of the same way traditional partitions are.  However a volume can be composed of one or more physical partitions, possibly on separate drives.  When a volume runs out of space you can just add another hunk of disk space to an existing volume to grow it.  Then the filesystem in that volume can be grown as well.  Although logical volumes may be grown (or shrunk) much more easily than with traditional partitions, using volumes well still requires careful planning of both the logical partitions and the underlying physical partitions.  (See about LVM for more information.)

A frequently asked question is how large should the XYZ partition be?  Unfortunately there is no simple answer.  Consider a partition for /var/log.  This needs to be sized to hold your logging data.  But how much data is stored?  If you have a central log server (loghost) you may not store any log data on some other host.  Otherwise you may need to store only a little log data (say for a printer server, anonymous FTP server, or a static content web server).  But if this host is your central log server or if you don't have a central log server and plan to keep 6 months of log data on-line, you will need a lot more space than a host that only keeps 1-4 days of log data.  So the size might be zero, less than 50 megabytes, or more than 10 gigabytes (or more; consider Apache access logs on a busy server).

Other partitions have similar considerations.  You may need no separate partition at all, a very small one, or a huge one.  If you have a separate partition for your database files, how much data do you plan to keep on-line?  For a web server's files, is this a home user's hobby web server, or a training site with gigabytes of video clips?  If you don't have a separate partition to hold crash dumps these end up in the swap partition, so that must be at least as big as your physical RAM.  (This is needed to support hibernate too.)

Consider sizing partitions on a mail server.  The critical partition will be the one holding the user's mail.  This may be stored in mailbox files in /var/mail or within the users' home directories.  Or even elsewhere on a database.  You need to estimate how much email will be kept on-line by your users, and also allow enough space for spam or a sudden burst of email traffic (or even storage over a break between semesters, when faculty rarely log in to read email.  A good guideline might me to use the same size limits per user as some other email servers, such as Yahoo! mail or Google's gmail.  (If you can afford such large disks as Google.)  So, how large is the sum of your mailbox sizes?  Whatever it is, add sufficient space for growth and multiple by the expected number of users.  At HCC it might be reasonable to expect 10 MB per student, and 100,000 students (counting current, future, and past students), or one TB total.

If using some sort of LVM it is usually not difficult to grow and shrink partitions later so there is little reason to worry about getting the size wrong.  Still you should be able to make a reasonable estimate on the initial size of your partitions:

• For some partitions you can guess a good size based on the vendor's recommendations.  You will often find those in the release notes.
• You can find sizing recommendations in the Internet (e.g., at doc.sun.com or fedora.redhat.com), for partition / filesystem sizes required or recommended for various applications.  For example, the swap partition should be at least as large as the amount of physical memory to support features such as hibernation and crash dump analysis.  VMware recommends making the /tmp filesystem at least one and a half times the amount of virtual memory on your system.  If running a database service, check the website (e.g., Oracle) for sizing recommendations.
• You can make an educated guess.  For example if you create a /boot partition it only needs to hold the OS image file(s) and boot loader file(s).  Each bootable OS may only require 20-30 megabytes.  So if you have only two OS images (the default for Fedora) you need less than 50 megabytes.  On the other hand if you (like me) like to play around with different kernel configurations you may have 3 or 4 such image files.  (Not forgetting the initial RAM disk image as well.)  In such a case, or with larger kernel images, you may want over 100 megabytes for this partition.
• For other partitions (such as the root partition) the size depends on how much software you plan to install there and what sub-directories are located on this partition.  For example, do you have a separate /opt partition?  If not any software installed there goes into the root partition.  That can be as much as 12 gigabytes or more.  On the other hand if you have separate partitions for /opt, /home, /usr, /var, etc., then the root partition can be quite small especially for a dedicated purpose server (that has no GUI and minimal software installed).
• You can estimate based on the size of current of similar systems.  If you are migrating a web server to a new host you should have a good idea of the current disk space requirements.  If you have been monitoring disk space (as you should) you will also have a reasonable estimate for the expected growth of your data, so you can allow for that.
• You can build the new OS on a test host without partitioning.  Then you can use the du command to see how much space is used for various directories.
• If you have excess space it pays to err on the side of caution and make the partitions larger.  So if you have 500 gigabyte drives it doesn't hurt to waste some by making some partitions larger: /var, / (the root partition), etc.
• Depending on your backup policies and available hardware you may not want to make any partition larger than will fit on a single backup medium.

A final consideration is that there are modern alternatives to per server disks (or DAS).  Using storage technologies such as NAS and SAN may mean your servers have no disks or just a small disk for booting only.  Even with these technologies you still need to plan the number and purpose of filesystems.  Organizing your storage well depends on available budget, technology, and local expertise.

A partitioning scheme (commonly called a partition map, partition plan, or disk layout) for your system must be well documented or it is useless.  Later on you will need to refer to this information and it may be difficult to recall details six or twelve months from now.

Your partition map should be neatly typed and include a description of your disk partitioning map and the scenario it is based on.  (That is the scenario might be this is a partitioning map for an at-home workstation, ... for a web server, ... for a multi-user development platform, etc.)

You must justify the choices you make.  (For example, for a student server: We have 5 classes of less than 30 students each, and low graphic web pages, Perl scripts, and general Unix shell scripting means each student is likely to need less than 5 MB, so /home needs 30 * 5 * 5MB = 750MB minimum, and to allow room for additional classes in the future 1GB will be used.)

You should summarize your partition map in a short table, something like these examples:

To find out how large the disk is you could look at the label, check the BIOS, or check the system invoice/system description (often obtainable on the Internet using a serial number or service tag number).  (In our case you could also ask a lab tech.)

The version of Fedora Linux we are installing is over 8 gigabytes in size, not counting space you reserve for user home directories, future additions and updates, log file space, database space, web site, ftp site, etc.  (Of course, if you don't install everything you can get away with a minimal system of under 1 GB.)  To see how much space is required for various directories you could log into a similar system (such as YborStudent.hccfl.edu) and use the du command.