CTS 2322 (Unix/Linux Administration II) Project
Build a Linux Kernel

(This is a group project; see below for details.)

Due: by the start of class on the date shown on the syllabus

Description:

You must rebuild the latest stable Linux kernel from source on your system.  While many distributions provide both binary and source packages for their version of the kernel, you will download the latest version of the stable kernel from www.kernel.org.  You will verify you have all the kernel building requirements (proper versions of tools, enough disk space), then configure, build, and (as root) install your new kernel.  Note you don't have to be root to build a kernel, just to install one.

Make sure your new kernel includes the features needed in this course, including:

(Building features you don't need, even as modules, takes extra time and disk space.  Don't include features you are sure you don't need (e.g., ISDN support).  If in doubt, you can include the features as modules.

Make a copy of your system journal pages that document the steps done and the kernel configuration you chose.  (Note if you get the configuration wrong, it is a simple matter to change it and then rebuild the kernel from that point to use the new configuration.)

(See the kernel resources on our class web page.)

Requirements:

For this project, you may work either individually or in a group.  Each student must submit their own copy of the assignment, which must include your name and the names of the others in your group.

Follow these steps and answer the following questions:

  1. Do any one-time prep, such as yum updates (and installing if needed gcc, make, and other required tools).  A good way to do this is to install the various yum groups, such as “Development Tools” and “Development Libraries”.  That will install most but not all items needed to run make xconfig.  Don't forget to add to your journal, all the software you installed.
  2. What version of the kernel are you currently running?  (Hint:  Use the uname command to check.)
  3. What is the latest stable version of the Linux kernel available?  (“Stable” kernel versions in the 2.6 series have version numbers such as 2.6.X.Y.  The version numbers of more recent kernels are 3.X.Y.)  (You can easily find out from www.kernel.org/kdist/finger_banner, or just from the home page.)
  4. Verify at least 6 GiB of disk space are available to build the kernel with the df -h command.  (Note the download, even after uncompressing, isn't this large: about 40MB to download and about 550MB to unpack).  But many large files are created during the install, so this much space is needed, at least temporarily.)  Which filesystem has sufficient space?  (Prefer $HOME if possible.)
  5. If installing in /usr/src (not recommended!) you must take care not to overwrite existing files there.  Depending on your distribution you may have to rename some directories and files.  Rename an existing source directory of /usr/src/linux to include the kernel version.  For example, if you currently have version 2.6.5-1.358 installed the source should be in a directory /usr/src/linux-2.6.5, and not /usr/src/linux.  Also check for a /usr/src/.config file, and copy it to /usr/src/config-version or /usr/src/config.old.  A backup of your grub.conf (or lilo.conf) would be a good idea as well.
  6. Now fetch the kernel source tar-ball and its digital signature into the directory you selected for the install (using the correct numbers for X and Y).  You may have to browse kernel.org to find the correct URLs to use, something like the following should work:
     mkdir ~/build; cd ~/build
     wget ftp://ftp.kernel.org/pub/linux/kernel/v3.x/linux-3.X.Y.tar.xz
     wget ftp://ftp.kernel.org/pub/linux/kernel/v3.x/linux-3.X.Y.tar.sign
  7. Verify the tar-ball is the one from the Linux Archives and that it hasn't been tampered with.  To do this you will need to import the public key from the Linux Archives, and uncompress the archive.  Visit www.kernel.org/signature.html to obtain a copy of the key, or import it directly with gpgWhat is the ID of the Linux Archives GPG key?  What was the result of the verification?  To save disk space, you can verify the file without uncompressing first, by using a command line similar to:
    xz -cd linux-3.4.2.tar.xz | gpg --verify linux-3.4.2.tar.sign -
  8. Next, unpack the source code using tar -xf linux-3.X.Y.tar.  (This may take a short while.)  (Note, doing this step can double the disk space used.  You can recompress the tar-ball (if you uncompressed it earlier), and unpack the smaller, compressed archive directly.)

    When installing under /usr/src, at this point you should to create a symbolic (soft) link to the /usr/src/linux-3.X.Y directory with the name /usr/src/linux.  Then you can cd /usr/src/linux and complete the install from there.  (Of course you should build in your home directory.  2.6 allows the source to be built in one place and installed in another.)

  9. Clean up the source with
    cd ~/linux-3.X.Y
    make mrproper

    (That step shouldn't be need for fresh source but it can't hurt!)  The source is now installed and ready to be configured and built.

    If you had any patches to install, now would be the time.  Note that Red Hat does provide a source RPM package containing the patches they used for the shipped kernel for Fedora, but we shouldn't need it for this project.

    To patch the Linux kernel, make sure you have the correct source code version for which the patch applies.  Linux 2.6 patches apply to the head source code (2.6.X), not the latest stable code (2.6.X.Y)!

    For example, if you have 2.6.5.2 and want to apply patches to get to 2.6.7.3, you need the 2.6.5 source code installed, then apply the 2.6.6 patch, then the 2.6.7 patch.  These kernel patches aren't cumulative (unlike Microsoft service packs).  You need all the patches between the version you have and the version you want, and must apply them all in order.

    Once you have patched your way to the correct 2.6.X version, you can then apply a single (cumulative) patch to that to get the latest stable version (2.6.X.Y).

    Unofficial patches are more dangerous than official ones.  Make sure you really need to do this!  In all cases use gpg to verify the originator and integrity of the patch file you have downloaded.

    To apply some patch called patch-2.6.X to the source do the following:

    # mv patch-2.6.X.bz2 ~/build  # the parent directory of the kernel source
    # cd /usr/src/linux
    # bzip2 -dc ../patch-2.6.X.bz2 | patch -p1

    For Solaris and other commercial Unixes the patches are binary files that modify the kernel image file (and LKM files).  To patch your kernel with a Solaris (binary) patch you can use the patchadd and patchrm commands.  Use patchadd -p to see list of installed patches.  Or you can use /usr/sadm/bin/smpatch {analyze|download|add} to manage your patches.  However the easiest way to manage Solaris patches is to use the popular (but 3rd party) pca script.  You can obtain pca from www.par.univie.ac.at/solaris/pca/.

    Solaris patches can be download from www.sun.com/sunsolve/patches/.  (Note Oracle no longer distributes Solaris patches for free.)

  10. Check that you meet kernel building requirements, documented in the README file:
    • Verify correct version of gcc is installed with gcc --versionWhat version of gcc do you have installed, and what is the minimal version required?
    • Gather hardware information that may be useful when configuring.  There are several utilities that could be used for this; here's one way:
      (lspci; dmesg; cat/proc/cpuinfo) |lpr
    (View an annotated listing of the hardware information collected by running the above command.  This will help you understand how to interpret this information.)
  11. Now configure your kernel.  This is the hardest part for newbies!  It is possible to configure using standard configurations, or old kernel configurations.  See make help for choices, and note that Fedora puts the installed kernels' config files in /boot.

    The current config is a good place to start.  Copy the current kernel's config file to ./.config.  Now run these commands to begin:

    make silentoldconfig # to update .config with new options; choose the defaults
    

    When it pauses to ask you about a new configuration item, you can hit the Enter key to choose the default choice.  When done, the original .config file is saved as “.config.old”.  Once you have updated your configuration, it is time to “tweak” it.  You need to run one of the various config user interfaces, for example:

    make xconfig   # to complete the configuration usig a Qt GUI interface

    The error message(s) you might get first when you try to run “make xconfig” will likely tell you what's missing, and some work with yum will tell you which package(s) you need to install (almost certainly some “foo-devel” package).  Hint:  A message about “can't find -lfoo” means the compiler couldn't locate the library file “libfoo.so”.  Again you can use yum to see what package provides the missing library.  Be sure to add to your journal all the packages you needed to install.

    For most choices you have the option of including the code in the base kernel (a checkmark shows), including it as a module (a dot shows), or not including it at all (the checkbox is empty).  The defaults include a lot of useless stuff as modules, which doesn't hurt a thing except take up extra disk space.  However it can cause the build to last several hours!  If you skip some code for now and decide you need it later, it is not hard to re-run “make xconfig”, and mark the code to be included as a module.  Then you build just the missing code with “make all modules_install”.  (Of course you will need to update the initrd image as well, or your new module won't be available to the kernel until after the root filesystem has been remounted.)

    If you're not sure which options to include, examine the system information you printed earlier for clues.  A number of changes to the default configuration are shown below, by way of illustration.  Your configuration will be very different than this!  I provide this information for an example only.

    • General setup
      Kernel .config support enabled
      Enable access to .config through /proc/config.gz enabled
      Set LOCALVERSION string to “-wp20120617
    • Loadable module support
      Module versioning support (EXPERIMENTAL) disabled
    • Processor type and Features
      Processor family
      Pentium-III/Celeron(Coppermine)/Pentium-III Xeon selected
      Preemptible Kernel enabled
    • Bus options (PCI, PCMCIA, EISA, MCA, ISA)
      PCI device name database enabled
    • Network support
      Network options
      Network testing
      Packet generator enabled as module
      Network Device Support
      Traffic Shaper (EXPERIMENTAL) enabled as module
    • Input device support
      Mouse interface (NEW)
      Provide legacy /dev/psaux device enabled
    • Filesystems
      DOS/FAT/NT Filesystems
      NTFS file system support enabled as module
      NTFS write support enabled
    • Security options
      NSA SELinux Support
      NSA SELinux boot parameter
      (1) NSA SELinux boot parameter default value (1, changed from 0)

    The kernel .config support and access means you can extract the config information from the running kernel, from /proc/config.gzLoadable module versioning can cause headaches and is only useful if you try to run modules from different kernel versions.  I selected the correct CPU type I actually have.  You also need to enable Preemptible Kernel or your GUI will be horribly slow!  (This is due to a change in the 2.6 kernel from 2.4.)  The SELinux options support SELinux in the kernel, but you still need to enable it by default when booting.  The other options I changed just for fun.  I also disabled some code built as modules by default, stuff like ISDN, frame-relay, battery monitor, and other laptop features.  A more through job of turning off stuff that isn't needed would have saved a lot of time!

    Note that features built as modules aren't available until the module is loaded.  For example, if the /proc/config.gz support is built as a module (which is the default), the file won't be there until you load the correct module (using “modprobe configs”).  You have the option to build most features (including that one) into the base kernel directly, if you wish.  (Or you can configure modprobe to load a module automatically at boot time.)

    Go through the configuration choices and read the help on each before deciding what to change.

    Remember you must include support for your hardware and IPv4, netfilter (iptables), Preemptible Kernel, a custom “LOCALVERSION” string (see below), and Kernel .config support → Enable access to .config through /proc/config.gz What configuration settings did you change from their default (or current kernel's) values?

    The installed kernel will have a version number of 3.X.Y.  If you install any patches or have custom configuration, you may end up with several kernels with the exact same name.  To avoid this, you should change the value for LOCALVERSION to something like this: –wp20120617.  Here I use a dash, my initials, and the date, but you can use any string you wish that makes sense to you.  The resulting kernel will have the name vmlinuz–3.X.Y–wp20120617.  You will find the setting for this under the “General Setup” section when you configure the kernel; double-click the option and you will be allowed to type in the new value.

  12. Take a look at the .config file produced as a result of your make xconfig.  You can also use diff to compare this configuration to the old Fedora one.  If you change your mind about any options, you run the “make xconfig” command again and make any needed changes.

    Make a backup copy of your .config file someplace.  Be sure to name it correctly with the kernel version (and local version string).

  13. Now build the kernel and modules (note the time when you start this):
    date >~/build-date
    make # This can take several hours!
    sudo make modules_install # you need to be root for this part!
    date >>~/build-date
    

    When I compiled my model kernel using the above steps, it took about 2.5 hours.  You can greatly speed up the build on a multi-core machine, by using the “–j number”, where “number” is the number of cores on your computer.  You can determine this by examining /proc/cpuinfo.

    When this runs a lot of messages will scroll by very quickly, including some “warning” messages.  This is normal.  The time to worry is when you see “error” messages.  (When I built a kernel, there were some warnings, and one error message for the third-party “VirtualBox” kernel module.  However the kernel works without it, and that module should be built later anyway.)  How long did the build take?  (Just look at the build-date file and compare the times shown.)

  14. Install the kernel documentation:
    make mandocs installmandocs
    

    You should expect to see many warnings, which you can ignore.  (This step can take about an hour!  I have never understood why it takes so long to format and copy documentation.)

  15. Now install the kernel.  The kernel supports a make install command (which must be run as root), but this may or may not work correctly on all systems.  This command invokes the shell script /usr/src/linux/arch/i386/boot/install.sh to do the install.  This script in turn runs an installkernel script if it exists.  (It does on Fedora, in /sbin.)  Otherwise, the script tries to install the kernel itself, unfortunately configuring LILO and not GRUB as the bootloader.  Fortunately, Fedora does include a /sbin/installkernel utility, which copies the files, makes the correct links, and updates the GRUB configuration for you (via another utility, /sbin/new-kernel-pkg.  That script invokes other scripts, and eventually a command called grubby to actually update the grub configuration file(s).)
  16. If you don't use make install, you can install the kernel manually.  This involves copying (or linking) two files into /boot, building an initial ramdisk, and updating the boot loader configuration.  Part of this process is to make sure you can boot from your previous working kernel if the new one doesn't work.
    1. Setup names and links for previous and current kernel files in /boot.
    2. Copy (install) the kernel to /boot, using the correct name.  Create the correct links.  The kernel can be found at ./arch/i386/boot/bzImage.
    3. Copy the ./System.map file to /boot, again using the right names and links.
    4. Run the depmod command to update the dependency information for the new kernel modules.  Check the man page for the correct options to use.
    5. Create an initial RAM disk with dracut (or the older mkinitrd).  For example using version 3.4.2 (and no local version):
              dracut /boot/initrd-3.4.2.img 3.4.2
    6. Update boot loader information to use new (link) kernel names.  In my preferred setup, I use the names of the form vmlinuz-version-localversion (if I used any local version name).  Then I use symlinks to the current and previous kernels, with the names vmlinuz and vmlinuz-old.  Then I update the boot loader to show both.
  17. After running make install, you still need to edit the Grub configuration.  This is because by default the system may boot your old kernel, not the new one.  You want the new kernel as the default, with the old (working) one available.  Changing this depends on which boot loader you use.  For old (legacy) grub,, change the “default=1” line to read “default=0”, assuming your new kernel is listed first in the file.  For the newer grub2, you can use grubby (if available), or (on Fedora anyway), grub2-set-default.  (For Fedora 16, by default grub2 will remember the last booted kernel, and boot that by default.  You can change the last remembered kernel using grub2-edit-env.)
  18. Test your new kernel by attempting a reboot.  Use the uname command to see if it is indeed your new kernel running.  (Also check out the /proc/config.gz feature!)  If it doesn't work, reboot using the old one, change the configuration, build additional (or fewer) modules, and/or patch the source code, and then try again.  Make careful notes as you go, so you can correctly update your journal later!

    If the kernel build was a success, you can reclaim most of the space used by running the command “make clean”, which will leave the .config and other files, sufficient to later build third-party modules.  (Or, you can delete the entire directory, but I would save a copy of your .config file first.)

To be turned in:

Answers to the above questions and the relevant system journal entries.  You can send as email to (preferred).  If email is a problem for some reason, you may turn in a hard-copy.  In this case the pages should be readable, dated, and stapled together.  Your name should appear on the first page.  See System Journal Hints for more details.

Don't turn in your whole journal, you will need to add to it every day in class!  It is common in fact to keep the journal as a text file on the system (with a paper backup of course).

Please see your syllabus for more information about submitting projects.