CTS 2322 (Unix/Linux Administration II) Project
Building and Installing Software From Source Code

In this project you will download and install a source code RPM package.  Next you will download and install a (standard) tar-ball.  You will then build binary packages from both and install them.  Finally, you will setup a local yum repository with your package and install it using yum.

The version of the pine package used (version 4.64) won't install by default on a “vanilla” Fedora system.  You will have to trouble-shoot the install to make this work!

Answer the following questions and perform the following tasks:

Part I — Installing pine from a source RPM:

1. Fedora currently doesn't include the pine MUA (email program).  What is the home URL for pine?  (Hint:  pine is maintained by the University of Washington.)
2. While binary packages exist for pine for Fedora, locate a source RPM from the home website of pine and download it.  What is the version of pine you downloaded?

   Note that “pine” is a discontinued project.  The replacement for pine is “alpine” but for this project you must use “pine” and not “alpine”!

   (Since a source RPM for pine may no longer be found on-line, here are links for pine-4.64-1.src.rpm, pine-4.64-maildir.patch.gz, and for pine-4.64-utf8-1.patch.)

3. If you don't have the rpmbuild command on your system, use what you have learned to locate the package for it and install that.  (You may need to install additional packages if you need to use some command that isn't currently installed on your system.)
4. Once installed, you need to create a build environment in your home directory (your non-root login).  You do this by running the command “rpmdev-setuptree”.  What files and directories were created in your home directory by this command?
5. Install the source package (as you, not as root) using the “rpm -U” command.  This will create a source tar-ball and a “spec” file under ~/rpmbuild/).  What is the exact command you used for this?  What files were installed?  (List the complete pathnames.)
6. Next, attempt to build a binary RPM from the installed source using the command “rpmbuild -bb spec-file”.  For the version of the pine source RPM used (version 4.64), this won't work on Fedora!  Now troubleshoot the problems and fix them:
   
   1. What error message is produced?  The error message includes the line number from the “spec” file that rpmbuild didn't like.  Before making any changes to the spec file, make a copy of it.
   2. Try to fix this problem by editing the “spec” file and commenting out the offending line.  Do this by pre-pending (adding to the front of the line) the “#” character, which is the start of comment character.
   3. Now try the rpmbuild command again.  What is the error this time?
   4. Run the rpmlint command on the spec file and note the error(s) it shows.  You can figure out what is wrong only by knowing the “spec” file syntax.  This can be found at www.rpm.org.  However, in the interest of saving you some time I will provide these hints:  Older versions of RPM required a “Copyright:” line in the “spec” file, but modern versions have replaced that with a “License:” line.  In addition, the list of valid RPM package “groups” for Fedora has changed.  You can find the current list of groups in the file /usr/share/doc/rpm-*/GROUPS.  You should pick an appropriate group from that list.  (Hint: in which group is the alpine package?)
   5. Edit the “spec” file and eliminate all the errors shown with rpmlint.  (It is okay to ignore warnings.)  What change(s) did you make to the “spec” file?  (Show the output of diff with the original and modified versions.)
   6. Try building the pine binary RPM.  You may discover missing development tools and libraries.  These can be solved one at a time, or you could use the groupinstall feature of yum, to install all the development tools and development libraries at once.  What (if anything) did you need to install?
   7. You may find a warning about how OpenSSL can't be found.  (That's because the DLL file for it has changed its name over the years.)  You can continue the build anyway without SSL, or (if time permits) attempt to modify the spec file to eliminate the error.  (See the hints section below.)  If the build is successful, the result is a binary RPM.
   8. Even once the spec file is correct, the build may not succeed due to missing libraries or other files.   If the build fails, examine the output in reverse order, to see the error that caused the failure.  What was the error?  On my Fedora system, the error was an undefined symbol.  Furthermore, the error included a helpful message about which library does include the missing stuff.  To include that library during the build, you should export the environment variable “LDFLAGS”, set to the name of the missing library.  For example, if the missing library is “/usr/lib/libfoo-3.5.so.1”, then set and export “LDFLAGS='-lfoo'”.  Once you set this correctly, you should be able to successfully build the binary pine RPM.  What was the environment variable setting required to successfully build the pine RPM?
7. As root, install the binary RPM you built for pine.  (You may have to uninstall apline first.  When done with this project, remember to re-install alpine if you plain on using it!)  What is the exact command(s) used?  Check to see if the install worked, by running pine, pico, and viewing the man pages.
8. By default, pine and Alpine use MBOX format.  However there is an available patch to pine and for alpine to enable support “Maildir” format as well.  Locate this patch on the Internet (or use the link provided above), download, and apply it to the pine source, and then rebuild.  This will take several steps, and you will need to examine the patch file to see where it should be located, and what the command line argument to patch should be.  Exactly what commands did you use to apply the patch?

Part II — Building from source:

1. Download the tar-ball myhello-1.0.tgz from this website.  (This is a simplified version of Gnu hello.)  Now unpack, compile, and install this tar-ball:
   
   1. What is the MD5 checksum for this tar-ball?  Does it agree with the following?

      bb3dc635c0023967e46f35130e96cde6  myhello-1.0.tgz
      

   2. Unpack the tar-ball using tar.  What is the exact command you used?
   3. Change your current working directory to myhello-1.0 and read the various installation documentation provided.  What files did you (or should you) look at before installing?
   4. Compile and install the software.  (Hint:  As this is a standard Gnu package, you use the standard steps to configure, compile, and install the software.) Which one of these commands is the only one that should be run as root?  What are the exact commands you used?
2. Check your installation by running the command hello and reading the hello(1) man page.  What files were installed (and where)?  What would you have needed to do, to have the files install under “/opt/” instead?
3. Now clean up the build files.  One way is to simply delete the entire build directory.  However, the Makefile generated usually has targets to clean up the files, while leaving the configuration files.  Examine the Makefile.  What “make” command can you use to delete the build files but not the configuration files?  What other targets look interesting/useful to you?
4. Change to your home directory, and delete the entire build directory (“myhello-1.0/”).  As root, delete all other files installed.  When you install from source, there is no record (except in your system journal) of what files were installed.  If you didn't have an accurate system journal, what could you do to determine what files were installed by some source (tar-ball) installation?

Part III — Building a binary RPM from a tar-ball:

Next build a digitally-signed, binary RPM package from the myhello tar-ball.  If you are lucky, a tar-ball will include the “spec” file required to build an RPM package.  Sadly for you, this is not the case here.  So you will need to create a spec file.  But first, you must create a gpg key to use to sign the package(s) and the repository meta-data.

1. Generate a GPG key-pair, to be used to digitially sign your package:

   $ gpg --gen-key  # and follow the prompts
   

   You should use the default key type and size.  Have your key expire in 4 months (long enough to last to the end of the term).  Use your real name, localhost email address (or alias), and a comment similar to “Software Packager for MyLocalRepo”.  Pick a good passphrase, and record it someplace (not in your journal).  What was the gpg output?  What is the public key's ID (the eight hex digits after the size and type)?  Where did you record the key info and password?

2. Set up the rpm build system.  You should have set up the basic build environment previously, in part I of this assignment.  Now you should edit (or create if missing) the file “~/.rpmmacros”, to contain the information needed, and to set default values, used by the rpmbuild tool.  Here is a copy of the one I use (the three lines I added to the default file are shown in boldface):

   %_topdir %(echo $HOME)/rpmbuild
   
   %packager       Wayne Pollock <wpollock@example.com>
   %_signature     gpg
   %_gpg_name      9F19C168
   
   %_smp_mflags %( \
       [ -z "$RPM_BUILD_NCPUS" ] \\\
           && RPM_BUILD_NCPUS="`/usr/bin/getconf _NPROCESSORS_ONLN`"; \\\
       if [ "$RPM_BUILD_NCPUS" -gt 16 ]; then \\\
           echo "-j16"; \\\
       elif [ "$RPM_BUILD_NCPUS" -gt 3 ]; then \\\
           echo "-j$RPM_BUILD_NCPUS"; \\\
       else \\\
           echo "-j3"; \\\
       fi )
   
   %__arch_install_post \
       [ "%{buildarch}" = "noarch" ] || QA_CHECK_RPATHS=1 ; \
       case "${QA_CHECK_RPATHS:-}" in [1yY]*) /usr/lib/rpm/check-rpaths ;; esac \
       /usr/lib/rpm/check-buildroot
   

   (Only the “%packager” line needs to be added, in general.  The other two added lines are to sign the resulting packages using the correct GPG key.  The rest are settings used to speed the compiles and run appropriate cleanup and post-build utilities.)  Be sure to edit this file, changing the information to your own name, GPG key, and home directory.  What are the contents of your ~/.rpmmacros file?

3. Copy the tar-ball to ~/rpmbuild/SOURCES/myhello-1.0.tgz.
4. Next, cd into the ~/rpmbuild/SPECS directory and create a spec file here, named “myhello-1.0.spec”.  This is the hard part!  You could create a template spec file with the command rpmdev-newspec.  An interesting feature of vim is that when you create a file with a name of the form “name-version.spec”, it will create a template you can easily edit.  Try this now and look through the result.
5. While a cruel instructor would make you read the tutorial and create your own spec file, you can just use this one:

   Name:      myhello
   Version:   1.0
   Release:   1%{?dist}
   Summary:   Gnu hello
   Group:     Applications/System
   License:   GPL
   URL:       http://www.gnu.org/software/hello/
   Source:    myhello-1.0.tgz
   Prefix:    /usr/local
   BuildRoot: %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX)
   
   %description
   Gnu hello is a sample project showing the Gnu build tool-chain.
   
   %prep
   %setup -q
   
   %build
   %configure
   make %{?_smp_mflags}
   
   %install
   rm -rf $RPM_BUILD_ROOT
   make install DESTDIR=$RPM_BUILD_ROOT
   
   %clean
   rm -rf $RPM_BUILD_ROOT
   
   %files
   %defattr(-,root,root)
   %doc %{_mandir}/man1/hello.1.gz
   %{_bindir}/hello

   The parts that need to be filled in are: Name, Version, Summary, URL (which is optional), and Source.  Then you need to add a description (can be multiple lines) after the “%description” line.  The last part you would need to edit is the list of files to be installed by this package (the lines following the “%defattr(-,root,root)” line).  These are the ones created when you did the make install step previously.  Note how documentation is marked with “%doc” tag.  What is the tag you would use to mark configuration files?  (You will probably need to do a bit of searching in the tutorials to find this.)

   Run rpmlint on your modifed spec file, and fix any errors reported.  (It is expected that the value for Source: will generate an illegal URL.  This is safe to ignore in this case.)

6. Now build a signed, binary RPM package using the rpmbuild command, with the correct two options.  What is the exact command you used?

   The signing of RPMs is done by a plug-in to the rpmbuild command.  On Fedora 16, this plug-in is not installed by default.  You will have to install the rpm-sign package first, or the rpmbuild command will skip signing the package.

   If this failed to work, you can use the rpmlint command on your spec file to see what might be the problem.  (You may need to install an additional package for this tool.)

   If you don't run this command as the same user that created the GPG key, the key won't be found.  You can export GNUPGHOME=~user/.gnupg to tell the command where to find the keyrings.  (Change “~user” to your home directory, of course.)

7. Locate the resulting RPM package.  Where did the system put the resulting RPM?  What is the exact name of the created package?  You should be able to query your package.  Use the rpm command to list the package information, and the list of files provided by this package.  (Note you will need to add the “-p” option in addition to the normal query options, to specify an un-installed .rpm file to query.)  What (two) commands did you use?  In the rpm output, what shows the package was correctly signed?
8. As root, attempt to install this package using the command “rpm -Uvh”.  (If you have SE Linux set to “enforcing”, this may not work.  Troubleshoot the problem.)  What change(s) did you make, if any?  Try running the hello command, and examine the man page installed.

Part IV — Creating and using a yum repository:

1. Remove the package you installed, using rpm -e.  Verify the command hello no longer works.
2. Next, create a new directory for your local yum repo.  This can be in any storage volume with sufficient space and appropriate mount options.  What is the pathname to the root of your new repository?  (For the remaining steps of this project, I will assume a pathname of “/var/myrepo/”.)
3. Copy your rpm package created in part III into your repository.  Make sure the file is read-only.  Then initialize the repo, by creating the appropriate meta-data files, using the command

   createrepo /var/myrepo
   

   (You may have to install some package to have this command.)  What was the output of running this command?  What new files were created in your repo directory?

4. To allow clients who do a “yum install” to validate digital signatures, you should sign the repo meta-data file, and also put the public key in the root directory of your repo (it doesn't have to be located here, but this is convenient):

   $ cd /var/myrepo
   $ export GNUPGHOME=~/.gnupg
   $ su -c 'gpg --detach-sign --armor repodata/repomd.xml'
   password:
   
   You need a passphrase to unlock the secret key for
   user: "Wayne Pollock (Software Packager for MyLocalRepository) <wpollock@localhost>"
   2048-bit RSA key, ID 9F19C168, created 2012-05-21
   
   Enter passphrase:
   
   $ su -c 'gpg --export -armor "Software Packager" > RPM-GPG-KEY-myrepokey.asc'
   password:
   

   Use su and not sudo, since sudo resets the environment (unles configured differently), and gpg will use root's keyring, not yours.  This can be addressed by adding the gpg option of --homedir ~ua00/.gnupg.  Using su (if GNUPGHOME is already exported in a login script) may be easier.  With either method, gpg will complain about “unsafe ownership on homedir”, since the user running gpg is different than the owner of the directory.  This is nothing to worry about.

5. To use this repo from other hosts, you would need to configure a web server or FTP server.  For this project, we will just access the repo locally.  (In the real world this is often the case; you use NFS to remotely mount the repo on your hosts.)

   Configure yum to know about your new repo, by creating a new file in /etc/yum.repos.d/.  See the man page for yum.conf(5) for the syntax and all the available options for this file.  At a minimum, your file must contain a unique repository ID, a human-friendly description, and the URL to your repo:

   [myrepo]
   name=Local yum repo for my company
   baseurl=file:///var/myrepo
   

   (Some options you should consider including are enabled=1, gpgcheck=0 (if you didn't digitally sign your RPMs), gpgkey=URL (if you did sign them), and others.  You can examine the other files, or read the man page, to see what directives can be used.

   What is the name and contents of your repo file?

6. Now install your package using the yum command.  What was the exact command line you used, and what was the output?  Verify the hello command from the package installed correctly.  (Note, if your repo is disabled by default, you will need to use the yum command line option to enable your repo.)
7. Update your system journal showing all changes made to your system.

• Do not work as “root” except when required.
• Active development on pine stopped some time ago.  (The new version is called Alpine.)  Since then the OpenSSL has split into two packages, and the default locations for various libraries and certificate directories (“certs”) has changed from where the pine spec file (and the pine build script) expects them.  By using various commands (rpm, yum, and locate, etc.), and examining build output for error messages, you can learn the new locations and libraries needed.  By reading the unpacked documentation with the source (in the “../BUILD” directory), you can learn how to modify the “build” command in the spec file, and/or define various environment variables, so the build can find the new locations.
• See docs.fedoraproject.org draft RPM Packagers Guide for details about building RPMs and using the build tools.  Also see fedoraproject.org/wiki/How_to_create_an_RPM_package, and the other tutorials in the class resources, for additional help with “spec” files.
• The rpmdevtools package includes several tools that can be very helpful in diagnosing RPM errors.  See the man pages for these, especially rpmlint.  On some systems, rpmlint may be provided by a different package that you should install.  (So is the “rpm-sign” package that provides the plugin needed to digitally sign packages.)

Trouble-shooting Compiles

• Installing from a package will automatically install required dependent software (tools, header files under /usr/include, and libraries in /usr/lib or /lib), but this doesn't happen when you install from source!  You must manually make sure the required development tools and software is installed.  The easy way to do this is to install everything, including all development packages (usually “name-devel”).  Otherwise you will get “command not found” when make tries to build and install software.  If that happens go and install the missing tool packages.  (Use “yum provides name” to find the package name.)
• If you have all the tools but don't have the required development software (headers and libraries), the compiler will show “not found” messages for included “header” files and/or DLLs.  (Missing header files will cause a cascade of error messages, so pay attention to the first one which says what is missing!)  Here's an example of failing to install the PAM development package when building some software that needs PAM:

  osdep.c:71:31: error: security/pam_appl.h: No such file or directory

  To solve this sort of problem, first search your system to see if the file is there but possibly in a different location.  (Note the default location for header files is “/usr/include/”.)  If not, use:

  yum provides /usr/include/security/pam_appl.h

  to get the package name (“pam-devel” in this case) and install it, or else find the source for that and install it.  When you restart the compile, it will either work or report something else is missing, in which case you repeat the process to install that.

  (To ensure the resulting package will work on any system, keep track of the dependencies, and add them to the spec file.  There is a tool called mock that creates a vanilla environment for rpmbuild to use, so you can be sure there are no hidden dependencies.)

Answers to the above questions and the relevant system journal entries.  You can send as email to (preferred).  If email is a problem for some reason, you may turn in a hard-copy.  In this case the pages should be readable, dated, and stapled together.  Your name should appear on the first page. 

Please see your syllabus for more information about submitting projects.