Staying Current with NetBSD

NetBSD is famous for its multi-platform architecture; it runs on everything from ancient VAXes to modern i386 and Sun servers. In this article we're going to upgrade a NetBSD system from the 1.6 release to the latest NetBSD-current, so I can eventually cross-build an hpcarm release and install it on my new palmtop. (If a UNIX-like palmtop won't ensure my reputation as local "alpha geek," nothing will...)

I've written about NetBSD on a couple of previous occasions, and I still have that same Alpha Multia running NetBSD in my home network providing DNS, filesharing, and other assorted small services. Most of what I touched on in those articles hasn't changed much--the installer is prettier, hardware support is updated, but NetBSD is still a basic, reliable BSD-based system. Cross-building releases on something as old as that Multia isn't such a grand idea, however, so I started over on a modern Athlon system.

The NetBSD Project provides comprehensive documentation on how to upgrade the operating system. As with many such comprehensive documents, it's frequently difficult to know which steps you should follow in your particular situation. This article isn't a comprehensive tutorial that covers every possible situation; rather, it covers the most common situation: updating your source with CVS, building that source code, and installing it on the build machine. While you could have your own source code repository, or tunnel the whole process through sup(1), most users just use this method. As the upgrade process is already sufficiently complicated, we'll cover the ways that are most likely to succeed. You may find, through further research, ways to optimize your particular case, but this way will work. We'll consider each of the following steps in turn:

• Back up your data
• Get the system and package source code
• Update the system and package source code
• Check /usr/src/UPDATING
• Build the new kernel and reboot onto it
• Build the operating system
• Install the system
• Update your configuration

As with any major system change, an upgrade can possibly damage your system or your data. Be certain that you have backed up all vital data. I define "vital data" very simply: if the data vanishes, how long will I spend worrying about it? If it's more than thirty seconds, the data is vital and might as well be included in the backups. In this particular case, I'm upgrading a new machine so I don't need to make backups. Never skip this step on a production system, however. While you may get away without backups on most occasions, their absence will cause you grief eventually. If nothing else, be sure you have a backup of your existing /etc directory. It's very easy during an update to, say, accidentally overwrite your password file. A convenient backup of /etc will let you recover easily.

# tar -czvf /etc.tgz /etc

A standard NetBSD install doesn't install the source code or the pkgsrc tree that allows easy compilation of add-on software. You could just download all of the code via anonymous CVS, but that's not an efficient use of network resources or of the anonymous CVS servers. Instead, start with recent source code and use anonymous CVS to update that. Grab recent source code from a NetBSD FTP mirror, in the NetBSD-current/tar_files/ subdirectory. These tarballs are updated weekly, so you should only require minor updates to get the very latest source code. You'll see a single tarball for the package source tree, and subdirectories for the source code and the XFree86 source code. For now, just grab the system source code and, if desired, the package source code. All of these need to be extracted under /usr, i.e.:

# cd /usr
# tar -xzf $HOME/bin.tar.gz

Now that you have a recent source code collection, you can update it with anonymous CVS. NetBSD provides a list of anonymous CVS mirrors. Choose one close to you that supports the SSH protocol. While you can also use anonymous CVS in clear-text, in my opinion anything that can be tunneled over an encrypted connection should be. The privacy of CVS updates may not be an issue, but the integrity of the data could be.

Using anonymous CVS over SSH requires two environment variables, $CVSROOT and $CVS_RSH. With NetBSD's default csh(1), set them like this.

# setenv CVSROOT anoncvs@anoncvs.netbsd.org:/cvsroot
# setenv CVS_RSH ssh

Once you have these in your environment, updating both the system and the package source code is simplicity itself.

# cd /usr/pkgsrc
# cvs -d $CVSROOT update -dPA

# cd /usr/src
# cvs -d $CVSROOT update -dPA

Related Reading The Complete FreeBSD Documentation from the Source By Greg Lehey Table of Contents (PDF) Index (PDF) Sample Chapter

These commands will take some time to run; cvs(1) compares the version of every file in your existing source code tree to those on the anonymous CVS server. When they finish, however, you'll have the freshest source code available to anyone. (On subsequent updates from the same server, you could delete the -d $CVSROOT portion of the command. It doesn't hurt anything to keep it.)

Now that you have the latest source code, you can check for special instructions for using the latest -current. The NetBSD team keeps special notes in the file /usr/src/UPDATING, in reverse chronological order. Notes in here may only apply to certain architectures or to certain types of programs. For example, when the new threading support was merged into the mainline -current, every threaded add-on program had to be rebuilt to eliminate the reliance on the old pthreads package. (Some people made extra sure they won't have problems by rebuilding all the packages on their system after an upgrade, but that may be overkill for you.)

The NetBSD team makes heavy use of cross-building to support all of their platforms. Obviously, building the operating system on a VAX or a Dreamcast would take quite a while; it's much simpler to build on a faster system and install the resulting binaries on older platforms. While this is a great idea, the actual cross-building process became quite complicated very quickly. The NetBSD build process now hides all of this complexity behind the build.sh shell script. We'll look at cross-building in a future article, but for now we'll just focus on building the software natively.

To build the operating system, you first need to build the tools to build the operating system. We want to start by building the latest compiler and associated components, and using those to build the latest NetBSD kernel. Much of this work will take place under /usr/obj/, so make sure that directory exists first. (If you don't have that directory, build.sh will die complaining.) When running an upgrade such as this, it's best to run the GENERIC kernel for maximum reliability. You can always build a new, customized kernel once you're sure that the generic kernel works.

# mkdir /usr/obj
# cd /usr/src
# ./build.sh tools kernel=GENERIC

Eventually, the build will finish. Before proceeding, install the new kernel and reboot to it. If you grabbed the source code at the wrong time, the kernel might have problems. It's best to know about those problems as soon as possible, preferably before installing the userland programs that require that version of the kernel. Also, the new userland programs might require system calls that are only available in the new kernel. Here, we install the latest i386 kernel. You'll want to use the proper directory for your architecture.

# cd /usr/src/sys/arch/i386/compile/GENERIC
# make install
# reboot

Now, wait to see if your machine ever boots again. If there's a problem with the new kernel, reboot to the old kernel. If your system crashes on the new kernel, or refuses to boot to multiuser mode, you should check the NetBSD-current mailing list archives and see if other people are experiencing the same problem with recent builds. If not, start debugging the problem. (Isn't the latest open source fun?) Often, simply waiting a day and re-upgrading your source code can resolve these issues.

Running a kernel that is newer than your userland will cause problems. Programs that rely on kernel data structures, such as top(1), systat(1), and IPFW may not function correctly. You need to get to a matching userland as soon as possible. Generally speaking, attempting to build the new userland is a good kernel test.

Once you are confident that your kernel is stable enough to stay up a while, rebuild the userland software using build.sh. You'll need to pick a directory to hold your newly-built software distribution. Here, we put everything in /usr/i386build/.

# ./build.sh -U -D /usr/i386build distribution

If the build does not finish, you might have grabbed source code at a bad time. Definitely check the NetBSD autobuild status page, which displays the results of the latest build attempts on the official NetBSD build machine. If there's a global problem, it will be visible there. Try again later, and see if the problem is fixed.

If the build completes successfully, you can then install it on your system at a convenient time. Some people prefer to do this step in single-user mode, but that's your choice.

# ./build.sh -U -D /usr/i386build install=/

Yes, you could install to a location other than the root directory. While this might seem senseless, you could use this to install on a hard drive that you intend to move to a different machine, or even to upgrade a system over NFS.

You could also do the build and install in a single step. I frequently build the updated code during production time and then install it during a maintenance window, but if you're upgrading your personal machine you just might want to do it all at once.

# ./build.sh -U -D /usr/i386build distribution install=/

As the last step in installing the build, NetBSD performs some basic system checks to see if obsolete files are left in the tree. This postinstall script generates output much like this:

...
postinstall check:
        /usr/src/etc/postinstall != ///etc/postinstall
defaults check:
        /usr/src/etc/defaults/daily.conf != ///etc/defaults/daily.conf
        /usr/src/etc/defaults/rc.conf != ///etc/defaults/rc.conf
        /usr/src/etc/defaults/security.conf != ///etc/defaults/security.conf 
...

First, the postinstall script runs through all of the standard "default" NetBSD configuration files, such as those in /etc/defaults, and compares them to the versions in the latest build. As we see above, it will print out a list of every default file that doesn't match. The script then churns through a list of user IDs and group IDs that have been added to the system recently, and checks to determine if they are in place in the new system. Finally, it will look for files, man pages, and libraries that have been obsoleted and make a list of them for removal. The process will end with a statement much like this:

...
postinstall checks passed: ssh wscons
postinstall checks failed: postinstall defaults mtree gid uid periodic \
	 rc makedev postfix obsolete
To fix, run:
    /usr/src/etc/postinstall -s /usr/src -d // fix postinstall defaults \
		mtree gid uid periodic rc makedev postfix obsolete

Of the various groups of postinstall checks, only the ssh and wscons checks have passed. The postinstall script has already displayed exactly what the problems are, and will automatically fix them for you if you just type the command given. If one of the issues shown isn't applicable on your system--for example, if you're running a customized version of Postfix and you don't want the postinstall script to touch anything--just remove that one test from the command.

After completing the postinstall checks, proceed to update the rest of /etc. While you could manually compare the various files in /etc to their updated counterparts in /usr/src, NetBSD uses the add-on tool etcupdate(8) (/usr/pkgsrc/sysutils/etcupdate) specifically for this purpose.

# cd /usr/pkgsrc/sysutils/etcupdate
# make install clean

Much like the mergemaster program used in FreeBSD and OpenBSD, etcupdate(8) builds a temporary directory populated by the /etc files from the latest source code. etcupdate(8) then compares the new files to the ones currently inhabiting /etc, and displays the differences. You'll then have the option to keep the current version, install the new version, or merge the old and new versions. Let's walk through one of each.

--- /etc/mail/aliases   2002-09-08 15:33:32.000000000 -0400
+++ /tmp/temproot/etc/mail/aliases      2003-03-15 15:41:46.000000000 -0500
@@ -1,4 +1,4 @@
-#      $NetBSD: aliases,v 1.15.2.1 2002/06/06 02:40:42 lukem Exp $
+#      $NetBSD: aliases,v 1.19 2002/06/06 00:21:57 lukem Exp $
 #
 #      from: @(#)aliases       8.2 (Berkeley) 3/5/94
 #

File: /etc/mail/aliases (modified)

Here we see the entire difference between the old and new files is a version number. (This particular example may be rare in the real world; most systems have a custom aliases file! It's true for any file with truly trivial changes, however.) If you were interested, you could use the NetBSD CVSweb tool to find out what changes were put in and later removed. In any event, after displaying the differences, etcupdate(8) gives the following options:

Please select one of the following operations:

  d  Don't install the new file
  i  Install the new file (overwrites your modifications!)
  m  Merge the currently installed and new files
  s  Show the differences between the currently installed and new files
  v  Show the new file

What do you want to do? [Leave it for later]

In this case, we want to install the newer version of the file. While there is no functional difference, etcupdate(8) will complain about this file every time you upgrade, so you may as well install the file and shut the program up. Other files have larger differences, such as this skeleton .cshrc file.

--- /etc/skel/.cshrc    2002-09-08 15:33:34.000000000 -0400
+++ /tmp/temproot/etc/skel/.cshrc       2003-03-15 15:41:47.000000000 -0500
@@ -1,4 +1,4 @@
-#      $NetBSD: dot.cshrc,v 1.1 2000/10/16 13:12:26 simonb Exp $
+#      $NetBSD: dot.cshrc,v 1.2 2002/09/29 14:52:57 grant Exp $
 #csh .cshrc file
 
 alias h                history 25
@@ -24,5 +24,4 @@
        set mail = (/var/mail/$USER)
        set mch = `hostname -s`
        set prompt = "${mch:q}: {\!} "
-       umask 2
 endif

Previously, new users had a umask of 2. This is no longer set in the skeleton .cshrc file. Do you want that set for new users? In this case, I don't want to change the previous system behavior, so I will tell etcupdate(8) to delete the new file with d.

Other files are not so easy, such as this sample from /etc/group.

--- /etc/group  2003-03-15 07:56:22.000000000 -0500
+++ /tmp/temproot/etc/group     2003-03-15 15:41:46.000000000 -0500
@@ -1,4 +1,4 @@
-wheel:*:0:root,mwlucas
+wheel:*:0:root
 daemon:*:1:daemon
 kmem:*:2:root
 sys:*:3:root
@@ -6,7 +6,6 @@
 operator:*:5:root
 mail:*:6:
 bin:*:7:
-news:*:8:
 wsrc:*:9:
 maildrop:*:11:
 postfix:*:12:
@@ -14,6 +13,7 @@
 named:*:14:
 ntpd:*:15:
 sshd:*:16:
+smmsp:*:17:
 staff:*:20:root
 guest:*:31:root
 nobody:*:39:

I need to remove the news group, add the smmsp group, and keep mwlucas in the wheel group. I must merge the old and new files, so I use etcupdate(8)'s m option. etcupdate(8) will compare the two files and display each line that conflicts, with the old version on the left and the new version on the right, like this.

wheel:*:0:root,mwlucas                | wheel:*:0:root
%

We want the entry on the left, so type l and hit ENTER. etcupdate(8) immediately displays the next conflicting line:

news:*:8:                             <
%

The group news has been removed from the base system, and I am not using that group on this computer. Type r and then ENTER to continue through the file. When you're finished, etcupdate(8) will let you install the file, try again, or view the combined file.

Similarly, you'll need to go through every file with local modifications and compare it to the new configuration. In some cases, such as /etc/master.passwd, you might be better off using vipw(8) to add and remove a couple of users rather than walk through dozens or hundreds of user accounts in etcupdate(8).

Once you have fully updated your /etc directory, reboot your system. Once it comes up, you'll have fully updated NetBSD-current on your computer! Over the next few articles, we'll work from that base to get this new NetBSD onto a palmtop computer.

Michael W. Lucas

Read more Big Scary Daemons columns.

Return to the BSD DevCenter.

Copyright © 2007 O'Reilly Media, Inc.