BSD DevCenter
oreilly.comSafari Books Online.Conferences.

advertisement


Building a FreeBSD Build System
Pages: 1, 2, 3

Export the source

The next step is to allow the clients to access to your sources and built binaries. Although I use NFS here, an interesting alternative for an internet-wide system would be to export over FTP.



For NFS, add to your /etc/exports file:

/usr/src -ro -mapall=nobody
/usr/obj -ro -mapall=nobody
/usr/ports -ro -mapall=nobody

Enable NFS by adding to /etc/rc.conf:

rpcbind_enable="YES"
nfs_server_enable="YES"
mountd_flags="-r"

Then start NFS by running /etc/rc.d/nfsd start.

Build the binaries

Now that your synchronized code is accessible remotely, you can start to build it. Normally, you can just cd to /usr/src and perform the make buildkernel and installkernel, but because you are now building for multiple machines, you may want to make it institute a locking mechanism as well as make it a little more efficient. I am including a shell script called build_all.sh that will disable make in /usr/src while a build is under way. It will also disable make installworld or installkernel outright if there is an error in the build process. For finesse, it even prints out a message when you run make to help you diagnose the problem.

Something else I found that traverses the border of the unsupported is the ability to build multiple make buildkernels simultaneously. While this might provide only a small benefit for uniprocessor machines, it's a huge benefit for a multiprocessor one. The build system will even let you know if an error occurred in building a particular kernel and prevent you from hosing your machine. Nonetheless, always test on your staging server before touching production.

To use these features, copy build_all.sh to your build server and put it in your crontab:

# build sources
0 10 * * * root ~root/bin/build_all.sh 4

The argument you give build_all.sh will be passed through to -j during the buildworld stage.

/usr/ports Power

Although I won't go into much depth on improving the spectacular ports system, there are a couple of tweaks you can now do to make it even more powerful and faster.

The first benefit is that you can keep an internal distfiles mirror. This reduces the time required in installing from ports. I have included another script called fetch_distfiles.sh, which will go through and run make fetch on every port in the system. You may want to modify the SUBPROC and CONCURRENTFETCH variables to suit your hardware, but basically they set a high-water mark for parallel process and parallel fetches you want to run simultaneously. Be forewarned that if you set these variables too high, you will come in to work to a build server running at the speed of hot buttered rum. The best strategy is to start low and turn up from there.

The second benefit is that you can start making local ports. You can keep a meta port that just builds all the fun packages you normally install, or build a port to install that anonymous binary application you have to run on all your servers.

The only change you need to make to a local port is to its makefile to be sure it doesn't get confused:

CATEGORIES=local
VALID_CATEGORIES+= ${CATEGORIES}

This will allow you to keep your port in new category called local.

Next, add the category to /usr/ports/.cvsignore so that it doesn't get accidentally erased on the next cvsup update. I have included a makefile for a local port I made called rcs_mergemaster.

Add the clients

The procedures for adding clients and the staging server are the same. Because the total update procedure takes about 5 minutes, it doesn't hurt to just run a test install on the staging server first, before the day starts.

Mount the remote filesystem

The first step is to mount the filesystems exported by the build server. Add to your rc.conf:

nfs_client_enable="YES"

Then to your /etc/fstab to mount the share automatically on boot:

build-ports:/usr/ports /usr/ports nfs ro,intr,bg 0 0
build-src:/usr/src  /usr/src   nfs ro,intr,bg 0 0
build-obj:/usr/obj  /usr/obj   nfs ro,intr,bg 0 0

Using the intr option allows you to recover from a NFS hang by sending an interrupt signal to the process trying to access the remote system. The bg option speeds up and prevents a hang during boot. The ro option is just a safeguard to verify the filesystem mounts read-only.

Pages: 1, 2, 3

Next Pagearrow





Sponsored by: