If you run CVSup on two identical servers, five minutes apart, you
will have a slightly different operating system on both systems. If
you're running several production machines, you'd be best served if
all the systems were absolutely identical. Even if they're running a
stable somewhere between 4.3-release and 4.4-release, being
able to eliminate differing software as a potential problem can help
troubleshooting immensely. You don't want to think "Gee, Server 1
keeps dying. Could it be because each server has a very slightly
different version of FreeBSD?" That way lies madness.
If you have several servers to upgrade, you're going to increase the load on the CVSup mirrors. And each load will be slightly different.
CVSup includes the ability to upgrade to the code as it stood on a particular date, but even that's not a guarantee. After all, the mirror operators are human, and the mirror hardware is fallible. It would be simpler to have your own source code repository on hand.
Having your own source code repository is pretty simple, actually. It's easy to do. It will make you popular with the mirror operators -- or at least, won't make you unpopular with them, which is almost as good.
The main mirror sites update their source code repository once an
hour. In many cases, I don't upgrade a local
cvsup server except by hand. When I have many machines and want to upgrade them all to the exact same version of
-stable, I can just upgrade them from the local
repository and be sure that they're all identical. I frequently
upgrade one server, put it through a few rounds of load testing, and then upgrade the rest.
cvsupd server isn't an easy task, but there's help to make it simpler. The port
/usr/ports/net/cvsup-mirror handles all the tricky bits of configuring a mirror.
When you install the port,
cvsup-mirror asks you some questions.
While there are default suggestions, you need to change many of them.
Master site for your updates [cvsup-master.freebsd.org]?
cvsup-master.freebsd.org, is reserved for official FreeBSD mirror use. Access to this system is tightly controlled by authentication keys; not just any doofus can use it. See the earlier article in this series for some hints on choosing a CVSup server.
How many hours between updates of your files ?
The script updates
/etc/crontab to run CVSup automatically. This default is intended for public mirrors. You probably don't need to upgrade more than once a day. Remember, every connection you make just increases the load on the mirrors. Are you really going to upgrade local systems every hour?
Do you wish to mirror the main source repository [y]? Where would you like to put it [/home/ncvs]? /repo Do you wish to mirror the installed World Wide Web data [y]? n Do you wish to mirror the GNATS bug tracking database [y]? n Do you wish to mirror the mailing list archive [y]? n
Most people just need the main source repository. If you want to mirror the whole www.freebsd.org site, including the PR database and the mailing list archives, you can answer the last three questions "y." (The mailing list archives are huge. Don't download them on a lark; this lark will peck out your eyes.) The source repository itself is about 1.3G at the moment, and grows slowly but continuously. Changes that make the size jump dramatically are referred to as "repo bloat."
Use unique user and group IDs for these. Don't use "nobody," "nonroot," or "nogroup;" these users are usually used by other processes. After all, you don't want a broken Apache server to compromise your CVSup mirror, do you?
Unique unprivileged user ID for running the client [cvsupin]? Unique unprivileged group ID for running the client [cvsupin]? Unique unprivileged user ID for running the server [cvsup]? Unique unprivileged group ID for running the server [cvsup]?
You can use the defaults, or use user and group names that fit your local scheme.
Maximum simultaneous client connections ?
The maximum simultaneous client connections is easy to change, so
don't sweat it. We'll see how in the
cvsupd.access file, below.
The make install process adds these usernames, sets the configuration, and generally gets you ready to go.
Also in Big Scary Daemons:
If you took the default during the
cvsup-mirror setup, your system
will upgrade its repository once an hour via
cron. If not, you must run the update script
/usr/local/etc/cvsup/update.sh by hand.
The first time the update script runs, it will download and configure the entire repository. This first download takes quite a while -- in my case, two hours over a cable modem. This initial download is a great candidate for a 3AM Sunday night
cron job. Updates are much quicker.
Once you're sure your repository is working, you can improve
performance by adding the
-s option to
update.sh. This turns on
mirror mode. In this mode,
cvsupd assumes that you have not touched any repository files by hand, and it can take some shortcuts that speed things up considerably.
This is simplicity itself. Just change the client's supfile to include
Just because you want to be a good Net citizen and have a private
repository, doesn't mean that you want every yokel downloading from
cvsupd server allows you to control access.
/usr/local/etc/cvsup/cvsupd.access controls which hosts may connect. The syntax is simple: a line beginning with
# denotes a comment. A
+ means that the client can connect, and a
- means that
the client cannot. A
* means that the client must authenticate (see below).
Each rule in
cvsupd.access can refer to either a host name or an IP address. IP addresses are preferred, for all the usual reasons. You can use netmasks with IP addresses as well.
For example, to allow access from the network 192.168.0.0/16 and reject clients accessing from elsewhere, use this.
You can also use
cvsupd.access to restrict the number of connections the system will allow at any one time. This is done by specifying a number after the network number. For example, to restrict the server to 10 connections per second you would use
As you might guess, you can use this system to throttle connections from blocks of IP addresses.
You can also use the
-C flag to
cvsupd to achieve the same effect, but that requires that you kill and restart
cvsupd. Changes to
cvsupd.access take effect immediately.
This system works well, until you want to allow people to connect by username. If you might want to connect to your CVSup mirror from arbitrary locations, you need to use authentication.
The CVSup server uses a challenge-response system for authentication, rather than handing passwords around in clear text. If someone drops a packet sniffer on the network, the cannot grab passwords. What's more, since the challenge-response system incorporates the time, the client IP address, some pseudo-random numbers, and a bunch of other system garbage that changes rapidly, a response cannot be used a second time.
You must use a password file,
/usr/local/etc/cvsup/cvsupd.passwd, to use authentication. This file should only be readable
cvsup user, or anyone could grab user information. (You can do this by running
chown cvsup cvsupd.passwd and
Blank lines and lines beginning with
# are ignored. The remaining lines are all for acceptable users.
The first line is the server name and private key, separated by a colon.
The server name is sent back to the client. The private key is used for additional randomness. You don't have to have a private key -- the
cvsupd password system is pretty random as is -- but you must have the colon. The private key cannot contain a colon.
After this, you have your legitimate users. Each user appears on a separate line, in the following format.
user ID:shared secret:class:comment
cvsup IDs are email addresses, i.e.,
"mwlucas@AbsoluteBSD.com." The shared secret is based upon a
cryptographic hash of your chosen password. The class is reserved for
future use, and should be left blank. Finally, the comment field can
be used by the administrator.
cvpasswd(1) command automates generating
You use it like this:
cvpasswd userID servername
# cvpasswd firstname.lastname@example.org magpire.blackhelicopters.org
Enter same password again:
Send this line to the server administrator at magpire.blackhelicopters.org:
Be sure to send it using a secure channel!
Add this line to your file "$HOME/.cvsup/auth", replacing "XXX"
with the password you typed in:
Make sure the file is readable and writable only by you!
Well, this is pretty straightforward. Copy the first line given to
/usr/local/etc/cvsup/cvsupd.passwd on the server. On your client system, create a
.cvsup directory and put the second line into
.cvsup/auth. Make sure that only you can read that file (
CVS master John Polstra says, "A common mistake here is to put the
'.cvsup' directory into the wrong user's home directory. It should be
in the home directory of the user ID under which the 'cvsup' command
runs. In the standard
cvsup-mirror installation, that would be
~cvsupin/.cvsup/auth." There speaks a man who has answered the same
question one too many times.
If you want to demand that all your users authenticate, it's pretty
simple. Create an empty
cvsupd.access file, and build a normal
cvsupd.passwd file. Requiring authentication is the standard in this case.
If you don't want to use authentication, don't create a
If you have neither a
cvsupd.access nor a
cvsupd.passwd file, anyone can connect to your server.
If you want to allow certain users to authenticate from anywhere, but allow some networks to update no matter who is running the system, things get only slightly more complicated.
There is an implicit "authenticate" rule at the end of
cvsupd.access. If your client hasn't been blocked out by an explicit "deny" rule based on IP address, you'll be allowed to authenticate. No special configuration is required.
If you're running a CVSup server behind a NAT or firewall, you
probably don't have to worry about either access control or
cvsup-mirror and go! This saves the mirror operator's resources, your own bandwidth, and generally reduces Internet traffic. And other people will be able to connect to the official mirrors just a little more easily. Besides, it will impress the other FreeBSD users just a little.
Michael W. Lucas
Read more Big Scary Daemons columns.
Return to the BSD DevCenter.
Copyright © 2009 O'Reilly Media, Inc.