ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


Subversion for BSD With All the Bells and Whistles

by Jeff Palmer
09/27/2007

I recently took a position with a new employer. One of the first tasks they had was to set up a full Subversion server with all the bells and whistles. My subversion experience was a bit limited, so the task seemed daunting. Luckily, I ran across an OnLamp article written by Dru Lavigne that helped get me up to speed relatively quickly. Because my employer wanted all the bells and whistles, the article only covered the basics of what I needed done.

My employer has several projects in Subversion. Some are closed source, others are open source. Subversion is a revision control system, that allows groups of people to collaborate on a project without overwriting another persons work/effort. Additionally, authentication was to be encrypted, and we needed Trac set up for each project. Trac is a web-based trouble ticket and repository viewer, among other things. It's rather useful for tracking bugs, feature/enhancement requests, and other similar aspects. It is exceptionally handy for collaborative efforts as well.

To add to the complexity, Trac was to run via Apache-2.2.x using mod_python and have a postgreSQL database backend. Lastly, I had some requirements of my own. I wanted to make sure each piece of software was in the FreeBSD ports tree, for easy administration. Also, this company does a lot of communication via IRC (Internet Relay Chat), so I wanted to setup a method to have announcements sent to the developers on the channel. As you can see, the number of requirements was quickly building, and there didn't seem to be an abundance of FreeBSD-related documentation.

Since I already knew I was going to be using Apache and mod_python for Trac, I decided to use mod_dav_SVN and have Apache handle everything including authentication to SVN. I proceeded to install Apache.

# cd /usr/ports/www/apache22
# make WITH_BERKELEYDB=db42 WITH_MPM=worker install

Subversion will use Berkeley DB as its backend, so we compile Apache with support. Through trial and error, I found that after a period of time "prefork MPM" left several unresponsive httpds running, and eventually, Apache would stop answering requests. "Worker MPM" seems to have fixed the problem rather nicely.

Next, we move on to installing Subversion itself.

# cd /usr/ports/devel/subversion
# make WITH_PYTHON=yes WITH_MOD_DAV_SVN=yes WITH_APACHE2_APR=yes install

We want Subversion to have python bindings for Trac. The other two options are to compile the DAV module and use Apache's APR, instead of the standalone.

Of course, the Apache installation isn't complete without mod_python!

# cd /usr/ports/www/mod_python3
# make all install clean

We don't need any extra WITH_* targets, just let it compile and install.

Next, we'll get postgreSQL setup. I only needed the client installed, because we run a central database server. You may want to run postgreSQL server on the same machine.

# cd /usr/ports/databases/postgresql82-server/
# make install /* Select the libc_r option in the menu

or

# cd /usr/ports/databases/postgresql82-client
# make install   /* Select the libc_r option in the options menu */

libc_r is used by plpython, and is desirable for Trac.

And finally, we need to install Trac.

# cd /usr/ports/www/trac
# make install  /* Select PGSQL support in the options menu */

Now that we have the software installed, we'll need to configure it.

Because each of the ports modified the Apache config, we'll need to change some things.

Using your favorite editor, open /usr/local/etc/apache22/httpd.conf

Remove the line:

LoadModule dav_module libexec/apache22/mod_dav.so

(The way we installed it, made 2 identical entries. Remove one.)

Also, add:

LoadModule python_module libexec/apache22/mod_python.so

to enable the python module for Trac.

We'll need to set apache to start on system boot.
Edit /etc/rc.conf. Add:


apache22_enable="YES"

If you installed postgreSQL server, we'll also want that to start.
Edit /etc/rc.conf. Add:


postgresql_enable="YES"

You'll also need to run a couple commands to initialize and start the postgreSQL database.

# sh /usr/local/etc/rc.d/postgresql initdb
# sh /usr/local/etc/rc.d/postgresql start

Configuration Details

First, we'll need to create the Subversion repository and set up the Trac environment.

# svanadmin create /home/svn/projectname
# trac-admin /home/trac/projectname initenv

The svnadmin command initializes the Subversion repository, and gets it ready to store all of the code for your new project. trac-admin will initialize the Trac environment, and make it accessible through the web browser. Make sure to create your PostgreSQL database prior to this step. Also, both subdirectories are owned by the wrong user:group. To fix this, we'll issue the following command:

# chown -R www:www /home/svn/projectname
# chown -R www:www /home/trac/projectname

This will allow the Apache process to make changes to those directories.

We'll also need to assign administrative permissions to the Trac environment. The easiest way to do this is with a plugin.

# cd /usr/ports/www/trac-webadmin
# make all install clean

You'll also need to assign yourself trac_admin privileges to use the plugin. After you run the following command, log out and back in again. You should see a new "Admin" tab on your Trac navigation menu.

# trac-admin /home/trac/projectname permission add yourusername TRAC-ADMIN

Next, we'll setup Apache and Trac to handle a common installation scenario. This would be suitable for an open source project where anyone is allowed to check out the repository, and create trouble tickets in Trac, while the write functions of SVN, and logging into trac are done over SSL.

In /usr/local/etc/apache22/extra/httpd-vhosts.conf:

<VirtualHost *:80>
ServerName svn.projectname.org
ServerAlias projectname.org
ServerAdmin webmaster@projectname.org
ErrorLog /var/log/apache/projectname.org/svn/httpd-error.log
TransferLog /var/log/apache/projectname.org/svn/httpd-access.log
<Location />
SetHandler mod_python
PythonInterpreter main_interpreter
PythonHandler trac.web.modpython_frontend
PythonOption TracEnv /home/trac/projectname
PythonOption TracUriRoot /
</Location>
<Location /login>
Redirect permanent /login https://svn.projectname.org/projectname/login/
</Location>
<Location /svn>
DAV svn
SVNPath /home/svn/projectname
AuthzSVNAccessFile /home/svn/ro_access
</Location>
</VirtualHost>

In /usr/local/etc/apache22/extra/httpd-ssl.conf:

<Location /projectname>
SetHandler mod_python
PythonInterpreter main_interpreter
PythonHandler trac.web.modpython_frontend
PythonOption TracEnvParentDir /home/trac
PythonOption TracUriRoot /
</Location>
<Location /projectname/login>
AuthType Digest
AuthName "svn"
AuthDigestDomain /
AuthDigestProvider file
AuthUserFile /home/svn/htdigest
Require valid-user
order allow,deny
allow from 127.0.0.4
Satisfy any
</Location>

<Location /svn>
DAV svn
SVNParentPath /home/svn
AuthzSVNAccessFile /home/svn/access
AuthType Digest
AuthName "svn"
AuthDigestDomain /
AuthDigestProvider file
AuthUserFile /home/svn/htdigest
Require valid-user
</Location>

You'll need to put the above code in the appropriate location of the Apache SSL config file. Also, pay attention to the 127.0.0.4, as we'll be modifying it later for the IRC announcements.

As you can see, checking out the SVN repository and creating/viewing tickets is all done over an unencrypted link. Meanwhile, logging into Trac, or Subversion, is handled via SSL encryption. Of course, you could modify the above configs to handle everything over an encrypted link.

We still need to create a user and set up the read-only access.

htdigest /home/svn/htdigest svn yourusername

In /home/svn/access:
[/] yourusername = rw

The rw will allow yourusername to read and write to anything in the repository.

In /home/svn/ro_access, add:
[/]
* = r

The * matches everyone, and sets them to read-only mode.

Now that Apache is configured, you have a user, and have set up anonymous access, you're probably interested in using them!

Trac will be available at http://svn.yourprojectname.org and the Subversion repository will be available at svn co http://svn.yourprojectname.org/svn/projectname.

For the authenticated features of the above, replace http with https.

Now that we understand what is going on, let's attempt a private or closed source project. Only authenticated users are allowed to browse the repository, check out code, and create/view/edit tickets. Also, everything is encrypted via SSL. This setup will be much easier than the last, as we don't have to worry about http/https or read-only users.

In /usr/local/etc/apache22/extra/httpd-ssl.conf:

<Location /admin>
SetHandler mod_python
PythonInterpreter main_interpreter
PythonHandler trac.web.modpython_frontend
PythonOption TracEnvParentDir /home/trac
PythonOption TracUriRoot /
AuthType Digest
AuthName "prometheus"
AuthDigestDomain /
AuthDigestProvider file
AuthUserFile /home/svn/htdigest
Require valid-user
order allow,deny
allow from 127.0.0.4
Satisfy any
</Location>

Again, pay attention to the 127.0.0.4, as it will be used for the IRC announcements. Your Trac and SVN should only be available to authenticated users over https now.

Lastly, for the configuration stuff, you may want to refer to this article to allow certain groups to specific parts of the repositories. (For example, web developers don't need commit access to the source code, only to the www directory.)

Commit Messages and New Ticket Notifications over IRC

I looked for quite a while to find an easy-to-use announcement method for our developers and analysts who use IRC as their primary means of communication. In the process, I found several software projects that would work, but ultimately, I decided on using a Supybot. Supybot is an IRC bot that has multiple plugins and features, allowing you to quickly and easily adapt it for specific needs. But wait, it's even better! Supybot has an available plugin to read RSS feeds, and Trac has RSS feeds. The fact that Supybot is written in Python is even better, because we're already using Python for the Trac installation. What more can you ask for?

# cd cd /usr/ports/irc/py-supybot-plugins/
# make all install clean

This will compile Supybot, and the extra plugins pack (which contains the RSS plugin)

Change directories to wherever you want your Supybot to run from. It doesn't matter where you choose, and it's all personal preference. I personally chose to create a new user account named Supybot and su'd as that user.

# supybot-wizard

Answer the questions it asks. When it asks about plugins, enable the RSS plugin. this is the main one we are interested in, but as I mentioned earlier, supybot has a lot of plugins to enhance functionality. Spend some time, and see which ones interest you. Now that you've got that done, it's time to start it.

 nohup supybot YourBotName.conf &

Your announcement bot should appear on your IRC channel within a few minutes. If not, check the logs and fix the problems. After he appears in your channel, you'll need to identify yourself to him with the username and password you chose a minute ago.

/msg YourBotName identify YourUsername YourPassword

He should respond with "The operation succeeded" or something similar.

Now it's time to configure the actual announcement plugin. You'll want to message (/msg) all of the following commands to your new announcement bot.

 config plugins.rss.announcementprefix ""
config plugins.rss.waitperiod 60
config plugins.rss.announce.showlinks true
rss add projectname http://svn.yourproject.org/projectname/timeline?ticket=on&ticket_details=on&milestone=on&changeset=on&wiki=on&max=5&daysback=1&format=rss
rss announce #channel projectname

The above lines do the following:

  1. Disable announcement prefixes
  2. Set the RSS feed check interval to 60 seconds
  3. Show the link to the ticket or changeset being announced
  4. Add the URL where the trac RSS feed resides
  5. Start the bot announcing on #channel

In closing, this has been a very educational project for me. I've never really used Python, Subversion, or Trac before. I hope this helps those of you who want to set up a similar environment for your project.

Jeff Palmer is a systems administrator with over 15 years experience with Windows and Unix operating systems. He has worked for several privately owned ISPs and other internet upstarts. Jeff is currently working as a contractor doing FreeBSD administration for various clients, and is active in the FreeBSD community. His hobbies include racing motorcycles (on a track of course!), riding jet skis, reading, embedded computers, and almost anything else computer related.


Return to ONLamp: BSD DevCenter.

Copyright © 2009 O'Reilly Media, Inc.