ONLamp.com: Custom-Compiling Apache and Subversion

Published on ONLamp.com (http://www.onlamp.com/)
http://www.onlamp.com/pub/a/apache/2005/07/07/apache_svn.html
See this if you're having trouble printing code examples

Custom-Compiling Apache and Subversion

by Manni Wood
07/07/2005

Custom-compiling Subversion and Apache is the best way of controlling what you want (and don't want) from your repository. I recently had to set up a Subversion repository for a project and found that custom compilation was the only way to get exactly what the project required.

The project requirements were:

Allow secure repository access from anywhere on the internet.
Ideally meet security requirements without having to ask the IT department to change any firewall rules.
Keep the repository on a local or an NFS-mounted filesystem.

Using Apache 2.0 as the front end for Subversion was the best way to meet the first two requirements; we could use https, which our firewall already allowed. Using Subversion 1.1, and specifically the new FSFS repository storage method, allowed us to satisfy the third requirement.

Custom-compiling Apache 2.0 let me exclude modules that most binary Apache distributions typically include but that seem at odds with our goals, such as mod_userdir and mod_status. Custom-compiling Subversion also let me use the very latest version--1.1.3 at the time of this writing--and be certain it integrated well with my custom Apache install.

Because I did my work with Fedora Core 1, this article will have the most relevance to that OS and distribution, but hopefully this article will get you most of the way there with other Unixes as well.

Prerequisites

Fedora Core is a reasonably feature-rich distribution; it already comes with gcc and OpenSSL, two essentials for getting Apache and Subversion working in the way that satisfied my three goals. If you do not have those packages on your Unix/Linux, install them. Your distribution media for Fedora Core contains these packages if you haven't already installed them; Sunfreeware.com has them for Solaris; other resources are available for other Unixes. Definitely get the binaries for these packages if you can; there's no benefit from custom-compiling these packages.

The exact versions I used were OpenSSL 0.9.7a Feb 19 2003, and gcc 3.3.2 20031022 (Red Hat Linux 3.3.2-1).

Please note that I'm assuming you can run all of these commands as root--it's necessary to make a lot of this stuff work.

Compiling Apache

I downloaded the latest Apache 2.0 source code (2.0.53 at the time of this writing) and copied it to /usr/local/src where I always do my custom compiling. If your system doesn't already have this directory, I encourage you to create it.

Red Hat/Fedora Core users note that distribution RPMs never install in /usr/local, which is nice: after all, /usr/local is for stuff that's "local" to your installation! Red Hat and Fedora Core's packages respect the proper use of /usr/local and leave it alone, which is particularly nice at upgrade time: your local modifications to your distribution survive upgrades intact. Complimentarily, almost all custom-compiled software that you will install from source code uses /usr/local as its base install path.

Next, I prepared Apache for compilation:

[root@localhost etc]# cp /downloads/httpd-2.0.53.tar.gz /usr/local/src
[root@localhost etc]# cd /usr/local/src
[root@localhost src]# tar -xzvf httpd-2.0.53.tar
[root@localhost src]# cd /usr/local/src/httpd-2.0.53

Note that only GNU tar recognizes the -z option. If your Unix has a non-gzip-aware tar, I recommend you use the following command instead:

[root@localhost src]# gunzip httpd-2.0.53.tar.gz
[root@localhost src]# tar -xvf httpd-2.0.53.tar
[root@localhost src]# # re-gzip to save disc space
[root@localhost src]# gzip httpd-2.0.53.tar

At this point, I was ready to configure Apache. I almost always run ./configure with arguments when I custom-compile software, and over the years I've come to make a wrapper script called runconfigure.sh so that I have a record of the arguments I used when I ran ./configure. In this script, I asked for all modules to be shared, for most modules to be compiled in (especially ssl and dav, which Subversion needs), but for mod_status and mod_userdir to be compiled out. Arguably, I could also have compiled out other superfluous modules such as mod_cgi. Here's my script:

#!/bin/sh
# runconfigure.sh -- wrapper script for ./configure
./configure --enable-mods-shared="most ssl dav" --disable-status 
  --disable-userdir

Then I compiled and installed Apache:

[root@localhost httpd-2.0.53]# chmod +x runconfigure.sh
[root@localhost httpd-2.0.53]# ./runconfigure.sh
[root@localhost httpd-2.0.53]# make
[root@localhost httpd-2.0.53]# make install

A Few Fixes to Apache's Configuration and Startup

Interestingly, the Apache install script did not remove the mod_userdir configuration from httpd.conf, even when I chose not to compile that module. I had to comment it out manually. There's only one line to remove: UserDir public_html/. Here's one way to do so, without having to use an interactive editor:

[root@localhost httpd-2.0.53]# cd /usr/local/apache2/conf
[root@localhost conf]# sed -i -e 's/^UserDir public_html/# &/' httpd.conf

I needed to fix another small problem with Apache in order for SSL to work smoothly. The apachectl control script needs to define the property SSL on the command line to make Apache read its SSL configuration from httpd.conf and ssl.conf. Although Apache needs to have SSL defined for both startup and shutdown, only the startup clause of the Apache control script defines it. To fix this problem, I opened /usr/local/apache2/bin/apachectl in a text editor, looked for the line startssl|sslstart|start-SSL, and added the following lines above that line:

stopssl|sslstop|stop-SSL)
    $HTTPD -k stop -DSSL
    ERROR=$?
    ;;

Once you have installed your own custom-compiled Apache with mod_ssl, don't forget to start and stop Apache with the commands /usr/local/apache2/bin/apachectl startssl and /usr/local/apache2/bin/apachectl stopssl, not /usr/local/apache2/bin/apachectl startand /usr/local/apache2/bin/apachectl stop.

Securing Apache Conveniently

To run using https, Apache needs a certificate to give to browsers. Compiling Apache does not generate this certificate; I had to create one myself.

First, I created the directories ssl.crt and ssl.key in the default locations expected by /usr/local/apache2/conf/ssl.conf (which /usr/local/apache2/conf/httpd.conf includes):

[root@localhost conf]# cd /usr/local/apache2/conf
[root@localhost conf]# mkdir ssl.crt
[root@localhost conf]# mkdir ssl.key

My project required only a self-signed certificate, not one from a bona fide certificate-generating authority. Interestingly, there's a lot of confusing information on the internet on just how to generate a self-signed certificate for Apache, and, sadly, most of that information is way more complicated than it has to be. It turns out I was able to generate the key and the cert all in a single openssl command (note that after some output, you will be prompted for a pass phrase twice):

[root@localhost conf]# openssl req -new -x509 -days 365 -keyout
    ./ssl.key/server.key -out ./ssl.crt/server.crt -subj
    '/CN=Test-Only Certificate'

Because of the certificate, whenever Apache started it prompted me for the pass phrase. This was not useful, because I planned on using apachectl (or some form of it) in my server's startup and shutdown scripts: I didn't want my server to stop and prompt me for a password every time I rebooted it--after all, what if I had to reboot it remotely? I wouldn't be at the console to enter the pass phrase. Happily, there's a way to get around this problem (note that the openssl command prompted me for the pass phrase I selected above):

[root@localhost conf]# cp ssl.key/server.key ssl.key/server.key.org
[root@localhost conf]# openssl rsa -in ssl.key/server.key.org 
   -out ssl.key/server.key
[root@localhost conf]# chmod 400 ssl.key/server.key

The new ssl.key/server.key above is insecure; I protected it by setting the permissions on the file as restrictively as possible.

Compiling Subversion

I downloaded the latest Subversion source code. At the time of this writing, that was 1.1.3.

[root@localhost conf]# cp /downloads/subversion-1.1.3.tar.gz /usr/local/src
[root@localhost conf]# cd /usr/local/src/
[root@localhost src]# tar -xzvf subversion-1.1.3.tar.gz
[root@localhost src]# cd subversion-1.1.3

(Again, note that if you are not using GNU tar, you will first have to gunzip the source code and then untar it.)

As you can see with my runconfigure.sh wrapper script for Subversion, I was very specific about which features I did and did not want in my Subversion installation. Because I knew exactly what I wanted, I could afford to do this, and it spared me from prerequisite hell.

#!/bin/sh
# runconfigure.sh -- wrapper script for ./configure
./configure \
--with-apr=/usr/local/apache2 \
--with-apr-util=/usr/local/apache2 \
--without-berkeley-db \
--without-zlib \
--without-jdk \
--without-jikes \
--without-swig \
--without-junit

Once I saved runconfigure.sh, I ran it and installed Subversion:

[root@localhost src]# ./runconfigure.sh
[root@localhost src]# make
[root@localhost src]# make install

For the convenience of whoever has shell access to the machine, I wanted to ensure that Subversion was in the path (and, in fact, first in the path, because I wanted it to run by default over any older version that may have shipped with my operating system).

First, I needed to ensure that the binary could find Subversion's shared libraries. On Linux, I did this by ensuring that /etc/ld.so.conf contained the directory /usr/local/lib. (The /etc/ld.so.conf layout is just a list of directories, one per line--a refreshingly simple syntax, in this age of XML.) If your copy of /etc/ld.so.conf is missing /usr/local/lib, add it, and then run ldconfig with no arguments. On Solaris, ensure that $LD_LIBRARY_PATH contains /usr/local/lib. If that directory is not present, add it in /etc/profile and export LD_LIBRARY_PATH so that all users automatically pick it up when they log in. Also, if you are on Solaris, change your current environment so that you have access to Subversion's libraries right away:

LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH

On any Linux/Unix system, be sure to add Subversion to your path. On my machine, I placed it first in the path, so that it would override any older Subversion that might be in /usr/bin:

PATH=/usr/local/bin:$PATH
export PATH

On any Linux/Unix system, you may want to ensure that anyone who logs on to the system has Subversion in the path. Here's a nifty way I did that without having to open a text editor:

[root@localhost src]# cat >> /etc/profile <<EOR
> PATH=/usr/local/bin:\$PATH
> export PATH
> EOR

Creating the Repository

Next, I needed to create a Subversion repository. In real life, I actually used cvs2svn to port an entire CVS repository to Subversion; as long as you already have Python installed on your system, cvs2svn is fantastic. You may not have an existing repository to migrate, though, so here's how to create a sample repository and check in a one-file project:

[root@localhost src]# mkdir /usr/local/svn_repository
[root@localhost src]# svnadmin create /usr/local/svn_repository
[root@localhost src]# cd /tmp
[root@localhost tmp]# mkdir -p test_project/trunk
[root@localhost trunk]# cd test_project/trunk
[root@localhost trunk]# cat > test_file.txt <<EOS
> this
> is
> a
> test
> file
> EOS
[root@localhost trunk]# svn import -m "initial checkin" .
    file:///usr/local/svn_repository/test_project/trunk

Integrating Subversion with Apache

I wanted to use basic authentication to restrict access to my repository, so I created a .htpasswd file. What better location for this than the repository's configuration directory? Please note that for illustrative purposes, I used htpasswd's ability to accept passwords on the command line; you may want to make htpasswd prompt you for the passwords instead. I also used ridiculously insecure example passwords for illustrative purposes. Note how the first invocation of htpasswd used the -c switch to create the password file, whereas subsequent ones did not. Finally, if you don't mind having the passwords briefly shown in clear text, and you have to create a lot of users, consider making this a shell script that you can just run once and then delete when you are done:

[root@localhost trunk]# /usr/local/apache2/bin/htpasswd -c -m
    -b /www/svn_repository/conf/htpasswd user1 password1
[root@localhost trunk]# /usr/local/apache2/bin/htpasswd -m
    -b /www/svn_repository/conf/htpasswd user2 password2
[root@localhost trunk]# /usr/local/apache2/bin/htpasswd -m
    -b /www/svn_repository/conf/htpasswd user3 password3

One step I didn't want to forget was making the repository readable and writable by the user nobody, which Apache runs by default. (Forgetting this step would produce sorts of errors from Apache.)

[root@localhost trunk]# chown -R nobody /usr/local/svn_repository

Finally, I changed httpd.conf to let Apache know about my repository and the fact that only authorized users should read and write to it. I plan to use my Apache installation only for Subversion, so I made the Subversion repository the root of Apache's filesystem. In /usr/local/apache2/conf/httpd.conf, I made the line containing DocumentRoot look like:

DocumentRoot /usr/local/svn_repository

Next, I added these lines to the end of /usr/local/apache2/conf/httpd.conf to let Apache know that /usr/local/svn_repository is a Subversion repository and that access uses SSL and basic authentication:

<Location />
    DAV svn
    SVNPath /usr/local/svn_repository

    AuthType Basic
    AuthName "Subversion Repository"
    AuthUserFile /usr/local/svn_repository/conf/htpasswd
    Require valid-user
    SSLRequireSSL
</Location>

After those lines in /usr/local/apache2/conf/httpd.conf, I added the following mod_rewrite lines as a nice way to forward browsers from http to https:

# redirect all port 80 requests to 443
RewriteEngine on
RewriteCond "%{SERVER_PORT}"    "^80$"
RewriteRule "^(.*)$"            "https://%{SERVER_NAME}$1" [R,L]

Ensuring Apache Starts When the Server Does

Chances are your Linux/Unix installation shipped with Apache already installed. My Fedora Core system did, so I disabled it to keep it from interfering with my custom Apache, which wanted to use the same ports.

The method for disabling the automatic startup of Apache is different from system to system. Fedora Core uses a spiffy system utility called chkconfig to manage the starting and stopping of system services at various runlevels. I asked chkconfig what runlevels Apache was running at:

[root@localhost etc]# chkconfig --list httpd
httpd           0:off   1:off   2:off   3:on    4:on    5:on    6:off

I told chkconfig not to run Apache at the three runlevels where it was on, and then I shut down Apache manually (as it was currently running).

[root@localhost etc]# chkconfig --level 345 httpd off
[root@localhost etc]# chkconfig --list httpd
httpd           0:off   1:off   2:off   3:off   4:off   5:off   6:off
[root@localhost etc]# /etc/rc.d/init.d/httpd stop
Stopping httpd:                                            [  OK  ]

At that point, I needed to put a different script in /etc/rc.d/init.d to start and stop my custom Apache. Fortunately, /usr/local/apache2/bin/apachectl is a shell script that already mostly follows the conventions of a Unix init script: it already takes start and stop as arguments. However, it uses the nonstandard startssl and stopssl to make Apache use SSL, so I had to do some editing.

First, I copied Apache's control script into Fedora Core's standard location for init scripts:

[root@localhost etc]# cp /usr/local/apache2/bin/apachectl
    /etc/rc.d/init.d/apache_svn

Next I borrowed, in altered form, the first few lines of Fedora Core's /etc/rc.d/init.d/httpd script and put them at the top of /etc/rc.d/init.d/apache_svn, so that Fedora Core's spiffy chkconfig program could use apache_svn. (Note that chkconfig's parameters in init scripts seem to be only comments, but they are more than that.) Here are the lines I added just below #!/bin/sh in /etc/rc.d/init.d/apache_svn:

#
# apache_svn        Startup script for the Apache HTTP Server
#
# chkconfig: - 85 15
# description: This Apache installation is really a Subversion repository.
# processname: httpd
# config: /usr/local/apache2/conf/httpd.conf

Then, I removed these two stanzas:

stopssl|sslstop|stop-SSL)
    $HTTPD -k stop -DSSL
    ERROR=$?
    ;;
startssl|sslstart|start-SSL)
    $HTTPD -k start -DSSL
    ERROR=$?
    ;;

Next, I changed this stanza:

start|stop|restart|graceful)
    $HTTPD -k $ARGV
    ERROR=$?
    ;;

to this:

start|stop|restart|graceful)
    echo -n $"Apache + SVN $ARGV, status: "
    $HTTPD -k $ARGV -DSSL
    ERROR=$?
    [ "$ERROR" -eq 0 ] && echo "OK" || echo "FAILED"
    ;;

to define the property SSL for all four targets captured by that stanza, as well as to give some feedback on the command line (though not as pretty as Fedora Core's scripts).

Next I told chkconfig to add apache_svn to its setup, and to make it active at runlevels 3, 4, and 5:

[root@localhost init.d]# chkconfig --add apache_svn
[root@localhost init.d]# chkconfig --list apache_svn
apache_svn      0:off   1:off   2:off   3:off   4:off   5:off   6:off
[root@localhost init.d]# chkconfig --level 345 apache_svn on
[root@localhost init.d]# chkconfig --list apache_svn
apache_svn      0:off   1:off   2:off   3:on    4:on    5:on    6:off

Then I started Apache using its new init script.

[root@localhost init.d]# /etc/rc.d/init.d/apache_svn start
Apache + SVN start, status: OK

Testing the Connection

The moment of truth came when I got to test repository access from my web browser. I went to https://localhost/, accepted the cert, entered one of the sample usernames and passwords, and browsed my Subversion repository.

The command-line client worked too:

[root@localhost init.d]# cd /tmp
[root@localhost tmp]# svn co https://localhost/test_project/trunk test_project
Error validating server certificate for 'https://localhost:443':
 - The certificate is not issued by a trusted authority. Use the
   fingerprint to validate the certificate manually!
 - The certificate hostname does not match.
Certificate information:
 - Hostname: Test-Only Certificate
 - Valid: from Mar 12 00:48:54 2005 GMT until Mar 12 00:48:54 2006 GMT
 - Issuer: Test-Only Certificate
 - Fingerprint: 56:f8:be:cc:df:69:c4:64:43:ba:d4:0b:5d:65:a2:0e:9f:9f:5d:ee
(R)eject, accept (t)emporarily or accept (p)ermanently? t
Authentication realm: <https://localhost:443> Subversion Repository
Password for 'root':  # there is no user root, so just hit enter here
Authentication realm: <https://svn1.digitas.com:443> Subversion Repository
Username: user1
Password for 'user1':
A  test_project/test_file.txt
Checked out revision 1.

Get What You Want

Custom-compiling Apache and Subversion can certainly be involved, but it's the best way to configure your repository just the way you want it. Hopefully this article will make it easier for you to get what you want out of Apache and Subversion.

Manni Wood leads teams at a Boston advertising company in building Java-based, database-backed web sites for clients like General Motors and FedEx.