BSD DevCenter
oreilly.comSafari Books Online.Conferences.


IPSec Certificate Basics
Pages: 1, 2

Creating and installing certificates

Now that you're set up to sign certificates, you can begin creating them for all the hosts using IPSec on your network. There are a few simple steps, repeated for each of the remote boxes and the tunnel server.

The first step is to create a public key and certificate request. On the CA server, run:

openssl req -new -nodes -newkey rsa:1024 -sha1 -keyform PEM -keyout privkey.pem -outform PEM -out newreq.pem

Next, the certificate must be signed. It's easiest to do this with the -sign command. Give the CA's password, then respond with y to the prompts, indicating that you want to sign the certificate.

You'll now have three files to copy to the appropriate machine: demoCA/cacert.pem, newcert.pem, and privkey.pem. Put them in /usr/local/etc/racoon/cert on the client you are configuring. Make sure the directory is only readable by root; otherwise, malicious parties could steal your private key and masquerade as you!

The final step is to allow the host to recognize other certificates signed by your CA. Login to the machine and run:

cd /usr/local/etc/racoon/cert
ln -s cacert.pem `openssl x509 -noout -hash -in cacert.pem`.0

Make sure to replace cacert.pem with whatever you named the file. The symbolic link allows OpenSSL to find your CA's certificate based on its hash, thus verifying that the certificate is authentic and acceptable.

Repeat the above steps for all hosts that you want to have connect to your tunnel server.

Racoon configuration

Your last remaining task is to configure racoon to accept the certificates. Edit your racoon configuration file, typically /usr/local/etc/racoon/racoon.conf.

First, make sure racoon knows where to find your certificates:

path certificate "/usr/local/etc/racoon/cert" ;

Next, you'll need to change the remote block(s) to use and accept certificates. The necessary changes are bolded below for IPSec over IPv4 on anonymous connections. The changes are the same for IPv6 and identified connections.

remote anonymous
        exchange_mode main,aggressive;
        doi ipsec_doi;
        situation identity_only;

        my_identifier asn1dn;
        peers_identifier asn1dn;
        verify_identifier on;
        certificate_type x509 "newcert.pem" "privkey.pem";

        nonce_size 16;
        lifetime time 1 min;    # sec,min,hour
        support_mip6 on;
        proposal_check strict;  # obey, strict or claim

        proposal {
                encryption_algorithm 3des;
                hash_algorithm sha1;
                authentication_method rsasig ;
                dh_group 2 ;

my_identifier asn1dn tells racoon that the local host should identify itself by using its Distinguished Name (DN) as defined by its certificate (that is, all of the fields in its certificate strung together). peers_identifier asn1dn states that the remote host should identify itself with its certificate's DN. verify_identifier on ensures that the identification the peer sends is the same type that is specified in peers_identifier; in this case, it's asn1dn. The certificate_type line tells racoon to use an X.509 certificate. cert.pem is the name of the local host's certificate file, while key.pem is its private key. These file names should be changed to whatever you named them.

The last line to change, authentication_method rsasig, tells racoon that it should use certificate-based identification of its peers.

Once you've made the necessary changes on each host's racoon.conf, you're ready to go. The next section discusses how to proceed and gives some troubleshooting tips.

Running and troubleshooting

Kill the racoon daemon on your server and client. Restart it, and tail your logfile. Ping one host from the other. You should see lines similar to:

2002-01-27 02:12:46: INFO: isakmp.c:1681:isakmp_post_acquire(): IPsec-SA request for queued due to no phase1 found.
2002-01-27 02:12:46: INFO: isakmp.c:795:isakmp_ph1begin_i(): initiate new phase 1 negotiation:[500]<=>[500]
2002-01-27 02:12:46: INFO: isakmp.c:800:isakmp_ph1begin_i(): begin Identity Protection mode.
2002-01-27 02:12:46: INFO: pfkey.c:1365:pk_recvexpire(): IPsec-SA expired: ESP/Tunnel> spi=122493290(0x74d196a)
2002-01-27 02:12:46: INFO: vendorid.c:128:check_vendorid(): received Vendor ID: KAME/racoon
2002-01-27 02:12:46: INFO: vendorid.c:128:check_vendorid(): received Vendor ID: KAME/racoon
2002-01-27 02:12:46: INFO: isakmp.c:2409:log_ph1established(): ISAKMP-SA established[500]-[500] spi:0dab7579614a1047:50c0edf86b1a
2002-01-27 02:12:47: INFO: isakmp.c:939:isakmp_ph2begin_i(): initiate new phase 2 negotiation:[0]<=>[0]
2002-01-27 02:12:47: INFO: pfkey.c:1107:pk_recvupdate(): IPsec-SA established: ESP/Tunnel> spi=23298201(0x1638099)
2002-01-27 02:12:47: INFO: pfkey.c:1319:pk_recvadd(): IPsec-SA established: ESP/Tunnel> spi=52337518(0x31e9b6e)

If you don't, there are a few things to check. First, make sure your path and file names are specified correctly in racoon.conf. Next, check that you created a symbolic link, mentioned above in "Creating and installing certificates." Double-check your configuration, making sure your identifier and your peer's are set correctly, along with any other lines you changed (certificate related or not.) Finally, check your security policies and make sure they still match.


In this article, you learned all the basics necessary to use X.509 certificates for authentication in IPSec on a FreeBSD box. You're now ready to enjoy the benefits, and you don't have to worry about creating, changing, and remembering all of your shared keys. Further, since the IETF may discontinue preshared keys in the next version of IPSec, it's one less headache if you upgrade (once IPSec version 2 comes out, of course).

Mike DeGraw-Bertsch is a security and Unix system administration consultant in the Boston, Mass. area. When he's not at a job, writing, hacking with Perl, or playing with his wireless network, he can usually be found playing goal in ice hockey.

Return to the BSD DevCenter.

Sponsored by: