In the last article, we configured the files necessary to negotiate an IPSec VPN using
racoon. I have a lot of troubleshooting scenarios
to cover this week, so let's carry on where we left off.
The one configuration I purposefully left out last time is the necessary
changes to the firewall ruleset. You might remember that we need to allow UDP
port 500 for IKE and protocol number 50 for ESP. The syntax for those rules
will depend upon whether your system is protected by
ipfilter, or is behind another type of firewall. It will also
depend upon the order of the existing rules in your ruleset and the degree of
paranoia dictated by your security policy.
For now, create two rules near the top of your ruleset. We may have to
tweak their placement later on when we start working with the negotiated
tunnel. Here is an example of some rules to an
#rules to allow IPSec VPN add 00201 allow log esp from any to any add 00203 allow log udp from any 500 to any
Until you are happy with your tunnel, you should use the
word. If your security policy allows it, consider starting with the
any keyword until your tunnel is successful. Afterwards, you can
tighten up those rules by specifying particular sources, destinations,
directions, and interfaces.
Here is an example of some rules to an
#rules to allow IPSec VPN pass in log quick proto esp from any to any pass in log quick proto udp from any port = 500 to any port = 500
Previously in FreeBSD Basics:
Once I've added my firewall rules, I'll reboot into the new IPSec enabled kernel. I'll also watch the startup messages as they go by. In particular, I'm looking for error messages. If you see something interesting in your startup messages, press the scroll lock key and use your page up key to go back to the error. Make sure all of your firewall rules load successfully. If you have a typo, your startup message will indicate which rule prevented the rulebase from loading.
You should also see this message in your output:
If you don't, doublecheck that
/etc/rc.conf contains this
You should also see your route being added when
Starting local daemons:add net 192.168.1.0: gateway 10.0.0.1
If you get this error:
racoon: failed to parse configuration file.
you have a typo in
carefully for missing semicolons or braces and typos in the keywords you added.
Once you've made your change, do a
shutdown now and re-check for
error messages. You want to be able to boot without error messages.
Once you have resolved any error messages, login and verify that
racoon has indeed started:
sockstat -4 USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS root racoon 5898 6 udp4 A.A.A.A:500 *:*
and that the "gif" was successfully created:
$ ifconfig gif0 gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280 tunnel inet A.A.A.A --> B.B.B.B
You can also verify your route:
$ netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire <snip> 192.168.1 10.0.0.1 UGSc 0 0 ed0
Finally, verify that the SPD contains the correct policy:
$ setkey -DP 192.168.1.0/24[any] 10.0.0.0/8[any] any in ipsec esp/tunnel/B.B.B.B-A.A.A.A/require spid=2 seq=1 pid=183 refcnt=1 10.0.0.0/8[any] 192.168.1.0/24[any] any out ipsec esp/tunnel/A.A.A.A-B.B.B.B/require spid=1 seq=0 pid=183 refcnt=1
If you don't have a policy, there is a problem with
/etc/ipsec.conf. You either have a typo or you forgot to tell
/etc/rc.conf to load that file at bootup. Once
is up at both peers, and both peers have a matching policy, you are ready to try
Whenever I negotiate a tunnel for the first time, I run several tests and
either take full advantage of my virtual terminals, or I use
/usr/ports/misc/screen if I'm restricted to one terminal. First, I
clear a terminal so I can watch the
$ clear $ tail -f /var/log/racoon.log
If you don't get anything back when you run that
not logging. Even though you haven't used
racoon yet, a log should
have been created for you and entries made indicating that
started. You do want a log when you're testing a tunnel, so kill
racoon, restart it with the logging switch and try again:
$ killall racoon $ racoon -f /usr/local/etc/racoon/racoon.conf -l /var/log/racoon.log
I then open a second terminal and start a
tcpdump session that
will just display the packets used in the IKE negotiations:
$ tcpdump port 500
Finally, I open a third terminal where I'll try to negotiate the tunnel. If your firewall rules allow you to ping, pinging the inside interface of the peer is a very good tunnel test.
If all goes well, your ping will hang for a moment while the tunnel is negotiated, then you'll see your ping responses displayed to your terminal. On your other terminals, you should see something like this:
$ tail -f /var/log/racoon.log 2002-12-29 13:03:09: INFO: main.c:172:main(): @(#)package version freebsd-20021120a 2002-12-29 13:03:09: INFO: main.c:174:main(): @(#)internal version 20001216 email@example.com 2002-12-29 13:03:09: INFO: main.c:175:main(): @(#)This product linked OpenSSL 0.9.6g 9 Aug 2002 (http://www.openssl.org/) 2002-12-29 13:03:09: INFO: isakmp.c:1358:isakmp_open(): 18.104.22.168 used as isakmp port (fd=5) 2002-12-29 13:04:31: INFO: isakmp.c:1684:isakmp_post_acquire(): IPsec-SA request for 22.214.171.124 queued due to no phase1 found. 2002-12-29 13:04:31: INFO: isakmp.c:798:isakmp_ph1begin_i(): initiate new phase 1 negotiation: A.A.A.A<=>B.B.B.B 2002-12-29 13:04:31: INFO: isakmp.c:803:isakmp_ph1begin_i(): begin Aggressive mode. 2002-12-29 13:04:31: INFO: vendorid.c:128:check_vendorid(): received Vendor ID: KAME/racoon 2002-12-29 13:07:21: INFO: isakmp.c:2412:log_ph1established(): ISAKMP-SA established A.A.A.A-B.B.B.B spi:2b7869ab380e7cc1:a3e29f4ac161cb25 2002-12-29 13:07:21: INFO: isakmp.c:942:isakmp_ph2begin_i(): initiate new phase 2 negotiation: A.A.A.A<=>B.B.B.B 2002-12-29 13:07:21: INFO: pfkey.c:1110:pk_recvupdate(): IPsec-SA established: ESP/Tunnel B.B.B.B->A.A.A.A spi=82702499(0x4edf0a3) 2002-12-29 13:07:21: INFO: pfkey.c:1322:pk_recvadd(): IPsec-SA established: ESP/Tunnel A.A.A.A->B.B.B.B spi=63238165(0x3c4f015)
Note that first the Phase 1, or
SA is established with a unique
spi. Then two Phase 2, or
SAs are established, one in each direction, and each with a unique
$ tcpdump port 500 tcpdump: listening on ed0 13:04:31.067156 A.A.A.A.isakmp > B.B.B.B.isakmp: isakmp: phase 1 I agg: [|sa] 13:04:31.067682 B.B.B.B.isakmp > A.A.A.A.isakmp: isakmp: phase 1 R agg: [|sa] 13:04:31.680474 A.A.A.A.isakmp > B.B.B.B.isakmp: isakmp: phase 1 I agg: (hash: len=20) 13:04:31.681046 A.A.A.A.isakmp > B.B.B.B.isakmp: isakmp: phase 2/others I inf[E]: [encrypted hash] 13:04:31.697564 A.A.A.A.isakmp > B.B.B.B.isakmp: isakmp: phase 2/others I oakley-quick[E]: [encrypted hash] 13:04:31.703306 B.B.B.B.isakmp > A.A.A.A.isakmp: isakmp: phase 2/others R inf[E]: [encrypted hash] 13:04:31.770199 B.B.B.B.isakmp > A.A.A.A.isakmp: isakmp: phase 2/others R oakley-quick[E]: [encrypted hash]
You may remember from VPNs
and IPSec Demystified that Phase One aggressive mode uses 3 packets, which
you can see from this
tcpdump. This is followed by Phase Two. Note that all
Phase Two packets are encrypted.
Finally, you know your tunnel is fully established when your
SAs. You can confirm that with:
$ setkey -D A.A.A.A B.B.B.B esp mode=tunnel spi=63238165(0x03c4f015) reqid=0(0x00000000) E: blowfish-cbc a24ac0e7 36f7e153 26f81300 43d0d333 A: hmac-sha1 6bb84116 e90d2b1b 2ac95285 0dd394fb afa0c3d8 seq=0x00000004 replay=4 flags=0x00000000 state=mature created: Dec 29 13:07:21 2002 current: Dec 29 13:15:32 2002 diff: 491(s) hard: 86400(s) soft: 69120(s) last: Dec 29 13:07:35 2002 hard: 0(s) soft: 0(s) current: 544(bytes) hard: 0(bytes) soft: 0(bytes) allocated: 4 hard: 0 soft: 0 sadb_seq=1 pid=50830 refcnt=2 B.B.B.B A.A.A.A esp mode=tunnel spi=82702499(0x04edf0a3) reqid=0(0x00000000) E: blowfish-cbc ece45b91 af659b1f 1031b8eb e6268c60 A: hmac-sha1 eb46c7b6 12051da0 567ca3a6 1c889e72 3faa5553 seq=0x00000004 replay=4 flags=0x00000000 state=mature created: Dec 29 13:07:21 2002 current: Dec 29 13:15:32 2002 diff: 491(s) hard: 86400(s) soft: 69120(s) last: Dec 29 13:07:35 2002 hard: 0(s) soft: 0(s) current: 336(bytes) hard: 0(bytes) soft: 0(bytes) allocated: 4 hard: 0 soft: 0 sadb_seq=0 pid=50830 refcnt=1
If your tunnel successfully negotiates, give yourself a pat on the back, and do the happy dance. If it doesn't, take heart and follow along as I demonstrate some common errors.
This is a common
racoon log error message:
2002-12-29 12:48:31: NOTIFY: oakley.c:2040:oakley_skeyid(): couldn't find the proper pskey, try to get one by the peer's address. 2002-12-29 12:48:31: ERROR: oakley.c:2054:oakley_skeyid(): couldn't find the pskey for B.B.B.B.
Fortunately, this one is fairly straightforward as it indicates a problem
/usr/local/etc/racoon/psk.txt. On both peers, check that:
You don't have to restart
racoon once you've fixed this
error. As soon as the typo/permission problem is fixed, Phase One will
If you don't receive any
tcpdump output at all, there aren't
any IKE packets being sent. This usually means that you forgot to allow
port 500 on your firewall, or forgot to tell your firewall about the changes to
your ruleset, so your firewall is discarding those packets. However, it could
also indicate a routing or policy misconfiguration. Remember, your policy tells
racoon when it is required to negotiate a tunnel.
For example, part of my policy in
spdadd 10.0.0.0/8 192.168.1.0/24 any -P out ipsec
This means that if I try to send packets (from a ping, for example) to a
host with an IP address starting with
192.168.1, those packets won't be sent
until IKE has negotiated a tunnel. It also means that if I send packets to any
other network, IKE does not come into play. If
tcpdump shows that
IKE isn't being used, double-check that your policy does require
ipsec for the
tcpdump will show your packets going out, but no replies coming
back. This usually indicates that the firewall at the peer is discarding the
packets. Check the firewall rules at the peer to ensure they allow IKE and ESP.
Sometimes Phase One will fail, and you can't quite tell why from the
racoon log. When this happens, I bring out the big guns and use
/usr/ports/net/ethereal utility. (See Using Ethereal) This allows me to analyze the policy negotiations in great
detail, as it records each policy parameter as sent by each peer. If Phase One
is failing, it will be because of a policy mismatch or a peer authentication
failure. If it is a policy mismatch, you should find it in the "remote"
racoon.conf. However, if the remote sections are
identical, you'll really need that packet sniff to pinpoint exactly what is
Things get a lot more interesting if Phase One succeeds but Phase Two
fails. Once Phase One is finished, all of the data is encrypted. A packet
sniff won't help you in pinpointing which policy parameter is failing to match
up. If both peers are running
racoon, the culprit will most likely
be a typo or mismatch in the "
sainfo" section of
However, if the other peer is not running
racoon, you may have
some detective work ahead of you.
Many commercial IPSec implementations include additional features. It's unfortunate that some of those features are only supported by that particular product and are usually enabled by default. If Phase One works, Phase Two hangs, and you know you've configured both peers with identical parameters, you've most likely stumbled upon such a feature. Scour that product's documentation and vendor's knowledge base looking for default configurations and how to disable them.
Failing that, you may be able to find working configs between
racoon and the particular product you are struggling with. I've
found the Interoperability
Profiles at VPNC to be useful.
The maintainers of
racoon also maintain a searchable list of
known and solved issues. They also provide prompt answers to queries.
Finally, if your blood, sweat, and tears results in a working tunnel, be kind and post your how-to somewhere on the Internet. You just might make a harried admin's day.
The last scenario I want to discuss in this article deals with VPNs and anti-spoofing rules. Depending upon the firewall(s) you are using to protect the gateways, this may or may not become an issue. This scenario can be the most frustrating as IKE is able to successfully establish the VPN. Unfortunately, all of the packets that enter the resulting tunnel end up being discarded by the firewall's anti-spoof rules.
If you're unfamiliar with anti-spoof rules, they are the set of rules that
drop all traffic entering a network from a private address range. Normally,
this is useful, as most legitimate traffic coming in from the Internet can't
have a source address in the
192.168.x.x ranges as Internet routers drop those addresses. A VPN tunnel operates
differently. While in the tunnel, the packets have the Internet routable
addresses of the peers' external interfaces. However, the data in the packet
itself is destined for an internal address, which is usually within the private
Whether or not this is a problem depends upon the firewall and the order in
which packet processing occurs. Several factors are at play here: a packet
needs to be encrypted or decrypted, it must be
NATed, and it also needs to be
compared to a firewall ruleset to see if the packet is allowed to pass. If the
NATing occurs in the wrong place, it is quite possible that a legitimate packet
will be mistaken for a malicious, spoofed packet.
How do you resolve the situation if your particular firewall's processing
order is working against you? Sometimes trial and error in rule order will
work. You've already created rules to allow the VPN to be established; those
were the IKE and ESP rules. You'll still need rules allowing your users to
access the services you wish to allow through the tunnel, otherwise you
wouldn't have created a tunnel in the first place. Try placing those rules
towards the top of a rulebase. Include the "
quick" word in the case of an
If that doesn't help, it may be time to reconsider the usefulness of the anti-spoof rules in regards to your particular security policy. Balance the need for the VPN against the benefits gained from the anti-spoof rules. If you log those rules for a week or two, you'll have an idea of how much those rules are used and how much protection they offer.
If your security policy requires the anti-spoof rules, there is some information that can be gleaned from the Internet, which might apply to your situation. As firewalls mature, they gain extra features. There may indeed be an extra switch or keyword that you can place in your rulebase that will resolve the anti-spoof problem. That extra knob may be in the works and available in another six months.
I'd like to leave you with some resources, which I have found to be helpful:
I hope you have enjoyed the cryptosystems series. I had a chance over the holidays to go through my "things I want to try out if I ever have more time" list. I'll share some of the results in the next article.
Dru Lavigne is a network and systems administrator, IT instructor, author and international speaker. She has over a decade of experience administering and teaching Netware, Microsoft, Cisco, Checkpoint, SCO, Solaris, Linux, and BSD systems. A prolific author, she pens the popular FreeBSD Basics column for O'Reilly and is author of BSD Hacks and The Best of FreeBSD Basics.
Read more FreeBSD Basics columns.
Return to the BSD DevCenter.
Copyright © 2009 O'Reilly Media, Inc.