Cryptosystems: Debugging IPSec

In the last article, we configured the files necessary to negotiate an IPSec VPN using racoon. I have a lot of troubleshooting scenarios to cover this week, so let's carry on where we left off.

The one configuration I purposefully left out last time is the necessary changes to the firewall ruleset. You might remember that we need to allow UDP port 500 for IKE and protocol number 50 for ESP. The syntax for those rules will depend upon whether your system is protected by ipfw, ipfilter, or is behind another type of firewall. It will also depend upon the order of the existing rules in your ruleset and the degree of paranoia dictated by your security policy.

For now, create two rules near the top of your ruleset. We may have to tweak their placement later on when we start working with the negotiated tunnel. Here is an example of some rules to an ipfw ruleset:

#rules to allow IPSec VPN
add 00201 allow log esp from any to any
add 00203 allow log udp from any 500 to any

Until you are happy with your tunnel, you should use the log word. If your security policy allows it, consider starting with the any keyword until your tunnel is successful. Afterwards, you can tighten up those rules by specifying particular sources, destinations, directions, and interfaces.

Here is an example of some rules to an ipfilter ruleset:

#rules to allow IPSec VPN
pass in log quick proto esp from any to any
pass in log quick proto udp from any port = 500 to any port = 500

Post-Configuration Tests

Once I've added my firewall rules, I'll reboot into the new IPSec enabled kernel. I'll also watch the startup messages as they go by. In particular, I'm looking for error messages. If you see something interesting in your startup messages, press the scroll lock key and use your page up key to go back to the error. Make sure all of your firewall rules load successfully. If you have a typo, your startup message will indicate which rule prevented the rulebase from loading.

You should also see this message in your output:

ipsec: enabled

If you don't, doublecheck that /etc/rc.conf contains this line:

ipsec_enable="YES"

You should also see your route being added when racoon initializes:

Starting local daemons:add net 192.168.1.0: gateway
10.0.0.1

If you get this error:

racoon: failed to parse configuration file.

you have a typo in /usr/local/etc/racoon/racoon.conf. Check carefully for missing semicolons or braces and typos in the keywords you added. Once you've made your change, do a shutdown now and re-check for error messages. You want to be able to boot without error messages.

Once you have resolved any error messages, login and verify that racoon has indeed started:

sockstat -4
USER	COMMAND	  PID	FD   PROTO	LOCAL ADDRESS	FOREIGN ADDRESS
root	racoon	 5898	 6   udp4	A.A.A.A:500	*:*

and that the "gif" was successfully created:

$ ifconfig gif0
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280
	tunnel inet A.A.A.A --> B.B.B.B

You can also verify your route:

$ netstat -rn
Routing tables

Internet:
Destination	Gateway		Flags	Refs	Use	Netif	Expire
<snip>
192.168.1	10.0.0.1	UGSc	   0	  0	  ed0

Finally, verify that the SPD contains the correct policy:

$ setkey -DP      

192.168.1.0/24[any] 10.0.0.0/8[any] any
	in ipsec
	esp/tunnel/B.B.B.B-A.A.A.A/require
	spid=2 seq=1 pid=183
	refcnt=1
10.0.0.0/8[any] 192.168.1.0/24[any] any
	out ipsec
	esp/tunnel/A.A.A.A-B.B.B.B/require
	spid=1 seq=0 pid=183
	refcnt=1

If you don't have a policy, there is a problem with /etc/ipsec.conf. You either have a typo or you forgot to tell /etc/rc.conf to load that file at bootup. Once racoon is up at both peers, and both peers have a matching policy, you are ready to try tunnel negotiation.

Testing the Tunnel

Whenever I negotiate a tunnel for the first time, I run several tests and either take full advantage of my virtual terminals, or I use /usr/ports/misc/screen if I'm restricted to one terminal. First, I clear a terminal so I can watch the racoon log:

$ clear
$ tail -f /var/log/racoon.log

If you don't get anything back when you run that tail command, racoon is not logging. Even though you haven't used racoon yet, a log should have been created for you and entries made indicating that racoon started. You do want a log when you're testing a tunnel, so kill racoon, restart it with the logging switch and try again:

$ killall racoon
$ racoon -f /usr/local/etc/racoon/racoon.conf -l /var/log/racoon.log

I then open a second terminal and start a tcpdump session that will just display the packets used in the IKE negotiations:

$ tcpdump port 500

Finally, I open a third terminal where I'll try to negotiate the tunnel. If your firewall rules allow you to ping, pinging the inside interface of the peer is a very good tunnel test.

If all goes well, your ping will hang for a moment while the tunnel is negotiated, then you'll see your ping responses displayed to your terminal. On your other terminals, you should see something like this:

$ tail -f /var/log/racoon.log

2002-12-29 13:03:09: INFO: main.c:172:main(): @(#)package version 
freebsd-20021120a
2002-12-29 13:03:09: INFO: main.c:174:main(): @(#)internal version 20001216 
sakane@kame.net
2002-12-29 13:03:09: INFO: main.c:175:main(): @(#)This product linked 
OpenSSL 0.9.6g 9 Aug 2002 (http://www.openssl.org/)
2002-12-29 13:03:09: INFO: isakmp.c:1358:isakmp_open(): 24.226.40.106[500] 
used as isakmp port (fd=5)
2002-12-29 13:04:31: INFO: isakmp.c:1684:isakmp_post_acquire(): IPsec-SA 
request for 24.226.16.88 queued due to no phase1 found.
2002-12-29 13:04:31: INFO: isakmp.c:798:isakmp_ph1begin_i(): initiate new 
phase 1 negotiation: A.A.A.A[500]<=>B.B.B.B[500]
2002-12-29 13:04:31: INFO: isakmp.c:803:isakmp_ph1begin_i(): begin 
Aggressive mode.
2002-12-29 13:04:31: INFO: vendorid.c:128:check_vendorid(): received Vendor 
ID: KAME/racoon
2002-12-29 13:07:21: INFO: isakmp.c:2412:log_ph1established(): ISAKMP-SA 
established A.A.A.A[500]-B.B.B.B[500] spi:2b7869ab380e7cc1:a3e29f4ac161cb25
2002-12-29 13:07:21: INFO: isakmp.c:942:isakmp_ph2begin_i(): initiate new 
phase 2 negotiation: A.A.A.A[0]<=>B.B.B.B[0]
2002-12-29 13:07:21: INFO: pfkey.c:1110:pk_recvupdate(): IPsec-SA 
established: ESP/Tunnel B.B.B.B->A.A.A.A spi=82702499(0x4edf0a3)
2002-12-29 13:07:21: INFO: pfkey.c:1322:pk_recvadd(): IPsec-SA established: 
ESP/Tunnel A.A.A.A->B.B.B.B spi=63238165(0x3c4f015)

Note that first the Phase 1, or ISAKMP, SA is established with a unique spi. Then two Phase 2, or IPsec, SAs are established, one in each direction, and each with a unique spi.

$ tcpdump port 500
tcpdump: listening on ed0 
13:04:31.067156 A.A.A.A.isakmp > B.B.B.B.isakmp: isakmp: phase 1 I agg:
[|sa]
13:04:31.067682 B.B.B.B.isakmp > A.A.A.A.isakmp: isakmp: phase 1 R agg: 
[|sa]
13:04:31.680474 A.A.A.A.isakmp > B.B.B.B.isakmp: isakmp: phase 1 I agg: 
(hash: len=20)
13:04:31.681046 A.A.A.A.isakmp > B.B.B.B.isakmp: isakmp: phase 2/others I 
inf[E]: [encrypted hash]
13:04:31.697564 A.A.A.A.isakmp > B.B.B.B.isakmp: isakmp: phase 2/others I 
oakley-quick[E]: [encrypted hash]
13:04:31.703306 B.B.B.B.isakmp > A.A.A.A.isakmp: isakmp: phase 2/others R 
inf[E]: [encrypted hash]
13:04:31.770199 B.B.B.B.isakmp > A.A.A.A.isakmp: isakmp: phase 2/others R 
oakley-quick[E]: [encrypted hash]

You may remember from VPNs and IPSec Demystified that Phase One aggressive mode uses 3 packets, which you can see from this tcpdump. This is followed by Phase Two. Note that all Phase Two packets are encrypted.

Finally, you know your tunnel is fully established when your SAD contains the SAs. You can confirm that with:

$ setkey -D

A.A.A.A B.B.B.B
	esp mode=tunnel spi=63238165(0x03c4f015) reqid=0(0x00000000)
	E: blowfish-cbc  a24ac0e7 36f7e153 26f81300 43d0d333
	A: hmac-sha1  6bb84116 e90d2b1b 2ac95285 0dd394fb afa0c3d8
	seq=0x00000004 replay=4 flags=0x00000000 state=mature 
	created: Dec 29 13:07:21 2002	current: Dec 29 13:15:32 2002
	diff: 491(s)	hard: 86400(s)	soft: 69120(s)
	last: Dec 29 13:07:35 2002	hard: 0(s)	soft: 0(s)
	current: 544(bytes)	hard: 0(bytes)	soft: 0(bytes)
	allocated: 4	hard: 0	soft: 0
	sadb_seq=1 pid=50830 refcnt=2
B.B.B.B A.A.A.A
	esp mode=tunnel spi=82702499(0x04edf0a3) reqid=0(0x00000000)
	E: blowfish-cbc  ece45b91 af659b1f 1031b8eb e6268c60
	A: hmac-sha1  eb46c7b6 12051da0 567ca3a6 1c889e72 3faa5553
	seq=0x00000004 replay=4 flags=0x00000000 state=mature 
	created: Dec 29 13:07:21 2002	current: Dec 29 13:15:32 2002
	diff: 491(s)	hard: 86400(s)	soft: 69120(s)
	last: Dec 29 13:07:35 2002	hard: 0(s)	soft: 0(s)
	current: 336(bytes)	hard: 0(bytes)	soft: 0(bytes)
	allocated: 4	hard: 0	soft: 0
	sadb_seq=0 pid=50830 refcnt=1

If your tunnel successfully negotiates, give yourself a pat on the back, and do the happy dance. If it doesn't, take heart and follow along as I demonstrate some common errors.

Troubleshooting

This is a common racoon log error message:

2002-12-29 12:48:31: NOTIFY: oakley.c:2040:oakley_skeyid(): 
couldn't find the proper pskey, try to get one by the peer's address.
2002-12-29 12:48:31: ERROR: oakley.c:2054:oakley_skeyid(): couldn't find 
the pskey for B.B.B.B.

Fortunately, this one is fairly straightforward as it indicates a problem with /usr/local/etc/racoon/psk.txt. On both peers, check that:

• The file contains the external IP address of the other peer
• The password is the same
• The permissions are set correctly

You don't have to restart racoon once you've fixed this error. As soon as the typo/permission problem is fixed, Phase One will continue.

If you don't receive any tcpdump output at all, there aren't any IKE packets being sent. This usually means that you forgot to allow UDP port 500 on your firewall, or forgot to tell your firewall about the changes to your ruleset, so your firewall is discarding those packets. However, it could also indicate a routing or policy misconfiguration. Remember, your policy tells racoon when it is required to negotiate a tunnel.

For example, part of my policy in /etc/ipsec.conf states:

spdadd 10.0.0.0/8 192.168.1.0/24 any -P out ipsec

This means that if I try to send packets (from a ping, for example) to a host with an IP address starting with 192.168.1, those packets won't be sent until IKE has negotiated a tunnel. It also means that if I send packets to any other network, IKE does not come into play. If tcpdump shows that IKE isn't being used, double-check that your policy does require ipsec for the remote network.

Sometimes a tcpdump will show your packets going out, but no replies coming back. This usually indicates that the firewall at the peer is discarding the packets. Check the firewall rules at the peer to ensure they allow IKE and ESP.

Sometimes Phase One will fail, and you can't quite tell why from the racoon log. When this happens, I bring out the big guns and use the /usr/ports/net/ethereal utility. (See Using Ethereal) This allows me to analyze the policy negotiations in great detail, as it records each policy parameter as sent by each peer. If Phase One is failing, it will be because of a policy mismatch or a peer authentication failure. If it is a policy mismatch, you should find it in the "remote" section of racoon.conf. However, if the remote sections are identical, you'll really need that packet sniff to pinpoint exactly what is happening.

Advanced Troubleshooting

Things get a lot more interesting if Phase One succeeds but Phase Two fails. Once Phase One is finished, all of the data is encrypted. A packet sniff won't help you in pinpointing which policy parameter is failing to match up. If both peers are running racoon, the culprit will most likely be a typo or mismatch in the "sainfo" section of racoon.conf. However, if the other peer is not running racoon, you may have some detective work ahead of you.

Many commercial IPSec implementations include additional features. It's unfortunate that some of those features are only supported by that particular product and are usually enabled by default. If Phase One works, Phase Two hangs, and you know you've configured both peers with identical parameters, you've most likely stumbled upon such a feature. Scour that product's documentation and vendor's knowledge base looking for default configurations and how to disable them.

Failing that, you may be able to find working configs between racoon and the particular product you are struggling with. I've found the Interoperability Profiles at VPNC to be useful.

The maintainers of racoon also maintain a searchable list of known and solved issues. They also provide prompt answers to queries.

Finally, if your blood, sweat, and tears results in a working tunnel, be kind and post your how-to somewhere on the Internet. You just might make a harried admin's day.

Interactions with Anti-Spoofing Rules

The last scenario I want to discuss in this article deals with VPNs and anti-spoofing rules. Depending upon the firewall(s) you are using to protect the gateways, this may or may not become an issue. This scenario can be the most frustrating as IKE is able to successfully establish the VPN. Unfortunately, all of the packets that enter the resulting tunnel end up being discarded by the firewall's anti-spoof rules.

Related Reading Virtual Private Networks By Charlie Scott, Paul Wolfe, Mike Erwin Table of Contents Index Read Online--Safari Search this book on Safari:   Only This Book All of Safari Code Fragments only

If you're unfamiliar with anti-spoof rules, they are the set of rules that drop all traffic entering a network from a private address range. Normally, this is useful, as most legitimate traffic coming in from the Internet can't have a source address in the 10.x.x.x, 172.16.x.x-172.31.x.x, or 192.168.x.x ranges as Internet routers drop those addresses. A VPN tunnel operates differently. While in the tunnel, the packets have the Internet routable addresses of the peers' external interfaces. However, the data in the packet itself is destined for an internal address, which is usually within the private address range.

Whether or not this is a problem depends upon the firewall and the order in which packet processing occurs. Several factors are at play here: a packet needs to be encrypted or decrypted, it must be NATed, and it also needs to be compared to a firewall ruleset to see if the packet is allowed to pass. If the NATing occurs in the wrong place, it is quite possible that a legitimate packet will be mistaken for a malicious, spoofed packet.

How do you resolve the situation if your particular firewall's processing order is working against you? Sometimes trial and error in rule order will work. You've already created rules to allow the VPN to be established; those were the IKE and ESP rules. You'll still need rules allowing your users to access the services you wish to allow through the tunnel, otherwise you wouldn't have created a tunnel in the first place. Try placing those rules towards the top of a rulebase. Include the "quick" word in the case of an ipfilter rulebase.

If that doesn't help, it may be time to reconsider the usefulness of the anti-spoof rules in regards to your particular security policy. Balance the need for the VPN against the benefits gained from the anti-spoof rules. If you log those rules for a week or two, you'll have an idea of how much those rules are used and how much protection they offer.

If your security policy requires the anti-spoof rules, there is some information that can be gleaned from the Internet, which might apply to your situation. As firewalls mature, they gain extra features. There may indeed be an extra switch or keyword that you can place in your rulebase that will resolve the anti-spoof problem. That extra knob may be in the works and available in another six months.

I'd like to leave you with some resources, which I have found to be helpful:

• Setting up a FreeBSD firewall with an IPSec uplink from BSD Today
• IPSec FreeBSD <-> Cisco PIX succesfull story.
• IP Filter FAQ

I hope you have enjoyed the cryptosystems series. I had a chance over the holidays to go through my "things I want to try out if I ever have more time" list. I'll share some of the results in the next article.

Dru Lavigne is a network and systems administrator, IT instructor, author and international speaker. She has over a decade of experience administering and teaching Netware, Microsoft, Cisco, Checkpoint, SCO, Solaris, Linux, and BSD systems. A prolific author, she pens the popular FreeBSD Basics column for O'Reilly and is author of BSD Hacks and The Best of FreeBSD Basics.

Read more FreeBSD Basics columns.

Return to the BSD DevCenter.

Copyright © 2007 O'Reilly Media, Inc.