Published on O'Reilly (http://oreilly.com/)
 See this if you're having trouble printing code examples

VPNs and Public Key Infrastructure

by Scott Brumbaugh

The virtual private network (VPN) is increasingly becoming an invaluable part of every business network. With broadband available in more and more places, small- and medium-size businesses are taking advantage of VPN technology and leveraging the investment they've made in their internal private networks, expanding services available to customers, partners, and staff. This article focuses on VPN tunneling. Because it is also necessary to understand the basic principles of data encryption, this article will also summarize the set of technologies that form a Public Key Infrastructure (PKI). We will see how to ensure privacy in a virtual private network.

A successfully deployed VPN will do two things for you. First, it will allow only known users to access your network. Second, it will encrypt all network traffic over the VPN, ensuring private communications. Certainly you have used the public Internet to access data or services dispersed across the globe. You are reading this article with a web browser located on your desktop, and the images and text have traveled perhaps thousands of miles from the web servers. The data has traveled as a sequence of digital packets; think of it as a stream. The stream has probably passed through at least a dozen network routing devices along the way. Due to the redundancy designed into the Internet, it is practically impossible to predict the path that a data stream will take. A motivated party can look at or change parts of this stream anywhere on the path. This lack of privacy doesn't matter much, given the nature of this web page, but for more sensitive information pertaining to your business or personal information, greater security is important. Using a VPN tunnel is a good way to ensure that.

A VPN can also extend your LAN. When working at the office, you likely take advantage of many computing services provided at your site. You can access files shared by your coworkers and read and send email using your business account. Your office network probably has specialized maintainers, allowing you to depend on applications to work, stay up to date, and remain findable.

Related Reading

Network Security Hacks
100 Industrial-Strength Tips & Tools
By Andrew Lockhart

Travel away from the office and try to do work that requires access to your office network. If you've done this, you know it can be frustrating. A typical non-VPN solution for smaller business is some type of dial-up access. That works on a limited scale and suffers from unreliability and cost. Modem banks are expensive. You need some safe, reliable way to extend the reach of your small-office network out to people who need access--you, your staff, your business partners, and your customers. Be it a remote affiliate site or a home office, a VPN will let you reach out and touch your local network resources from practically anywhere in the world.

VPN Technologies

A VPN tunnel connects two locations. In this case, it's your site and a remote site that needs network access. When an outsider wants to connect to your local network through a VPN, he must first communicate with a known service running on your local network by authenticating to it. That is the process by which a remote user proves his identity to the VPN service running on your network. This way you control who accesses your network.

After successful authentication, the VPN service establishes a direct communications channel to a compatible VPN service running at the remote site. This channel will provide the tunnel through the Internet between the two networks. From now on, until the outsider disconnects, any network message that needs to travel between the two networks will pass through this tunnel. The VPN services at the end points encrypt and decrypt all messages that pass into the tunnel.

At the nuts and bolts level, many different techniques are available for actually implementing the tunnel, each with advantages and disadvantages. Two of the most often used techniques for constructing the VPN tunnel are Point to Point Tunneling Protocol (PPTP) and Layer 2 Tunneling Protocol encrypted with IPSec (L2TP/IPSec). Current Microsoft Windows Server operating systems provide everything necessary to set up VPN tunnels using one of these two techniques. However, this article covers a third method, implemented by James Yonan's OpenVPN.

Why not use Microsoft's built-in PPTP or L2TP/IPSec packages? In many cases you can, but be aware of the limitations of these. First, PPTP was initially a proprietary protocol by Microsoft with notoriously weak security. Later revisions of the authentication protocol fixed the problems, but interoperability with non-Microsoft platforms remains difficult to this day. A more subtle potential problem with PPTP has to do with the construction of the VPN tunnel. The PPTP tunnel packet uses the General Routing Encapsulation (GRE) protocol. Sometimes routers between the two end points block this protocol. In a typical VPN scenario, you don't have control over the channel between the two end points, so this can be a real problem. This situation is hard to diagnose and harder to fix, since you may not be able to convince a third party to allow your GRE packets to pass. OpenVPN will use either UDP or TCP for constructing its tunneling packets, so there is more assurance that the VPN will work everywhere.

Considering L2TP by itself, it does not provide security but is simply an encapsulation protocol. So, in the VPN scenario, IPSec is used to add encryption and authentication. IPSec is an industry standard and as such is extremely robust and reliable when implemented correctly. Fortunately, most platforms have good implementations available.

The L2TP/IPSec VPN suffers from one major problem: it cannot operate over basic Network Address Translation (NAT) routers. Small networks often connect to the Internet using NAT; most small office/home office (SOHO) routers implement NAT by default. Some newer SOHO DSL routers provide Universal Plug and Play (UPnP) NAT Traversal, which allows IPSec to operate--but again, you want the VPN to work everywhere.

The complexity of L2TP/IPSec is another minor disadvantage. It's more difficult to troubleshoot and maintain relative to other protocols. This has to do in part because IPSec is part of the underlying network operating system and is difficult to separate. Contrast this tight binding to the lower levels of the network stack with PPTP, which operates at a higher level. With PPTP it is easier to isolate problems either to the VPN protocol or the underlying network. This final disadvantage is minor because we assume that a business network will employ specialists that, given time, can work through these technical difficulties. Major disadvantages involve third-party equipment that your network staff will have not access to, such as routers at some unknown off-site location. Here, OpenVPN's use of the ubiquitous UDP and TCP protocols is clearly advantageous.

As mentioned previously, OpenVPN provides a third, widespread alternative: the software runs on all widely used computer platforms. It uses either UDP or TCP for reliability and ease of maintenance, and employs the Secure Socket Layer (SSL) protocol developed by Netscape to secure e-commerce applications. More often, you may hear of this protocol by the acronym TLS (Transport Layer Security). In fact, the Internet Engineering Task Force renamed SSL as TLS in 1999. The rest of this article uses the terms SSL and TLS interchangeably. SSL/TLS enforces security using X.509 certificate technology.

Earlier releases of OpenVPN lacked a couple of key features unrelated to security, but the latest beta releases of OpenVPN-2.0 have added them. Specifically, the first new feature is support for certificate validation versus a Certificate Revocation List (CRL) during client authentication. This leads to easier maintenance and greater security. The second new feature is that the OpenVPN service now runs as a single server instance. Previously, each VPN tunnel connected to a different server port and needed a separate OpenVPN service to manage it. This new feature greatly simplifies configuration work. The second part of this series will cover both features.

Public Key Cryptography Basics

As mentioned previously, OpenVPN uses the SSL protocol for security. The SSL protocol relies on digital objects called X.509 certificates for authentication and encryption. Once your site has its PKI in place, you can provide many more services, including secure web services and email.

The technology that serves as the foundation for all of this security is public key cryptography. This is a complex subject, worthy of far more than this brief summary. Public key cryptography is a technique for secure communication that uses two different keys: public keys made globally available to the world, and private keys guarded secretly by individual parties. Both public and private keys are very large numbers generated simultaneously as a pair by sophisticated computer algorithms.

These key pairs have a very useful property: encrypting a message with one key requires the use of the other key to decrypt it. This is the most important property of these keys, and it bears repeating. After generating a public/private key pair, a user can decrypt a message encrypted by the private key only by using the public key. This also works in reverse; messages encrypted with the public key require the private key to decrypt.

Suppose that Alice wants to send a secret message to Bob, and Bob alone. Suppose also that Bob has generated a public/private key pair. First, Alice will create her message. Then she will encrypt it using Bob's public key. Finally, she will send the encrypted message to Bob. Bob can decrypt this message only by using the private key that matches the public key used to encrypt the message. As long as Bob keeps his private key secret, any message encrypted with his public key is safe. When Bob receives the message, he uses his private key to decipher it. He then can read the message from Alice.

For Bob to reply securely to Alice, she would need to create her own public/private key pair. Why? What if he encrypted his message to Alice using his private key thinking that Alice could then decrypt it using her copy of Bob's public key? Alice could indeed decipher Bob's message, but so could anybody else who had Bob's public key. That defeats the purpose of encrypting the message in the first place. Bob needs to encrypt his reply with Alice's public key to ensure that only Alice, the holder of her private key, can decrypt it. That's public key cryptography in a nutshell.

Digital Certificates

A gaping hole in the above scenario has to do with the management of public keys. How does Alice know that a given public key really does belong to Bob? You can imagine that an interloper may create a public/private key pair and masquerade the public key as belonging to Bob, thereby intercepting secured communications by tricking Alice into using the fake public key to encrypt her message. It is just this security hole that a digital certificate fills.

A digital certificate is a data structure that authenticates a public key and contains, at the minimum, the following pieces of information:

An agency creating a digital certificate guarantees that the public key belongs to the party that requests the certificate. The requesting party is the subject, and the party issuing the certificate is the issuer. The X.509 certificate format encodes those names according to a special syntax, calling them distinguished names (DNs). The issuer creating the certificate collects information from the requesting party until it knows the requestor's identity sufficiently. Then it combines the name of the requesting party, the requesting party's public key, and its own name into a data structure, and calculates a digital checksum on that structure. The certifying agency encrypts the result of this checksum using its own private key and appends this encrypted checksum to the certificate's data structure.

After completing this process anyone can examine the digital certificate and verify that a certain public key does in fact belong to a particular individual. Of course, all parties involved in the transaction must trust the certifying agency.

Assume that Frank runs a reputable digital certification service and that both Alice and Bob trust him. Bob can contact Frank and ask him to issue a digital certificate. Frank will ask Bob to provide some kind of identification proving that he is indeed Bob. Once Frank is satisfied that Bob is indeed Bob, he will construct a digital certificate using a computer program. The inputs to this program are Bob's name as the subject, his public key, and Frank's name as the issuer. The computer program will calculate a digital checksum over the combination of these items and encrypt that value using Frank's private key. Now when Alice wants Bob's public key, she locates his digital certificate and validates it using Frank's public key.

For validation, Alice separates the encrypted checksum from the digital certificate and calculates a checksum on the remainder of the data structure. She then uses Frank's public key to decrypt the original checksum and compares it with her result. If the checksums match, Alice knows that Frank and no one else actually put his information into the certificate. How does Alice know that she really has Frank's public key? She can validate Frank's public key by using a digital certificate issued to Frank by some other trusted agency. Typically, this chain of trust will terminate at a well-known public entity that has saturated the Internet with a public key. A few such entities are VeriSign, Entrust, and GlobalSign, which have public keys built into many software products including Internet Explorer. Incidentally, the certificate at the top of the hierarchy is a "self-signed" certificate.

Public Key Infrastructure

Earlier, this article claimed that using a PKI could support access to and operation of our VPN. A Public Key Infrastructure is a set of computer programs, databases, and network services that allow an organization to issue, publish, and revoke digital certificates. I've described issuance and publication previously, but revocation is a new topic. (It's also one of the important new features found in OpenVPN-2.0--support for verification against a Certificate Revocation List or CRL.)

If a certificate issuer decides that a certificate is no longer valid, it will add that certificate to its CRL. As hinted by the name, the CRL is a list of revoked certificates. It is the responsibility of the party using the certificate to check the CRL and verify that a given certificate remains valid.

Suppose that you have implemented a VPN and secured it with your Public Key Infrastructure. You have given many of your staff members VPN access to your internal network by issuing them digital certificates. Everything works great, and your staff can now work easily from home and access electronic material stored locally from remote customer sites. Now, assume a member of your staff moves on to another job. You no longer want her to have access to your network. Revoke her certificate and add it to the CRL maintained as part of your PKI. Because OpenVPN-2.0 can validate against this CRL, you can immediately and easily terminate that staff member's VPN access.


This article has attempted to lay out conceptually all of the pieces required to put together a VPN using certificates for security. If you have followed along and want to see what it takes to put these concepts into action, the second part of this series will work step-by-step through a complete real-world implementation.

Scott Brumbaugh has worked professionally as a software/systems engineer since 1987.

Return to the Security DevCenter

Copyright © 2009 O'Reilly Media, Inc.