Choosing a VPN-Type Program for Mathnet

James F. Carter, UCLA-Mathnet, 2003-09-26


Candidate VPN Programs

Issue FreeS/WAN vtun cipe OpenVPN PoPToP
Protocol IPSec (ESP, AH) Its own Its own TLS + its own PPTP (GRE)
Encryption Excellent Weak (3) Weak (3) Excellent See (7)
Authorization X.509 (4) SSH kludge Static keys X.509 Windows
  (Pre-arranged keys also work for most programs)
Ad-hoc users Yes Kludge (1) No Yes (8) Yes
Resists IP collisions No Kludge (1) No Yes (8) Yes
Clients for: L,W,M (2) L,S L,W L,S,W,M-x L,S,W (6)
  (L=Linux, S=Solaris, W=Windows, M=Mac)
Works through NAT Patch (4) Yes Yes Yes Yes
We used how long? 1 year 0.5 year None 0.1 year None
Blocked by BOL Works Works (ssh) Prob. blocked Asked for unblock Probably works
Included with SuSE Yes Yes Yes Yes (8) No
Links Regular,
vtun cipe openvpn poptop
Good features Has been reliable Reliable (with 1 user) Net buzz Rock solid, easy setup Microsoft uses it
Bad features Hard to set up. Diarrhea of the log files. Requires kludge script. TCP meltdown (5). Shell accounts on bastion host. No ad-hoc mode None (8) Microsoft uses it
  1. For vtun, each user must be preassigned an IP address for the tunnel. A script has been written to dynamically assign these addresses when the connnection begins. But it's a kludge.
  2. Any IPSec implementation should be able to talk to FreeS/WAN, and successful interoperation testing is extensive. It is said that Windows XP can do this natively. A kernel module is required, and I don't know if one is available for Solaris or Mac OS-X, nor whether it works with non-UNIX MAC OS's. There is a commercial IPSec client for Mac OS-X: VPN Tracker by Equinux.
  3. Vtun uses a crypto algorithm (Blowfish) from the SSL library, but does not use the full-blown SSL to encrypt its channel. There has recently been discussion on the vtun mailing list about a weakness in how the initialization vector is chosen. (A similar problem led to the exploit against 802.11b WEP.) The same criticism is said to apply to cipe.
  4. FreeS/WAN development is divided into two branches: "pure" FreeS/WAN (for which political goals are significant that require code that may be subject to USA export restrictions to be avoided), and "super" FreeS/WAN, which has patches for (among other things) X.509 key exchange and authentication, and NAT traversal. The NAT patch is present in SuSE Linux 8.2 but unfortunately there is a technical disagreement in the kernel community so that SuSE judges it imprudent to put the needed ESP-in-UDP patch in the stock kernel. You have to recompile the kernel yourself if you want it.
  5. When tunnelling TCP in TCP, as with tunnelling vtun (or anything else) over a ssh forwarded port, when the load is high enough to make packets die in collisions, both TCP layers will retry, and the result will be a runaway overload, referred to as TCP meltdown. Special care is needed to resist this, which ssh is not designed for. Presumably kernel-level IP-in-IP tunnels can do better. See this link for more.
  6. While PoPToP is for UNIX, Windows (95 and later) includes a native PPTP client, with which PoPToP interoperates. The HOWTO mentions a Mac client but you have to purchase it.
  7. See this link for criticisms of security in Microsoft's implementation of PPTP. The issue is not weak encryption but weak authentication. Microsoft has put out a new, improved authentication protocol, which PoPToP interoperates with.
  8. OpenVPN version 2.0beta$N has recently been released; we've installed N=5. In this version the server has a pool of IP addresses which it pushes to the clients. The client's local IP address is not used. The protocol is immune to NAT. The 1.x version requires each tunnel to be prearranged using the client's IP address. SuSE 9.1 includes OpenVPN-1.6, so we're distributing our own RPM of 2.0.

Additional Discussion

IPSec is divided into the kernel support for AH and ESP protocols (with associated tunneling code), and the key (ISAKMP) daemon. The FreeS/WAN distribution provides the kernel modules and hacks. SuSE 8.2 (kernel 2.4.20) includes these (except see above about ESP-in-UDP). IPSec is now mainstream in the kernel as of 2.5. The ISAKMP daemon for FreeS/WAN is called "Pluto". People say that a competitor "Racoon" (sic) is just as good and is easier to set up. The FreeS/WAN developers would disagree.
While vtun has a number of users and partisans on its mailing list, its developer is in finished mode, and response to the security criticism mentioned above has not been swift. For us, the biggest problem with vtun is the likelihood of remote IP collisions, which it cannot handle natively. It was intended for use with prearranged connections.
It is often mentioned as being similar to vtun but better or more suited to real sysadmins. It is generally described in the context of prearranged connections.
It is often mentioned in the context of you really ought to take a look at OpenVPN. They're right, we should, and will. It uses OpenSSL for TLS crypto on the control channel, a block-encrypted main channel via UDP, and TUN/TAP for tunnelling (mainstream in the kernel, originally written for vtun). No other kernel support is required. Performance quote: Pentium-2 266 MHz, TLS/Blowfish/SHA1, large precompressed files, data rate was 1.455 Mby/sec. It can do chroot, and/or not run as root, and can lock key info in memory. Version 1.x is intended for static point-to-point tunnels, and hence is designed so it has to be individually configured for each user. Version 2.0 includes the road warrior multi-user configuration that we need.
PPTP was developed by Microsoft and is present natively in Windows since Win98 (patches available for Win95). It apparently is in use by a lot of people. If more than one PPTP session have to traverse the same NAT box (not a common scenario), it needs special handling; Linux iptables includes a PPTP helper module. Microsoft provides their employees with a PPTP client which does more than the native one; it can use a smart card for authentication, and refuses to connect unless the machine is up to date on patches.


FreeS/WAN (Super)
Super FreeS/WAN has a number of very positive features: Crypto is believable; a single configuration stanza can be used on the server for all user tunnels; the security association is automatically re-established if a laptop is suspended and reawakened; endpoint authentication is both effective and flexible. Recently Harlech's kernel was recompiled with the NAT traversal patch. And, supposedly it interoperates with Win2K and WinXP natively. A web interface to produce the X.509 certificates is (still...) in beta test. The only remaining flies in the ointment are, in the default configuration the remote host uses its public IP address ( on the remote connection. The research was done to learn how to configure WinXP for IPSec, and it actually worked, but the procedure is so horrendous that it's not credible to expect a user to do it. For over a year Mathnet used FreeS/WAN exclusively as our VPN solution. But for the present, see the discussion of OpenVPN below.
Native vtun requires individual configuration of each client's tunnel. A script has been created to kludge up dynamic address assignment. While my experience with vtun is good, I'm afraid that it isn't robust enough for us to deploy in production.
Since it requires individual configuration of each client's tunnel, cipe has to be rejected.
We were discouraged by version 1.x because each tunnel had to be configured individually, but we have found that v2.0beta5 is easy to set up for both generic clients and our server, and is totally reliable. The Windows client in particular is excellent. Mathnet has settled on OpenVPN as the VPN client that we will support with our users.
PoPToP appears to give solid cryptography and it used to be the only solution that can natively avoid IP address collision of the remote clients. (OpenVPN's new version also can do this.) However, authentication is done via PPP, specifically authenticating to a Windows domain controller (which Mathnet does have). Extensive auxiliary scripts are needed at the server end to make the connection work, e.g. filling in the ARP table manually. Thus, PoPToP unfortunately has to be rejected.