Valid HTML 4.01 Transitional

Activating IPv6 on Mathnet

James F. Carter <>, 2014-08-10

Starting about 2014-08-05, the UCLA Mathematics Department (Mathnet) has received a transit connection to the global IPv6 Internet. We are going to activate IPv6 service on Mathnet forthwith. (For this purpose, Mathnet includes PIC.)



Which Mathnet hosts are IPv6 capable?

For general purpose computers under Mathnet's administrative control, all are capable of IPv6. This includes Linux (kernel 3.11.10), Windows-7, and two Macs. Other equipment ranges in age from antediluvian (no IPv6) to modern (yes IPv6). This includes managed network switches, wireless access points, printers, and closed-source NAS boxes. The project does not include upgrading ancient equipment to get IPv6 capability. However, we need to work around non-IPv6 hosts; specifically, they must not be given an AAAA record alleging nonexistent IPv6 capability.

How do we support rogue machines?

Users' own equipment (referred to as rogue machines) may include back-version hosts, which should continue to function on IPv4. Non-antique rogue hosts should participate in IPv6 on the same terms as Mathnet hosts. This includes PC's, cellphones (on Wi-Fi) and tablets. IPv6 is a mandatory feature on 4G-LTE, so all 4G cellphones should be capable of IPv6. We have been warned that Mac OS-X 10.5 "Leopard" (2007) has a bug in IPv6 so it will try to use IPv6 but may fail. Mathnet will deal with this situation by telling the users to upgrade their OS.

Where do we get a router?

Our current plan is to use a Linux box (Harlech) to route IPv6. On the main switch our Cisco operating system is back version and is not capable of IPv6, and the current operating system version will not run on our hardware, which would therefore have to be upgraded. We plan to do both these steps but it will take a lot of time, whereas the Linux box is ready to go and has been waiting for this role for years.

Update: Max says that the local switches do not need to know about IPv6; only the main PSnet router does, and it was just upgraded. So the plan is being revised: Deploy on Harlech while we are providing service only to test machines, because Linux's radvd can be told to do so, but when activating IPv6 for the whole department we will turn on routing at the PSnet level.

Chicken and egg issues

Subsystems have to be activated in the correct order to avoid service breaks. Specifically, we need to configure the hosts' network interfaces to have IPv6 addresses before we publish AAAA records for them, inviting partners to communicate using those addresses.


These action items are approximately in the order in which we are going to do them.

Basic Design Issues

Monitor Software

Pinger shows a strip chart of connectivity to a list of hosts. It has a color code for nine states: all, some or no ping packets returned (during a history interval), for each of IPv4 and IPv6. It has already been enhanced to do IPv6, but it needs one more feature: the ability to reinitialize and re-resolve the hosts on command, so when hosts gain or lose AAAA records it can respond without being completely restarted. [Done.] Command line for pinging the test machines:

pinger tabbycat 2607:f010:2a8:8fe4:1a03:73ff:fe1e:a881 2607:f010:2a8:8fe4:0221:70ff:fe31:cf5c 2607:f010:2a8:8fe4:baac:6fff:fe83:d420 &>/dev/null

We need a simple script that takes data in the same format as /etc/hosts, and pings (IPv4 and IPv6) all the hosts listed. The IP addresses should be used so it can be run before AAAA records are published. A report should be produced listing hosts that respond to both families, to only one, or which are completely dead.

A related script would go through /etc/ethers, generate link-local addresses for all known hosts, and ping them. Responders are IPv6 capable. This would have to be done on each subnet and the union taken. A related script could go through hostdata and enable IPv6 AAAA records for those hosts.

We also need something to monitor how much traffic we are handling. We don't have that for IPv4. Likely we'll do something very simple, only on the router. Likely this will be delayed until after most of the project is finished.

Configuration Management Automation

We have a package called hostdata which uses a common database of data about hosts, and generates various derived files, in particular the DNS zone files. It needs to generate AAAA and matching PTR records. Beware on testing: these must not be installed and published until after the hosts have these addresses configured; otherwise partners will try to connect using IPv6 and will fail. It needs three states for the IPv6 addresses: implicit from RFC 2462 auto-configuration (the most common), explicitly excluding hosts incapable of IPv6 (next most common), and explicit fixed IPv6 addresses. I don't presently know of any case where the latter is needed, but inevitably that case will come up. We do have the Ethernet MAC address in hostdata, and it's accurate and checked daily, so we can do the RFC 2462 thing in the hostdata installer.

The first step will be to configure every host to be excluded from IPv6. Then particular hosts (test machines) will be moved to the implicitly activated state. Finally, all hosts that respond to ipv6-icmp echo (ping) will get activated.

Mketchosts extracts key IP addresses from DNS and/or NIS and creates a local /etc/hosts file with just enough information to boot the machine. We need to take a look to see if it's going to extract IPv6 addresses if available. I'm pretty sure it will but it needs to be checked.

Mathnet has modified /etc/sysctl.conf and makes sure that the modified version stays installed. It has settings for IPv6 networking. These should be reviewed. If a change is needed, it should be emplaced before router advertisements go out. Remember to set the Martian defense strategy on Harlech, similar to what's on Papyrus now. (If a client sends to Harlech from other than its primary net (4-net), replies will go direct to the VLAN pseudo-NIC where the client is, from the IP address that the client sent to, i.e. 4-net. That pseudo-NIC is not on the 4-net, so such a source address is impossible and a hacker defense strategy is activated.)

Since /etc/ethers is a key resource for IPv6 management (to generate the EUI-64), we need to have it served by LDAP, but making that happen is not part of this project. Presently ethers is served by NIS.

Router Hardware

The IPv6 main router [during testing] is going to be Harlech (see issues section for why to pick this box). Several of its NICs are going to do 802.1q trunking, i.e. there will be a separate pseudo-NIC on each relevant VLAN. Conventionally these are named after the VLAN, e.g. vlan101, vlan66… The ideal NIC assignment would go like this:

There is one fly in this ointment: Harlech has only two onboard NICs. I hope we can get an add-in card for it. If not, an alternative is to use 802.1q on eth0 for all internal subnets including the 4-net and the priv net. eth1 would be on VLAN 66 for the default route. Update: we have the add-in card and it appears to work.

Charlie has connected VLANs and NICs as follows. The addresses and route come from RFC 2462 autoconf, since radvd is not running yet.

I plan to use YaST LAN configuration (SuSE administration package) rather than to configure IPv6 by cowboy programming like I have done previously. I've been using 6RD, which YaST doesn't know how to handle, but with a normal IPv6 configuration YaST can deal with it.

Update: Harlech will be used during initial testing because radvd can be told to respond only to specific listed hosts' router solicitations, but when we activate IPv6 for the whole department we will do that on the PSnet main router (sup720 on 5th floor), which is new and which knows about IPv6. The departmental or floor switches do not need to know what's inside the packets, only which VLAN they are on.

Details of Configuring Harlech

I'm going to configure eth3 to do 802.1q, then configure vlan98, vlan99, vlan101 with these static addresses. 207:e9ff:fe01:dfc5 is the EUI-64 of eth3, the trunk port.

yast2 lan takes a long time to Detect Network Devices and to Read Device Configuration, at least a minute for each. Be patient.

When configuring a VLAN, get in the toplevel dialog. Add an interface. Device type: VLAN. Configuration name: text to follow the word VLAN in the device name; by convention it's the VLAN number. (Hit Next.) On Network Card Setup: Pick the underlying interface (eth3 here) and fill in the static IP from the list above. Fill in /64 as the subnet mask.

To enable forwarding you need to do this command, or add this line in /etc/sysctl.conf:

echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
net.ipv6.conf.all.forwarding = 1

We also don't want the complication of RFC 4941 privacy extensions, which has the host sourcing from a random IPv6 address. On Harlech they are already set up in /etc/sysctl.conf.

echo 0 > /proc/sys/net/ipv6/conf/default/use_tempaddr
echo 0 > /proc/sys/net/ipv6/conf/all/use_tempaddr
net.ipv6.conf.default.use_tempaddr = 0
net.ipv6.conf.all.use_tempaddr = 0

If eth0 is in a bridge and both eth0 and br0 are auto-configured, specifically if a bridge member (eth0) has a route onto the LAN, it eats IPv6 packets. This is the case on Tabbycat.

echo 0 > /proc/sys/net/ipv6/conf/eth0/accept_ra
net.ipv6.conf.eth0.accept_ra = 0

Routing issues: Both eth2 and eth3 have routes onto VLAN 66 (2607:f010:2a8:8042::/64). Likely we just need to disable auto-conf on eth3. They should not have routes at all, because when you turn on router status (net.ipv6.conf.all.forwarding = 1) that disables auto-conf. Note that this is per NIC; we could in theory turn it back on, just for eth2. Nonetheless, I've set (using yast2 lan) the default route explicitly to 2607:f010:2a8:8042::1 dev eth2.

Firewall on Harlech

I recently upgraded the firewall on Harlech to handle IPSec tunnels, and I took the opportunity to propagate the same changes to IPv6. It will need tweaking to take into account the new IPv6 addresses and additional NICs. As soon as /etc/hosts contains the IPv6 address of Harlech, the firewall can be reloaded and it should function on IPv6. However, it should be reviewed and checked for proper functioning.

Router Advertisement Daemon

In IPv6 the router needs to send out router advertisements, which tell the hosts on each subnet these kinds of information:

Note for testing: it is possible to tell radvd to send to only specific listed clients and to ignore router solicitations from all others. This should be done to test the configuration.

Once radvd is running, clients will automatically configure their RFC 2462 addresses and will be able to originate IPv6 connections to the global Internet.

On 4-net (VLAN 97), every iPhone seems to be spewing out dhcpv6 solicits.

Radvd Configuration

Here's a nice writeup about ULA (Unique Local Addresses, RFC 4193) and IPSec with pre-shared keys. When radvd was installed it assigned ULA prefix fd56:6668:6b94:0001::/64. Actually the first 48 bits are our prefix and the 1 is for our specific routing domain. We aren't really going to use the ULA.

Publishing the AAAA Records

Once radvd is running we can use the hostdata installer to add IPv6 AAAA and PTR records to our zone files, for hosts that are not excluded in hostdata from IPv6. Once this is done, internal and foreign partners that are IPv6 capable will use IPv6 to communicate with our hosts.

A final step will be to run mketchosts -c on all hosts with AAAA records, so the host's own IPv6 address will be available at boot time before the network comes up.

Message to Users

Here is (a draft of) the message that we will send to our users, on the mcg-announce alias.

On (fill in date here), the Mathematics Computing Group is going to enable IPv6 (Internet Protocol version 6) on the entire Math and PIC network.

Please see this Wikipedia article giving an overview of IPv6, with statistics on the unavailability of IPv4 addresses, and extensive discussion of the technical issues making IPv6 preferable to IPv4.

All Mathnet hosts, and personally owned computers, tablets and cellphones with non-antique operating systems, will be able to participate. Antiques will continue to use IPv4 (the version made obsolete by IPv6). UCLA campus sites and forward-looking services such as Wikipedia, Facebook, Google and its subsidiaries such as Youtube will be reached over IPv6. For retro services like Amazon and Twitter, the machines on Mathnet will automatically fall back to IPv4.

Users should not notice any differences except that we can reach IPv6-only partners and they can reach UCLA-Mathnet services. To see if your machine has become capable of IPv6, try connecting to either of these URLs:

The programming staff have been using the IPv6 infrastructure for the last week and have not experienced problems; we expect that department-wide deployment of IPv6 will similarly be a non-event. However, if you experience connectivity problems that might be related to IPv6, please contact the MCG Support Office (bugs manager) at