Starting about 2014-08-05, the UCLA Mathematics Department (Mathnet) has received a transit connection to the global IPv6 Internet. We are going to activate IPv6 service on Mathnet forthwith. (For this purpose, Mathnet includes PIC.)
All Mathnet hosts that are capable of originating IPv6 connections
shall be provided with the infrastructure so they can do so.
Capable
means that IPv6 is supported in the operating system
and/or firmware, and the project includes any necessary configuration
of our hosts to activate IPv6. Due to the mandates in RFC 2462,
hand labor will only be needed on the main router.
All of our foreign and internal partners that are capable of IPv6
(and properly configured) shall be able to make IPv6 connections to any
of our hosts that are capable of IPv6. Here, internal
means on
Mathnet, and foreign
means not on Mathnet.
IPv4 service in and out shall continue undisturbed, and in
particular, hosts that are not capable of IPv6 (internal and foreign)
shall be able to make and receive IPv4 connections to or from our
hosts whether or not IPv6 capable. This is commonly referred to as
the dual stack
paradigm.
Access control rules (firewall) shall be similar for IPv6 as they now are for IPv4.
We shall provide IPv6 configuration control automation, monitoring tools, and documentation, similar to what we have now for IPv4.
For general purpose computers under Mathnet's administrative control, all are capable of IPv6. This includes Linux (kernel 3.11.10), Windows-7, and two Macs. Other equipment ranges in age from antediluvian (no IPv6) to modern (yes IPv6). This includes managed network switches, wireless access points, printers, and closed-source NAS boxes. The project does not include upgrading ancient equipment to get IPv6 capability. However, we need to work around non-IPv6 hosts; specifically, they must not be given an AAAA record alleging nonexistent IPv6 capability.
Users' own equipment (referred
to as rogue machines
) may include back-version hosts, which should
continue to function on IPv4. Non-antique rogue hosts should participate in
IPv6 on the same terms as Mathnet hosts. This includes PC's, cellphones (on
Wi-Fi) and tablets. IPv6 is a mandatory feature on 4G-LTE, so all 4G
cellphones should be capable of IPv6. We have been warned that Mac OS-X 10.5
"Leopard" (2007) has a bug in IPv6 so it will try to use IPv6 but may fail.
Mathnet will deal with this situation by telling the users to upgrade their OS.
Our current plan is to use a Linux box (Harlech) to route IPv6. On the main switch our Cisco operating system is back version and is not capable of IPv6, and the current operating system version will not run on our hardware, which would therefore have to be upgraded. We plan to do both these steps but it will take a lot of time, whereas the Linux box is ready to go and has been waiting for this role for years.
Update: Max says that the local switches do not need to know about IPv6; only the main PSnet router does, and it was just upgraded. So the plan is being revised: Deploy on Harlech while we are providing service only to test machines, because Linux's radvd can be told to do so, but when activating IPv6 for the whole department we will turn on routing at the PSnet level.
Subsystems have to be activated in the correct order to avoid service breaks. Specifically, we need to configure the hosts' network interfaces to have IPv6 addresses before we publish AAAA records for them, inviting partners to communicate using those addresses.
These action items are approximately in the order in which we are going to do them.
We will use RFC 2462 auto-configuration, and will not provide a DHCPv6 server (except in the remote possibility that a situation is discovered where neither RFC 2462 nor a fixed address is feasible). All presently known operating systems can do RFC 2462.
(RFC 2462 has been superceded by RFC 4862; the procedures are changed very little, but some robustness issues are improved.)
We will provide a /64 subnet for each IPv4 subnet (most are /24, some are smaller). Each one of these has its own VLAN. So there is a 1-1 correspondence between VLANs, IPv4 subnets, and IPv6 subnets. This includes the priv net, used for backups.
Max has given us 2607:f010:2a8:8fe0::/59 for our network.
Even though auto
configuration is used and the host is
authoritative for its own IPv6 address, we will publish
administratively generated AAAA and PTR records (DNS) for the same
hosts. When a defective NIC or motherboard is replaced, we need to
remember to promptly update hostdata with the new MAC address, from
which the RFC 2462 address is generated. mDNS (multicast DNS) could
be considered: each host sends out DNS records for its own name and
address, and clients query and believe in this material. But this is
not useful for us, since foreign partners do not have access, and since
spoofing attacks are too easy.
Radvd (router advertisement daemon) can be configured to send router advertisements to only specific listed clients. When IPv6 is set up, this feature will be used to first activate test machines (tabbycat and mulberry), then MCG hosts, and finally all clients. The final step can be done near-atomically: HUP radvd, spot check that clients have their IPv6 addresses, then publish the AAAA records.
Pinger
shows a strip chart of connectivity to a list of hosts.
It has a color code for nine states: all, some or no ping packets returned
(during a history interval), for each of IPv4 and IPv6. It has already been
enhanced to do IPv6, but it needs one more feature: the ability to reinitialize
and re-resolve the hosts on command, so when hosts gain or lose AAAA records
it can respond without being completely restarted. [Done.] Command line
for pinging the test machines:
pinger tabbycat 2607:f010:2a8:8fe4:1a03:73ff:fe1e:a881 2607:f010:2a8:8fe4:0221:70ff:fe31:cf5c 2607:f010:2a8:8fe4:baac:6fff:fe83:d420 &>/dev/null
We need a simple script that takes data in the same format as /etc/hosts, and pings (IPv4 and IPv6) all the hosts listed. The IP addresses should be used so it can be run before AAAA records are published. A report should be produced listing hosts that respond to both families, to only one, or which are completely dead.
A related script would go through /etc/ethers, generate link-local addresses for all known hosts, and ping them. Responders are IPv6 capable. This would have to be done on each subnet and the union taken. A related script could go through hostdata and enable IPv6 AAAA records for those hosts.
We also need something to monitor how much traffic we are handling. We don't have that for IPv4. Likely we'll do something very simple, only on the router. Likely this will be delayed until after most of the project is finished.
We have a package called hostdata
which uses a common database
of data about hosts, and generates various derived files, in particular the
DNS zone files. It needs to generate AAAA and matching PTR records. Beware
on testing: these must not be installed and published until after the
hosts have these addresses configured; otherwise partners will try to connect
using IPv6 and will fail. It needs three states for the IPv6 addresses:
implicit from RFC 2462 auto-configuration (the most common), explicitly
excluding hosts incapable of IPv6 (next most common), and explicit fixed IPv6
addresses. I don't presently know of any case where the latter is needed,
but inevitably that case will come up. We do have the Ethernet MAC
address in hostdata, and it's accurate and checked daily, so we can do the
RFC 2462 thing in the hostdata installer.
The first step will be to configure every host to be excluded from IPv6. Then particular hosts (test machines) will be moved to the implicitly activated state. Finally, all hosts that respond to ipv6-icmp echo (ping) will get activated.
Mketchosts
extracts key IP addresses from DNS and/or NIS and
creates a local /etc/hosts file with just enough information to boot the
machine. We need to take a look to see if it's going to extract IPv6 addresses
if available. I'm pretty sure it will but it needs to be checked.
Mathnet has modified /etc/sysctl.conf and makes sure that the modified
version stays installed. It has settings for IPv6 networking. These should be
reviewed. If a change is needed, it should be emplaced before router
advertisements go out. Remember to set the Martian defense strategy on
Harlech, similar to what's on Papyrus now. (If a client sends to Harlech
from other than its primary net (4-net), replies will go direct to the VLAN
pseudo-NIC where the client is, from the IP address that the client sent to,
i.e. 4-net.
That pseudo-NIC is not on the 4-net, so such a source address is
impossible
and a hacker defense strategy is activated.)
Since /etc/ethers is a key resource for IPv6 management (to generate the EUI-64), we need to have it served by LDAP, but making that happen is not part of this project. Presently ethers is served by NIS.
The IPv6 main router [during testing] is going to be Harlech (see issues section for why to pick this box). Several of its NICs are going to do 802.1q trunking, i.e. there will be a separate pseudo-NIC on each relevant VLAN. Conventionally these are named after the VLAN, e.g. vlan101, vlan66… The ideal NIC assignment would go like this:
There is one fly in this ointment: Harlech has only two onboard NICs. I hope we can get an add-in card for it. If not, an alternative is to use 802.1q on eth0 for all internal subnets including the 4-net and the priv net. eth1 would be on VLAN 66 for the default route. Update: we have the add-in card and it appears to work.
Charlie has connected VLANs and NICs as follows. The addresses and route come from RFC 2462 autoconf, since radvd is not running yet.
I plan to use YaST LAN configuration (SuSE administration package) rather than to configure IPv6 by cowboy programming like I have done previously. I've been using 6RD, which YaST doesn't know how to handle, but with a normal IPv6 configuration YaST can deal with it.
Update: Harlech will be used during initial testing because radvd can be told to respond only to specific listed hosts' router solicitations, but when we activate IPv6 for the whole department we will do that on the PSnet main router (sup720 on 5th floor), which is new and which knows about IPv6. The departmental or floor switches do not need to know what's inside the packets, only which VLAN they are on.
I'm going to configure eth3 to do 802.1q, then configure vlan98, vlan99, vlan101 with these static addresses. 207:e9ff:fe01:dfc5 is the EUI-64 of eth3, the trunk port.
yast2 lan
takes a long time to Detect Network Devices
and
to Read Device Configuration
, at least a minute for each. Be patient.
When configuring a VLAN, get in the toplevel dialog. Add
an
interface. Device type: VLAN. Configuration name: text to follow the word
VLAN in the device name; by convention it's the VLAN number. (Hit Next.)
On Network Card Setup: Pick the underlying interface (eth3 here) and fill
in the static IP from the list above. Fill in /64
as the subnet mask.
To enable forwarding you need to do this command, or add this line in /etc/sysctl.conf:
echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
net.ipv6.conf.all.forwarding = 1
We also don't want the complication of RFC 4941 privacy extensions, which has the host sourcing from a random IPv6 address. On Harlech they are already set up in /etc/sysctl.conf.
echo 0 > /proc/sys/net/ipv6/conf/default/use_tempaddr
echo 0 > /proc/sys/net/ipv6/conf/all/use_tempaddr
net.ipv6.conf.default.use_tempaddr = 0
net.ipv6.conf.all.use_tempaddr = 0
If eth0 is in a bridge and both eth0 and br0 are auto-configured, specifically if a bridge member (eth0) has a route onto the LAN, it eats IPv6 packets. This is the case on Tabbycat.
echo 0 > /proc/sys/net/ipv6/conf/eth0/accept_ra
net.ipv6.conf.eth0.accept_ra = 0
Routing issues: Both eth2 and eth3 have routes onto VLAN 66 (2607:f010:2a8:8042::/64). Likely we just need to disable auto-conf on eth3. They should not have routes at all, because when you turn on router status (net.ipv6.conf.all.forwarding = 1) that disables auto-conf. Note that this is per NIC; we could in theory turn it back on, just for eth2. Nonetheless, I've set (using yast2 lan) the default route explicitly to 2607:f010:2a8:8042::1 dev eth2.
I recently upgraded the firewall on Harlech to handle IPSec tunnels, and I took the opportunity to propagate the same changes to IPv6. It will need tweaking to take into account the new IPv6 addresses and additional NICs. As soon as /etc/hosts contains the IPv6 address of Harlech, the firewall can be reloaded and it should function on IPv6. However, it should be reviewed and checked for proper functioning.
In IPv6 the router needs to send out router advertisements, which tell the hosts on each subnet these kinds of information:
Note for testing: it is possible to tell radvd to send to only specific listed clients and to ignore router solicitations from all others. This should be done to test the configuration.
Once radvd is running, clients will automatically configure their RFC 2462 addresses and will be able to originate IPv6 connections to the global Internet.
On 4-net (VLAN 97), every iPhone seems to be spewing out dhcpv6 solicits.
Here's a nice writeup about
ULA (Unique Local Addresses, RFC 4193) and IPSec with pre-shared keys.
When radvd was installed it assigned
ULA prefix
fd56:6668:6b94:0001::/64. Actually the first 48 bits are our prefix and the
1 is for our specific routing domain. We aren't really going to use the ULA.
Once radvd is running we can use the hostdata installer to add IPv6 AAAA and PTR records to our zone files, for hosts that are not excluded in hostdata from IPv6. Once this is done, internal and foreign partners that are IPv6 capable will use IPv6 to communicate with our hosts.
A final step will be to run mketchosts -c on all hosts with AAAA records, so the host's own IPv6 address will be available at boot time before the network comes up.
Here is (a draft of) the message that we will send to our users, on the mcg-announce alias.
On (fill in date here), the Mathematics Computing Group is going to enable IPv6 (Internet Protocol version 6) on the entire Math and PIC network.
Please see this Wikipedia article giving an overview of IPv6, with statistics on the unavailability of IPv4 addresses, and extensive discussion of the technical issues making IPv6 preferable to IPv4.
All Mathnet hosts, and personally owned computers, tablets and cellphones with non-antique operating systems, will be able to participate. Antiques will continue to use IPv4 (the version made obsolete by IPv6). UCLA campus sites and forward-looking services such as Wikipedia, Facebook, Google and its subsidiaries such as Youtube will be reached over IPv6. For retro services like Amazon and Twitter, the machines on Mathnet will automatically fall back to IPv4.
Users should not notice any differences except that we can reach IPv6-only partners and they can reach UCLA-Mathnet services. To see if your machine has become capable of IPv6, try connecting to either of these URLs:
The programming staff have been using the IPv6 infrastructure for the last week and have not experienced problems; we expect that department-wide deployment of IPv6 will similarly be a non-event. However, if you experience connectivity problems that might be related to IPv6, please contact the MCG Support Office (bugs manager) at bugs@math.ucla.edu.