Valid HTML 4.01 Transitional

Sun's Network Filesystem (NFS) on CouchNet

James F. Carter <jimc@jfcarter.net>, 2016-12-06

My net has several hosts. I want to be able to execute on any of them and to get my home directory: not a copy, but the actual files delivered over the network. I also have an immediate need for two people who log in to different machines to edit a big web document (set of web pages).

I already have tried several schemes where the home directory (or part of it) is copied between machines. None were really satisfactory. I do have a requirement that throws a monkey wrench into the network filesystem solution: my laptop needs to be functional without a network connection, e.g. inside an airplane. I see two solutions here: either the laptop will have the exported instance of the homedir (versus a real fileserver), or I will do the copying thing before and after my journey and temporarily use the laptop's local copy.

What network filesystems are possible? See my writeup on the Asus Transformer Pad Infinity (a tablet and mini-laptop) which has a discussion of the homedir issue.

SSHFS

I initially wanted to try sshfs, using this design:

I ended up abandoning the sshfs approach: too many kludges, too many ways for things to go wrong in the authentication area, too uncertain that a SSH agent or Kerberos ticket could be found reliably when needed.

Back to NFS

Instead I imported the design I use at UCLA-Mathnet for Sun Microsystem's Network File System (NFS), with some improvements that Mathnet doesn't have. If $XDIR is the exported directory and $SERVER is the 1-component name of the server that exports it, the clients are going to find it as /net/$SERVER/$XDIR . I make this happen by these files and steps. Of course many of the files can be renamed but these are either standard or are the ones I picked.

Autofs (Automounter)

NFS Server and /etc/exports

The NFS server needs to be told what directories to export, and to who, in /etc/exports; see exports(5) for all the details. Here is a sample stanza. You give one export per line (blank lines and #comments ignored), but long lines can be wrapped using backslash newline.

/s1 -rw,root_squash,async,no_subtree_check,mountpoint \
*.cft.ca.us \
2001:470:1f05:844::4:0/112 \
2001:470:1f05:844::2:0/112 \
192.9.200.128/26

Instead of actually relying on the mountpoint parameter, I have a script that reads /etc/fstab and adds the mount points to /etc/exports, all with the same options. The policy, that gives the right exports on my net, is that only ext4 or btrfs filesystems are exported, the root is not exported, and /home is exported (not with the mountpoint option) if it is not a separate filesystem, which is the case on my virtual machines. With NFS you export directories and they don't have to be mount points.

NFS Client and /etc/nfsmount.conf

The NFS client negotiates several parameters with the server, guided by /etc/nfsmount.conf, which it reads at every mount. See nfsmount.conf(5) for details. The non-default values I picked were these:

Firewall

While NFS protocol version 4 handles the whole interaction multiplexed on port 2049, earlier protocols have a collection of service daemons on the server: rpc.mountd which handles mount requests, kernel module lockd which handles NFS file locking, rpc.statd which reestablishes or clears locks when either the server or the client reboots, and of course the NFS server itself. This means that the firewall needs to be open for these ports, both TCP and UDP. However, except for NFS itself the ports are randomly assigned when the daemon starts.

My first solution was to write a script that runs rpcinfo -p, extracts the actually assigned port numbers, and updates a firewall rule to let them through. This works, but is fragile and takes some systemd trickery to get the list updated if NFS were to be restarted.

My second solution was to do enough Google searches to find how to set the daemons' ports. This is for OpenSuSE Leap 42.1, but is identical in prior SuSE versions, and forum postings suggest that the same configuration file and variable names are used for the Red Hat family including CentOS, and possibly also Ubuntu. I assigned ports sort of arbitrarily, and added these variables to /etc/sysconfig/nfs:

Then these ports were opened in the firewall. An alternative for kernel module lockd is to create /etc/modprobe.d/50-lockd.conf containing options lockd nlm_udpport=903 nlm_tcpport=903.

Also you need to add the fixed ports (except $STATD_PORT) to /etc/bindresvport.blacklist . When RPC ports are assigned at random, this will prevent your fixed ports from being used. Other services, like CUPS, also need to be in this file.

User Mapping

In NFS the server relies on the client's operating system (running as root) to not be hacked and to honestly report which user is trying to access the exported files, so the server can use normal access control methods like mode bits and POSIX ACLs. For protocol versions 3 and earlier the client reports a numerical UID, which therefore has to be in sync between the client and the server. The biggest can of worms with NFS version 4 is user mapping: the client's alphabetic loginID is sent to the server. So the loginIDs still have to be in sync, but not the UIDs; the server maps the loginID to a UID in its own way.

The main use case where UIDs can't be synced is where the client and server are in different organizations, e.g. client at home, server at work. Of course there's also the problem of syncing the loginIDs, but that is a little easier. However, you have a security issue: if the client is controlled by other than the server's organization, its sysop can configure whatever UID or loginID he pleases, and could steal files to which his organization is not entitled. So the server's sysop usually refuses to export to clients not under its control, i.e. expected to dishonestly report the UID or loginID.

Work to home mounting is requested frequently, and Microsoft Windows can handle it: the client gets a Kerberos ticket for the server's realm by giving the foreign loginID and password. But unfortunately in the NFSv4 implementation in Linux, user mapping has turned out (for me) to not be reliable: the map daemon at random times gets into a mode where it maps all the users to nobody, and you have to reboot the server to recover; restarting all the daemons is not sufficient, nor is it effective to signal the kernel to clear its ID map cache.

Also on my home net but not at work, when I use NFSv4 the server exports nothing about half the time, then recovers without intervention, whereas NFSv3 never fails. So I have reverted to protocol version 3.

System Monitor

I also wrote system monitor scripts. The one for autofs picks a partner that's up, and checks if /net/$PARTNER/$MTPT can be accessed, i.e. can be mounted. /home is always exported and it has a copy of /etc/exports at toplevel, so the script knows which filesystems to test. The script also checks autofs access from $HOSTNAME, i.e. from itself, and makes sure a bind mount is being used.

The test script for nfs-server picks a partner and uses SSH restricted execution to run itself there. The partner tries to mount each exported filesystem, and if this fails, the script restarts nfs-server.