Security Plan for UCLA-Mathnet
James F. Carter
The Mathematics Department has done well in repelling the vast inimical
forces on the Internet, and this plan will incorporate much of what
we are already doing. However, it's helpful to have a written plan,
for several reasons. First, we need to review periodically whether
current threats match what we are defending against, and whether changes
in the system may have reduced the effectiveness of our defenses.
Second, it is easier to improve our defenses when we have the existing
system available clearly for review.
What do we want to prevent hackers from doing? There are four basic
Who are we protecting ourselves against? Most highly publicized attacks
of the last several years have come from viruses, which pick victims
at random, and whose main goal is to reproduce, leaving behind a payload
of more or less subtle vandalism. Random attacks are also mounted
by hands-on hackers, but though the attacks can be more sophisticated,
the goals are pretty much the same; they use victim sites as transfer
points to attack more victims. In contrast, if we have something valuable
and someone knows we have it, he or she can make a targeted attack
on us, and the hacker will be justified in devoting more resources
in the effort than for a random attack. Particularly significant for
us, our targets are mainly of interest to people (students) who have
legitimate access to our machines, who legitimately know all details
of our systems, and who have plenty of time on their hands.
- Hiding data. Our users would prefer to share with others only the
data which they have decided to share. We have an obligation under
FERPA to keep private the information about who is a student and what
classes they are taking. Perhaps a more serious problem is students
trying to steal examinations before they are administered. But in
the department the emphasis is more on sharing data, rather than hiding
- Data integrity. The most likely threats in this area are probably
where a student tries to change his grade in the instructor's online
gradebook, or vandalism of web pages. Also in this area is sending
disinformation by e-mail (e.g. an exam has been postponed) and attributing
it to one of our professors.
- Theft of services. In this category would be for unauthorized people
to use software licensed solely for our use, or to use our computing
power for projects not related to the department.
- Hostile activities. Viruses generally use the infected machine to
reproduce, imposing a considerable load on it. Spam is sent by misconfigured
mail servers utilized by unauthorized third parties. Writeable FTP
areas and virus-type FTP and IRC servers are used to share illegally
copied files. Besides the loss of our resources, to avoid legal liability
for complicity in the hostile activities, we need to exercise due
diligence to prevent them.
A particularly troublesome aspect of this exercise, for me, is to
get a feel for what the attackers can do to us. I discovered a web
site, http://project.honeypot.org/. These people have
set up a variety of instrumented networks of machines, with production
operating systems and tools, to attract hackers. They then analyse
what was done, and post the most interesting exploits in a lesson
or contest type format, so sysadmins can learn how to deal with them.
Here is a summary of these ``scans of the month'' since 2000.
Many of the reported scenarios (not shown) are port scans, where the
hacker looks for a vulnerable system. The successful exploits were
all of vulnerabilities reported in CERT advisories. I have several
- Looking for FTP and Front Page vulnerabilities.
- Looking for an exploitable debug tool in the cgi_bin directory.
- The ``ramen'' worm. Send a nasty string to a wu-ftpd server with the
vulnerability, and get a root shell. Similarly to an unpatched rpc.statd.
- Identifying a Windows IIS server before attacking its vulnerabilities.
- Windows IIS Unicode directory traversal exploit.
- Another rpc.statd exploit succeeded.
- On Windows NT, exploiting the Unicode directory traversal bug, and
an RDS (remote data service) exploit.
- Yet another rpc.statd exploit succeeded.
But a question remains: is there a large class of exploits that can
only be done by an interactive user, which would not be seen in this
collection? I think not, but I cannot be sure.
- A stock, unpatched machine is hopeless. One of the test systems, running
Red Hat 6.2, was broken into within 8 hours of being put on the net.
- A system with the latest patches is not vulnerable to the exploits
- When a new exploit is developed, someone has to be the first to be
hacked with it, but with defense in depth and attention to patches,
it's likely that we will not be the unlucky someone.
- It is not true that ``anyone can get root access on your system''.
Vulnerabilities are quickly used, reported, and plugged. But so many
systems are misconfigured, or hardly attended to at all, that successful
exploits are very common.
This is a plan for security of the computing network. I would like
to mention up front some areas of security which are important, but
which are outside the scope of this plan.
- Patches. We put a high priority on patching reported vulnerabilities
both on Windows and UNIX. Only by doing this can we resist the inevitable
- Firmware. Vulnerabilities are regularly reported in printers and routers,
and we are at the mercy of the vendors in getting patched firmware.
These important devices should be regarded with grave suspicion.
- Backup. Disasters can be natural, e.g. earthquakes, or vandalism in
the course of hacking. We keep multiple backup copies of user data
so it can be restored, often to a point within 24 hours of loss, or
in other cases within a week. We regularly access these backups, and
we know they actually can be used. We ensure that the backups go away
after a specified time, to limit the amount of work we have to do
in response to a subpoena.
- Mail viruses. These generally work by inducing the user to execute
code on a vulnerable machine. We intercept executable content delivered
by our mail server (but not on outside servers downloaded by our users).
We run virus scanning software on Windows machines (which sees mail
from outside as well as inside), and we update the virus signatures
regularly, not relying on user action. The default configuration for
Windows mail readers does not automatically execute the content. (UNIX
mail readers never execute messages.) But we need to make sure that
the users do not voluntarily execute the content, and that they keep
the virus software running.
- Permissions of data. The default permissions should be (and are)
sufficient to protect sensitive data from disclosure or alteration.
When opening up files for sharing, users have to be careful not to
open it up too much. It might be a good idea for us to add a capability
for user-managed groups, so the choice can be more than just ``secret''
vs. ``world access''. A capability for user or group authentication
to receive web data would also be helpful.
- Mail identification. For many years the computing group has had a
goal that all mail originating in the department should be cryptographically
signed. But the available user agents have not been up to the task.
Perhaps the most recent generation can be made to authoritatively
identify senders, cleanly and by default.
- User authentication. In the past there were several nasty incidents
where a user's account at another institution was compromised, and
by remote execution privilege (.rhosts) the hacker got to us also.
We need to insist on better inter-system authentication, for example
by forbidding .rhosts and requiring ssh. We are pretty fascist about
requiring passwords that should be hard to crack, but we could do
better, for example by actually cracking passwords on a regular basis.
We very much need to get rid of NIS for user authentication, because
it makes encrypted passwords available to a locally active hacker
even without root access, but our previous attempt ended in failure.
This needs to be worked on again. The obvious candidate is Kerberos.
- Appropriate computing. Users need to exercise good judgment about
what activities are appropriate for a State agency; for example, it
would not be appropriate to maintain a mailing list for a political
party. It is possible to annoy other users by visual imagery, for
example a pornographic screen background. Users need to remember our
privacy obligations; for example, TA's and instructors should not
post students' names, registration numbers or grades on the web.
The threats we mainly defend against are frequent random attacks from
the global internet, and rare but sophisticated targeted attacks by
insiders with interactive accounts, or who have compromised our user's
account on another system. In some cases the hacker will be executing
as root on one of our machines. Thus ``defense in depth'' is essential:
multiple layers of defense, all of which the hacker has to work through.
Mainly this plan describes the relations between different machines
and subnets, so a sucessful exploit on one machine is contained there.
In the ideal configuration, we would have a perimeter network, to
which we would connect a screening router (passing only selected traffic)
to the PSnet-Internet, screening routers to each of our subnets, and
one or more ``bastion hosts'', which would do most of the interacting
with outsiders including hackers. Thus the bastion host is the most
likely one to be damaged by enemy action. Preferably the perimeter
net would be physically implemented rather than depending on Cisco
VLANs. It could be carved out of a fraction of the 70 or 19 nets,
similar to the two Beowulf clusters. Here is a diagram of such a network.
The bastion host(s) will serve all content that we intend to give
to outsiders, such as web pages and anonymous FTP. Incoming e-mail
(SMTP) will be accepted by the bastion host. In general, outsiders
are not allowed to initiate connections to internal hosts, but internal
hosts are allowed to initiate most connections to the outside. Users'
home directories are on internal file servers and are not accessible
from the bastion host. (This section describes an ideal design and
it is understood that some security aspects may need to be relaxed
due to operational requirements.)
Security flaws are discovered with discouraging regularity in the
firmware of network printers. In this plan all the network printers
are on a separate subnet, probably implemented as a VLAN. It would
not be accessible at all from the Internet, but would be considered
to be as untrustworthy as the Internet by internal hosts. Printers
attached to actual computers would not be affected.
It's important to our users to be able to mount home and software
directories from and to most internal machines. In an ideal system
a strongly authenticated remote filesystem, such as AFS, would be
used. Deployment of AFS is not very likely; instead, Sun's NFS, notorious
for the weakness of its authentication, will probably continue to
be used. NFS has its own host-based access control list, which would
(and does) only allow internal machines to mount the files.
The following table lists the protocols and ports to be passed by
the routers. Services not listed are blocked (until requested by users).
In general, UDP ports for TCP-only services are blocked; however,
UDP services often have TCP fallbacks which are restricted similarly
to the UDP ports. These listings refer to packets that initiate connections;
subsequent packets are generally between arbitrary ports, and have
to be let through. ``Ext'' is the router from the perimeter net to
PSnet and the Internet, ``Sub'' is a router between a client subnet
and the perimeter net when the partner is external, and ``Int'' refers
to the same routers communicating between our client nets through
the perimeter net. Hosts which are capable (e.g. Linux) will individually
implement the ``Int'' rules as well. ``I'' and ``O'' mean the connection
is allowed inbound (Internet to local) or outbound. ``n'' means blocked
in both directions. ``m'' means ``maybe''. ``TPB'' in the comments
means ``a truly paranoid sysop would block this''.
||Possibly selective by type
||Also called ipv6-crypt
||Cisco needs this?
||Multicast Transport Protocol
||TPB; sftp (ssh extension) could substitute
||ssh to home sites is allowed
||We will try to insist on ssh
||Mail exchange via Bastion
||This is DNS (53); UDP and TCP
||To internal server only
||To internal server only
||TPB, presently useless
||Internet can only contact Bastion
||Internet can only contact Bastion
||Internet can only contact Bastion
||Bastion relies on internal server
||TPB; I wish we could block this
||Secure imap is OK
||TPB; I wish we could block this
||Secure POP is OK
||Net printers on separate subnet
||For hosts with printers
||NFS filesystem mount, and other services
||UDP; internal hosts mount Bastion FS
||Reject, not just drop packets
||We have no news server
||Perimeter net has no SMB services
||Encrypted v3 can be considered
||Requires bidirectional connections
||Until we have our own LDAP server
||Until we have our own LDAP server
||X-Windows: require SSH port forwarding
In some cases security has to be sacrificed for useability. The major
modifications we need to make are these. In this discussion it is
assumed that a hacker is executing as root on the bastion host.
- Fractional Subnet Perimeter:
- At present, each subnet has a designated
fraction (e.g. IP addresses from 240...254) which is ``outside
the firewall'', subject to traffic restrictions as listed above for
the external router, and all others are restricted more severely.
The outside hosts are acting effectively as bastion hosts. The purpose
is to allow users free access to their home sites from outside, and
thus we are exposing our most valuable machines to hacking. We need
to make the political effort to educate our users how ``free access''
can be set up to be both better and more secure, e.g. with sftp or
slogin; the goal is to put the home sites under maximal restriction,
- Internal NFS Mounting:
- In general, a user expects to get access
to his/her UNIX and Windows home directories from any machine in the
department. Non-system software (e.g. Matlab) is served from a crossmounted
directory, both on UNIX and Windows. But the root partition and basic
system software are not accessible across the net. In the past there
was a host-based restriction against crossmounting PIC and Math, but
numeric user IDs are maintained carefully so there are no duplications,
and this restriction has been relaxed in the past few years. Since
the bastion host is most likely to be hacked, it would be a good idea
if it could not mount users' home directories.
We would really like to use a more secure inter-host
filesystem mounting scheme, such as AFS. AFS also has the advantage
that readonly volumes (for software) can be replicated with transparent
failover when a server goes down; writeable (home directory) volumes
get a local write-through cache, and files can be exported reasonably
safely to alien hosts.
- Mounting Web Pages:
- Our users expect to be able to create web pages
for consumption over the Internet. Presently they go in $HOME/public_html,
which is exported by NFS to the web server. Users would resist if
they had to go through a high security interface to publish web pages.
One possibility is to export the home directory filesystems (readonly)
to the web server on the bastion host. This is actually done now,
except access is read-write. If that were done, the hacker could read
any user's file. If specifically the public_html directories were
exported, the hacker could only see material intended for public consumption
(plus .htaccess and .htpasswd files), but the administrative work
would be greater. Perhaps a better solution would be for the bastion
host to export (read-write) to the internal hosts a filesystem containing
web directories, one per user named for and owned by the user. To
preserve the illusion of local access, the user could put a symbolic
link to it in his home directory. A similar arrangement could be (and
actually already is) implemented for material to be published via
- CGI Execution:
- This refers to executing a program on the web server
in conjunction with showing a web page or responding to data on a
web form. A number of users have ``vanity'' CGI scripts of low value
such as hit counters, but a few have real operational needs for CGI,
such as mathematical or statistical tools and demo software. But such
software is often written with naive assumptions about security. The
threat is that nasty input could induce the CGI script to execute
arbitrary commands, as if the hacker had an interactive session. I
believe that the SUexec mechanism is strong enough to ensure that
such execution occurs as the user, so that only the user's own pages
could be defaced. Another problem is that execution is on the web
server (bastion host) itself, which gets overloaded when the application
is computationally intensive or goes into an infinite loop. In PIC
40 the students learn to write CGI scripts, so the capability has
to be supported on the server.
- Remote NFS Mounting:
- In the past we have had requests from politically
influential users to let them mount their Mathnet home directories
on a non-Mathnet system. The exported users were relegated to a separate
machine and filesystem, for damage control in case of possible hacking.
We should vigorously resist any future such requests. We should be
pro-active and develop the tools to support remote mounting securely,
such as AFS, and we should provide the client tools for our users
to mount alien filesystems locally.
- Remote SMB Mounting:
- In PIC it has been the policy to allow students,
at least those in the dormitories (on campus), to mount their PIC
home directories on their home computers. Since the dorms are a hotbed
of both virus activity and motivated targeted attackers, this policy
should be terminated. A working and safe substitute should be provided.
We presently have a FTP server product, and we let students deposit
their homework, connect to the server via telnet, and compile their
programs. An alternative might be for the users to execute the sftp
component of ssh on a PIC UNIX machine, and access the home directory
via Samba. This provides a nice drag and drop interface, and so far
as we know it is reasonably resistant to nastiness sent down the connection.
An AFS client for Windows NT is available.
- Remote Printing:
- Network printers cannot be trusted to resist attacks,
and should not be allowed any contact with the Internet. If we decide
to allow printing from alien sites -- and I believe this is an operational
requirement for delivering hardcopy confirmations of financial transactions
from Murphy Hall -- BSD style print requests should be directed to
the bastion host, which would forward the data to the actual printer.
It is possible to enumerate the sending sites on a special case basis.
- Realtime Collaboration Tools:
- Users (faculty, staff and students)
need to be able to use the computer system to collaborate; here are
a few application categories:
- Instant Messenger: For ``talking'' to colleagues in real time, and
keeping track of whether they are available online.
- Calendar Management: To display and alter the times of someone's
appointments. Only the user or a designated secretary should be able
to alter the calendar.
- Group Authorship or ``Net Meeting'': Preparing a document with
each author doing his or her part. The work in progress is visible
at all times. It is particularly useful as a written record of a discussion.
- Discussion Lists: The same kind of thing but the participants post
unitary messages, which are answered by similar postings.
There is software implementing each of these tools, and in general
the software is notoriously insecure. We need to be pro-active in
providing secure variants or alternative implementations of these
functions for our users, and particularly, we need to resist demands
to put the insecure versions on our system or let their traffic through
- Mail Delivery.
- On UNIX it involves running arbitrary user-provided
delivery agents and writing into arbitrary users' files, and thus
it requires a process (sendmail) running as root to ``su'' to the
actual user. Here is a potential for abuse, but since the defeat of
the Morris Worm, exploits against sendmail have been rare. Our mail
model would be that incoming mail goes to the bastion host which relays
it to an internal host, and vice versa; internal hosts would no longer
do SMTP to the global Internet.
- We would, however, like to be more paranoid about
the mailboxes themselves. Placement on the bastion host is not secure
because we need to safeguard mail same as other home directory content,
and also, we don't want spam filters or other mail management software
to execute as the user and mount the home directory on the bastion
host. Thus the mailbox has to be on the home server, preferably in
the home directory itself. We are not happy with the security of the
IMAP and POP3 protocols for moving mail to the user agent (reader).
Secure (encrypted) versions of both protocols exist, and we should
deploy the daemons, and should encourge or require that they be used.
Some user agents such as pine can use ssh to execute a remote IMAP
server specific to the owning user. If all reasonable user agents
can do this, we should consider blocking IMAP and POP3 at the firewall,
or even decommitting the server daemons for them.
> Jim Carter,
- Define with our user community the types of service which have to
function through our firewall, and particularly, get rid of dispensible
services like telnet and remote SMB (Windows) mounting.
- Educate our users about more secure and functionally better solutions
to their problems, such as ssh for remote login and sftp for file
transfer. Make sure sftp works for Windows directories. Investigate
secure mailbox access.
- Move outward-facing services, like mail, web and FTP, to one or at
most two bastion hosts. In part, this is already done.
- Move users' UNIX home machines, and all Windows machines, inside the
firewall. Organize the subnets for special protection of, and against,
network printers. A physically implemented perimeter net could be
- Obtain and install the more secure variants of modern ``groupware''
and educate our users to get the most benefit from them.
- Improve our presently inadequate and insecure user authentication
procedures (e.g. NIS replaced by Kerberos) and inter-host filesystem
export (NFS replaced by AFS).