Installing Citadel

After installing Citadel I reinstalled it and wrote up the procedure, in case it has to be done over. (Filename is /usr/local/citadel/README.UCLA)

This is Citadel-7.85, installed on 2011-01-27 by jimc. Sources are in simba:/s1/citadel-7.85

Downloaded sources for Citadel and libsieve:

• http://sourceforge.net/projects/libsieve/files/libsieve/2.2.7/libsieve-2.2.7.tar.gz/download
• http://easyinstall.citadel.org/libcitadel-7.85.tar.gz
• http://easyinstall.citadel.org/citadel-7.85.tar.gz
• http://easyinstall.citadel.org/webcit-7.85.tar.gz

Check dependencies, have: libdb-4_5 libical0 libexpat1 libcurl4 libopenssl0_9_8 shared-mime-info gettext-tools

Check devel packages, need: libical-devel libcurl-devel openldap2-devel Have: libdb-4_5-devel libexpat-devel libopenssl-devel pam-devel

Installation directory: /usr/local/citadel which is a symlink to /net/simba/m1/citadel-7.85 .

When configuring, I used --prefix=/usr/local/citadel (which is the default for citadel itself), except /usr/local/citadel/webcit . No other ./configure options were needed except --with-pam for Citadel.

For ./configure I used these arguments:

• --prefix=/usr/local/citadel (except for webcit, /usr/local/citadel/webcit)
• --with-datadir=/var/lib/citadel (for the database and per-user material such as user's photo)
• --with-rundir=/var/run/citadel (for sockets, PID file, etc.)
• --with-pam (just for building Citadel itself)

To get ./configure to work, I needed to set these environment variables so the compiler and linker would find sieve2.h and libcitadel.h in /usr/local/citadel/include, and libsieve.so and libcitadel.so in /usr/local/citadel/lib. I made a wrapper script so as to avoid errors typing them in repeatedly; it sets the variables and then execs all the command line arguments as an actual command. Since the build host has demands on its CPU I use nice -19; most people would not need this.

#!/bin/sh
c=/usr/local/citadel
CFLAGS="-g -O2 -I $c/include" \
        CPPFLAGS="-I $c/include" \
        DEPEND_FLAGS="-I $c/include"  \
        LDFLAGS="-L $c/lib" \
        LD_RUN_PATH="$c/lib" \
        nice -19 "$@"

Compilation (on Simba):

libsieve

--prefix=/usr/local/citadel

configure 8 secs; make 11 secs; install.

libcitadel

--prefix=/usr/local/citadel

configure 5 secs; make 7 secs;

citadel

./configure arguments were:

• --prefix=/usr/local/citadel
• --with-datadir=/var/lib/citadel (for the database and per-user material)
• --with-autosysconfdir=/var/lib/citadel (for citadel.config citadel.control refcount_adjustments.dat )
• --with-rundir=/var/run/citadel (for sockets, PID file, etc.) Note, somehow this specification was ineffective and the sockets and PID file ended up in /var/lib/citadel, which is not a bad location.
• --with-pam (for host-integrated authentication)

Apply CouchNet patches -- patch -p0 < textclient.pat etc. Configure 11 secs; make 30 secs;

When upgrading an existing installation, after make install you need to fix these items in the likely case that you've customized them: (relative paths relative to /usr/local/citadel)

• Custom edited page fragments in messages/*
• network/mail.aliases (if custom hand edited)
• /etc/pam.d/citadel
• Custom hacks to the schemas in openldap/*.schema
• Note, /etc/init.d/citadel is produced by setup, not here, but if you have customized it make sure the directory pathnames match what you told ./configure.
• Do these before running ./setup (described below) because it's going to start Citadel.

webcit

./configure arguments were:

• --prefix=/usr/local/citadel/webcit
• --with-datadir=/var/lib/citadel (for the database and per-user material) (probably irrelevant for webcit)
• --with-rundir=/var/lib/citadel (for sockets, PID file, etc.) (webcit looks here for the Citadel socket)

configure 4 secs; make 15 secs; install

Port assignments on Mathnet:

Setup:

• Shut off competing ports like SMTP because Citadel will freak and try to restart once per second.

• Pre-add to /etc/passwd:

  citadel:x:490:12:Citadel daemon:/usr/local/citadel:/usr/local/citadel/citadel
  

• If you have compiled Citadel to put everything in /usr/local/citadel (which is the normal way) but you want the database to be elsewhere, do the following. With the parameters above, i.e. --with-datadir=/var/lib/citadel the database is automatically in that directory. But someone had a very active site and wanted the database on a separate disc entirely, and this technique was suggested. In ${datadir} (/var/lib/citadel for us, or /usr/local/citadel if you take the defaults), create ${datadir}/data and create ${datadir}/data/DB_CONFIG which says: (#comments OK, fill in the directories you actually want to use)

  set_data_dir	/var/lib/citadel/data
  set_lg_dir	/var/lib/citadel/data
  

• Link /etc/pam.d/citadel to com-aaonly (the one they give you is useless, being for a back-version of Red Hat).

• Install our custom /etc/init.d/{citadel,webcit}.

• In /usr/local/citadel/keys , make symlinks as follows (specific to how Mathnet manages its certs):

  • ln -s /etc/ssl/private/host.key citadel.key
  • ln -s /etc/ssl/hostcerts/host.crt citadel.cer
  • ln -s /etc/ssl/hostcerts/host.csr citadel.csr

  If you forget this you will get some very strange error messages from the clients, since Citadel silently synthesizes a selfsigned cert for cn=*, which any sane client should reject. Citadel and Webcit read these files as root, so no special group membership is needed.

• Make sure everything is ready, since setup is going to start citadel at the end. Execute /usr/local/citadel/setup using these answers:

  Homedir	/usr/local/citadel (is preset)
  Sysop	Bugs Manager (give GECOS, not loginID of bugs)
  Sysop password	leave blank
  Citadel UID	citadel
  Server IP	leave blank
  Server port	Use default of 504
  Auth mode	1 (host integration)
  Disable Postfix	No
  Disable Saslauthd	No
  Retain boot script	Yes

• Execute /usr/local/citadel/webcit/setup, just 2 questions:

  Install directory	/usr/local/citadel/webcit (is preset)
  Retain boot script	Yes
  (Port number	Set in /etc/init.d/webcit )

Checking out services:

• webcit on 2443 -- works.
• webcit on 2080 -- not used because of plaintext password issue.
• citadel on 504 -- implicitly tested by webcit.
• XMPP on 5222 -- works including partner doing chat on Citadel.

Setting up Thunderbird:

• Set up an IMAP account on Citadel. It auto-discovered ports 143 and 587 on external servers; edit to put 143 on Citadel, but leave 587 on Sumac.

• KolabSync can deposit calendar(s) in an IMAP folder. Don't even need a specialized calendar server.

• Install Lightning and SyncKolab in Thunderbird.

• Run Tools/SyncKolab Options (setup wizard).

• This section needs to be fleshed out.

Lingering Issues with Citadel

I'll try to annotate the issues as I fix, or don't fix them.

Coarse grained presence

Pidgin lets you set several kinds of presence, e.g. do not disturb, and custom status messages, e.g. out to lunch. With ejabberd these would be propagated to the partner, but with Citadel you are either available or offline. The transition between these is reported promptly to the partner, but status variants are never seen by the partner.

Restart after Page

In Administration - Restart after paging users, it shows a box titled Message to your Users whose content is didn't find Template [box_serverrestartpage] 21 21, and it doesn't restart.

Doesn't Send Mail

It's supposed to send to a smart host at otter.mine.nu:587. Viewing the outbound queue: connection refused. Guess what, Postfix is not listening on that port. Because that feature is not turned on, because CFT never uses it anyway for incoming mail. (Until now, all mail was outsourced.) Exponential backoff running the queue is good for scalability but a pain for debugging. I changed to localhost:25 and did /usr/local/citadel/sendcommand SMTP runqueue and the message was sent out.

Wrong Host Name

The message was rejected by the recipient because the envelope said it was from jimc@jacinth.cft.ca.us. This is not publicly resolvable. It needs to use otter.mine.nu, which is a public name, although it doesn't accept incoming mail. This is set in Administration - Site Configuration - General. Changing the hostname requires restarting the server. Now the recipient swallows it.

No Tree Structure for Admin Mail Menu

Administration - View the Outbound SMTP Queue: when you get into this page the only way you can get out is to hit a button (like Administration) in the left side bar.

__CitadelSMTPspoolout__ as a Page Title

When you hit Advanced in the left sidebar, the page has this identifier as its page title. The same page title appears on all the pages under Your Info, and likely other dependent pages as well.

HTML Mail, Yuck

How do you make it send mail as text/plain? There was some discussion of this on the support forum.

Invalid HTML Mail, Yuck

The message has some formatting and header problems. Here's what it looks like, omitting Received headers after Postfix got it.

Return-Path: 
Received: from otter.mine.nu (localhost [127.0.0.1])
	by jacinth.cft.ca.us (Postfix) with ESMTP id 7DC3740FE0
	for ; Thu,  3 Feb 2011 22:42:32 -0800 (PST)
To: jimc@pic.ucla.edu
Date: Thu, 03 Feb 2011 22:41:42 -0800
Subject: Test message from Citadel
Message-ID: <0000000136@otter.mine.nu>
From: "Jim Carter" 
MIME-Version: 1.0
X-Mailer: WebCit 7.85
Content-type: multipart/alternative; boundary="Citadel--Multipart--jacinth.cft.ca.us--215d--0004"
X-UID: 185025                                                  
Status: RO
Content-Length: 1072

This is a multipart message in MIME format.

--Citadel--Multipart--jacinth.cft.ca.us--215d--0004
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Citadel@otter.mine.nu -> Postfix@otter.mine.nu -> Laguna, will laguna swa=
llow
it? =20

--=C2=A0
James F. Carter          Voice 310 825 2897    FAX 310 206 6673
UCLA-Mathnet;  6115 MSA; 405 Hilgard Ave.; Los Angeles, CA, USA 90095-155=
5
Email: jimc@math.ucla.edu  http://www.math.ucla.edu/~jimc (q.v. for PGP k=
ey)
--Citadel--Multipart--jacinth.cft.ca.us--215d--0004
Content-type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable



Citadel@otter.mine.nu -> Postfix@otter.mine.nu -> Laguna, will l=
aguna swallow it?
--=C2=A0
James F. Carter          Voice 310 825 2897    FAX 310 2=
06 6673
UCLA-Mathnet;  6115 MSA; 405 Hilgard Ave.; Los Angeles, CA,=20=
USA 90095-1555
Email: jimc@math.ucla.edu  http://www.math.ucla.edu/~=
jimc (q.v. for PGP key)


--Citadel--Multipart--jacinth.cft.ca.us--215d--0004--

    
My objections to the message are these: 

    
    
I wish the user could have a choice to send one, the other, or
    both of the text/plain and text/html parts.  In some circles the use of
    HTML mail gives a very negative impression, and it's a fact that most
    spam (that I see when checking our spam suppression software) is sent 
    in HTML format.  I would turn off the HTML if the choice were available. 

    
Horde/IMP (webmail) puts in a header identifying the submitting user
    and the host from which he connected.  This is very useful when a user's
    password is stolen by a keystroke logger, and I have to identify which user
    it is, even though the body sender is forged, pointing back to the
    spammer's fraud hosting site.  Of course Webcit has to tell Citadel the
    connect host at login time.
        
    

Lame Use of Browser Cache

Using Opera-11.00. The browser takes a long time to render each page, and from tcpdump it looks like it does at least a HEAD request for each of at least 40 elements (Javascript, CSS and icons). The main page has don't cache headers, but the rest appear to have a lifetime of at least 1 hour.

Mandatory Mail Alias

The mail composer wants to send from Jim_Carter@otter.mine.nu, but I need at least a Reply-To and preferably the actual From, saying jimc@math.ucla.edu.

1 Minute Wait to Run SMTP Queue

If you submit a mail message when the queue is empty, Citadel will not ship it out immediately but rather will wait about 1 minute to run the queue. (Based on a sample size of 1 message.) This is not too unreasonable, but . . .