Installing Kolab

Kolab is not available in the main SuSE distro, but it is on the SuSE Build Service. It appears to be written in PHP. The required or optional packages appear to be these (for the server:Kolab:STABLE/openSUSE_11.2 version group, 884Mb compressed):

• kolab-2.2.2.90_cvs20091219-2.1 : The core server
• kolab-storage-0.4.0-1.1 : Stores data in the user's IMAP (mail) folder
• kolab-filter-0.1.7-1.1 : Server resource management and sender policies
• horde-icalendar-0.1.0-1.1 : iCalendar protocol handler (by kolab-filter, kolab-freebusy)
• kolab-z-push-0.2-5.1 : Handles ActiveSync protocol (from server:php:applications)
• z-push-1.2.2-2.1 : ActiveSync protocol in PHP (for kolab-z-push)
• kolab-webadmin-2.2.0.99_cvs20091125-1.1 : Web based GUI for user interaction and server administration
• php5-smarty-2.6.22-1.1 : Template engine for PHP (for kolab-webadmin)
• kolab-format-1.0.1-1.1 : Converting Kolab data objects from XML to hashes
• horde-browser-0.0.2-1.1 : Data about user's web browser (for horde-framework)
• horde-date-0.1.0-3.1 : Date formatting for Horde (for kolab-format)
• php5-pear-channel-horde-1.0-1.1 : PEAR installation channel (by horde-date)
• horde-dom-0.1.0-1.1 : Convert's PHP4 domxml calls to PHP5 DOM (for kolab-format)
• horde-nls-0.0.2-1.1 : Horde localization (for kolab-format)
• horde-prefs-0.0.3-1.1 : Access user preferences (for kolab-format)
• kolab-freebusy-0.1.5-1.1 : Handles free/busy (presence) info
• kolab-server-0.5.0-1.1 : Server module stores user data in LDAP
• horde-sessionobjects-0.0.2-1.1 : Store session keyed data (for kolab-server)
• horde-ldap-0.0.2-1.1 : LDAP communication (by kolab-server, kolab-storage)
• horde-util-0.1.0-1.1 : Various useful classes (for all Horde stuff)
• cyrus-imapd-kolab-2.3.14-8.1 : The version of Cyrus IMAP daemon approved for use with Kolab
• perl-Cyrus-SIEVE-managesieve-kolab-2.3.14-8.1 : (for cyrus-imapd-kolab)
• perl-Cyrus-IMAP-kolab-2.3.14-8.1 : Perl module to do IMAP transactions with the Cyrus IMAP daemon (architecture dependent)
• perl-Mail-Box-2.070-1.1 : Mail folder manager (for kolab)
• kolab-fbview-1.2-1.1 : Horde plugin (?) that shows free/busy info
• perl-kolab-2.2.2.90_cvs20091230-1.1 (architecture dependent): Perl modules used with Kolab server
• horde-kolab-0.0.6-1.1 : Helper functions for Horde to work with Kolab
• horde-framework-0.0.2-1.1 : Core classes for Horde

Currently available version groups, quoted for the base server package, appear to be:

• kolab-2.2.2.90_cvs20091219-2.1 : server:Kolab:STABLE/openSUSE_11.2
• kolab-2.2.2.90_cvs20100212-2.1 : server:Kolab:UNSTABLE/openSUSE_11.2
• kolab-storage-0.4.0-4.1 : server:php:applications/openSUSE_11.2

These additional packages (not in the main SuSE 11.2 distro) are needed:

• (added to above list)

Trying to install from the Kolab repo,
zypper addrepo -r http://download.opensuse.org/repositories/server:/Kolab:/STABLE/openSUSE_11.2/server:Kolab:STABLE.repo
zypper install kolab

• Will toss imap-2006c1_suse-131.0.i586
• It's going to install 103 packages including amavis and clamav
• PHP startup, got hold of /usr/lib/libc-client.so.2006c1_suse and couldn't find symbol gss_nt_service_name in /usr/lib/php5/extensions/imap.so
• Creates /etc/syslog-ng/syslog-ng.conf.rpmnew for kolab-webadmin
• Creates group pki containing wwwrun ldap cyrus postfix

To do:

• Identify the real keystone package(s). The ones that probably are optional are marked with (*).

  • kolab-filter-0.1.7-1.1
  • kolab-freebusy-0.1.5-1.1
  • kolab-webadmin-2.2.0.99_cvs20091125-1.1
  • perl-kolab-2.2.2.90_cvs20091230-1.1
  • perl-Cyrus-SIEVE-managesieve-kolab-2.3.14-8.1
  • perl-Mail-Box-2.070-1.1
  • perl-Net-Netmask-1.9015-105.1
  • perl-TermReadLine-Gnu-1.16-104.1
  • php5-ctype-5.3.3-0.1.1
  • php5-hash-5.3.3-0.1.1
  • php5-json-5.3.3-0.1.1
  • php5-xmlreader-5.3.3-0.1.1
  • php5-xmlwriter-5.3.3-0.1.1
  • libgda-4_0-postgres-4.1.2-2.5
  • cyrus-imapd-kolab-2.3.14-8.1 (*) RPM doesn't complain when I substitute imap-2006c1_suse-131.0 (UW IMAP) for this.
  • amavisd-new-2.6.2-3.4 (*)
  • amavisd-new-docs-2.6.2-3.4 (*)
  • clamav-0.96.1-0.1.1 (*)

• These appear to not be optional:

  • perl-Cyrus-IMAP-kolab-2.3.14-8.1 (needed by perl-kolab)

• Make sure wanted features like kolab-z-push are installed. (Isn't.)

• Compare provides of Kolab and Mathnet imap packages, see if we can forget about their imap.

• See if the pki group can be made useful; maybe install globally.

Obvious prerequisites are a webserver with either mod_php or binfmt_misc (/proc/sys/fs/binfmt_misc) actiated, and at least the ability to install Horde packages. On Papyrus we have horde-webmail-1.0.6, but installed by cowboy programming. The SuSE Build Service has

• horde-framework-0.0.2-1.1 from server:Kolab:STABLE
• horde-framework-0.0.2-9.1 from server:php:applications
• horde-3.3.5-5.16 from server:php:applications
• kronolith: unavailable (only has dependencies of Kolab)

Status of Horde on Papyrus: It was last upgraded in 2008-03-28. Per the Horde website, the latest version is Horde Groupware 1.2.6 (or Groupware Webmail Edition, same version).

Clearly we need to upgrade Horde. However, that's going to be impossible this month, and I want to make progress on the PIM server, so maybe we should try this on a different webserver.

The webservers having mod_php5 are arachne malibu papyrus zuma; those without it are bamboo01 sumac sunset simba. In reality, binfmt_misc should be sufficient for testing. I'll try this on my personal workstation, Simba.

Configuring Kolab

Having installed the packages, now you have to configure the system. Relevant web links:

• Kolab (Kolab2) wiki start page.

• Kolab2 Installation from Source -- This is the default installation mode, and apparently they're up to version 2.2.4 while we have 0.5.0.

• Kolab CVS Repo -- Has more documents about server configuration.

• Clients for Kolab -- Clients listed as supporting most Kolab features are: Kontact, Horde, Outlook, Thunderbird.

From the first link above: To configure the Kolab server, run kolab_bootstrap. This is a Perl script and is going to ask the following questions:

• /etc/kolab/kolab.globals and /etc/kolab/kolab.conf are important configuration files.

• It expects to utilize these ports and wants you to get rid of existing services on them:

  80	http webserver
  443	https webserver
  143	imap server
  993	imaps server
  110	pop3 server
  995	pop3s server
  25	smtp server
  465	smtps server
  10024	amavis server
  10025	postfix reinjection from kolabfilter
  10026	postfix reinjection from amavis
  9999	kolab daemon
  389	ldap server
  636	ldaps server
  2000	sieve server
  2003	lmtp server

• It plans to run slapcat on the existing LDAP database, and back up the existing content. If not found, it assumes a fresh install. Looks like it can use a remote LDAP server though.

• It will put certificates in /etc/kolab/ca and /etc/kolab/*.pem

• See /etc/kolab/templates for the templates it will use.

• Asks for the server's FQDN.

• Decide: master or slave server.

• Asks for domain name (to be appended to email adr e.g. luser@dom.ain)

• Asks for the base_dn of the Kolab database (LDAP).

• Asks for a password for the manager account.

• Starts up slapd on localhost. But I think you can get it to use a remote LDAP server.

• Uses Net::LDAP to connect to the LDAP server.

• Creates a bunch of LDAP stuff.

• *** Left off at "apache"

Kolab via OpenPKG

It would appear that I stumbled on an obsolete project to make Kokab native in SuSE, which is no longer supported. The officially supported version is hosted in OpenPKG. Key factoids from its FAQ:

• OpenPKG is a virtual distro in which all infrastructure is installed in a special prefix (/kolab in our case). The native OS's binaries, libraries, config files, etc. are not used and are not raped.

• It uses RPM for package management. The packages come from the OpenPKG build service and the app vendor (Kolab).

• The only bone of contention is ports: you either have to kick off the native network daemons, or configure Kolab to use nonstandard ports. (And sockets? Or are they created in a private directory?)

• Hacking Kolab config files: Edit /kolab/etc/templates/* These receive textual substitution and are then copied to operational locations such as /kolab/etc/openldap/ldap.conf when the Kolab server starts up, overwriting hacks in the derived operational files. There's a FAQ item that says to do this. In our version the directory is /kolab/etc/kolab/templates .

• Typical Kolab installation: 200Mb download, 850Mb to compile the source packages, about 3 hours.

So let's let 'er rip on Simba, doing a default installation. We'll hack the configuration later to be integrated with our system. Key steps:

• Kolab2 Installation - Source is the default installation method. The current version really is 2.2.4.

• Deactivate apache and postfix. Use insserv -r so restarter won't try to restart them. Here's a list of the servers and ports that it wants to not be running (of which Apache and Postfix are the only ones on Simba):

  • http webserver on port 80
  • https webserver on port 443
  • imap server on port 143
  • imaps server on port 993
  • pop3 server on port 110
  • pop3s server on port 995
  • smtp server on port 25
  • smtps server on port 465
  • amavis server on port 10024
  • postfix reinjection from kolabfilter on port 10025
  • postfix reinjection from amavis on port 10026
  • kolab daemon on port 9999
  • ldap server on port 389
  • ldaps server on port 636
  • sieve server on port 2000
  • lmtp server on port 2003

• Make the residence directory (1.5Gb free space needed, so it says):
  mkdir /m1/kolab-2.2.4
  ln -s /m1/kolab-2.2.4 /kolab
  mkdir -p /kolab/srcx

• Download the source from CVS. Use http://files.kolab.org in Germany; there are mirrors in Belgium, another German site, and United Arab Emirates.
  wget -r -l1 -nd --no-parent http://files.kolab.org/server/release/kolab-server-2.2.4/sources/

• Or by rsync: List first, then if looks reasonable, download.
  rsync -L -rltzv rsync://rsync.kolab.org/kolab/server/development-2.2/current/sources/
  rsync -rltzv --partial --copy-unsafe-links rsync://rsync.kolab.org/kolab/server/development-2.2/current/sources/ /kolab/srcx

  You get a (list of a) pile of symlinks to source RPMs, for these package sets: Horde, Kolab itself, PEAR and PHP5 infrastructure, amavisd, Apache, clamav, build environment (binutils, make, gcc, etc. etc.), openldap, openpkg, openssl, perl (and modules), postfix, SASL, spamassassin, sqlite, etc, etc.

  Downloaded (rsync) 381Mb at 7.7Mbyte/sec (62Mbit/sec), took 50 secs. It would be a lot longer on a home DSL line.

  Follow instructions for how to check integrity. The key hex on the wiki is not up to date; you need to get the current key. Everything was OK.

• Start the installation:
  nice -19 sh ./install-kolab.sh >& /tmp/kolab-install.log < /dev/null &

  It appears to be using these host system build tools: tar (or gtar), make (or gmake), cc (or gcc), ar, ld, as, strip, /bin/sh.

  Started about 15:10, finished 16:43 (93 mins) on a Dell Optiplex GX-755 with Intel Core 2 Duo E8400 at 3GHz.

• What you get:

  • /kolab/RPM/PKG: 130 RPM package files, 357Mb (units 2^20 bytes), plus symlinks to the source packages.

  • The packages are installed. Disc space allocation:

    • 1725 Mb: Total
    • 376 Mb: Source RPMs (don't have to be in /kolab directory)
    • 374 Mb: Binary RPMs (units of 1000 * 1024 bytes)
    • 975 Mb: Everything else

  • You use /kolab/bin/openpkg command args to execute things.

    • rpm
    • rc (to start a service)
    • man (it has a few man pages, not very useful)
    • openpkg --help to see all the commands

• Run the initial configuration command:
  /kolab/sbin/kolab_bootstrap -b
  (You can't capture the output through tee; there must be a block buffered program and you can't see what it's asking for.) Here is the list of configuration choices that I made. * indicates that the default was correct already.

  • This host's FQDN: simba.math.ucla.edu (*)
  • Master or slave: 1 (master, *)
  • Mail domain (to append to unqualified loginIDs): math.ucla.edu (*)
  • LDAP DN corresponding to the mail domain: dc=math,dc=ucla,dc=edu (*)
  • Base DN is the same; bind_dn is cn=manager,cn=internal,dc=math,dc=ucla,dc=edu (auto generated)
  • Password for user manager (bind_pw): the bugs password
  • FQDN of slave server(s): None, press return.
  • (It creates LDAP database and initial service configuration files.)
  • Do you want to create a CA and certificates? No (use our own).
  • (Initial configuration is finished.)

• Start Kolab and its various related services:
  /kolab/bin/openpkg rc all start
  Problems:

  • OpenLDAP and Apache did not start. Apache complained that there was no certificate file; LDAP also needs one and that is probably the problem.
  • Make symlinks from /kolab/etc/kolab/cert.pem and key.pem to the actual certificate and key. Also change the permissions so group kolab-r can read the key, including the containing directory which most likely is 700. This is simple on Simba but not on a production server, probably POSIX ACL will be needed.
  • Insert Mathnet's CA cert into /kolab/x509/public-ca.crt.pem That fixed it: Apache and LDAP do start.
  • The admin page was not served. Scavenger hunt for the log files: /kolab/var/apache/log/apache-error.log and apache-access.log. Problem is transport: need to start the VPN.

• Do subsequent configuration; navigate to the URL https://simba.math.ucla.edu/admin/ and log in as the user manager. Choices made were:

  • Must create a recipient for administrative mail first. jimc on Nasturtium. You need to tell it your password (for LDAP). I left all the user info stuff blank (title, physical address...)
  • Turned off Amavis virus scanning.
  • Privileged networks: localhost, 4-net, 19-net, 70-net.
  • Relay host: Sumac
  • Did not turn on Accept Internet Mail.

• Housebreaking Kolab: it's necessary that Kolab coexist with our existing system, meaning particularly that the host's Postfix and Apache should function. For this purpose:

  • I stopped Kolab's Postfix and Apache, and started the host's.

  • Rewrote /etc/init.d/kolab to have LSB dependency comments and to support restart and status actions. The status action is to print an excuse-type message and exit with status 0 (success). It looks like you're supposed to execute something with an argument of status and a filename of /kolab/etc/rc.d/rc.$service . Try: /kolab/bin/openpkg rc all status and look for output like ${service}_enable="yes" and ${service}_active="no".

    Update: I wrote a status section in /etc/init.d/kolab that parses this output and gives a reasonable return code. /kolab/bin/openpkg rc can never decide if openpkg is active, and kolabd won't start yet, so these were bypassed in the script.

  • Hack files in /kolab/etc/kolab/templates to customize ports:

  • Postfix: Hack master.cf.template. lmtp is already on 11565 (default is 24). We have smtpd on @@@bind_addr@@@:smtp (need to specify bind_addr). There's also one on 465 (deprecated smtps), 10025 (loopback from kolabfilter), 10026 (loopback from Amavis). We'll kill the one on 465, and change smtp to 11025.

  • Apache: hack httpd.conf.template. Changed port 80 to 11080 and 443 to 11443.

  • OpenLDAP: Doesn't seem to be running. Change port to 11389 . This isn't done in slapd.conf -- you need to start slapd with a command line option: -h "ldap://$FQDN:11389". May be a space separated list. Possibly the FQDN can be a null string, definitely it can be an IP address such as 127.0.0.1. 0.0.0.0 also works. See /kolab/etc/rc.d/rc.openldap openldap_url . Better yet, see /kolab/etc/rc.conf and its template copy. [Done.]

  • Trap for the unwary: /kolab/etc/kolab/kolab.globals also has the LDAP port. [Hacked]

  • saslauthd is already using port 10087. Host system only runs SASL on a UNIX domain socket.

  • Imapd: notify socket is on 11571.

  • Amavis: already on 11147; we don't want to run this.

  • Clamav: already on 11354; we don't want to run this.

  • To transfer the hacks to the operational files you need to run... See ./lib/perl/vendor_perl/5.10.0/Kolab/Conf.pm : Changes can be activated by running .$Kolab::config{'kolabconf_script'}. Per etc/kolab/kolab.globals this is /kolab/sbin/kolabconf .

  • When running kolabconf: Error loading configuration. Maybe the LDAP server is not running. Duh, it is neither running locally nor has the master server had the Kolab schema installed.

  • Let's try running kolab_bootstrap -b again. This clears the LDAP table, sets a new host FQDN and mail domain, gets the admin password, and runs kolabconf (with its own fake LDAP server). -b is required to get this latter behavior. It also removes /kolab/etc/kolab/{cert,key}.pem so re-create the symlinks to the real cert files.

  • Try starting kolab: /kolab/bin/openpkg rc all start

    • It apparently successfully started these enabled services: apache openldap postfix sasl
    • These started and the daemons are running, but they are not listening to their assigned sockets: amavis clamav
    • While these may have started initially, the daemons are not running now: imapd (was using port 143 which is occupied).
    • These are not active even though enabled: kolabd (wrong LDAP port)
    • These are not active and are not enabled: spamassassin
    • It says openpkg is enabled but activeness is unknown.

  • Configuration issues discovered:

  • /kolab/etc/imapd/imapd.conf -- ldap_uri: ldap://127.0.0.1:389 should be either port 11389 or sunset:389.
  • There is a ldap_password, I'm not sure for what, but Sunset isn't going to honor it.
  • I don't see where to set imap's own port number. Try listen 0.0.0.0:11571. Why are we using 11571 and not 11143 or 11993?
  • See this howto about Cyrus IMAP. We should be looking at /kolab/etc/imapd/cyrus.conf . It needs to say:

    		SERVICES {
    		imap cmd="imapd" listen="$ipaddr:$port" prefork=0
    		}
    		

    (substituting values for $ipaddr and $port). Presently it listens to 0.0.0.0:143 which of course dies. Also 993 and 995. I modified the template /kolab/etc/kolab/templates/cyrus.conf.template to use ports 11143, 11993, 11110, 11995 (IMAP and POP).
  • Modified kolab.conf.template to use port 11389 rather than 389 for LDAP. Also slurpd_port: 11389 (was 9999). I don't know why they have both.
  • Chicken and egg prob: /kolab/sbin/kolabconf uses /kolab/etc/kolab/kolab.conf to find the LDAP server, which it has to do before rewriting kolab.conf from the template. Hand-edit that file. Well, it still gets an error even though the LDAP server is listening to the port.

  • More on the chicken and egg problem: you need to change 389 to 11389 in both of:

    • /kolab/etc/kolab/kolab.globals
    • /kolab/etc/kolab/kolab.conf
    Now you can run /kolab/sbin/kolabconf successfully. And now kolabd can start! But there are a lot of other files containing ldap://127.0.0.1:389 which kolabconf does not regenerate, such as /kolab/etc/kolab/kolabfilter.conf .

• Now logging in to the Kolab web interface for the first time. See previous section where this was done. Connect to https://simba.math.ucla.edu:11443/admin/ and log in as the user manager, password is the current bugs PW. Setup steps (copied from previous setup attempt):

  • Must create a recipient for administrative mail first. mathtest on Nasturtium. You need to tell it your password (for IMAP). I left all the user info stuff blank (title, physical address...)
  • Turned off Amavis virus scanning and Sieve mail filtering. Remaining enabled: POP3 (SSL on 995), IMAP (on 143), IMAP (SSL on 993).
  • Privileged networks: localhost, 4-net, 19-net, 70-net.
  • Outgoing relay host (smarthost): Sumac
  • Did not turn on Accept Internet Mail.
  • I also created jimc as an ordinary user and added jimc to each of the 5 admin distribution lists.

• Logging in as jimc: the URL is https://simba.math.ucla.edu:11443/client/

  You get a Horde login screen, .../imp/login.php . Login fails (wrong password). Logging in as manager: it doesn't reject the password but goes right back to the login page.

• We seem to be getting output from cron jobs where the command is: /kolab/etc/rc all quarterly ; output is fatal: Unable to connect to local Cyrus admin interface. This is for kolabd specifically. In /etc/crontab.

• Next step would be to compare the LDAP content on 11389 with the real LDAP tables on Sunset:389. Propagate Kolab-specific stuff to Sunset. See if it will be happy using Sunset's LDAP. Go through all the other conf files and change the LDAP URI to whichever one we're using.

• Guess what, kolabd wants to open a socket on 11389 and gets upset when slapd is also using it. Test command line: ldapsearch -H ldap://localhost:11389 -x -z 3 It times out and gives up, not like on a real LDAP server. And now I can't start openldap, nothing in syslog about why. I did insserv -r /etc/init.d/kolab and left it for next week.