Evaluating SOGo

SOGo versus Ideal PIM Client

Comparing SOGo, mostly via its web interface, with my design for an ideal PIM suite:

PIM Object Types

Supports contacts (vCard), events (vCalendar), tasks (vTodo), but as far as I can see it doesn't do generic notes nor vJournal. The provided web client displays webmail from multiple external IMAP servers. It has semantically appropriate formatting for vCard and vEvent, but it is less advanced for vTodo; it displays them as events.

Multi-User Access

Multi-user access to the PIM objects is a standard feature. Read or write access can be given both to an explicit ACL or globally. I don't think host OS groups can be used in the ACLs.

Network Service to Wild Side

SOGo itself does not provide any offsite service; it's recommended to enforce this with a firewall blocking SOGo's port. Instead it relies on a separate webserver to act as a proxy and feed requests to it. Normally the server is Apache but nginx is also supported. Thus the wild side service is identical to what the host provides for any other web content. In particular, TLS is handled by the webserver using its own host certificate.

Protocols Supported

SOGo's wire format is RFC 2445 (iCalendar). The preferred request protocol is CalDAV. The CardDAV variant is supported for the contact list. GroupDAV is also supported, and a generic WebDAV client can extract the PIM objects, e.g. for backup. (Thunderbird's iCal protocol choice really means CalDAV, this being the native protocol of Mac OS-X's iCal PIM client/server.)

Outgoing mail is delivered by SMTP and is normally passed to a separate mail transfer agent such as Postfix. Incoming mail is stored by a separate delivery agent, and SOGo retrieves it via a separate IMAP server. Although SOGo is not supposed to talk to outside clients, it communicates with the local proxy webserver by normal HTTP.

Web Interface

There is a complete web UI from which all functions can be performed.

Client Interface

SOGo does not include a dedicated client. However, a major use case is through the Lightning plugin for Thunderbird. There is an enhanced version of Lightning with more features and/or better integration with SOGo.

Transitive Authentication

If SOGo is doing authentication itself, it uses the host OS's LDAP service. SOGo itself does not do transitive authentication. However, it can be configured to believe in authentication performed by the proxy, and Apache has several transitive authentication modules including X.509 and GSSAPI. If SOGo does the authentication and if conditions are met which need more investigation, it will have a Kerberos ticket for the authenticated user and can use it to get service from the IMAP mail server.

Client Support

Assuming correct client and server configuration, the iPhone's native PIM client can utilize the SOGo server for contacts, calendar, and (presumably) tasks. Thunderbird is the usual dedicated client on desktop Linux and on Windows. Android needs investigation but I'm optimistic.

PIM Object Storage

The PIM objects are stored in a central database belonging to SOGo. Not my preferred arrangement.

Object Extraction

The web UI has an easy control to export an entire address book or calendar (to be saved as a file by the web browser). It looks like the right procedure for exporting a single object is to create a temporary address book or calendar, copy the object into it, and export the temporary container.

Now, how do you attach a PIM object to outgoing mail? Surprisingly, importing and exporting mail attachments of PIM objects is not currently supported; you need to go through the above file intermediary. The procedure is detailed later on.

Also for backup, can the server enumerate the objects to be backed up, e.g. users who have address books? How can the backup process authenticate so as to extract those files, which would not be globally readable?

Conclusion: SOGo is the kind of PIM server I am looking for.

Usage Scenario

I'm repeating the trial I did of Citadel, by going through a sequence of operations that a client might actually do. Mostly these are using the web interface, and in fact I'm using the Android browser for the initial part of this test.

General Remarks

A general complaint about the SOGo web UI pages: they all set the body font including the size. I need a magnifying glass to see the glyphs on Android. It is incorrect to set the body font size. For ADA compliance you need to go along with the default size selected by the user, who may be visually impaired or who may have equipment different from what the web designer used.

Also, there is an awful lot of chatter; for example the login page has 2Mb of non-obvious content which comes from the server and most of which is sent back in the login form. This is a burden on a EDGE connection from a mobile phone. Similarly, the various UI pages transfer a lot of material even after it looks like the displayable images and form have been displayed.

Although there is an Installation Guide, there is no written documentation for the users, nor context help in the web UI. I'm afraid that there are functional and important features which my users will never discover, and which even I had trouble to get working, or failed to get working, or never noticed. Jimc paraphrasing a blog posting by Ray Ozzie, a then-new senior manager at Microsoft, dated 2005-10-28: The demand for compelling, integrated user experiences that just work. There is so much hardware and software in our life, we can't spoon-feed each item.

Connect via TLS

The Apache webserver handles the TLS connection. Pro forma, it has been tested with these clients:

• Android
• Desktop Linux (Opera and Firefox)
• iPhone (tested with Citadel)
• Windows (Firefox and MSIE-7)

Logging In

This test is using SOGo native authentication with LDAP. It works reliably.

Compose Webmail

Use the UI to compose a mail message and send it out.

Does composition go smoothly?

Yes, no problems.

Is the mail sent out?

Yes.

Did the MTA use the authenticated mailing feature?

The mail was handed off to the local SMTP server (Postfix). From the work domain authentication is not required or used. From home, Postfix is configured to authenticate (with a X.509 certificate and key) on the work MX and relay the mail through it.

Did SOGo get the sender address right?

It sends from user@realm, using the realm it was configured for. At work this is the correct sender address. In my home setup I use my work IMAP and SMTP hosts. With SOGoMailAuxiliaryUserAccountsEnabled YES the individual user (me) was able to configure nonlocal IMAP account(s) and to specify the from address to be used in connection with mail from that account, i.e. when mail from that account is being displayed.

[NEED TO MOVE] At work each user's LDAP Person record includes, as a local hack, an attribute called mailSite naming the IMAP server host, and given the IMAPHostFieldName = mailSite declaration in the local SOGoUserSources list, SOGo uses this attribute to find the server, and uses the user's login password as a credential to the server.

Did SOGo get the recipient address right?

Yes.

Does the message look like spam to the recipient?

Each user can configure SOGo to send the mail either as text/plain or text/html. vCards in the sender's address book also have a selection for which format the recipient prefers to receive, which I assume (without testing) overrides the sender's global preference.

The message does not have any gratuitous spammy garbage features.

Read Webmail

This worked competently.

• The only fly in the ointment was that SOGo did not have access to the inbox, but it did get access to the folders in the user's home directory. I did the tests by using Pine to watch the inbox, and then I moved relevant messages to a folder.

  I'm sure this can be straightened out, but it would be a showstopper if not.

  Duh, it's using localhost as the IMAP server! Which of course does not have the mail. To fix this requires (A) configuring LDAP with the udata schema, and (B) configuring SOGo telling it which field in the Persons record has the mailhost. I'd better get on the stick.

  When SOGo webmail checks for mail, Pine (and/or the Dovecot server) reports that every message in the inbox was expunged and a whole new tranche (of the same messages) is served up.

• SOGo and Dovecot cooperated to deliver the configured namespace, i.e. it got the folders and not the whole homedir. On the other hand, using UW IMAP as the server and with the namespace setup wacko, SOGo got the whole homedir and there's no way to override the namespace to be used.

• The message index is reasonable.

• Messages are displayed normally and readably. Messages appear promptly, noticeably better than with Horde webmail, even over a not particularly swift DSL link.

• One attached JPEG photo was shown inline. A different one, possibly larger, was represented as a generic icon which, if clicked, opens the photo in a separate tab or window.

• a text/calendar was shown with proper semantic formatting. However, the whole idea of that test was to get the event it contained into my personal calendar, and I couldn't figure out how to do that. According to the developers, that activity currently isn't implemented.

  Here's how to make it work:

  • Turn to the Calendars tab (vs. Date) in the upper left panel.
  • Hit the New Calendar icon. Give the calendar a name; I used Attachments.
  • Cause the selected event to be visible in list view.
  • Right click on the row. Pick Edit Event.
  • Under Calendar, pick Attachments (your temporary list).
  • Hit Save and Close.
  • Again on the Calendars tab, select (left click) Attachments, then right click. Pick Export Calendar.
  • Your browser's save/open dialog should appear. Save the file.
  • Remember to move the PIM object back to the container (calendar) that it came from.
  • Open the Mail view (top nav bar). Hit Write.
  • Compose the recipient(s) and body of your message. Hit Attach.
  • At the right end of the recipient area an empty box appears for attachments. Right click in it and pick Attach Files.
  • Just above the box, an entry widget appears with a Choose button. Use one or the other to provide the name of the calendar that you saved, or on Windows use drag and drop to provide the filename. The path is shown as C:\fakepath\filename.
  • Send out the message.

  Now the recipient logs in to the SOGo web client.

  • Turn to the Mail module and read the message. The attachment is represented by an icon. (For a vCard it may be shown semantically.)
  • Click on the icon. The browser's save/open dialog will appear. Save the attachment.
  • Turn to the Calendar module (top nav bar).
  • In the upper left pane pick the Calendars tab (vs. Date). Select (left click) the calendar you want to import to. Right click. Pick Import Events. A box will appear for a filename, including a browse button. Provide the filename of the saved calendar.
  • Hit Upload. The event(s) in the file are imported. Existing events are not disturbed (unless unknown but probably strict matching criteria are met).
  • Beware, there is a bug in timezone processing; the event was exported with seemingly correct times in the file, but when imported the effect was as if it was in UTC when exported (wrong) but was supposed to be shown in civil time (correct). In other words, in Los Angeles (timezone -0800) it was shown 8 hours earlier than expected.

• If the attachment is not in the web UI's displayable repertoire, a link is shown, and if you click it, your browser will obtain the referent from the SOGo server, and what happens then depends on what mime-types the browser can handle: display natively, open with an external program, or save the file.

Calendar using Web UI

This is all using the web client.

• Create an event in your personal calendar.

  Created, no problems. The UI looks a lot like Lightning. Probably this is a deliberate choice of the SOGo developers.

• Evaluate what fields are under-supported in the provided form.

  I think all the relevant fields are there.

  For category codes, select Preferences - Calendar. You can add your own category codes (use the + icon), you can toss theirs that you find useless (use minus), and you can edit the names (double click on the name). You can colorize categories. Double click in the color rectangle and a color picker will pop up. Click in one of the rectangles to select it (the color patch on the RGB widget). This was one of the operations that I had to try on several days before I figured it out.

• Create all-day and repeated events.

  Seems to work fine. Only one complaint: when I adjusted the start and end dates of a repeating all-day event, the event appeared twice on some days in week and month view, but not at all in day view. In the list view it appeared on the correct days and only once per day. This is believed to be a timezone issue: if created mistakenly in UTC it would be split between two days in any other timezone.

• How do they handle expired events? Can you use your calendar in a kind of diary mode, to record past activities?

  I created an event in the past (2011-02-01) and also one which is going to expire in a few minutes (2011-02-26 21:00). We'll see how long they survive. Both of them survived at least two days.

  It's important to purge old garbage, but also important to be able to preserve (some) old events. I would suggest a section in calendar properties with these choices:

  • Keep events within the last N days/weeks/months/years.
  • Auto purge older events.
  • Notify user, display a list of old events, offer a toss all button or let him select.
  • Events should individually have a purge property similar to this one. In other words, events should inherit the calendar's default, but their purge setting should be adjustable individually.

• Finding events: How is this done?

  There is a search box for the list view. While it did work some of the time, I managed to get it thoroughly confused and after that it wouldn't recover.

• Create a shared or group calendar with some events in it.

  A calendar's properties includes a sharing tab, on which you can designate named users, all authenticated users, or the general public as having access to events by default. I don't know about groups. There is a page of links to the calendar's public or private aspect (described as authenticated user probably meaning any authenticated user, or named users). These links can be given to another user to subscribe to the calendar. Available formats are CalDAV, WebDAV ICS (probably equivalent to GroupDAV), or WebDAV XML (not sure which clients want this: Microsoft Outlook?)

  Events can individually be designated as public, private or confidential. I'm not sure what these actually mean, but my guess is that public events appear in the public aspect, private events are accessible to the named users or to any authenticated user, as configured, and confidential events are visible only to the owner and possibly to the named other users.

  I didn't see any control over whether a non-owner can add events to a shared calendar or whether he can alter existing events. I haven't yet tested what the default behavior is for this.

• Can the individual client (different from the creator) view it and/or create events?

  Hit Calendars (vs. Date). Select (left click) the calendar. Right click. Pick Sharing. Hit the + icon. A sub-dialog will appear with a search box. Type an initial segment of the user's full name; stop when you see your target. Click on it and hit Add. That user appears as an authorized user. Do another user, or close the window when done.

  In the list of authorized users, mark Subscribe User. I'm not sure what this does, but it does not unilaterally give the person a subscription. He has to subscribe (see below).

  In the list of authorized users, double click on a row. A sub-dialog will appear. You can mark can create objects or can erase objects. You can individually select the activities which the particular user can do to events marked Public, Confidential or Private. Presumably (I didn't test) the activities are nested in this order: modify, respond to, view all, view date+time, none. You can get the same dialog for any authenticated user or public access.

  It is necessary to give the user some kind of access beyond none to the calendar, otherwise he cannot even see the calendar to subscribe.

  Trying again. Hit Calendars (upper left corner) - Subscribe to a Calendar. A selection box appears. In the top search bar the down triangle lets you pick Name or Email versus Category (I used the former). Type; it does Ajax pre-queries and shows the full name and e-mail address of matching users (matching only on the full name). When you see your target, click on the little plus sign at the left edge and it says Please Wait. But the promised list never appears.

  Trying again, and giving a search string that actually matches the full name: it promptly says No possible subscription.

  Trying again. This time the recipient was given access, and now the calendar appears in the expansion list. Click and the calendar will be added.

  Having been authorized, the recipient can modify the events. The modification is visible instantly for the person editing, but other users (including the owner) have to repaint the screen to see the update (not surprising).

• Can he view the personal and group calendar at the same time? (Union calendar view.)

  Yes, union view works. You can turn on calendars individually, just like in Lightning.

• Can you move events from one list to another?

  Yes. In the list view (but not week view), right click, pick Edit Event, and find a list of calendars it could be moved to. I didn't test moving to someone else's calendar, but in week view (not list view) if you right click on someone else's event, you are offered the choice to copy (not move) the event to your own calendar.

• Can you add participants? From your contact list?

  Yes, by typing in the e-mail address, but I couldn't see how to add attendees from the contact list.

• SOGo can subscribe to a web calendar. How well does this work?

  I tried and failed to subscribe to a foreign calendar, served by DAViCal, but the problem may be with authentication: SOGo never asked me for the password and DAViCal failed to deliver the content.

  I tried to subscribe to a (public) calendar hosted on the same SOGo instance using web public access. It worked on this for a while, but got a proxy timeout.

• Receive mail with a vEvent object as an attachment. Can you add this event to your calendar?

  This was kind of hard. I couldn't figure out how to attach a vEvent to a mail, nor to export an individual event. What I ended up doing was like this:

  • Select the (entire) calendar, pick Export. This is, or turns into, a link to the ICS calendar aspect. Tell the browser to save it as a file.
  • Edit the file keeping only the one vEvent.
  • Send mail, attaching the vEvent.
  • The vEvent was mailed with mime-type text/calendar, even though file called it ASCII text. Probably the extension was used to guess the mime-type.
  • It appeared as a blank form with a space (no content) for the time, organizer and attendees.

  Trying this again but sending the whole calendar (with 1 event). Again it was text/calendar. This time the event had content: title, time, organizer. The description was something generic, probably because I didn't fill out the description in the vEvent. Attendees is a blank mailto link, whatever that means.

  If I click on the mailto link, nothing happens (duh). I'm not sure what I'm supposed to do with this notice. I was not able to take the event into my calendar.

• Microsoft Exchange has elaborate facilities for accepting or declining events by mail. What does SOGo provide here?

  Probably SOGo has similar commands but I was not able to activate them. Yet.

Calendar using GUI Client

Do these clients behave normally with calendars served from SOGo? (Both viewing and creating events.)

• Lightning plugin for Thunderbird and/or Firefox.
• iPhone calendar client
• Android calendar client -- No way to configure it for other than Google Calendar, hiss, boo!

Contact List using Web Client

The evaluation is similar to the calendar case with these additions:

• Create some contacts in your personal contact list.

  Yes, you can create contacts.

• Receive some mail; can you transfer the sender to your contact list?

  Yes, you can import any of the various mailto links.

• Send some mail; can you attach a vCard? Can you receive that vCard into your contact list?

  (Need to test this.)

• Create a group contact list with some contacts in it.

  You can create an object called a list, which appears in the selected address book analogous to an individual vCard. To add members, select the list in the address book (left click), then right click, and pick Properties. (You get the same dialog after New List.) Hit the + icon to add a member. In the text entry widget, slowly type the first few letters of the full name (display name) of someone in your contact list, and the rest of the name and e-mail address will appear; hit return. I assume that you can continue to type an otherwise unknown recipient. (Hit Save when finished.)

• Compose some mail; can you get recipients from both of these contact lists?

  Yes, you can add recipients from your contact list. If you pick a list object, all members are added.

  From the address book, select an individual vCard or a list object and hit write; a message form opens with that item pre-filled as the recipient.

• Can you declare one list to be the union of several others?

  I don't think so.

• Can you use a contact list in the mode of a mailing list?

  That's what you use a list object for.

Contact List using GUI Client

Do these clients behave normally with contacts served from SOGo? (Both viewing and creating them.)

• Thunderbird -- Lightning is irrelevant; identify the right plugin.
• iPhone contact list client
• Android contact list client, and check from K-9 mail reader.

Task List using Webcit

Basically, the evaluation is the same as for the calendar, with these additions:

• Can the tasks be ordered by priority?
• How do they handle due dates?

The task object for SOGo is handled identically to a calendar entry. There isn't any special task list semantics.

I would prefer to have tasks (with specific dates) appear on the calendar, but also to have a proper task list. Tasks should have more than just 3 priorities. Undated tasks are common.

Questions for the Developers

Webmail: Alternate Roles

I need to send mail from several from addresses, and in fact at home I always send from the realm at work. How does the individual user configure this? Can one of the addresses be set as the default?

See if alternate e-mail addresses does anything. No, that's for recipients, not senders.

Mailing PIM Objects

User A wants to mail a vEvent, vTodo, vCard, etc. to user B. When composing mail, how does A attach a PIM object? If A uses his hacking skills to accomplish this, it is displayed semantically appropriately (tested for vEvent), but how does B take the attached object into his PIM collection (calendar or address book)?

Purging Ancient Events

How do you purge ancient events? How do you configure the survival time? Can it be per event?

Calendar Search Box

Calendar list view, there is a box with a magnifying glass icon and a down triangle which, if clicked, shows a list with one member, Title or Description. When you type in it, it appears to be doing Ajax pre-queries. What is it for? I typed keywords from event titles (with the correct case), and entire event titles, but never got any matching events.

Foreign Mail Accounts

At home my mail is outsourced; at work my IMAP server is not the machine where I'm testing SOGo. I see the configuration option to identify the IMAP host from a field in the LDAP Persons record, but I have not yet gotten that configured. The GUI is set up so it could represent foreign IMAP servers such as Gmail. How can we make that happen for particular users?

Thanks to Pascal Gienger for this tip:

In the config file turn on SOGoMailAuxiliaryUserAccountsEnabled YES. Beware, passwords are stored in cleartext in your sql backend.

Then in Preferences - IMAP Accounts, a + and - button will appear in the panel listing accounts. Hit +. Fill out the form specifying the server. If you leave the password blank... It does not ask the user for a password. It attempts to connect to the IMAP server without a password, which the server rejects. This is poisonous for sogod: log file says ---. Restarted sogod; now the password is auto filled with 9 centered dots, same as for the primary account (which is not the length of my password). Here's exactly what happens when I hit save+close in Preferences, which causes a mail refresh:

• In sogo.log;
  localhost - - [01/Mar/2011:21:22:46 GMT] "POST /SOGo/so/jimc/Mail/0/mailboxes HTTP/1.1" 200 170/0 0.081 - - 0 (client gets messages from primary mail server; there aren't any (correct), but the folders are shown.)
• In process circle circulates, then gives up.
• Hit Preferences; nothing is logged, and the window eventually shows 502 proxy error, timeout reading from remote server. But the SOGo master process is still running, and after 10 minutes it will restart the worker thread, if you don't restart it by hand.
• After 3 minutes maillog on the remote IMAP server says:
  Mar 1 21:25:46 nasturtium dovecot: imap-login: Disconnected: Inactivity (no auth attempts): rip=71.107.56.108, lip=128.97.19.118, TLS handshaking

The cure: change TLS to SSL. What other bad consequences does this have?

In the middle panel if you change your e-mail address, then whenever a folder from that account is being shown, outgoing mail is sent from the configured address.

Task List

So far I haven't found any difference between tasks and calendar events. Is there a real task list view? A simple but effective example is Astrid on Android. If I were using SOGo's tasks at all, I would pick an arbitrary week, and use days to represent categories, and times within the day to represent the order of doing the tasks, kind of like the priority but (manually) accounting for how one step depends on a previous one being finished.