Valid HTML 4.01 Transitional

Disentangling Kerberos from LDAP

James F. Carter <jimc@math.ucla.edu>, 2014-11-03

Former versions of MIT Kerberos could not do incremental propagation, so we set up various substitute schemes. The most recent was to use the LDAP database backend and to rely on LDAP incremental propagation.

In OpenSuSE 13.1 we have krb5-1.11.3 dated about 2013-11-xx. It would appear that MIT Kerberos is now able to do incremental replication. So we want to revert from the LDAP backend to native files, turning on native incremental replication.

About examples in this writeup: the hostnames and realm refer to jimc's home network. The reader will need to substitute his own realm and KDC hostnames. Also some steps refer to local features such as /m1/custom/restarter.conf , which are absent on a stock OS. This SuSE system uses systemd (systemctl) for booting and system process control. If you have LSB startup scripts (e.g. /etc/init.d/kerberos) or upstart, substitute the appropriate commands for your process control framework.

Research on Features

kprod propagation daemon: This is for Kerberos version 1.14 (2014) but has not changed a lot since 1.11. kpropd runs on the slave and is normally socket activated. In conventional propagation a cron job on the master executes kprop occasionally, sending a full dump to kpropd.

When using kprop I hit this error: 

kprop: Server rejected authentication (during sendauth exchange) 
while authenticating to server
Generic remote error: Wrong principal in request
The master authenticates as host/$MASTER@$REALM, and the slave needs this principal's key in its default keytab -- with the same key version (KVNO). I failed to use the -norandkey option of ktadd (in kadmin) when extracting the key to the slave's keytab.

If incremental propagation is enabled, kpropd runs continually on the slave and periodically polls kadmind on the master. kdc.conf parameter iprop_enable turns it on; iprop_slave_poll is the poll interval, default is 2 minutes. The slave's keytab must include kiprop/$SLAVE@$REALM and /var/krb5kdc/kpropd.acl needs the principal as which the master is sending the updates.

If incremental propagation is enabled, kprop can no longer be used. The symptom is this error mesage: kprop: Software caused connection abort while reading response from server. This message means that kpropd received the dump that kprop sent, and then execed kdb5_util load -i $kdir/from_master, which returned a nonzero exit code. Various issues could make kdb5_util fail, but in this case the -i option means to look for the serial number of the last entry in the update log that is included in the dump, so kpropd knows where to start polling from, but the dump lacks the serial number.

To debug propagation the program kproplog is handy. In case you don't have convenient symbolic links, its full path (for SuSE) is /usr/lib/mit/sbin/kproplog . kproplog -h gives just the serial numbers and timestamps of the first and last updates. On the master only, kproplog -e 5 -v gives more details of the last N updates.

Table of parameters for incremental propagation. For incremental propagation, the default poll interval is 2 mins, and there is a lockout of 10 secs; if the DB was modified this recently the slave (kpropd) will pause and retry. If a principal's policy is altered, it cannot be propagated incrementally and a full update will be done.

Tidbit: For cross-realm trust, MIT recommends the password be 26 bytes of random ASCII text. Assuming this means an alphabet of 96 choices, that would be about 168 bits.

Tidbit: How to destroy a realm in LDAP: You have to do this while the krb5.conf file has the LDAP parameters; otherwise the error message will be LDAP container not specified or something like that. Do it on any server and rely on LDAP replication to delete entries on the other servers. You will need to type in the password from ldaproot, generally stored in /etc/ldap.secret and/or /etc/openldap/ldap.secret. (It doesn't know about -y.) For a big realm it may run as long as a minute. 

    kdb5_ldap_util -D $rootdn -H ldaps://hostname destroy -r REALM
To test if it's gone (do before and after) (it will print nothing, if the realm is gone): 
    ldapsearch -x -H ldaps://jacinth.cft.ca.us \
    -D uid=ldaproot,dc=cft,dc=ca,dc=us -y /etc/ldap.secret \
    -b cn=CFT.CA.US,cn=Kerberos,dc=cft,dc=ca,dc=us -LLL \
    '(krbPrincipalName=jimc@CFT.CA.US)' krbLastSuccessfulAuth

Outcome: it says kdb5_ldap_util: Realm Delete FAILED: Operation not allowed on non-leaf deleting database of 'CFT.CA.US'. But the stuffing is gone even so, on all 3 servers due to LDAP incremental propagation. I think it tried to remove the realm container before it was empty. It left these objects in place, all of which I formerly created by hand:

Procedure

These web pages describe the procedure to set up a new KDC:

I re-created new KDCs with a native file backend, copied the payload to them, and then removed the LDAP instances.

Converting the Master KDC

Converting the Slave KDCs

A recommended strategy is to configure the master and slaves identically. This means that if the master dies you can edit /etc/krb5.conf specifying a different machine as the master, and Kerberos will pick right up until you can order a replacement. But this means, for example, that kadm5.acl needs permission granted to all slaves to suck updates, but the master should be included in case of a future role switch. There are several places where permissions and credentials are included that are not strictly necessary, just so the configuration can be the same on all KDCs.

Cleanup Steps

Setting Up Incremental Propagation

See: Incremental Propagation Parameters

Issues During Testing