Motorola Droid 3
Settings and Setup

Change history:

• 2011-12-24: Originally for my upgrade to CyanogenMod-6 Froyo.
• 2012-03-30: Revised and split up for CyanogenMod-9 Ice Cream Sandwich.
• 2012-08-27: Revised again for upgrade to CyanogenMod-10 Jelly Bean

Setup After Upgrading

These notes are very specific to jimc's preferences and configuration, but other readers can get an idea of the procedure which I've found to be efficient.

Since CyanogenMod-9 for Droid 3 is in alpha test mode, new distro images will be frequent, and usually it will be necessary to wipe data, to prevent inappropriate settings from the previous version from messing up the new one. After you have installed the new CyanogenMod, follow these procedures.

Setp Wizard

• When the boot animation shuts off, the user interface runs setup, which results in a box saying Complete this action with com.android.provision.DefaultAction or Setup Wizard. They're really the same, but for luck let's pick Setup Wizard.

  Simultaneously or immediately after you pick it, a box pops up saying Unfortunately com.android.phone has stopped. Dismiss the box. Sometimes this happens once more, but for me it starts and stays running on or before the third try. This behavior is new in CM9-01-30 and I hope it gets fixed later.

• Setup wizard wants you to pick your language. Then hit Next.

• Give your loginID@gmail.com and the password for your Google account. You could also create a Google account at this point, though jimc recommends pre-creating it using another computer. Setup in CyanogenMod also lets you bypass this step entirely, although you need the Google account to use the Market app.

• It has to do stuff on the cloud server. Give it time.

• On the next screen you're offered the chance to join Google+. Not a chance. Hit Next.

• Do you want to back up your stuff in the cloud? Consider your security policies carefully here. Jimc insists on backing up to my own cloud server, but many other people think nothing of storing their personal files on a service with no service level agreement, whose business model is world domination through advertising. The prudent user will un-check both boxes and hit Next.

• Google Location Service. While a lot of people are paranoid about revealing their point of physical presence, I am a little looser in this category than for the backups, and left both location boxes checked. These are to make your location available to apps, and to include your location in Google search queries. Hit Next.

• They say they personalize your user experience by using your full name, although I haven't yet seen any of that. If you're paranoid you could give a screen or gaming alias. Hit Next.

• Setup is complete. Hit Finish.

Basic Settings

Next step is to set a few settings to make the UI a little less annoying while you restore your apps and data. Hit Menu - Settings and pick:

• Display

  • Brightness (automatic is on by default, good)

  • Automatic Backlight

    • Enable light sensor filter (turn on)
    • Allow light decrease (turn on). Inability to turn down the brightness was one of the more annoying features of the stock Verizon image and, presumably, standard Android (Gingerbread and Ice Cream Sandwich).

  • Sleep, set to 5 minutes.

• System

  • Date & Time

    • Use 24 hour format (turn on)
    • Select Date Format: 2012/01/31, hiss boo.

  • Developer options

    • Android Debugging (turn on)
    • Stay awake when charging (turn on)

Reinstalling Apps and Data

On your laptop, un-tar your backups of /data/app and /data/data . For people just getting started with UNIX/Linux, here's the procedure. This is copied off my screen using my filenames, and you will have to substitute your own.

mkdir -p /home/scratch/restore-0130 #Make a place to un-tar the backups.
cd /home/scratch/restore-0130
mkdir app # Extracting downloaded apps
cd app
tar xf ~jimc/upgrade/selen/files/data.app-0130.tar
cd .. # Return to containing directory, extract data
mkdir data
cd data
tar xzf ~jimc/upgrade/selen/files/data.data-0130.tgz # This one is compressed
cd ../app # Get ready to restore the apps.
ls -1 > ./install.sh # Makes a list of your apps
chmod 755 ./install.sh

Use your favorite editor on ./install.sh and exclude several items that are already installed or have other contraindications. Likely the already installed apps are specific to me, being carried over from Gingerbread. There's no reason to preserve them.

• com.android.vending-1.apk -- Not already installed, but scary
• de.android.telnet-1.apk -- Not sure what this is
• install.sh -- Not an app :-)
• com.google.earth-1.apk -- Installed (/system/app/GoogleEarth.apk)
• com.koushikdutta.rommanager-1.apk -- Installed (/system/app/RomManager.apk)
• jackpal.androidterm-2.apk -- Installed (/system/app/Term.apk)

Now turn the list into a useful script, along these lines.

#!/bin/sh
apps="
com.amazon.kindle-1.apk
com.eclipsim.gpsstatus2-1.apk
etc. etc. Remove items to be excluded.
"
for a in $apps ; do
echo === Installing $a
adb install $a
done

Execute the script. I always capture the script output and check carefully later to see if anything strange happened. On the phone, the apps all end up in /data/app owned by system:system, and the app's data directory is created, /data/data/${APP} owned by its paranoid UID and group, mode 751.

./install.sh 2>&1 | tee /tmp/logfile

Now the real trick is to restore the data. In Android each app has its own user ID and group, as an aid to keeping a hostile app from stealing data from others and sending it to the mother ship. The UIDs are assigned in an arbitrary order which changes from one OS version to the next, so the UIDs in your backup will mostly be wrong, and most of the apps will not be able to read or write their data. Here's how I automated fixing the UIDs.

First, only a small subset of the data will be restored: I'll exclude apps that I didn't configure and didn't save a useful history in, and I'll also exclude apps that I put some investment into, but which I suspect have been upgraded in such a way that it would be dangerous to carry over settings from the previous incarnation. I have 129 directories backed up, but I'm restoring only these 27:

• com.amazon.kindle
• com.android.bluetooth
• com.android.browser
• com.android.contacts
• com.android.deskclock
• com.android.keychain
• com.android.providers.calendar
• com.android.providers.contacts
• com.cadreworks.bible_le
• com.eclipsim.gpsstatus2
• com.fsck.k9
• com.googlecode.andoku
• com.happydroid.bookmarks
• com.neddashfox.nightclock
• com.whatsapp
• com.xabber.android
• eu.kowalczuk.rsync4android
• jp.sblo.pandora.jota
• net.healeys.lexic
• org.connectbot
• org.dmfs.caldav.lib
• org.dmfs.carddav.sync
• org.linphone
• org.thetomahawk.spreadsheet
• ru.org.amip.ClockSync
• tkj.android.homecontrol.mythmote
• uk.co.nickfines.RealCalc

Specifically I excluded these items which look too scary to restore:

• com.android.phone
• com.android.providers.settings
• com.android.providers.telephony
• com.android.provision
• com.android.vending
• com.android.vending.updater
• com.koushikdutta.rommanager
• com.noshufou.android.su

Create this script (suggestion: on your laptop and copy it). On the phone put it in /data/data and call it untar-data, mode 755. The next script expects to find it there. Link to premade script, but I'm not giving any warranties etc.

#!/system/xbin/sh
# $1 = name of tar file, from "tar cf $1 $2"
# $2 = basename of data directory.
# Copy this script to /data/data (executable)

cd /data/data
if [ ! -d $2 ] ; then
echo " On phone: missing directory $2 skipped"
exit 4
fi
uid=`ls -ld $2 | awk '{print $2}'`
if [ -z "$uid" ] ; then
echo " On phone: can't determine UID of $2"
exit 4
fi
tar xpf $1
# We chown $uid $1 because xargs runs the command once even with no input.
find $2 ! -type l ! -user system -print | xargs -n 25 chown $uid $1
find $2 ! -type l ! -user system -print | xargs -n 25 chgrp $uid $1
rm $1

(end of script)

Create this script, which I called send-data. Link to premade script, but I'm not giving any warranties etc.

#!/bin/sh
apps="
uk.co.nickfines.RealCalc
etc. etc., fill in the list of app directories to be restored
"
tfil=/tmp/app.tar
for a in $apps ; do
echo === Restoring $a
if [ ! -d $a ] ; then
echo " Missing directory skipped"
continue
fi
tar cf $tfil $a
adb push $tfil /data/data/app.tar
adb shell "/data/data/untar-data /data/data/app.tar $a"
done

(end of script)

Execute the script. You have to change to the data directory first because the script looks in the current directory for the stuff to be restored.

cd ./data
../send-data |& tee /tmp/errs #I put the script in the containing directory

Check for problems, fix, and restore them again. Edit the list of apps to include only the ones you're going to restore again. I had seven apps with weird symlinks, probably broken in a previous unsuccessful restoration attempt.

Here are the outcomes when I tried each app:

Setting the Settings

Now that the apps are restored it's time to set the rest of the settings. The current version of CM9 has fewer settings than CM7 (Gingerbread) had, and likely some of those will be brought forward in future versions. I've tagged these CM7 in these notes. Also, I'm listing only settings that I changed, with only a few exceptions.

• Wireless and Network:

  • WiFi: On; and configured access point. Note: if you have WiFi on and are not in range of a connectable access point, the OS will do repeated wireless scans, which eats battery: almost half of capacity in 8 hours. Use your widget (see below) to turn it off.

  • VPN: Need to test this, but later. See below for procedure.

  • System Select: choice of Home Only (default), Affiliated Networks, and Automatic. In CM7 I tried Automatic; this did not spring the phone to Sprint. Later on Verizon I left it at the default, which has changed to Automatic.

• Call Settings: (I can't find this section in CM9)

  • Voicemail Settings: leave on My Carrier (CM7).

  • Defaults are to vibrate on answer and hangup. I turned on keep screen awake and show nick/organization. (CM7)

  • Left hand mode: turned on. I'm not sure what the difference is, but that's how I hold the phone when calling. (CM7)

  • Evidently the default behavior when a call comes in is to start the call UI in foreground. You can have it stay in the background until summoned. I didn't turn this on.

  • There is a section to blacklist particular numbers. (CM7)

• Device Section:

  • Sound Settings:

    • Phone Ringtone: Pyxis. Other non-music tones are Digital Phone and Bell Phone.

    • Notification Ringtone: Missed It is the default; this is the same as in standard Android. Some others might be better.

  • Display:

    • Automatic Backlight: This submenu is mostly or all about controlling the light sensor.

      • Enable (turn on).

      • Allow Light Decrease: Turned on; this was one of my complaints about the Verizon image, that it wouldn't decrease the light.

    • Sleep: change to 5 minutes, default is 1 minute.

    • Screen-on animation: turned on (CM7). It's unobtrusive and adds to the richness of the user experience :-)

  • Interface Section:

    • Launcher (Trebuchet)

      • Homescreen

        • Homescreens (number of panels): default is 5, I changed to 3.

        • Search Bar: Turned off. This is the Google Search widget which is annoyingly ubiquitous.

      • System

        • Notification Drawer: Settings for what appears when you open the notification window.

          • Widget Buttons: I turned on Media Play/Pause, Airplane Mode, Bluetooth, Sound, Wi-fi. For Sound Modes (at the bottom): I changed to vibrate, sound+vibrate.

        • Status Bar:

          • Battery Status Style: I set percentage. It's now just the number, not superimposed on the icon. (This in CM7; not yet obeyed in CM9-01-30.)

          • Alternate Signal Display: Choices are bars (default), number (what I chose), number with dBm label. (This in CM7; not yet obeyed in CM9-01-30.)

        • Wallpaper: You can also get to this menu by long pressing in the home screen background, or it's on the menu key's menu. For CM7 I picked Cyanogen wallpapers #2, the skating droid on colored shale. This is missing in CM9, and I kept the default, called Phase Beam from Live Wallpapers.

          Here's the procedure (for CM7) to use an externally generated picture, specifically android-2.1/packages/apps/Launcher/res/drawable-hdpi/wallpaper_ sunset.jpg (or any version that I have from 1.5 to 2.3):

          • At least for the 480x320 screen (MDPI) of the Dream, the photo should be 640x480px in size. JPEG is acceptable; I don't know which other formats it can handle. If the size or aspect ratio is wrong, the Gallery app will let you crop or scale it.

          • Copy it to the SD card's VFAT partition, possibly in any subdirectory.

          • Hit the Menu key; Wallpapers; it asks complete action using file manager or Gallery. I tried the file manager, but after I selected the item, the response was not recognized and the dialog box popped up again. Trying Gallery:

          • Give it a moment to search for displayable photos. Find the one you want. Tap it.

          • The gallery app will show a larger image with a crop rectangle. Drag the edges to cover the entire photo, unless you actually want to show only part of it. Hit Save.

          • It will scale the cropped image and save it to some hidden place, and use it as the desktop wallpaper.

    • Personal Section:

      • Security

        • Lockscreen Options (in Screen Security):

          • Screen Lock (style): Secured with pattern. A non-null lock style is required to use the X.509 certificate storage.

          • Automatically Lock (how long after the screen is turned off for inactivity, to require the password etc.). Default is immediate, changed to 10 min.

          • Owner Info: This is displayed on the lock screen. I configured Jim Carter's Selen.

          • Screen Timeout Delay (how long after the screen is turned off for inactivity, to require the password etc.). Default is immediate, changed to 10 min. This looks like the same contingency as for Automatically Lock.

          • Screen Turned Off Delay (how long after the screen is voluntarily turned off to require the password). Default is 5 sec, changed to 10 min.

        • Unknown Sources (turn on): allows installing packages from other than the Market. There's a lurid warning; blow it off.

        • Install (X.509 certificates) from the SD card. I need my home and work root certificates, and my personal certificates at home and at work. All (?) TLS connections check the partner's host certificate against this keystore, and if the browser is asked for a user certificate it will come from here.

      • Accounts and Sync

        • Gmail and Whatsapp will add themselves automatically. Use the Add Account button at the bottom to add the CalDAV and CardDAV accounts. It will want the server's URL, your loginID, and password. To set the URL, use manual mode and imitate their example for SOGo, substituting your loginID where it says user, and give the port where the SOGo server or its Apache proxy is actually listening, which in my case is 443, not the one they gave,

        • Gmail: All sync categories are turned off. I use Gmail as little as possible.

        • Whatsapp: Sync contacts, turn off. I have my own cloud server.

        • CFT Address Book: Edit account settings, and turn on sync.

          • One way sync: Turn off (so phone can send contacts to the server).
          • Policy: phone always wins (is the default).
          • Periodic auto sync: leave it on.
          • Sync interval: change to 1 day (I think the default is 15 mins.)
          • Data service types: leave them all turned on.

        • Jimc's Calendar: almost identical to CFT Address Book.

  • System Settings -- Anonymous statistics: In CM7 I drank the Kool-Aid. Our system version is CyanogenMod-7.2.0-RC0-SOLANA-SELFKANG . But I can't find this item in CM9. I'm sure it's there somewhere and I opted in, but I've misplaced it now.

  Application Settings:

  Since I was able to restore my settings this time, I only had to re-do a few application settings.

  • Clock: Set up alarms. Using Rooster Alarm for the ringtone.

My Desktop Icons

I've set for 3 pages of icons (labelled 0, 1, 2).

Setting Up Dropbear

In CM9-12-24 Dropbear is not preinstalled as it usually is in CyanogenMod. And it wouldn't be installed in the stock image. If this is your situation, download a recent version of CyanogenMod that has it. Follow this procedure to extract it, executing on your Linux laptop or desktop machine. The procedure for Windows should be not too different. The procedure requires that the pocket computer be rooted with a working su (superuser) program and with adb running as root. If you have su but adb runs as an ordinary user, you can cope by modifying the procedure.

The example below shows extracting dropbear from CM4D3-nightly-20111116-1445.zip. This worked up to the 2012-01-12 version, but then there was some kind of ABI conflict which prevented dropbear from starting. As of the 2012-03-16 version, I obtained 7.2.0-RC1 for Droid 2 off the CyanogenMod Wiki Latest Version page (image is for Gingerbread), and once again the procedure described below worked.

mkdir /tmp/cm7sys #Pick any convenient name
cd /tmp/cm7sys
unzip ../CM4D3-nightly-20111116-1445.zip #Substitute name of CM image
find . -name "dropbear*" -ls #Gives the filenames shown below
adb remount
adb push ./system/xbin/dropbear /system/xbin/dropbear
adb push ./system/xbin/dropbearkey /system/xbin/dropbearkey
adb shell #Now you're executing on the pocket computer
chmod 755 /system/xbin/dropbear /system/xbin/dropbearkey
ls -l /system/xbin/dropbear /system/xbin/dropbearkey #To check your work
#Continue with the instructions below to set up your keys.

Summarizing the CyanogenMod wiki page on SSH:

• It's assumed that you have your personal private and public SSH keys. If not, the wiki page shows how to create them. Copy the public key to any convenient location on the phone, like the SDcard.

• Use your favorite hacking tool, like ADB or the terminal app. On CyanogenMod, ADB's shell is automatically run as root; otherwise use su to get root. All of the files below should be owned by root. Your umask is 000, so be sure to fix the modes of the files created. Create /data/dropbear (mode 755) and /data/dropbear/.ssh (mode 700).

• Copy (or append) your public key to authorized_keys in this dir (mode 600).

• Generate the host key(s):

  • dropbearkey -t rsa -s 2048 -f /data/dropbear/dropbear_rsa_host_key
  • dropbearkey -t dss -f /data/dropbear/dropbear_dss_host_key

  The page says dss, not dsa; it has a fixed size of 1024 bits.

  Alternatively, if you are replacing your pocket computer or are doing a major version upgrade requiring wiping /data, and if you had the forethought to save the old host keys, you should restore them to the files given above, owned by root and mode 600.

• For testing, first determine the IP address of your phone. Determine the interface names by either of: ls /sys/class/net or cat /proc/net/dev. On the Dream as well as the Droid 3, wireless is tiwlan0. On the Dream the cellular data interface is rmnet0, 1 or 2 (guess which), while on the Droid 3 it is called ppp0. These names vary between devices. Now do ifconfig tiwlan0 to get the IP address. If the phone is napping it may have shut off WiFi; if so, wake it up.

• Manual operation of Dropbear for testing. Run dropbear with these command line options: -s = no passwords accepted, publickey only. -v = verbose output, -F = no daemon, stay in foreground. It will use the default key files, which are the ones you just created.

• Now try to slogin to your phone. For me it worked the first time, with much verbosity in the debug output. If the sending host already has a (different) host key in ~/.ssh/known_hosts, you will get a nasty message, including the line number. Get rid of the obsolete key and try again.

• For normal operation you can start it when needed by just dropbear -s. Recommended to redirect stdin-stdout-stderr because otherwise it holds open your TTY and you can't exit from the ADB shell until you kill Dropbear. (The current version, 0.53.1, closes these file descriptors when forking.)

• To make it start at boot, do this (ineffective in CM9):

  • mount -o remount,rw /system

  • Append this stanza to /etc/init.local.rc (actually in /system/etc):

    # start Dropbear (ssh server) service on boot
    service sshd /system/xbin/dropbear -s
        user  root
        group root
        oneshot
    

  • This done, Dropbear starts at boot, and I can execute commands and get a shell session on it, using its IPv6 (or IPv4) address, just like a normal host. Yay!

    Fly in ointment: the boot scripts have changed in CM9, not executing /etc/init.local.rc . Watch this space for a fix. Until then, use the terminal emulator app, do su to get root, then dropbear -s to start the server.

Miscellaneous Setup

My Web Home Page

I restored my backed-up web materials, that formerly were on the SDcard of Cupcake. I edited the home page removing the Google Gears hack; the version of Gears that I'm using is obsolete and needs to be upgraded, but not today. A URL of file:///sdcard/jimc-web/index.html was honored by the browser and was functional as my home page -- unlike on Cupcake where the developers, in a fit of unjustified paranoia, suppressed file URLs.

The only problem was that I use some symlinks to indices of contained directories, and FAT32 does not do symlinks. So I duplicated the referent pages instead, hiss, boo.

Setting Up Root and User Certificates

This is for CyanogenMod-9 based on Android-4.0 Ice Cream Sandwich. It is considerably improved over Gingerbread.

First copy the certificates to /sdcard. The app does not give you a choice of source directories; it must be the root of /sdcard, and the expected extension is .crt . Example: /sdcard/ca-mathnet-2024.crt

Navigate to Settings -> Security -> Credential Storage -> Install from SD Card. It shows a list of certificates that it found. Click on them one at a time to install. You can edit the friendly name. These certs will be honored for validating SSL partners.

For your personal certificate, use the web browser to open your Certificate Authority's web interface and do whatever it requires to elicit a PKCS#12 file, mime-type application/x-pkcs12. The browser has an association for this mime-type and will feed the cert to the certificate installer. You can edit the friendly name when installing.

When you connect to a web site that demands a certificate, the browser will show a list of installed user certs; you are supposed to pick one. On subsequent visits, that cert will be preselected in the connection dialog.

VPN Connection

CyanogenMod-6 comes with OpenVPN preinstalled. (Not sure when it came in; I think CyanogenMod-5.) If you have the stock Android, a rooted phone, and a tun.ko module compatible with your kernel (that's the hard part), you can install OpenVPN from the Market. Recommended to just upgrade to CyanogenMod.

I installed OpenVPN Settings and OpenVPN Installer by Freidrich Schäuffelhut, from the Market (free; the latter is ad supported). The two OpenVPN GUI packages don't look promising; they appear to attempt to install their own binaries, two different ones.

CyanogenMod-7 includes the OpenVPN binary, the tun.ko network tunnel module, and a settings module, found in Settings / Wireless and Net / VPN Settings / Add VPN / Add OpenVPN. I believe that stock Android-2.2 does not have any of this, but you can install them from Market packages. To load tun.ko requires root access, that is, r00t your phone.

On CM7 I was able to create a configuration that would connect; however, the provided configuration does not pull settings like the MTU from the gateway, and the cipher and HMAC algo must match exactly and cannot be pulled, but there is no way to influence them in the client, so no data can be transferred and the connection eventually dies.

These are the specific complaints, in the unlikely case that you are able to adjust the gateway to match what the stock client wants to do. The client wants a MTU of 1541 bytes; it does not do LZO compression; it uses the BF-CBC cipher (Blowfish?), and it uses SHA1 for HMAC.

On CM9 you will need to use the OpenVPN Installer by Friedrich Schläuffelhut, which you execute as an application. It does (automatically) an extensive sequence of steps that require superuser privilege, installing the binary. Now execute OpenVPN Settings by the same author. It opens with a settings-type dialog. It wants to see /sdcard/openvpn/ populated with certs and *.ovpn. The latter is the configuration file extension expected by this program. I installed these files from /etc/openvpn on Jacinth, which works:

• mathnet.conf (as mathnet.ovpn) (includes name of gateway, and filenames of the files below. Relative paths are relative to /sdcard/openvpn.)
• ucla-math.crt (root certificate that signed the gateway's host cert)
• to-mathnet.key (user's secret key)
• to-mathnet.crt (user's certificate signed by the above CA)
• dh1024.pem (Diffie-Hellman parameters) (not sure if these are required)

Procedure:

• Launch OpenVPN Settings (ad supported).
• Turn on OpenVPN.
• Click on the conf file basename (mathnet.ovpn).
• It asks for superuser privilege; allow it (remember is marked so you only have to do this the first time).
• The thread is lost after superuser escalation, so click it again. This occurs whenever the superuser privilege expires.
• Now it connects. I was able to display a non-public web page at work which normally is not accessible.

Here are some details:

• In Menu / Advanced (at the bottom) you can turn off ads, but they bring revenue to the author, so I left them on.
• You can long-click on the ovpn filename and pick edit ovpn file. The editor is not elaborate but you can fix a misspelled filename, for example.
• To diagnose problems you need to look at the Android log. Connect the USB cable, and on the host computer do adb logcat > /tmp/andr.log & (in the background). Then view the log which is continuously appended to that file.
• My configuration file uses the mlock command but that system call is not available. Ignore the warning.
• The user key is group readable. What do you expect on a FAT32 filesystem? Ignore the warning, or move the openvpn homedir to a real filesystem.
• The Advanced menu lets you edit the configuration directory and the path to the binary. With CyanogenMod you don't need to have it load tun.ko by itself.

XMPP/Jabber Client

On Cupcake I used Jabiru as a XMPP client, but it appears to be discontinued. I replaced it with Xabber by Redsolution (free). When you add an account, the server is the realm that is being served. If the realm is a domain name, and that domain has a SRV record for XMPP, it will be honored, but if not, specify the actual server's hostname (or IP) in Custom Host.

I also tried Beem and Yaxim as XMPP clients, but I had trouble getting them to work with my server.

All the XMPP clients require a saved password, hiss, boo! This is going to be a security hazard and a royal pain in the arse when I change my password.

Multimedia Checkout

Results of my standard web browser test:

Using the Pocket Computer as a Media Player

This is using the stock player, and not through the web browser. It is not possible to type in a URL in the media player; you have to go through the web browser, or else deposit music files on the SD card by other means.

Procedure to copy media without wires: Get a SSH session on Selen, display on laptop. Using the laptop's web browser, get the URL to the TAR file of the desired album; see below for accepted formats. It will be about 60 to 70Mb each. On Selen:

cd /sdcard/Music
wget -O - $URL | tar xf -

How does the music player know where the music is? Execute Dev Tools, pick Media Provider, hit Scan SD Card. It takes a few seconds to scan /mnt/sdcard (doesn't scan /sdcard-ext). On the HTC Dream /sdcard would be the external card, and you could unmount it, then remount, and the indexer could be run. But that is impossible on the Droid 3.

The Music app offers (requires) you to link it to a cloud account, for which Google's service is either the most convenient or the only one that is feasible. It is, however, possible to skip actually using Google's music service.

To play the music: My albums include pre-made playlists, with tracks in the correct order. Start the music app. Pick the Playlist tab. Long-press on a playlist filename. Pick Play from the context menu.

Alternatively, pick the Album tab. Click on an album's name. The list of tracks is shown, not necessarily sorted lexically, in the form of a playlist. Click on one track and it will be played; or more typical for my usage, hit Menu / Play All. A shortcut for this action is to long-click on the album's name and pick Play.

The playlist is expected (or at least is acceptable) in M3U format (extension .m3u). The content may/must be relative filenames, relative to the playlist. The content may be in Ogg Vorbis or MP3 format; others are promised but I did not test them.

If the playlist contains a URL to a streaming audio site, it is removed! Hiss, boo!

Playing Streaming Audio

So far I have not found a really satisfactory player for streaming audio. The scenario goes like this:

• Using the web browser, navigate to a website such as KUSC Listen Online.

Hit the URL of the form http://host/basename.mp3 . It will be passed to the music player, which will play it! At least it will play KUSC, but it will not play Jimc's OTA stream: it plays a few seconds of content, then reports Cannot play this kind of audio file. Reason: KUSC is sending out Icecast, while Jacinth is sending plain MP3. The music player will play Jacinth's Icecast service.

Mounting the SD Card

I partitioned my SD card with (in order) VFAT, swap, and ext4 partitions. The VFAT partition is mounted at boot; the others aren't. I deferred the issue of getting them mounted, to this point.

The intended design in CyanogenMod-6.1 (I am not sure how much of this is inherited from the stock Android-2.2 Froyo) is that the scripts in /system/etc/init.d/* are executed one by one (in lexical order) as a very early step in booting. Once all of these finish, vold starts (though I haven't found the invocation). One of these scripts is 05mountsd. It waits up to 6 seconds for /dev/block/mmcblk0 to appear. It goes through the partitions on this device, and the first one of type 83 (any Linux filesystem) is checked and mounted, if there is one.

The fly in the ointment is that vold creates the inodes for both /dev/block/mmcblk0* etc and /dev/block/vold/179:[0-9] etc (the same referents). Thus 05mountsd waits however long, exits having done nothing, and vold then creates the devices.

I did these hacks:

• Deactivated /system/etc/init.d/05mountsd by changing the mode to 640 (not executable). It is then ignored.

• Copied a hacked version to /data/local/bin/mountsd . Local copy of this script.

• Turned the script into a service by adding this stanza to /system/etc/init.local.rc . The service is started (once) whenever init feels like it; the log messages appear early, just after vold starts up.

  # Mounting the additional partitions on the SD card (jimc/CouchNet hack)
  service mountsdext /data/local/bin/mountsd
      oneshot
  

• The script was hacked to also search for a swap partition, and to add the first one found. This was effective.

• The ext and swap partitions are not unmounted when you use Settings to unmount /sdcard. If you want to remove your card, you need to unmount them by hand. Hiss, boo!

• The partition (independently verified to be ext4) is mounted as ext2! Busybox mount is stupid: it apparently tries all filesystem types in /proc/filesystems (excluding nodev), and ext2 appears earlier, and (amazingly) succeeds. I hacked the script to try the filesystem types in reverse order. Fixed.

Once I have CyanogenMod-7.0 running on my new pocket computer, I should verify that the same issues are present, and should send this in as a bug report.

Restoring User Data from Cupcake

Now that I have CyanogenMod stabilized, I want to restore user data from Cupcake, wherever the format is compatible or reasonably hackable. In particular, I want my contacts back. Here is a list of saved databases. On Android, an extension of .db always cues a SQLite database.

alarms.db

I re-created these by hand; no need to restore.

apns-conf.xml

(Haven't found the Froyo file yet)

assets.db

This is /data/data/com.android.vending/databases/assets.db . The database has 1 table called assets10 with one row per installed program. It represents the actual state on the machine and should not be messed with. Restore by re-installing missing programs from the market. [Done.]

browser.db

This is /data/data/com.android.browser/databases/browser.db . The only relevant table in here is the bookmarks. I have rebuilt my bookmarks by hand; no need to restore, or to save the old database.

com.android.settings_preferences.xml

This is /data/data/com.android.settings/shared_prefs/com.android.settings_preferences.xml . Some (but not all) items from the main settings app are saved here. The XML schema is very similar, but the keys differ considerably. It would be a big mistake to restore this file. I've set it all up by hand according to CyanogenMod's current capabilities. OK to overwrite the saved file from Cupcake.

contacts.db

In Cupcake the format is about as distant from vCards as you can get: separate tables for many of the aspects such as the phone numbers, street addresses, and photos. In NCCN (Froyo) the database is found in /data/data/com.android.providers.contacts/databases/contacts2.db which has a different but equally messy format. It won't be possible to just copy the contacts, even by a minorly clever Perl script.

The correct solution is cloud sync, but my Sogo server was not ready to take the Cupcake information.

Am I going to have to stupidly re-enter everything by hand, using the database dump as input?

Brainwave: if someone upgrades without wiping data, it may be able to convert the database format.

cdb=/data/data/com.android.providers.contacts/databases
cd $cdb
ls -l #Note the owner, group and mode of contacts2.db
mv contacts2.db Xcontacts2.hideme
adb push contacts.db $cdb/contacts.db #Do this on connected laptop
chown app_3:app_3 contacts.db #Use same owner, group and mode as contacts2.db
chmod 660 contacts.db
Reboot the pocket computer

And wonder of wonders, it comes through and imports all the old contacts. The old contacts.db and Xcontacts2.hideme are still there; remove them when contacts have been spot-checked for accuracy. You don't want it trying to convert the old database a second time.

mmssms.db

This is /data/data/com.android.providers.telephony/databases/mmssms.db . This is where SMS messages are saved. I clear threads as soon as I'm finished with them so the database is empty. No need to restore, no need to save the old one.

user_dict.db

This is /data/data/com.android.providers.userdictionary/databases/user_dict.db . The one table in this database is completely empty, both the saved and new version. No need to restore or to save the old one.

Backup Script

On Cupcake I had a script that copied all relevant stuff to /sdcard/BackupRoot, and a cockamamie procedure would transfer it (by sneakernet) to the backup server. On my normal Linux boxes I have a script called backup-host which I execute from the central site by SSH, and I hope to make that work equivalently on the Android machine.

Untoward Events

Poor Battery Life

Since starting with CyanogenMod I've noticed that my pocket computer eats battery and the percent charge changes non-smoothly. Other people report the same issue. Here's a thread on poor battery life with Gingerbread on OG Droid (OP: tkrug, 2011-05-21).

• Boriquaman says, cpuset if not configured correctly will drain the battery. When he stopped using it his battery issues went away. (Correct spelling: SetCPU?)

• DontPanic says, battery statistics might be wacked. Try this procedure:

  • Fully charge the battery.
  • Boot into recovery and wipe battery stats (under Advanced).
  • Let the phone run until the battery expires and it shuts down.
  • Recharge it fully again.

  Tkrug reports that this cured his problem. Others report that the wacked percentages may return in the future.

• Rrrrrrredbelly posts links to other threads with hints for what to do about an overactive CPU.

• Hater says he uses CyanogenMod 7.03 (on a OG Droid) with a stock kernel and has great battery life, double what he had with stock Android.

• Nesquik says, try Spare Parts / Battery History and look for partial wake. His dialer was in this state and it was eating battery.

I have a lot of anecdotal data and curses; here's a record of my objective test, before resetting the battery stats.

No Home - No Receive Syndrome

After I installed CyanogenMod-6.1, I got these very annoying symptoms:

• The home button seems to do nothing.
• I can't receive voice calls (can originate voice and receive SMS).
• It never ran the setup wizard and so there is no Gmail account and therefore I'm not allowed to access the Market.
• There is also unexplained instability in the UI (ADW Launcher).

Forum postings suggest that this is a syndrome, and a fix for one element fixes the others too.

Cause and Solution: I was using the non-tiny version of Google Apps. When I did a factory reset, re-flashed CyanogenMod-6, and used gapps-mdpi-tiny-20101020-signed.zip (the correct Google Apps for HTC Dream), it ran the setup wizard as it should, the Home button goes to the home screen, I can receive incoming calls, and the UI has stopped crashing. The following material is kept for historical interest.

In a Google search for cyanogenmod home button not working, this forum post (OP jsam69, 2010-09-01) indicates that re-flashing the Cyanogenmod image will fix it. Works for some people, not for others. Another item in the same thread from (anonymous) says to run the terminal emulator, type bin and press the home button. Doesn't say what's supposed to happen then.

In another thread (2011-06-13), the problem was caused by restoring apps using Titanium Backup and indiscriminately restoring the data for everything. They haven't found out which app's data is poisonous but it appears to be a system app.

At the end of this thread, the person says he went into recovery and fixed permissions and this helped. The Amon_Ra Recovery has an action item to fix permissions; basically this sets a bunch of directories to mode 755, where some of them formerly were not world readable. (Didn't help me.)

OP WaVeS (dated 2010-04-28) says he has lost his home button. One respondent says he had this symptom repeatedly (on a G1). Boot into recovery (power off, hold down home and power on). Do a full wipe, then reboot. Fixed the problem every time. (What you're really after is to wipe cache and data.) It didn't help me.

OP says if he blows cold air onto the home button, it would work. Do a factory reset and reflash the image. It works again. Another procedure: Install AnyCut from the market. Execute it. Run Setup Wizard. Go through the procedure. This should fix your dial (green), home, hangup (red) keys gone wacko.

JoshuaE says (paraphrasing): Make /data/data/com.android.providers.settings/databases/settings.db available on a real machine. Use sqlite to insert a record with name=device_provisioned value=1. Copy it back to the original location. Reboot.

My experience: Booted into Recovery, wiped cache, Dalvik cache, and Factory reset/data (which I think includes the 2 caches). The Home key worked in Recovery (other people's symptoms suggest corrosion in the Home key). This wiped my bookmarks (and other settings) but didn't cure the Home key problem.

More drastic: Boot into recovery. Wipe data factory reset. (Also wipes the cache.) Flash CyanogenMod. Flash Google Apps (MDPI). Reboot. Outcome: it did not run the setup wizard. This doesn't seem to be on the phone. The home key is still inoperative.

Got into Koushik's ROM Manager and hit Fix Permissions. (And rebooted afterward.) It changed 64 files and dirs always to mode 755; some had more restrictive permissions before. Home key is still inoperative.

Comparing JoshuaE's fix and actual phone behavior, the problem is that the setup wizard is not being run, and the software is insufficiently idiotproof against missing values. Another syndrome element is that the market and map apps do not appear in the launcher page. Nonetheless, all of these are present in the Google Apps (MDPI) image (gapps-mdpi-20110501-signed.zip). I had an idea to obtain the APK's and install them separately, but this is obviously bogus.

More Plans to Recover

More on the home key - receive calls syndrome. Doing the right thing, i.e. wipe and reflash, was ineffective on repeated trials. I'm going to try several lines of attack. Of course it's perverse to stick with the G1 when the current OS version won't fit on it, and it's definitely on the agenda to get a new phone, but it's also not prudent, and it's a waste of a good learning experience, to accelerate that step. So, here's what I'm going to try in the short term.

• Dump the dmesg log and see if there's a clue (not likely). No clues found.

• Dump the Android error log and see if there's a clue (more likely).

  adb logcat > nccn.log

  It works like tail -f. In another window check if it seems to have delivered all the messages up to present time, and if so, kill it.

  Unfortunately no clue was found. There was no evidence that it tried to run the Setup Wizard.

• AnyCut: put a shortcut to it on your screen, and it will launch anything: make phone call, send SMS, run the Setup Wizard . . .

  Filename is com.appdroid.anycut.apk . Normally you get it from the Market, but since I haven't run the setup wizard I can't access the Market. Google search for the filename; found it on an extremely ad-supported site that I normally wouldn't install software from. I downloaded it to my laptop and dug around:

  
  (make and change to a subdirectory, for easy cleanup later)
  unzip ../com.appdroid.anycut.apk #(it lists the files)
  

  The app icon is a 48x48 px PNG file, a picture of a house with a transparent background. I was not able to figure out how to check integrity of the package. ADB may or may not check it when installing.

  To install a package, connect the USB cable and then, on the PC:

  adb install com.appdroid.anycut.apk

  On the phone, launch the app from the regular launcher (or long-press the desktop background, pick Shortcuts, and pick AnyCut from the list). Select New Shortcut / Activity. From the unedited list of installed packages pick Setup Wizard. There are two instances (many programs have multiple subpackages such as providers or data collections); I chose the first one. It pops a box to edit the shortcut; don't change anything. Hit Back to leave AnyCut.

  Now you have a shortcut to the Setup Wizard. Expose the keyboard. Make sure you have wireless data connectivity. Hit the shortcut. The wizard starts. The wizard from Froyo is going to ask you your Gmail loginID (not including @gmail.com; they will fill that in for you), and the account password. If you don't have an account, you can create one. The wizard exits after that but syncs your account in the background; look for a statusbar icon of a circle with two arrow-like triangles. Wait for it to disappear. In my case with absolutely nothing in my Gmail account, it took about 10 minutes.

  Curse, curse, that intervention was ineffective: home button does nothing; can't receive calls.

• Assuming I can't bring it back from those clues, I'll use a hybrid approach:

  • Wipe, and load an authentic unhacked Froyo image. Now where do I get it?

    • Quoted in forum post (no link): HTC says they won't issue an image of v2.2 for Dream; not enough space.
    • Za-pai custom version of CyanogenMod: Howto by Ali Waqas, 2010-07-07; Froyo image.
    • Built from source by Chann, posted 2011-03-03, uses DangerSPL and Amon-Ra Recovery. Howto; system image; kernel; Google Apps
    • Link to CyanogenMod forum for Dream
    • List of HTC Phones, check out the HTC Doubleshot (myTouch 4G Slide).

  • Very likely this will run the setup wizard. Assuming success:

  • Back up system data but not apps.

  • Wipe, load the CyanogenMod image and Google Apps, and restore the backup.

  • With any luck, CyanogenMod will inherit the provisioning data from the authentic Froyo.

Miscellaneous Tidbits

Top 10 phone models

As of 2011-08-25 by count of CyanogenMod installations (in thousands):

Model	Installs	Comment
bravo	67.8	Mid level phone by Motorola for AT&T.
umss_jordan	20.9
morrison	26.2
supersonic	26.1
passion	22.6
blade	20.5	Low-end phone by ZTC
sholes	17.5	The OG (original) Droid
buzz	16.9
galaxysmtd	15.8
encore	14.9
Other	109.7
Total	358.9	(All official CyanogenMod installs)

 

Items detected in logcat output:

This was for diagnosing the home key syndrome, but it provided a number of useful tidbits:

• This dump is definitely for an initial boot after installation; lots of data files do not exist.

• Our chassis is called trout. Alternatives are goldfish (the emulator) and sapphire (the HTC Magic?)

• The keyboard is the trout-keypad-v3 whose layout can be found in /system/usr/keylayout/trout-keypad-v3.kl

• We're using a virtual screen size of X = 320px, Y = 960px (actual Y = 480px). Quoted dimensions: 45 x 67mm, about 180dpi. 16bit color 5:6:5 (6 bits green).

• Touch screen pixel range: X 80-3400, Y 32-5336, pressure 0-255, size 0-15.

• We intrinsically have a backup service. There is no saved data and backups are not enabled.

• W/ActivityManager( 170): Unable to start service Intent { act=com.google.android.location.NetworkLocationProvider }: not found

• It uses sntp north-america.pool.ntp.org to set the clock, but cellular data is disabled and so this doesn't work.

• No sign that it tried to mount the ext4 filesystem.

• It appears to have mounted /dev/block/vold/179:1 on /mnt/sdcard

• Our DNS server: 172.18.7.170 (RFC 1918 address)

• There is no sign that it tried to start the Setup Wizard.

How to install Debian on CyanogenMod

You need an ext[34] partition. There's a chroot program called "Run Linux", and you can bind mount. Get the picture?

List of 30 indispensible apps (francophone)

Several that I already picked are on this list, and several on the list look like I will want them.

Android Bootscript System

Link to a writeup about the Android bootscript system.

• Unlike SysV init which uses a horrid syntax for /etc/inittab, Android's init has a semi-programmatic syntax for init. One commandoid per line, words are whitespace separated, #comments and empty lines are ignored, double quotes and backslash escapes are recognized.

• There are two kinds of sections: actions and services. (Everything has to be in a section.)

  An action section has the form: on [trigger] \n [command]...
  It is possibly true that there can be only one action per unique trigger.

  A service section has the form: service [name] [fullpath] [args]* \n [option]...
  The service name definitely must be unique.

• There are several options; the most significant is oneshot which means to run the service once at boot but not restart it if it exits, which would be the normal behavior. The disabled option suppresses the service on boot; it can be activated by a command.

• There are five triggers: boot, name=value, device-added, device-removed, service-exited. name=value fires when a property is set using the setprop command, e.g. setprop cm.userinit.active 1. device-added-/dev/block/mmcblk0p2 did not fire for me.

• There are a lot of commands, but to execute arbitrary binary programs you have to define them as one-shot services and use the start command. Types of commands:

  • Normal filesystem commands such as chmod, mkdir...
  • Import another init script file
  • Set the hostname or domainname
  • Start or stop services or classes of services
  • Mount a filesystem
  • Set a property (setprop)
  • Trigger an arbitrary event
  • Write static data into a file, including files in /proc

• The CyanogenMod init script has these features, starting with the on init section:

  • Set clock offset to 0 (UTC).
  • Set PATH and similar environment variables
  • Creates mount points anew (is the root not a real filesystem?)
  • Set a bunch of proc items
  • Mount internal partitions (excluding SD card)
  • (Re)build directory structure in /data and /cache
  • Bring up lo:, set host and domain name (evidently this is not contemplated as being the YP domainname)
  • Set a bunch of memory management properties
  • Set more /proc parameters
  • Set permissions of power management files in /sys/power and /sys/android_power
  • import /system/etc/init.local.rc (user defined init script)
  • Finish by execing sysinit (whatever that does)

• Services include:

  • adbd (Android debugging daemon)
  • /system/bin/servicemanager
  • vold
  • netd
  • debuggerd
  • rild (looks like it manages the radio)
  • zygote (not sure, but it looks very important)
  • mediaserver (likely the music indexer is one of its jobs)
  • dbus-daemon
  • bluetoothd (runs only when explicitly enabled)
  • Bluetooth profile daemons for HFP HSP OBEX(push) PBAP(whatever that is) FTP
  • installd (presumably to install a package)
  • racoon (only when explicitly started)
  • mtpd (a VPN service)
  • keystore
  • dumpstate

• /system/etc/init.local.rc starts these services:

  • compcache (uses RAM as a swap file; somehow this is supposed to be helpful)
  • dropbear (jimc/CouchNet hack)

Boot Init Scripts

At some early point, something kicks off the scripts in /system/etc/init.d (probably using runparts):

00banner

Shows a banner at the start of the logcat information; it spells out cyanogenmod in unreadable ASCII art.

01sysctl

Sets kernel parms from /etc/sysctl.conf, which exists but is empty.

02audio_profile

Sets audio parameters, picking the init file according to the hardware model.

03firstboot

If existing, execs then removes /data/firstboot.sh

04modules

Loads (modprobe) modules listed in the script, currently just ipv6.

05mountsd

Identifies the first partition of type 83 on the SD card (if any), runs e2fsck, and mounts it on /sd-ext . But it did not find /dev/block/mmcblk0 .

20userinit

Executes each of these scripts, if existing:

• /sd-ext/userinit.sh
• /sd-ext/userinit.d/* (using run-parts)
• /data/local/userinit.sh

Open Issues

• When you unmount the SD card (VFAT partition), it does not unmount the ext4 or swap partitions. This complaint is from CM6 on HTC Dream; I need to check if it's been fixed in CM9.

• Add CM9 open issues . . .

Formerly Open Issues Now Finished

• Root Certificates in System Keystore (CM7) -- I installed CACertMan, which can delete a CA cert. Watch for an update allowing insertion. I installed my certs by cowboy programming. CM9 has a proper certificate installer which was described earlier.

• Mount ext4 Partition of SD Card on Boot -- Traced it to a design problem in the script that mounts the partitions. Fixed. Now we need to get it unmounted.

• XMPP/Jabber Client -- all of Xabber, Beem and Yaxim fail to connect to the server, and sooner or later force close. The server alleges it accepted a connection, but it (seemingly) never goes beyond the 3-way handshake. (Fixed, I was using the wrong server name.)

• CM7 on HTC Dream (G1): I'm getting horrible battery life, compared to Cupcake. The best clue so far is that the cell radio has no signal as much as 46% of the time, and the radio coprocessor has to do frequent expensive searches for a tower, eating battery. Proposed cure: switch to a carrier that has a better signal at my house. Successful, helped a lot. Also, see the issue of Dovecot versus K-9; putting a Band-Aid® on this also helped a lot.

  CM9 on Moto Droid 3: Battery life is excellent, generally no lower than 70% at the end of the day when I recharge. I'm using a carrier that has a semi-adequate signal at my house. Also I upgraded Dovecot and configured it to not be overactive when pushing e-mail (for which you need the new version).