Sony Xperia XA2 (2020)
Setup

Table of Contents

• When the New Phone is Received
• Setting Up the Stock OS
• Battery Tests
• Backing Up the Trim Area
• Struggles With Unlocking the Bootloader
• Installing CyanogenMod (LineageOS)
  • How the Installation Step Worked Out
  • LineageOS Initial Setup
  • Installing Magisk and Procedure Confirmation
  • Setting Up LineageOS Settings and Apps
• Launcher Icon Layout
• Google Pay on Rooted Phone
• Upgrading to LOS-17 (separate page)

When the New Phone is Received

The old phone's hostname is selen. The new phone will have the hostname orion until it's set up and reasonably operational, whereupon Selen will be renamed to Selen-S5 and Orion will be renamed to Selen. The IP addresses follow the names; the MAC address stays with the chassis. I've paid for one phone line, and the micro-SIM is in Selen. When I do the changeover I will need to get a nano-SIM for the new phone. And to make sure that the phone number isn't randomized. This will take a trip to the brick and mortar store. For reference:

• Mini-SIM: 25x15mm
• Micro-SIM: 15x12mm
• Nano-SIM: 12.3x8.8mm

There are hints in the documentation about dual SIM operation on some models. To find out, dial the phone, give it the number *#06# and it will show the IMEI(s), one per SIM slot. Too bad, I have only one slot. And if it were dual, both would be nano size. The second SIM goes in the SD card slot (despite different sizes), so you can't have both at the same time.

What's in the box:

• The phone.
• Generic charger (Sony branded), 1.5A 5V out, 100-240V in, USB type A connector. Plug is NEMA 1-15P (North America, Andean South America, Japan). See IEC World Plugs website.
• USB cable, type A to type C. Don't try to cram a micro-USB connector into the phone; you'll break something. Use a cable with a type C connector.
• Setup guide plus various regulatory statements.

First steps:

• Copy down serial numbers etc. Particularly the MAC address and the IMEI. The box label has the IMEI but not the MAC address.
• If I have the MAC address, add it to hostdata.db and install files. Reload the firewall so it will accept the phone's packets on Wi-Fi.
• Charge the phone. Usually a new phone is mostly charged. Mine was 95% charged.
• Keep the SIM out of the phone until after checkout.
• Turn on the phone with the stock OS. Go through Setup. Specifically, give the Wi-Fi password.
• If the MAC address is not yet known, check Jacinth's firewall log, then do hostdata.db and firewall setup. Finish phone setup.
• Go through the checkout procedure on the stock OS.
• Try moving Selen's SIM and see if it works. It doesn't; this model doesn't have dual SIM capability, and if it did, both SIMs would be nano size.
• Get a nano-SIM at the brick and mortar store. The customer service person waived the cost of the SIM because I was moving to a different phone on the same account and line. Phone number was unchanged (good). Surprisingly he didn't confiscate the old SIM. Never allow both phones to be powered up (with a SIM) at the same time; that will totally confuse the infrastructure. Best to take out the old SIM, label it, and store it.

Setting Up the Stock OS

Going through the steps in Sony's Startup Guide:

• Charge the phone using the provided charger and cable (type C). It looks very similar to micro-USB but it's not, and if you try to cram in a micro-USB connector you'll break something. Out of the box the phone was 95% charged.
• To view the FCC ID, drag out the SIM and SD trays, which are at the left upper corner of the phone. Turn the screen downward. Remove the SIM/SD cover, which includes the SIM tray; don't drop and lose your SIM. You could break a fingernail doing this; you'll probably need something metallic. Once you get the cover off, you'll see a white layer at the bottom of both slots, and there's a cutout in the one under the SD tray, by which both together can be dragged out. Copy down the various numbers: Model, IMEI, SI, Type, FCC ID, IC. Drop the SIM and SD cards in the trays, if you have them, with the contacts up, i.e. toward the rear of the phone, and reassemble. (My strategy involves keeping the SIM out of the phone until after checkout.)
• You can also see the phone's idea of its IMEI by dialling *#06# . Also the regulatory compliance logos can be displayed by *#07# .
• To get started, basically, power it on by holding down the power button (center of right side, round button) until it vibrates, wait for it to get started, and follow the extensive on-screen instructions. Some details of setup follow.
• It takes about 40 secs to start up.
• The default language is Spanish; there's a drop-down list to change it.
• Confirm that you've read the warranty and safety info, and that data charges may be incurred. Opt out of sharing diagnostics.
• Insert SIM. Since I don't have a nano-SIM yet, I'll hit Skip.
• Copy your data from an iPhone or Android device. I'm going to try it, see if it can be made to work. Prerequisite is…
• Connect to Wi-Fi. Pick your SSID from the list. I have this great QR code all ready to go, and there's no option to scan it, hiss, boo. Typing in the password.
• Data source, choice of backup from Android, iPhone, or cloud server. Obviously I'm choosing Android.
• Connect by NFC, put devices back to back. The antennas have to be aligned: Pioneer is near the camera, Galaxy S5 is in the lower center.
• There was a problem on the Android end. I tried turning on Bluetooth manually and pairing, but the Pioneer did not pop a dialog box to enter the pairing code, and pairing timed out. Giving this up: hit the Back icon (left pointing triangle) several times. Choose Set up as new.
• It checks for updates. Looks like it's not going to do any now (it will do a lot of them later).
• Sign in to your Google account. Accept the terms of service.
• I'm going to skip fingerprint setup and do it later.
• Turn off unwanted data transfers. Back up to Google Drive: off (I have my own backup procedure.) App location: on. Improve accuracy: on. Send system data: off. Hit Agree.
• Google Assistant setup -- Skip. No yapping animated circle like for Cortana.
• Xperia Services: Sony Mobile will collect and combine your personal data… Hit Not Now (or ever). If you do not sign in, you will miss out on a lot! Skip.
• It offers to open Google Play so you can install apps. Skip.
• All finished! (So it says.)
• It's installing a bunch of apps with a default icon collection. It also says a system update is about to happen. It's managed to run the battery down to 65% (from 95%); I don't really believe that 30% of battery was used. Wait for it to finish downloading the apps. It's got the time and timezone correct; lacking a SIM it must be using NTP (and GPS for location?) But I was not able to find the NTP settings in Setup.
• At this point I checked for updates; at least 27 app updates were available. I updated. A system update is also in the offing.
• Now run the welcome app. Well, it doesn't look too promising. I didn't get through it.
• Setting up the fingerprint reader. The screen lock choices are None, Swipe (current), Pattern, PIN, Password. Setting PIN: at least 4 digits (I have 6). Setting pattern: my standard pattern. Apparently you can't have both at the same time. Setting up fingerprint: It's hard to hit the sensor when you can't see it, but accuracy improves with practice. Trying to unlock: 2 failures of 10 tries, and in those I clearly had my finger off center. I think with practice this is going to work out, assuming CyanogenMod supports the print reader at all. Later, as reported by other users, the print reader got more finicky and I had a lot more failures. Could the reader or my finger actually be dirty, as the error message says?
• The stock image does not mention face recognition.
• System (OS image) updates continued through the evening. Updates are downloaded only on Wi-Fi (possibly you can enable cellular data and use up your quota).
  • The original image I think was dated sometime in 2018.
  • 50.1.A.13.83, 1 to 2 minutes (I think, probably an underestimate).
  • 50.1.A.13.123, 334Mb download, 8.5min to download, install and reboot. Some updates have postprocessing steps after the reboot.
  • 50.2.A.0.379, 963Mb download, 14.5min.
  • 50.2.A.0.400, 312Mb download, 6.5min.
  • 50.2.A.3.22, 312Mb download, 6.6min.
  • With these updates it has Android 9 Pie.
• Standby overnight with no SIM: used 2% of battery. With a SIM the stock image (updated) fairly consistently uses 3% overnight.

Battery Tests

Five years ago I did a battery test on the Galaxy S5 like this:

• Determine the number of cores, 4 on the Galaxy S5.
• Print the time, the reported percent charge of the battery, and the current out of the battery.
• For each core, find each plain file in /system/app, and make a SHA-512 sum of it. This runs in the background. There is 254Mb of data in /system/app, which will fit in the buffer cache so it doesn't have to be read repeatedly.
• Wait for all the threads to finish, taking about 5 minutes on the Galaxy S5.
• Repeat until killed manually.
• Command line (run as root):
  cd /sdcard/jimc/bin-gingerbread;
  bash ./batt-test 2>&1 | tee ./logfile

When new, the Galaxy S5 would run this for 151 minutes, taking the battery from 100% to 4%. Test conditions: executing in a local terminal, no other active tasks, Wi-Fi and cell were enabled. Dropbear and a remote shell session existed but were doing nothing, screen was about 50% bright with a dark background, battery started fully charged to 100%. Here is the battery use history, interpolating for the time (in minutes) at each percentage. Cap'y is the run time as a ratio to the Galaxy S5 when new. Run time is the relevant functional measure, but the Pioneer has more nominal battery capacity (3.3 amp-hours at 3.15V) than the Galaxy S5, and its 8 core processor is supposed to use less energy for a standard app-like task, whereas this test procedure doubles the number of cores doing sums, cutting in half the run time. And the time to completion of the same file on one core of the two devices is going to be different but is not measured.

Recharging the Pioneer's battery using the provided 1.5A charger: similarly times are in minutes, interpolated to even percentages. On both the provided charger and my smart charger both devices announce that they are charging slowly, implying that high voltage on the power bus has not been negotiated. The charge time (0 to 80%) could be speeded up substantially if that were done, i.e. with a USB cable with type C connectors on both ends (and a charger with such a connector), vs. the provided cable with type C on the phone end and type A at the charger.

Backing Up the Trim Area

There's an issue which for me is spiralling out of control: what to do about the Trim Area or TA partition. It contains your DRM keys for licensed preinstalled Sony content, plus other items like the IMEI and the Wi-Fi and Bluetooth MAC addresses, that are not license related.

• If you unlock the bootloader it trashes the TA partition, or at least the DRM keys in the TA partition. I get the impression that they are encrypted and the key is wiped when you unlock. Confirmation from expert, see post #291 (at end of page 29).

  The discussion of trimarea_tool by p0kemon (2016-12-09) (read lurid warning) suggests that the whole thing is in a weird format and is encrypted, but there are multiple keys, only one of which is deleted when you unlock the bootloader. This is jimc's interpretation and should not be taken as authoritative.

• Warning: never flash a backed up TA partition from a different phone! People report that this bricks your phone, including at least two people who actually did it and now regret.

• There are at least three programs available that can back up the TA partition and restore it. The LOS installation guide has no guidance for doing the backup; they wouldn't touch it with a 10 foot pole. This is going to be hard.

• TWRP Recovery can back up the TA partition, but you can't flash TWRP until after you've unlocked the bootloader. You're saving and restoring an exact copy of the bytes, not decrypted, not interpreted, and not alterable (unless you have Sony secret keys to re-encrypt the altered items). So your DRM keys are gone forever, but there are important items that are preserved and that can be restored in extremely rare failure events like if malware vandalizes your TA partition.

  You can't just use dd to do the same thing because you don't have, and aren't going to get, a SELinux security context that allows you to read the raw device. Whereas Recovery does not have SELinux (or more likely, didn't put it in enforcing mode) and can read-write whatever it pleases.

• To back up your TA partition in the main kernel, preserving your DRM keys, you need to downgrade to the stock image for Android Marshmallow, which is vulnerable to the DirtyCow exploit (Wikipedia article about it) (see CVE-2016-5195). This exploit is used by the recommended backup program to steal the contents of the partition.

  Now a key question is, when you put a custom OS (CyanogenMod) on the phone, can you restore the TA partition and have the DRM keys work? Does anything bad happen if you do this? Are you going to be able to use whatever depends on the DRM keys, without support from the stock OS? Can you even extract and install, e.g. the stock camera app?

• Another question: what do I lose if I botch (or don't do) the TA backup? Answering this will take some work.

• Tutorial for backing up and restoring the Trim Area by rickwyatt (2013-04-14). The backup-restore program (Backup-TA) executes on Windows. Run it and make sure ta.img (the backup) has been written. Run it with a different option to restore. In between, he unlocked his bootloader and the DRM keys were lost, also the ba2 (whatever that is, I couldn't find a reference). And presumably he put on a custom image. He says, I then reflashed it with adb and rebooted; my Sony Z was locked and DRM and BA2 were working again as if I never unlocked it. Is he saying the custom image was still there, or that he reflashed the stock OS plus TA area?

  It looks like rickywatt did the procedure and wrote it up. See this tutorial by Yunlong (2014-10-13). DevShaft developed the BackupTA program. And the phone has to be rooted first. Yunlong also tells how to check if the keys are still working. On the Service Menu, run Service Test-Security, except the Pioneer doesn't have this test. It's under Service Info, and more stuff is shown. Under Suntory Blobs, I recognize two keys that other people have mentioned: CKB and WIDEVINE (I'm not sure what they do). On my stock image, both of them are OK[ACTIVE], which is good.

• Xperia X: backup TA partition before unlocking bootloader (wiki page, OP not stated, initiated 2017-10-09, last update 2018-06-10). It's a very detailed howto, telling how to downgrade the stock image to Marshmallow (Android-6) and then do the backup. There's also a link to a different howto and to the Marshmallow stock image. Note, this is not for the Xperia XA2 Pioneer but for Xperia X. The introduction talks about returning your phone to original factory condition, i.e. the stock OS plus the backed-up TA area.

  naytsyrhc says: He followed a different howto (in German, his language), and it went smoothly. He put Sailfish on it and it looks like his DRM keys are still unavailable but Sailfish works.

  Giacomo says: This is a PITA. I'd rather purchase another Xperia X than break my head with that process. (Jimc says: I kind of agree.)

• Tidbit from somewhere else: To check if your DRM keys have been trashed, open the service menu, service info, verify certificates. On jimc's Xperia XA2 Pioneer with the stock image, it reports 138 platform, 43 apps, 5 media, 7 shared packages, and nothing that looks like an error message. Update: after my DRM keys were trashed, this menu item gave the same count and no error message.

• At one time, on a particular model, if you trashed your TA partition, the camera would take pictures all in green, but when this model was upgraded to Android 8 Oreo, the camera worked normally with a trashed TA (within the reporter's ability to judge normalcy). Other sources say Oreo is still vulnerable to green photos but in Pie this is fixed.

• Android Pie and DRM keys (OP rogger888, 2019-01-09, on together.jolla.com, sponsor for Sailfish custom ROM). laubblaeser replies: XA2 camera does not break when unlocking the bootloader. rozgwi replies: when you're using another system (such as Sailfish) you'll be left with a basic camera… jimc combines several sources: when the origin OS was Android-8.1 Oreo the camera could have annoying issues like taking everything in green, but this has been fixed in Android-9 Pie. rozgwi in his Answer says (paraphrased by jimc): it matters which stock OS you jumped ship from, when you put on a custom ROM. When you unlock from Pie, the important stuff (non-DRM) is not trashed, as it might have been from Oreo.

• Unlocking the bootloader no longer breaks the camera on Sony Xperia devices running Android Pie by Mishaal Rahman on XDA-Developers (2018-11-20). He gives a very complete discussion of the TA partition mess, and in particular he passes on a report from LazerL0rd telling what works and what doesn't, after you unlock the bootloader. (This is on an Xperia XZ Premium.)

  • What works: Color Gamut profiles, Camera no longer takes green pictures (quality not guaranteed to be the same, but it works at least), White Balance, Camera2API (FULL without RAW for rear camera, LIMITED for front camera)
  • What doesn't work: X-Reality Video Enhancement, DSEE HX, ClearAudio+, Widevine L1
  • Jimc also saw a post mentioning the Bravia Engine (in what model?) as non-working.
  • Jimc spotted something: lineage-16.0-20200229-nightly-pioneer-signed.zip changelog notes that Widevine L1 support was added to nile-common. Few people say what Widevine does.
  • Sony Xperia XZ2 gets Android P Beta 3 with Widevine L1 support by Adam Conway (2018-07-05) on XDA-Developers. He says that Widevine L1 is what prevented many flagship phones from playing Netflix or Amazon Prime Video above 480p. (Jimc assumes he means lack of Widevine L1 prevented…)
  • Wikipedia article about Widevine. A lot of media vendors (e.g. Netflix, see the article for a list) use it for DRM. When the client's Widevine software proffers some kind of proof that the client has the right to see the video, the media vendor will send out an encrypted stream, which Widevine will decrypt. Jimc does not play DRM-controlled video (or any other kind of video except test clips) on his pocket computer.

• Another serious issue: in reports of successfully backing up the TA partition, nobody clearly states that they have been able to actually do the DRM-restricted activities in their custom image like LineageOS or Sailfish, e.g. play Netflix movies at over 480p. Success is reported in reverting to stock, but not in using the custom image.

• Jimc judges that he is extremely unlikely to even notice the absence of the non-working items. Thus I'm cutting off this project with the decision to not back up the DRM keys. However, I will use TWRP Recovery to back up the TA partition (minus DRM keys) after unlocking the bootloader.

• Procedure to back up the TA partition:

  • First, which partition is going to be backed up? TWRP offers to include these in the backup: /system /data /vendor /storage (excluding user storage, what would be FUSE mounted on /storage/emulated/0), and image variants of some of these.
  • On the web, a keyword search for backup trim area TA partition yields (ancient) procedures to back it up before unlocking the bootloader, which is not what I'm doing. Nobody actually says which partition is being backed up, by name.
  • Based on various hints it looks to me like /vendor is the most likely one. A second choice is the non-user-accessible region of /storage.
  • Connect the USB cable; see Struggles With Unlocking the Bootloader (next section) for nasty details in this step. Confirm ADB debugging on the phone.
  • adb devices (says CQ30010NGY device) (starts daemon if not done yet)
  • adb reboot bootloader — after about 10sec it starts the bootloader, indicator light is dimly red, then blue.
  • fastboot devices (says CQ30010NGY fastboot)
  • fastboot boot /s1/sony-xperia-xa2/twrp-3.3.1-5-pioneer.img
    It goes through these steps: lock your bootloader (ignore), SONY splash screen, TeamWIN splash screen, enter your pattern (to decrypt /data, which it takes a while to do), finally the main screen.
  • In Settings - middle tab (sun+moon), uncheck Enable Screen Timeout (very annoying).
  • Hit Backup. Turn on/off partitions to be backed up. Already decided to back up Vendor Image. Hit Select Storage and choose the SD card. The numbers shown are the amount of free space in Mbytes. Don't mess with the other tabs; compression does nothing for the images, though it probably helps the tar files a lot.
  • Swipe to start backup. It shows the output dirname which you should memorize. It's not all that fast: 174sec in my case, about 5Mb/sec. 850Mb not compressed.
  • Accept the option to reboot. Do not install the TWRP app if asked.
  • To retrieve the result, start dropbear.
  • On the backup server (Diamond):
    rsync --rsh="ssh -4 -p 2222" --log-format="%o %f" -a selen:/storage/B2A3-0150/TWRP /scr/.
  • Check for damage:
    sha256sum -c vendor_image.emmc.win.sha2 #Says it's OK
  • gzip vendor_image.emmc.win
    Output is 145MiB, 16% compression.
  • Write a README file. Move everything to a directory in the backup area. For me: /home/backup/selen/vendor-ptn

Struggles With Unlocking the Bootloader

(Numbers refer to steps in LineageOS' howto for unlocking the bootloader. The next section, Installing CyanogenMod, has preliminary steps that should be done first; then keep in mind the lessons learned in this section when you unlock.)

• 0. WARNING, your DRM keys will be gone forever (see previous discussion of the consequences), and the rest of the phone is totally wiped, so back up user data. (It isn't clear if the SD card is included, but one forum post suggested that it is, so I removed the SD card; follow the link for suggestions how to do it. However, other documentation says it isn't.)

• 1. Dial the Service Menu code. I did this already, at the very beginning.

• 2. Go to service info - configuration - rooting status. Yes, bootloader unlocking is allowed.

• 2a. The instructions mention this after you've started fastboot: you need OEM Bootloader Unlocking to be turned on. Find it in Settings - System - the Advanced section.

• 3. Connect the USB cable to the host. How you connect makes a difference: peek ahead to step 5.

• 4. We're supposed to run the bootloader (fastboot). On the host:
  adb reboot bootloader
  Screen goes black for about 30 secs, then shows the boot splash screen Sony for about 10 secs, shows battery charging icon for 10 secs, then goes black again. Not in the instructions, but the notification light should have changed from red to blue, very dim, hard to see.

• 5. On the host, execute:
  fastboot devices
  Blecch, no devices are shown. Various interventions produced partial success, but never enough to actually unlock the bootloader.

  Condensing a page of turgid story line: My laptop has one USB-2.0 port and two USB-3.0 ports. On the sub-Reddit for LineageOS I got some help from LuK1337, the manager for this build: he suggested that I try using the USB-2.0 port or a USB-2.0 hub. Just using that port helped but did not get to successful unlocking. But with the hub, I got the phone unlocked. Thanks to LuK1337. Here are some details that may be useful:

  • To start fastboot, with the USB cable connected directly to the USB-3.0 port, adb reboot bootloader just hung.

  • But the key combination works: turn off power and disconnect the USB cable. Hold down volume-up, and connect the USB cable while holding it. Watch the notification light (upper right corner). It will come on very dim red, then change to dim blue, hard to see. When it stays blue, you can release volume-up. 15sec was enough. fastboot devices reported the phone's ID number.

  • With USB connected to the USB-3.0 port, the key combination resulted in fastboot starting up but immediately rebooting (red, then blue again). If I held down volume-up through that, fastboot would be useable. Sort of.

  • With USB connected to the USB-2.0 port or the hub, adb reboot bootloader would successfully start fastboot.

  • To go through the whole installation successfully, I needed to connect USB through the hub to the USB-2.0 port.

  • In a different context a different user reported that a particular USB-3.0 hub, presumably badly designed, emitted enough EMI (electromagnetic interference) to prevent Wi-Fi and Bluetooth in the 2.4GHz ISM band from working on his laptop. Being aware of this issue, I did not blow off suggestions to isolate possible EMI from the Pioneer.

  • Another user installing LOS-16 did not succeed with his USB-2.0 port, but one of his machines, presumably newest, had a USB-3.2 port, and that worked. He recommends, try every port on every machine you have until one of them works.

• 6. Wing off to Sony's official unlocking site.

  • Read lurid warnings and discouraging texts.
  • If you haven't chickened out yet, select your device (Xperia XA2) from the dropdown list. Follow the instructions that appear.
  • Fill out the form: provide the IMEI (digits only, no spaces or hyphens) and click the boxes for voiding the warranty and service fees for warranty repairs.. You don't have to subscribe to the newsletter. Hit Submit. Respond to Captcha. Copy down your code (16 hex digits).
  • How to use the code: this is found on "how to unlock the bootloader" (separate page):
    fastboot oem unlock 0x0123456789ABCDEF
  • I did that command. Fastboot is waiting for something, but not using measurable CPU time. For 15 minutes. I unplugged the USB cable and it died having done nothing good or bad. See above for an intervention that brought this to life (thanks to LuK1337). On the retry this step took about 5-6 sec.
  • When I rebooted to the OS it showed a page saying Erasing, then rebooted again into the stock image, and it ran initial setup. This was the point where the phone's former internal flash was wiped.
  • Checking in the Service menu, my Suntory Blobs were in [ERROR] state (DRM keys gone). App data was gone. Etc etc, as promised.

Installing CyanogenMod (LineageOS)

Procedure for installing CyanogenMod (LineageOS), from LineageOS product page for Sony Xperia XA2 and the linked-to installation guide.

• This process is destructive. You need to back up any user files that you want to preserve. Is the SD card wiped too? HOWTOs vary in what they say, but I'm going to physically remove my SD card after doing backups that end up there. Also see the note after install custom recovery; I'll reinsert the SD card, back up the TA partition (unfortunately after DRM keys have been trashed), and remove the card again.

  In the stock image internal flash is encrypted, whereas in AOSP it is plaintext. This does not refer to decrypting files; it refers to wiping the partition and creating a new empty filesystem.

  Update: see Settings - Security & Location - Advanced - Encryption & Credentials - Encrypt Phone. It says Encrypted. There is no option to decrypt in place. This says to me that major parts of the phone's storage are encrypted with a key that is released (decrypted) when you enter your PIN, lock pattern, or fingerprint. This key is different from the one that formerly encrypted the stock image, which is erased (key and data) when LOS is installed.

• You need to decide whether to back up your TA partition (Trim Area) which contains your DRM keys for licensed preinstalled Sony content. See a previous section for extensive discussion of how and whether to do that. The conclusion was that this backup is too horrendous to be accomplished, and it is very dubious whether anyone has actually been able to use the saved DRM keys in a non-stock OS image like LineageOS or Sailfish.

  However, backing up the non-DRM content is prudent and important, and that backup will have to wait until after the bootloader is unlocked and the DRM keys have been trashed, since I'm going to do a TWRP backup.

• Just before the second try at installation (when LOS-16.0 was running), I backed up the configuration to Google Drive, which I normally don't do. Setup - Google - Account Services - Backup - (turn on). It says the backup is encrypted with your Google Account password and includes apps, app data, call history, contacts, device settings (incuding Wi-Fi passwords), and SMS. After enabling this, go to Settings - Accounts - (your Google account) - Account Sync. It will list backup categories, skimpy since you just turned on general backups. Dotdotdot menu, hit Sync Now. Well, it doesn't re-sync in any other categories than these. You have to add a backup account (the phone account isn't automatically used). Now it's doing it. Let's save my 2 photos as well.

  Whatever may or may not have been backed up, restoring it did not seem to produce the desired result, and I had to set up the phone anew.

• 2nd pass: For backup I installed Backup Your Mobile By Artur Jaszczyk. It couldn't get permission to put the backups on SD partition 2, but did work on SD-1. Took about 1 min. 760Mb, about 0.1Gb free on this partition, blecch. Installed AndFTP. Now copying it off the SD card into /tmp/jimc.junk -- Done. So far I haven't had an opportunity to actually test whether restoring this backup is useful.

• I've now started doing ADB backups onto my normal backup media. Again, this needs a test whether restoration is going to be useful. [Needs work]

• LineageOS product page for Sony Xperia XA2; it has a link to the installation guide.

• Follow link to Get the Builds Here. Image filenames have the form: lineage-16.0-202002XX-nightly-pioneer-signed.zip (where XX is the day), and you will need to download the SHA-256 checksum also. Do any of these builds have customer complaints? By Google search, there were no hits mentioning any of the download filenames.

  Downloading the 2020-02-23 nightly image and SHA-256 sum. Saved in /scr/orion-downloads-200208/ , 596Mb, took 7.4min. Checksum: OK, use sha256sum -c.

• Make sure the host computer has adb and fastboot installed. On Suse the package is android-tools. Version 9.0.0_r52 in hardware (experimental). The included programs are:

  • adb -- Android Debug Bridge, sends commands and file transfers from the host to the device, and can suck files.
  • fastboot (x86_64) -- Do fastboot --help | less for docs. It communicates with a lowlevel component of the phone's control programs, below the Linux kernel (bootloader?), and can flash partitions, reboot the machine, etc.
  • avbtool (Python) -- For working with Android Verified Boot Images. It can do things with SHA-xxx sums and RSA keys.
  • mkbootimg (Python) -- Packs a kernel, ramdisc, etc into a boot image.
  • mke2fs.android (x86_64) -- Creates a ext234 filesystem, presumably with Android idiosyncracies.
  • e2fsdroid (x86_64) -- looks like it can tweak an Android ext234 filesystem with Android idiosyncracies.
  • ext2simg (x86_64) -- Reads an input image or block device (I assume it has such an image installed), and writes an output image, optionally compressed and with CRC.

• On the phone, activate Developer Options. Procedure: in Settings - About Phone - Build Number, tap 7 times on it. Now turn on Settings - System - Advanced - Developer Options - Android debugging (also Debugging Notify). Also you need OEM Bootloader Unlocking (in Settings - System - Advanced), which the instructions tell you after you've already started fastboot. Once you turn this on and actually unlock the bootloader, this menu item disappears.

• Review the previous section, Struggles With Unlocking the Bootloader.

  Unlock the bootloader, see Sony's instructions. This means to make a setting so the phone will let you install a custom (non-stock) Recovery. Warning: when you do this, your DRM keys are gone forever. See prior discussion of the consequences. Once unlocked, it stays unlocked; you only need to unlock once per lifetime. You need an unlock key, which depends on your IMEI, which you enter into a form on the instruction page.

  Yes, looking on the Service menu, bootloader unlock is allowed.

• Download and temporarily install a custom Recovery other than the one provided by the vendor (which will refuse to install non-vendor images). TWRP is recommended, being reliable and easily available.

  • Select the most recent image from the list. The link is to a download page, not the image file itself.
  • Download the image (31Mb). And the PGP signature and the SHA-256 sum.
  • The PGP public key is on the index page.
    gpg --import TeamWin-TWRP-pubkey.asc
  • Now verify the signature:
    gpg --verify twrp-3.3.1-6-pioneer.img.asc twrp-3.3.1-6-pioneer.img
  • Oops! gpg: BAD signature from TeamWin <admin@teamw.in> and twrp-3.3.1-6-pioneer.img: FAILED; sha256sum: WARNING: 1 computed checksum did NOT match. But twrp-3.3.1-5-pioneer.img (1 week earlier) has a good signature. Using that one. Bug report was submitted.

  Why don't we flash TWRP permanently? Jimc's speculation: the LineageOS (CyanogenMod) image includes the AOSP Recovery, which will overwrite TWRP, and each periodic update will do the same. If I need TWRP's special features later, I'll re-flash it then.

  Update: If you follow the instructions you never really install TWRP; you send the image from the host computer and execute it on the spot. You'll need to do this twice during the installation procedure. LuK1337, the developer in charge of the LineageOS build for Pioneer, recommends against permanently installing TWRP.

  More update: The Pioneer has an A/B arrangement of basic partitions, so while executing out of one of these, the active slot, it can write a new system image into the other slot. If anything goes wrong, the active slot is unchanged and you can just do it over. But there is no traditional Recovery partition into which a non-stock Recovery program could be installed, and alternatives apparently are hard to get right.

• I'm going to back up my TA partition (minus DRM keys) using TWRP. The backup ends up on the SD card, so it has to be reinserted now. And removed after the backup. The procedure to do the backup can be found at the end of Backing Up the Trim Area.

• Download and install your LineageOS image. (See the installation guide for the command lines.) Repeat the process for Google Apps and AddonSU (to root the phone). Since I want to make Google Pay work, I'm going to use Magisk as the rooter. Images to be downloaded, used in this order:

  • twrp-3.3.1-5-pioneer.img (Recovery)
  • lineage-16.0-202002XX-nightly-pioneer-signed.zip (OS image) (adb reboot sideload, after flashing it). Don't reboot LineageOS itself, until after you've flashed Google Apps.
  • Google Apps, the nano size variant, recommended by LOS instructions.
  • Magisk root framework (and the Magisk Manager app when the phone is running). See the section for Magisk for how and when to flash it.

  The procedure totally wipes your phone and makes internal flash unencrypted. Update: this is what the instructions say, but the phone claims that it's encrypted.

How the Installation Step Worked Out

• Booting TWRP. On the Pioneer you shouldn't actually install TWRP due to the A/B partition layout; instead, do the adb boot $file thing, see below.
• Connect the USB cable between the phone and the host. Jimc connected a USB-2.0 hub to the laptop's USB-2.0 port, and plugged the cable to the phone into that. See Struggles With Unlocking the Bootloader for why this was necessary.
• adb devices # It starts the adb daemon on the host, and the device ID is shown.
• adb reboot bootloader # Watch the indicator light: it goes dim red, then the phone vibrates and it goes dim blue. It's hard to see.
• fastboot devices # The device ID is shown.
• fastboot boot twrp-3.3.1-5-pioneer.img # Shows complaint can't be checked for corruption, lock your bootloader (which you ignore). Shows Sony splash screen. Shows TeamWIN splash screen. You need to authenticate with your unlock pattern (or PIN?) There's a screensaver, don't waste time taking notes :-) If the screen goes dark, press the power button and then swipe. There's an option under Settings to turn off the screen saver.
• Keep system readonly? Yes (if you want to use Magisk later). Now you're on the main screen.
• Hit Wipe. Select Format Data. Swipe to Wipe. Back (twice) to the main screen. Advanced Wipe. Select System. Swipe to Wipe.
• Back to the main page (3 times). select Advanced. Select ADB Sideload. Swipe to begin.
• On the host, adb sideload lineage-16.0-20200308-nightly-pioneer-signed.zip # It proceeds, not too swiftly, about 1% per each 4 secs, once it gets going. The screensaver turns off the screen; ignore it. The progress readout seems to only go up to 47% (??)
• The instructions say to adb reboot sideload so you can install Google Apps; you must do that before rebooting. But that was a mistake; you'll end up in LOS Recovery which is useless for this. When I tried…
• adb sideload open_gapps-arm64-9.0-nano-20200307.zip # Signature verification failed! Install anyway? Not a chance. For reference, we're on slot A. I'm not sure why this worked enough to download the zip file but botched the signature, but I think we were supposed to get back into TWRP. Update, when upgrading to LOS-18 I blew off this warning and have not noticed GRU hackers on my phone.
• Repeating the temporary installation of TWRP. Hit Advanced. Select Reboot to Bootloader.
• fastboot boot twrp-3.3.1-5-pioneer.img # (On the host.) Work through to TWRP main page. Hit Advanced. Hit ADB Sideload. Swipe to begin.
• adb sideload open_gapps-arm64-9.0-nano-20200307.zip # Goes faster. Shows GAPPS progress titles. Eventually the screensaver blanks the screen. Progress readout goes up to 57%.
• Should I follow instructions and do adb reboot? Or use the reboot button on TWRP? I'll follow instructions. Lock Bootloader, Sony, repeats these screens. "Can't load Android system", I selected Try Again. Same result. 3rd time, I told it Factory Data Reset.
• Now it shows a LOS boot animation, and after an unusually long time (about 120sec) it started Initial Setup.
• At various points there were complaints about can't bind-mount /sdcard/ on somewhere. I ignored these, hiss, boo.

LineageOS Initial Setup

• EN_us is preselected (can be changed). No cute and colorful toucan graces the initial screen. Hit start.
• Connect to Wi-Fi. Give password. It connects, and checks for updates. Can't tell if it actually found any. The circle that reassures you that it hasn't died looks very much like Sony's. Actually the whole setup sequence looks very similar, meaning that Sony didn't hack it up too much.
• Copy apps and data. Yes, hit Next. Choice of cloud backup or iPhone, neither of which I have. Sony could successfully copy from an Android phone with NFC. Hit Don't Copy.
• Enter your old lock pattern (or PIN?) to decrypt any encrypted data. LOS instructions say user data is supposed to be not encrypted now, but per Settings - Security & Location - Advanced - Encryption & Credentials - Encrypt Phone, it's still encrypted.
• Authenticate on your Google account. It does a bunch of non-obvious stuff.
• Google Services: Back up to Google Drive: No. Automatic freeing space: No. Apps use location: Yes. Allow scanning (even if Wi-Fi or Bluetooth is set to off): No. Send usage and diagnostic data: No. Automatically install updates and apps: you can't turn this off. Later I'll try to put this in manual [succeeded].
• Fingerprint setup: skip, do later.
• Set PIN. You can also set other options, like a pattern or password.
• Android Assistant: Not now (or ever).
• Etc: font size, wallpaper: Skip, do later.
• LineageOS features: Diagnostic and usage data: Yes (preselected). Privacy Guard: Yes.
• Final screen with LOS logo: hit Start. OK, the UI is running with a wallpaper that looks like a cow with blue splotches.
• From Sony stock to working LOS used 5% battery (89% to 84%) but the phone would have been charging from the host for some of the time. Elapsed time: about 1.5 hours.

At this point I shut down the phone and reinserted the SD card. I tried to use sticky tape again to re-extract it but could not get it underneath the tray.

Installing Magisk and Procedure Confirmation

Help with Installing Lineage on Xperia XA2, OP davidovski on Reddit (about 2019-03-xx). He describes the procedure he followed, which to jimc seems to not agree with LineageOS instructions. He gets a public key verification error on (I'm pretty sure) the LineageOS zip file. shamanonymous replies: You definitely don't need to install TWRP permanently. The XA2 uses the new A/B system image method, so there is no recovery partition… I only needed TWRP for reinstalling root, but now Magisk is also A/B aware, so I just have to run it after the Lineage OTA.

He continues with a summary of the install process (with extra notes from jimc):

• fastboot boot TWRP (run fastboot on the host, tell it to send the TWRP image and execute it, but it doesn't get installed in the nonexistent Recovery partition.)
• ADB push the LineageOS image, and Google Apps. (Jimc says: the official instructions have you sideloading instead.)
• (Jimc says: he omits wiping steps that are in the LOS installation guide.)
• Install the Lineage zip. (Jimc reminds: don't reboot into LineageOS before installing Google Apps.)
• Install Google Services/Apps.
• Boot, do initial setup.
• If you want root, do the fastboot thing again to run TWRP, and install the Magisk zip. Jimc's addition: you first have to become a developer and turn on USB Debugging. Progress readout went up to 68%. It takes a while to install itself but does finish. Yes the phone reboots successfully.
• (Jimc's addition:) I also downloaded the Magisk Manager APK; I'll use adb install MagiskManager-v7.5.1.apk (after rebooting).

He continues with how to handle a Lineage update. It will write the new image to the other (A/B) set of partitions. If botched it will say so and exit. If OK it will change that partition to be active. Before you reboot, start the Magisk Manager app and click Install - On The Other Partition. Do this, then reboot. Lineage OTA updates work just fine in this configuration.

thinkofdoc responds: LuK1337 is the primary developer for this build. He advises against installing TWRP, and provides a link to a custom TWRP image that works better with the XA2 image. (jimc says: looks like the reported issue is very minor and in 2020 it will be long forgotten.)

Respondent moroi (2018-08-14 on XDA-Developers) (this is post 311, last on page 31 of this very long thread). He tells why you mustn't boot LineageOS before installing Google Apps: on an A/B system, TWRP installs the LOS zip into the inactive partition, then (on success) marks it active. But it installs Google Apps in the active partition. So after installing LOS you have to reboot back into TWRP, not into LineageOS. This seems a little strange to jimc. My speculation (without objective evidence): TWRP installs everything to the inactive A/B set of partitions. On success with LOS it swaps the active/inactive markings. But the partition isn't really active until you reboot, so if you install something else (Google Apps), it will again go in the currently inactive partition where LOS was just installed, which is what you want. Whereas if you rebooted into LOS (without Google Apps), then got back into TWRP and installed Google Apps, it would go into the then-prevailing inactive partition, where LOS isn't.

Running Magisk Manager to set up Magisk:

• It says that both Magisk and Magisk Manager are up to date.
• To do the SafetyNet test, you have to hit the circle-arrow icon precisely, not just tap the text.
• Default settings (from hamburger, upper left corner):
  • Dark theme
  • Language EN_us
  • Didn't hide the manager yet
  • Yes check for updates, stable channel
  • Not core only
  • Yes hide Magisk
  • Systemless hosts (I clicked, it added the systemless Adblock module, don't do that in future unless you really want Adblock.)
  • Biometric authentication (greyed out, fingerprints not set up)
  • Superuser access, apps and ADB
  • Multiuser mode, only owner has root access
  • Mount namespace, inherit from requester
  • Automatic response: prompt, timeout 10sec, toast notification
• I turned on hide manager. Default name is Manager, I used MgkMgr.
• Manager exits. I started it under its new name. It says, Upgrade to full Magisk Manager to finish the setup. Download and install? (I hit OK.) It reinstalled. No visible difference in the GUI. Settings appear unchanged except for Restore Magisk Manager, which you would use to change back to the default app name.
• Magisk Manager's Java namespace prefix was com.topjohnwu.magisk, changed to com.Y3.AI.Xv.bqnX.Gf .
• SafetyNet check still does nothing.
• In settings, turning on Magisk Core Only Mode, as recommended for Google Pay. (Reboot to apply settings, it says.) Rebooting.
• Now SafetyNet check does something: Magisk Manager is FOSS. OK to install a proprietary extension from Google containing the GoogleApiClient that can do the check? (OK) Result: ctsProfile is false, basicIntegrity is true. Got to fix this.

Update: The following hodge-podge of forum posts documents unsuccessful attempts to get the phone to pass ctsProfile. Suggestion, pick up at Google Pay on Rooted Phone. A lot of the forum posts linked here refer to how to take an image that Google won't certify, and make it look like one that can be. But I had no problem registering this LOS image with Google.

[FIX] SafetyNet Failed: CTS Profile Mismatch Errors — Full Guide! by Arvind Rana on DroidHolic (2020-03-05). CTS means Compatibility Test Suite. Among quite a lot of other items to check/fix, he says USB Debugging (in Settings - System - Developer Options) is a red flag for CTS. For me, turning it off did not solve the problem.

Cant find Enable Busybox in Magisk settings and ctsProfile not succesfull, OP rolferikalfheim on XDA-Developers (2017-09-14). He has my symptom, no Busybox option in Magisk Manager, and ctsProfile test fails. Digeridoohan (moderator) replies, that setting has been absent for some time. Use the Busybox module in the Downloads section of Magisk Manager.

Magisk and MagiskHide Guide - SafetyNet maintained by didgeridoohan. One issue: check in Play Store settings (at the bottom) for Play Protect certification: Device is not certified. The linked page gives some fixes. Basically, make sure you can pass SafetyNet. Jigger some props to match a known certified kernel/system. Clear data for Play Store. Reboot. The main symptom of this failed certification is that certain apps won't appear, or will appear but won't be installable. Netflix is the one most often complained about. I did this but still can't pass ctsProfile. The item for Play Protect Certification has disappeared, and the Netflix product page can be displayed. (I didn't try to install it; I didn't try to see it before improving my compliance level.)

Following instructions in the above wiki and various forum posts, I disabled Magisk Hide, rebooted, tested SafetyNet (ctsProfile false, basicIntegrity true), enabled it again, rebooted, and tested SafetyNet again. Still ctsProfile false.

LineageOS is supposed to be certified by Google. But from time to time updated versions fail to be certified. Basically, install the Magisk modules for Busybox (prereqisite) and MagiskHide Props Config. Guess which device to emulate. That got him past SafetyNet, except for one game. He had forgotten to spoof the name of MagiskManager; doing that brought that game to life.

After clearing data for Play Store and rebooting, I configured settings again, then selected My Apps and Games. It showed a panel saying Install apps you've used before. It found 10 apps: apps from the Sony stock image (that I never opened) like PlayStation App, three that I had on the old phone, and one that I actually installed on the Sony stock image. Selectively installing them on the Pioneer.

I found out a better way to install previous apps: Play Store web URL, likely you can click to install, on the phone. It has most of them, including some that I must have had at one time but then uninstalled.

In MagiskManager I installed Busybox by osm0sis (prerequisite) and MagiskHide Props Config by didgeridoohan. Reboot afterward. Following instructions: start a terminal (LOS has one, unless you suppressed it in initial setup), command props and grant root access, choice 3 to enable better hiding (and reboot if it prompts), then choice 1, and then 'f', and pick your fake OS.

Setting Up LineageOS Settings and Apps

Now I have LineageOS running on the phone, with Magisk. Here's a journal of what I did to set it up.

• Settings - Network - (advanced) - Private DNS - It was set to Automatic, and hostnames in the internal cft.ca.us domain could not be resolved (by Firefox). Turning this off brought DNS to life. Update: Off means to use the DNS server given by DHCP. Automatic means to attempt DoT (DNS over TLS on port 853) on DHCP's DNS server with a 15sec timeout, then fail over to UDP or TCP on 53. I don't know why this was failing. Another option is to give the hostname of a working DoT server, but there's a chicken and egg problem: how is it going to be resolved? An IP won't do because the server's certificate has to certify the name used to contact the server. Update #2: Later I set up a DoT server on 853 and Automatic worked.
• Install Firefox. I'm installing this first to get the mail Google sent to pick up my list of installed apps. But the list was useless: a combination of Sony stock apps, and just a few from the Galaxy S5.
• It can't find http://jacinth.cft.ca.us/~jimc/ . Obviously a DNS issue. http://jacinth.jfcarter.net/~jimc/ is fine. Why didn't it use DNS from DHCP? See the note about Private DNS, 2 steps previous.
• Temporarily set my Firefox home page to http://jacinth.cft.ca.us/~jimc/
• Import the CFT root cert. It's on https://diamond.cft.ca.us/~certauth/ and Firefox doesn't have the trust anchor. Make an exception and download it. That done, Jacinth's secure server page (port 1447) is shown. So is the Roundcube mux page.
• Import jimc's user cert. But it couldn't open the PKCS#12 file, probably a permission problem. The cert installer interface with OpenVPN For Android succeeded in installing it.
• Following the link to install all my 100+ apps. Using IMAP login (works). This is the union of what was on the Sony stock image and the old phone. Got to suppress some of them. I skipped over these apps: Ring, Wyze,… All the apps are allegedly installed. Look for GPS Time (sourcewalker.net), I don't remember installing that.
• So where is the app drawer? Long press in launcher background. Older versions can be made to emit an Appdrawer icon, but not this one. Update: swipe upward from the bottom to open the app drawer.
• Settings - Accessibility - Screen Rotation - turn on.
• Installing Yet Another Phase Beam by Davide Nabais. Follow the link for how to adjust the colors. The AOSP wallpaper app for Android-9 (LOS-16) did not show the live wallpapers, and Wallpapers by Google was needed. On Android-10 (LOS-17.1) the stock app does show live wallpapers.
• It looks like ClockSync is no longer available. Installing NTP & GPS Clock [ROOT] by (name in Japanese) (new). Both NTP and GPS work as clocksources, and when you hit the clock icon or long press on the offset text, it does sync the system clock to them, but apparently only if the offset is greater than a limit, not yet determined but somewhere between 1 and 180 sec. Reqires root, of course. Sync on a schedule is coming soon.
• Importing my bookmarks and pinned sites for Firefox. There's got to be a better way (short of creating a Mozilla sync account).
• Setting up Dropbear. Path is ~jimc = /storage/emulated/0/jimc . There is 1.9Gb used out of 19.5Gb total, card total is 32Gb. I copied the authorized_keys file and host keys from the SD card. And it worked the first time!
• Restoring jimc's homedir. The script to back this up also has an option to restore, but it was vaporware. I added that section, and it worked.
• Now in Firefox I revised the bookmarks and pinned sites to use the local web dir as my home page.
• Pairing my Bluetooth headset. Works. I can now listen to music. This makes the phone ready for production use (mostly).

Launcher Icon Layout

The app launcher (Trebuchet) has an unlimited number of pages. It wants to open on the leftmost of them, whereas formerly I used the middle of three as my home page. For launcher settings, long press in the background and a menu will open with items for settings, widgets and wallpapers. I haven't found a setting to tamper with the home page. Here are my settings; * indicates other than the default.

• Allow edit (user can move or remove icons)
• * Show Google app (launch by a left-to-right swipe): turn off
• * Allow home screen rotation: turn on
• Grid size 4x5 (option is gone in 17.1; it's always 4x5.)
• Add icon to home screen (when an app is installed)
• Show icon labels on desktop
• Show icon labels in app drawer
• * Show icon labels in landscape mode (turn on)
• Swipe down to show notifications (gone in 17.1, on by default)
• Hidden and protected apps (left at default). Hidden means they don't appear in the app drawer. Protected means authentication is required to run them.
• * Change icon shape. Default is circle, changed to square, which even so has rounded corners. (Gone in 17.1)
• Do not disturb (submenu) -- mostly defaults except:
  • * Schedule: Turn On Automatically: edit (click on) Sleeping or add a new rule, then turn it on and set Days (all), Start time 23:00, End time 07:00, alarm can override end time (turn on).
• Notifications (submenu); click on Notification Dots in the containing menu.
  • On lock screen: default is to show all notifications. I tried to limit what's shown, but notifications might need to be responded to with no delay, like from the Shake Alert L.A. app. So I reverted to the default.
  • Allow notification dots (a bubble over the app's icon, if it's active.)
  • Battery light (you can change the color, brightness, and blinking.)
  • * Notification light (ditto). The default is green, and I changed the light for SMS message waiting to blue flashing, matching the stock configuration.
  • * Notification sound -- default is Argon, changed to CyanDoink. (Gone in 17.1, probably moved to Sounds. Yes it is.)

My icon layout. Many apps are reached only from the app drawer. I'm going to duplicate what's on the old phone (Samsung Galaxy S5) as much as possible but exchanging pages 0 and -1.

Miscellaneous settings: Sounds. In LOS-17.1 based on Android-10 Quiche, they have a ton of new sounds. When you set a sound you can complete action using Sounds (i.e. the new sounds) or Media Storage (the old sounds). Find the settings in Settings - Sound - Advanced - Phone ringtone, with the notification and alarm sounds just after. You can also set the ringtone from the phone app (Settings in dotdotdot menu).

• Ringtone. Default is Orion which is not too bad. There are 111 to choose from. The one I used before is (presently) called Draco; the names and choices seem to change per Android version. I chose Draco again, with Sheep and Canis Major as 2nd choices.

• Notification. I changed to CyanDoink, which is annoying but aggressive enough to not be missed.

• Alarm. The default is Hassium, which is horrible and has none of the cachet of Nuclear Launch. Among the provided sounds, Platinum is the best of a bad lot. I would really like to use Walk_in_the_forest.mp3 (in ~jimc), which you could add by Add Ringtone, once it's been restored from backups. Confirmed that Huge Digital Clock (digiclock) can find and play this file when the alarm goes off.

How to disable battery optimization: This is needed by apps that run in the background and have time sensitive activities, specifically Kitchen Timer and Shake Alert LA, To set: Settings - Apps & Notifications - Advanced - Special App Access - Battery Optimization - (change from Not Optimized to All Apps) - select app - Don't Optimize.

Google Pay on Rooted Phone

On this phone I would very much like to use Google Pay, a digital payment app which has different security exposures than the traditional use of a physical credit card with a static and stealable account number. (Apple Pay pioneered getting merchants to install the software and a NFC reader, but Google Pay and possibly others can use the same protocol even if the reader is Apple branded.) The digital payment protocol uses a one-time code so the thief cannot steal the transaction record off the wire and use it for a nefarious transaction, a notorious vulnerability, but the credit card's number resides in the phone, where malware could steal it if able to subvert very difficult security involving SELinux. Obviously a rooted phone is a serious threat to this security model, and Google Pay will clam up if it detects root capability. The system service that handles root checking is called SafetyNet.

Therefore if you want root, it has to be stealthy. Of course the easiest solution is to not root your phone. So what do I use root for?

• ClockSync by Sergey Baranov. It requires root to sync the time automatically. It does have a mode for manual syncing on a non-rooted phone, but you have to watch the display and hit a button at the optimal moment. The Android Developer site shows a class called TimeManager which can set the current time, set the timezone, set 12 vs 24 hour time, and turn NTP sync on and off. That suggests that a recent AOSP image should have these items in Settings. But I find no trace of them anywhere.

• StrongSwan is supposed to provide the server pushed DNS server addresses to the VPN API, but this doesn't seem to be happening. The DNS Forwarder app requires root to establish the DNS server manually by cowboy programming. Not having DNS for the VPN is kind of a showstopper.

• Running roughshod over file permissions, particularly storage permissions. This is for ad-hoc tasks like copying crypto keys onto the device in the required locations.

• I use Total Commander for such file operations, which uses root if available.

• Backups are done with user permissions. This means they aren't very complete. I could do better with root permission.

• I think my conclusion is, if I can deal with the clock sync issue, it would be fine if I could manually turn root on and off, and have Google Pay work when it's unrooted.

• Magisk is the currently popular rooting app/framework, by topjohnwu. Version as this is written (2020-02-20): v20.3; you should also have Magisk Manager v7.5.1. Magisk Release and Announcement Thread on XDA Developers; distributed from Magisk Github site.

• Note, it's a cat and mouse game; Google improves SafetyNet and the developer improves Magisk to get around it. Do not update Google Pay (both the front and back end) until Magisk is correspondingly updated.

How to Use Google Pay on Rooted Android Phones by Rohail Khan (2018-07-18). Use Magisk root. Configure Magisk to use BusyBox, Magisk Hide, and Systemless Host. Enable Magisk Core Only (and reboot). Look for the SafetyNet lack-of-root report. It should pass, and both Google Pay and rooted apps should work. Jimc's note: I think turning on Core Only would preclude spoofing the OS type, if that's needed to pass the ctsProfile check.

Google Pay no longer worked, OP henban89 (2019-03-03). It turns out that an updated version of Google Pay was aware of Magisk. A respondent gives instructions to revert to a back version and prevent it from being updated.

Google Play Certification: What it is, and how it affects you by JavelinAndArt (2018-04-09). This is what SafetyNet checks. You need to register your Google Services Framework. If you aren't passing SafetyNet, you will be prevented from downloading some apps, and others like Google Pay will not work at all. See also SafetyNet: What it is, and how it affects you by JavelinAndArt (2017-06-05).

• I followed the link in the Google Play Certification knowledge base page (Google authentication required).
• To get your Google Services Framework ID, you need to do adb root.
• However: adb: unable to connect for root: device unauthorized. This adb server's $ADB_VENDOR_KEYS is not set.
• I had Do Not Disturb mode on. Turned off. Now adb root yields adbd cannot run as root in production builds. adbd cannot run as root in production builds #425, OP peyer (2018-07-29): didgeridoohan responds, probably caused by MagiskHide changing ro.debuggable to 0. It needs to be set to 1 for adb root to work. Turn off MagiskHide (and reboot)…
• Continuing that response: …change just that prop back to 1 with MagiskHide Props Config. Jimc will do that once I get through framework registration.
• Now it says root access is disabled by system setting - enable in Settings -> System -> Developer options. This setting (Root Access) was not available previously.
• Now I can run the command (in a shell):
  sqlite3 /data/data/com.google.android.gsf/databases/gservices.db "select * from main where name = \"android_id\";"
  Result looks like: android_id|1234567890123456789
• Fill the decimal number (not including the '|' symbol) into the input field and hit Register. It thinks for about 30sec and then pops a toast saying Device Registered.
• Per the LineageOS blog post, every time you do a factory reset you will have to register again, presumably because the number has changed.
• Now, turn Magisk Hide back on (and reboot), and test SafetyNet. Ta-da, ctsProfile true, basicIntegrity true! (Now let's try to keep it that way.)

Working: Magisk with Google Pay as of gms 17.1.22 on Pie OP BostonDan (2019-05-14). He gives a credible looking procedure for getting around the security check.

Before I try to make Google Pay work, I need to install Magisk Manager, try the SafetyNet check, and get it so it passes SafetyNet. (And don't forget to obfuscate the name of the MagiskManager app, which some programs check for explicitly.) On the first installation attempt, I ended up with a failure in ctsProfile, a server-side test; basicIntegrity (client side) passed. Forum posts about troubleshooting this invariably say, wipe your phone, reinstall the OS, and step by step, check if you're passing SafetyNet. The point at which it starts failing gives a clue what intervention may fix it.

Repeating installation up to the point of activating Magisk.

• Turn USB debugging back on.
• Reboot into Fastboot, either by adb reboot bootloader or from the option on the power menu (long-press power button).
• fastboot devices -- device ID is shown.
• fastboot boot twrp-xxx.img -- reboots into TWRP.
• Wipe - Format Data - yes - swipe to wipe. (Back 3 times)
• Wipe - Advanced-wipe - System - swipe to wipe. (Back 3 times)
• Advanced - ADB Sideload - swipe to begin. On the host, adb sideload filename.zip (for LOS.zip). Screensaver will blank the screen. Progress readout goes up to 47% and stays there for a while.
• Although you can go back and run sideload again for Google Apps, its install script says Incompatible device detected… please download the correct version for your device: arm. Error code 64.. Tried twice, same result. Instead, adb reboot bootloader, which works on TWRP). Start up sideload same as before. (Don't wipe anything of course.)
• On the host, adb sideload open_gapps-xxx.zip
• Now, hit Reboot System.
• Can't load Android system. Your data may be corrupt. You may need to do factory data reset… That's right, hit that line. It reboots, and this time we get the LOS boot animation. Slow! Takes about 120sec until setup gets going.
• Go through initial setup.
• Turn on USB debugging.
• Install Magisk Manager: adb install MagiskManager-v7.5.1.apk
• Run Magisk Manager. Yes, Magisk is not installed, don't install it yet. Do a SafetyNet check. Allow it to download the proprietary Google extension. ctsProfile false, basicIntegrity true. Blecch.
• Turned USB debugging off again. For the hell of it, disabled the master switch for developer options. Didn't help, same result.
• Can't make progress until I actually install Magisk. Hit Install on that line item. This is version 20.3 (the one I downloaded). Method: download zip. Re-do with select and patch a file. It drops you in some dir with no contents. Hit the hamburger (upper left) and pick Downloads. Click on the zip file you just downloaded. Blecch, unknown image format. adb push Magisk-v20.3.zip /sdcard/Downloads/Magisk-v20.3.zip Ended up in /h3123/Downloads/ (h3123 is its idea of the hostname, and is the internal flash. /sdcard/ is at best useless for this. Again: unsupported/unknown image format. Consensus of forum posts on this error message is that the procedure to select and patch a file in MagiskManager does not mean to select the downloaded zip file and sic MagiskManager on it. You need to do an arcane procedure to unpack the LOS zip, merge it (how?) with the Magisk zip, repack, and select the result. There are reports of one or two people accomplishing this, but the usual way is to start TWRP and sideload Magisk-v20.3.zip. This worked once for me and I'm going to try it again. The installer script actually does the repacking and patching operation for you.
• Now in Settings I can Hide Magisk Manager. And download whatever additional stuff for the Full version. Still fails ctsProfile. Leaving Core Mode off.
• Downloading and installing modules for Busybox and MagiskHide Props Config. Reboot once this is done. [Didn't do this yet.]
• If you have a custom system image like LOS, you need to register your Google Services Framework ID, see separate section.
• Now it passes SafetyNet.

Now to see if Google Pay will actually function. Since there's an unconfigured instance of Google Pay in my backup, I'll restore state first. I'm using Backup Your Mobile By Artur Jaszczyk. Re-installing this app. There's also an active instance of Magisk Manager. The current instance is called MgkMgr; the backed-up one is … It restored 0 application data, 1 Wi-Fi password, 48 system settings, and the app list. Please restart device. It should have restored application data; I don't know why it didn't. Play Store did not seem to know which apps it was supposed to reinstall. Neither did it restore the icon load of my home screen. In Settings - Apps - See All Apps - (name of app) - Permissions - turn them all on; in the dotdotdot menu there's one item for all permissions. Restore again. This time it went much faster probably because the target data didn't need to be changed. Not much improvement. I think this has failed. Next time around I'll try ADB backup.

Installing Google Pay. With Magisk core only mode on or off, and with the Google Services Framework ID registered, it passes SafetyNet. But Google Pay still says This phone can't be used to pay in stores. This may be because it is rooted… Turned on Google Pay in the Magisk Hide list, joining Google Play Services. Didn't help (but I left it on). Turned on Google Play Services. And rebooted. Didn't help (but I left it on).

Google Pay Magisk Discussion Thread, moderated by Didgeridoohan, response #5 by JarlPenguin (2019-03-03). Discussing similar symptoms seen last year, an update to Google Play Services caused it. Wait for a version of Magisk that it can't resist. Temperorarily revert to the previous version by using the procedure shown.

How to get GPay to work on rooted Xiaomi Mi9 by smohanv (2019-09-24). He links to a Magisk module and lists the operations that it does if you want to do it by hand. Basically, lie about whether (something) has been attested, and change the mode to 440 so your fix can't be reverted. See response #11 by 73sydney and follow the links there, if downloading the module.

Conclusion: I need to just be patient. The procedure for reverting to a prior version requires a prior version to revert to, which I don't have. Once I have a working version (of Magisk, Google Play Services, and Google Pay), I'll save them so I can revert if needed.