Sony Xperia XA2 (2020)
Upgrading to LOS-17

LineageOS-17.1 is based on Android-10 Quiche.

Before doing this upgrade I'm going to do a TWRP backup, depositing the result on partition 2 of my SD card. Steps:

• Connect the USB cable for ADB debgging. I'm running it through the hub, like I did for unlocking the bootloader.
• Got a problem: TWRP doesn't know about the second partition of the SD card. I'm going to back out of this, take out the SD card, reformat it with 1 partition, and start over.
  • parted /dev/mmcblk0 (on the host)
  • print (make sure you have the correct disc!)
  • mklabel gpt
  • quit
  • yast2 partitioner -- In hindsight it would have been easier to not use the GUI, just make the partition in parted.
  • select mmcblk0
  • Create partitions: 1: 1Mb BIOS boot partition, do not format. Guess what, the YaST partitioner automatically leaves 1Mb unallocated at the beginning. Forget the BIOS boot partition.
  • 1: entire disc, for data/ISV, ext4, partition ID 82 (Linux), label 8GB-1 (not settable unless you mount it which you aren't going to do on the host), yes format.
  • fsck -C /dev/mmcblk0p1
  • tune2fs -L 8GB-1 /dev/mmcblk0p1
  • Reinstall the SD card in the phone. Yes it's mounted; ext4 is OK and the format doesn't have to be fat32. Total Commander can create the backups directory on it.
  • The SD card's various partitions (in my case, just one of them) are mounted (bind mounts) in various arcane locations. Let UUID = the UUID identifying the filesystem on a particular partition on the card (hyphen separated strings of 8,4,4,4,12 random-ish hex digits). Look in /dev/disk/by-uuid for a symlink to the block device's own inode. The original mount is to /mnt/media_rw/$UUID, and this is mounted to /mnt/runtime/{default,read,write}/$UUID. It is also in /storage/$UUID. /sdcard is not a link to the SD card; it's a link to the user's area in internal flash, the part that is not included in the TWRP backup. There is no convenient symlink to any of the partitions on the SD card.
• Running TWRP again and trying the backup. Go through initial screens and enter your pattern to decrypt data. Arrive at TWRP's main screen.
• Hit Backup. On the backup tab, pick partitions to be backed up. I'm including Boot (64Mb), System Image (2896Mb), Vendor Image (850Mb), Data (3466Mb), 7276Mb total estimated. Oink. The non-image variants are (said to be) tar files, but I'm backing up everything including free space, to avoid, after a restore, possible problems with paranoid security that would keep it from passing SafetyNet.
• Options: enable compression (compression isn't used on Images). Select Storage: Micro-SD (7362Mb free, just barely fits). You don't get an opportunity to naviate to a directory -- unless it's OK to prefix the output filename with a directory or path. Swipe to back up. Took 980sec (16.3min).
• The result ends up in $RealSD/TWRP/BACKUPS/$DevID/$prolix_dirname/ where $RealSD is one of the UUID-based pathnames mentioned above, $DevID is the phone's serial number as reported by adb devices, and the prolix dirname includes the date, time, kernel build ID, and something that looks random. Inside that dir you find separate image-type files (dd from the raw device to this file, no compression) for each of boot, system and vendor, plus three different segments of the /data dump, which may or may not be tar files and may or may not be compressed. Each has an accompanying sha256 checksum.
• If I'm going to do this kind of backup repeatedly, or if (as is likely) the content grows, I'd better get a bigger SD card.

I have a modest amount of confidence that the backup can be restored and will work, unlike confidence and experience with other backup methods.

I/you also need to back up user content in /data (/storage/emulated/0) using my/your normal procedure. This content is not included in the TWRP backup. Aargh, the backup bloated up to 2.30Gb from 0.39Gb and took 35min vs 4min. Probably one of my various backup attempts that won't be restorable.

Now for the upgrade. Read the version-to-version upgrade instructions first. They are almost but not quite identical to the instructions for installing LOS over a stock image. Here's a summary adjusted for jimc's situation:

• 0.5. I checked the available LOS-17.1 packages. A search on Google revealed no complaints or warnings about any of the 4 nightly versions. So I picked the most recent one.

• 1. Download the install package (.zip file) for the new version, as well as Google Apps (Nano variant for arm64, Android-10) and Magisk. I'm going to use the Magisk app to install Magisk, once the OS is installed. For reference, I'm storing the downloaded files in xena:/s1/sony-xperia-xa2/los-17.1 .

  URLs of download directories:

  • https://download.lineageos.org/pioneer
  • https://wiki.lineageos.org/gapps.html

  These files were downloaded. The LineageOS image had a checksum file, which was checked successfully, but Google Apps didn't.

  • lineage-17.1-20200425-nightly-pioneer-signed.zip
  • open_gapps-arm64-10.0-nano-20200422.zip

• 3. In Settings - System - Advanced - Developer Options, make sure USB Debugging is turned on, and also Root Access Options should be at least ADB Only. (Not clear if there's a reason not to have ADB and Apps. Hey, the ADB and Apps choice is gone.) If you're coming from LOS-17.x this setting is called Rooted Debugging.

• 2. Make sure ADB is still available on the host. Connect the USB cable from the host to the phone; see Struggles with Unlocking the Bootloader for nasty details of this seemingly simple step. Do adb devices to check the connection. Confirm ADB execution on the phone. Then you need to do adb root and confirm again.

• 4. You don't need TWRP to do this upgrade; you can use the LOS recovery program. On the host do adb reboot sideload. Update: this is true for installing the LOS upgrade, but if you want to also install Google Apps, you will need to boot TWRP for that.

  The phone reboots (unlock bootloader; Sony splash screen; screen goes black and no indicator light).

• 5. Run adb sideload package.zip. This is the install package for the new version.

  Screen comes on with LOS logo. Ignore message about can't mount (whatever), stopping adbd (which it has to do, to reboot into fastboot). It shows steps in the installation. Verification took 56sec. Installation took about 3min (estimated). It ends by dropping into LOS Recovery.

• 6. For Google Apps, repeat steps 4 and 5. Do not allow the phone to reboot into the OS until after Google Apps is installed.

  If you do adb reboot on the host, it doesn't happen. adb devices reports the device is unauthorized (so how do you authorize it?) Hit Advanced. Choices are reboot to bootloader, reboot to recovery, etc. Trying reboot to recovery. No improvement, same options but without nice GUI. Reboot to bootloader. It doesn't go through the splash screens; screen goes black but the notification light is dim blue.

  fastboot devices shows the phone (whew!) Now booting TWRP: fastboot boot twrp-3.3.1-5-pioneer.img. Proceeds normally through the various splash screens. On the main screen select Advanced - ADB Sideload. On the host, adb sideload open_gapps-arm64-10.0-nano-20200422.zip It shows progress messages from the gapps installer. Installation successful.

• 7. Finally, adb reboot.

  Actually, I'm going to hit TWRP's Reboot System. Splash screens. And it shows the LOS boot animation, same as in LOS-16.0.

  We're setting up permissions. Keep existing permissions. In Settings - System - Advanced - Updater, it says we're updated to Android-10. We don't have developer options. About Phone - Android Version says it's 10 also. Give self developer privileges, tap the Build Number 7 times. It says you've already enabled developer mode (???) And now Developer Options is in the System menu.

  In Settings - System - Developer Options, turn on Android Debugging (should already be on from the old OS), and Rooted Debugging (not yet on, allows running adbd as root, i.e. doing adb root on the host).

So what have I noticed that's new in LOS-17.1?

• All(?) of my apps seem to still be there. I haven't yet run Google Play updater.

• Trebuchet icon layout is unchanged. Live wallpaper (Yet Another Phase Beam) is still alive.

• We're using the correct DNS, what DHCP on Wi-Fi would have told it. According to Hurricane Electric Network Tools, it has the Wi-Fi IPv4 address (but not IPv6), but it's not clear what interface has this IP. There also does not seem to be an interface with other than a link-local IPv6 address. Whatever this display is telling me, ping to an internal host's global IPv6 address succeeds. Also succeeding: an IPv6 ping to the internal host's alphabetic name. Firefox to http://[:IPv6:addr]/ gets the requested index page. But Firefox failed to resolve http://ipv6.test-ipv6.com/ which only has an IPv6 address. Firefox on Gnu-Linux (desktop) can do so.

• The notification bar has a new icon: a microphone. It vanishes when a normal app has focus, and returns when Trebuchet has focus. Here's a list of apps which have (or want but don't have) permission for the microphone. See Settings - Apps & Notifications - Permission Manager - Microphone. Some need to be turned off, some need to be turned on. I think, but didn't confirm in detail, that the permission for Google is what causes the microphone to be opened on the home screen (not Trebuchet itself). With Google's (etc.) permission revoked, the microphone icon is gone.

  • Camera (to record video)
  • FM Radio (why? station selection? Turn off. Can't turn it off.)
  • Google (kill that!!!) Ooo, you'll miss out on all our stuff! Kill.
  • IP Webcam (to snoop on your kids)
  • Messaging (why? MMS with a voice clip? Just in case, keep it on.)
  • MyChart (for telemedicine, turn on)
  • OpenCamera (to record video, was off, turn on)
  • Phone (for voice chat)
  • Recorder (to record voice notes or spy stuff)
  • Ring (to cuss out the burglar; was off, turn on)
  • Sensors (to test the microphone)
  • Smart Checker (the hardware suite includes an ambient noise meter which I didn't buy, turn off)
  • Sound Analyzer App (turned off for some reason, turn it on)
  • T-Mobile Visual Voicemail (why? Probably to record the voicemail greeting. Leave it turned off until actually needed.)
  • VLC (why? Leave it turned off.)
  • And a bunch of others which shouldn't be asking for the microphone.

• Let's see about updating apps. Apparently there isn't a big API revolution in the jump to Android-10. Only 2 to update: Maps and Google Pay. Updating Maps; refusing Google Pay because I want to see if the latest version of Magisk can make it work. If so I'll save the relevant APKs to restore if Google improves it.

• It's possible that Dropbear has become inoperative. I can't do ssh to port 2222. Update: fixed later.

• What bloated stuff has gotten into my shared storage (/storage/emulated/0)? I have a feeling I'm going to have to wrap the whole 2.3Gb in a tar file and un-tar it on the host, then use normal Gnu-Linux tools to analyse it. Well, let's get that started…
  rsync --rsh="ssh -4 -p 2222" --log-format="%o %f" -a -r selen:/storage/emulated/0/ /scr/selen-data/

  The content is there, but only 86Mb. Various cruft that is totally obsolete and needs to be deleted, but is not exactly huge and doesn't take 2.3Gb.

  Let's retry this, attempting to store everything in /data. Total content: 3.55Gb, took about 2min to transfer. It's all miscellaneous stuff, no backups, just cruft and bloat. The biggest offender is Firefox (cache 0.44Gb and profile 0.16Gb). Also notable are Google Quick Search Box (0.30Gb), Amazon Kindle (0.20Gb), Ring (0.18Gb), and several others.

  I need to work some more on getting a balance in my backups between saving important stuff and bypassing worthless stuff like the browser cache.

Tidbit: someone in a forum post mentioned adbfs. You connect to the phone with adb, then run adbfs which uses FUSE to mount your phone on a specified host mountpoint, similar to sshfs. A prime use case is backups. It's available on SBS: adbfs FUSE Filesystem for Android Debugger. (It's a community package.) adbfs-rootless by spion is the Github repo for the source code.

Growl! Night Clock is out of action, no way to open the menu to set alarms.

Repeating Checkout

Repeating the checkout tests:

• Does it boot? Yes (whew!)
• Battery life: Seems similar to LOS-16.0; no obvious power hogs.
• Startup times: Unchanged.
• Display: Brightness is managed just about identically to LOS-16.
• Memory: No major change.
• Processor: Still working on the tests…
• Voice chat: Working, voice and SMS. Still no sign of VoLTE (reverts to HSPA), but this is model specific and depends on the baseband software, which is not part of Android.
• Cellular data: Works, preferring LTE if so configured.
• Wi-Fi: Works.
• Wi-Fi in master mode (access poing on phone): Works.
• Bluetooth: Works, tested with A2DP to headphones.
• Playing streaming audio: VLC to Bluetooth (A2DP) works.
• GPS: It's having a problem. Taking a long time to get a fix, despite cache clear and re-download AGPS data.
• NFC: Not tested, no partner.
• Fingerprint reader: I cleaned my finger and the sensor with 70% isopropanol. I removed my previous finger and re-enrolled it. Then I tried unlocking the machine 20 times. 16 successes, 4 allegedly dirty fingers, and no really rejected prints. But the finger and/or sensor does get dirty quickly.
• Compass (magnetometer): Works (after re-calibrating it).
• Accelerometer: Works.
• Proximity sensor: Works.
• Notification LED: Works normally.
• Ambient light sensor: Works and influences the display.
• Rear camera: Works, tested with OpenCamera.
• Front camera: Works, tested with OpenCamera.

So what's going on with the GPS? These apps have permission to use location:

• Camera (AOSP) -- GPS is on by default, but location recorded was Xian, China, no latitude and longitude.
• DaFit (fitness monitor) -- Not tested.
• Domoticz Lite -- Location notification appeared, but no GPS icon.
• Google -- Not tested.
• Google Pay -- Location notification appeared, but no GPS icon.
• Google Play Store -- Location notification did not appear.
• GPS Status -- Shows expected number of satellites but never gets a fix.
• Jog Tracker -- Waits forever for GPS position.
• LTE Discovery (was off, turned on) -- Sees no satellites.
• Maps (by Google) -- Promptly shows the correct location, similar performance to the Galaxy S5. But no GPS notification.
• MyChart -- Not tested.
• NTP & GPS Clock -- GPS attempt times out.
• OpenCamera (was off, turned on) -- I turned on mandatory location and the location was stored in the picture. Can't tell if it was using GPS though. Latitude and longitude were correct but the location as text was alleged to be Xian, China.
• ShakeAlertLA -- It was using Location Services, but no GPS (doesn't need that much accuracy). I'm not confident that I've got all the needed services set up so it will really alert me.
• Speedtest (by Ookla) -- Not re-tested.
• Street View (by Google) (was off, turned on) -- Uses location but no sign of GPS.
• WiFi Analyzer (was on, turned off) -- No sign of what it might use location for. Maybe for labelling saved graphs.

Settings - Location - Advanced - Google Location Accuracy: It says it may use Wi-Fi, mobile networks, and (other?) sensors to estimate the location. If you turn this off, it only uses GPS, resulting in low accuracy if GPS is not actually available. Spot testing with this off: did not bring GPS Status back to life. I also tried turning the Location master switch off. This resulted in no satellites at all being shown by GPS Status.

Although GPS Status (actually the Location daemon) has AGPS data, suppose it requires a real Almanac from the satellites. Let's run it for half an hour and see if anything improves. No, GPS Status is still trying to turn the locations of the 23 satellites into a fix.

At the exact same time and location, the old Galaxy S5 took 303 secs. It needed to get new AGPS data (54 days old) and a new ephemeris. It found 4 or 5 satellites fairly quickly, but after five minutes it finally got the whole constellation, and promptly delivered a fix. This shows that GPS has not been disabled as a part of the USA's response to COVID-19.

In the Launcher (Home) Settings, these items were present in LOS-16.0 but not in 17.1: Grid size, Change icon shape (it's now fixed as circles).

Re-Activating Magisk

Redoing Installing Magisk…, possibly with modifications. Here's what I actually did.

• Starting after upgrading to LOS-17.1 with no root package, but Magisk Manager is installed under the hidden name MgkMgr.
• Executing MgkMgr for the first time in LOS-17.1. It says Magisk Manager is up to date, v7.5.1 (267)(7) (present date is 2020-04-30). Magisk is not installed (duh), latest is 20.4 (20400).
• Advanced Settings (which I didn't change): Preserve Force Encryption (is on), Preserve AVB 2.0/dm-verity (is on), Recovery Mode (is off).
• Hit Install. You have to hit the word Install, not the body text. Installation mode: download zip only (I picked this), vs. patch a file. OK, it's downloaded. It's in /storage/emulated/0/Download (not Downloads).
• Covering various bases: I'm copying the zip file to the root of my SD card. No I'm not, seems to be a permission problem which was not fixed by granting write access for the SD card to TotalCommander.
• Instead I'm going to copy the zip file to the host, boot TWRP, and then sideload the zip file. adb pull '/storage/emulated/0/Download/Magisk-v20.4(20400).zip' (into current directory).
• adb reboot sideload -- Goes through lock your bootloader and Sony splash screen, ends up in LineageOS Recovery with ADB Sideload action item.
• adb devices -- Says CQ30010NGY sideload. Just being paranoid.
• adb sideload 'Magisk-v20.4(20400).zip' -- Signature verification failed, failed to verify whole-file signature. Deja vu. Let's try this with TWRP instead. Hit No (don't install), Advanced, Reboot to Bootloader.
• fastboot devices -- says CQ30010NGY fastboot.
• fastboot boot twrp-3.3.1-5-pioneer.img -- Goes through lock your bootloader, Sony, TeamWin splash screens. Give your pattern. It gets to the main screen. Advanced, ADB Sideload, swipe to begin.
• adb devices -- Says CQ30010NGY sideload. Just being paranoid.
• adb sideload 'Magisk-v20.4(20400).zip' -- Now here's a question: the install log says nothing about verifying a signature on the Magisk zip file. Did it verify and just keep silent? Or did it not verify at all? It does mention that the boot image is signed with AVB 1.0, and it re-signs the patched image with verity keys.

Infinite loop while booting:

• Reboot System. (Cross fingers.) Yes it boots. The boot animation runs for a long time (2 mins?) and there's a scary 20 sec pause in the middle. After 10 mins it's still doing the boot animation. Something's wrong.
• I disconnected the USB cable, then held down the power button until the phone turned off power. 30 secs, did not power off. It looks like I'm going to have to just let it do its thing until it runs down the battery. Details on the outcome of this:

After about 24min it ran the battery down again and the screen went black. This attempt to reboot has failed.

Trying to reinstall LOS-17.1:

• Goal is to start the bootloader, then load TWRP. Hold down volume-up, and connect the USB cable while holding it. Indicator light comes on dim red, then dim blue. Keep holding for 15sec; just because the light is blue doesn't mean that it's started fastboot. fastboot devices says CQ30010NGY fastboot, good.
• Oops, 15sec might be not quite enough. In another attempt the light ended up dim green and fastboot devices says ???????? fastboot, bad. I pulled the USB cable (light went out after a few seconds), retried with the same 15sec hold with the same result, and tried with holding 20sec, and this time it got into fastboot.
• fastboot boot ./twrp-3.3.1-5-pioneer.img -- Boots up normally (whew!)
• We're at 31% battery. It is not charging, now it's at 30%. To be safe I should charge it to at least 50%. Hit reboot, then Power Off. Given that the USB cable is still connected, it rebooted into autonomous charging mode. Battery is now at 60%.
• fastboot boot ./twrp-3.3.1-5-pioneer.img -- Boots up normally.
• adb devices -- reports CQ30010NGY recovery.
• What should I wipe? I'm following the instructions for upgrading to a new major version, vs. installing from stock. You don't wipe anything in this case. In the first Sideload I did check Wipe Cache and Wipe Dalvik Cache. Unable to find partition for path /cache, but it did wipe /data/dalvik-cache. Better to do just dalvik-cache in Wipe - Advanced. That claimed to have worked.
• In Sideload, it sucks in the zip file and stores it in /sideload/package.zip, then installs it as if separately uploaded. Or at least that's what the progress notices imply and it corresonds with what I've seen on forum posts.
• Sideloading these images, rebooting TWRP each time:
  • lineage-17.1-20200425-nightly-pioneer-signed.zip (progress to 47%) (current slot is B)
  • open_gapps-arm64-10.0-nano-20200422.zip (progress to 58%) (current slot is A)
  • Magisk-v20.4(20400).zip (progress to 71%) (current slot is still A)
• Idiot, no you don't hit Reboot System. Re-doing from the beginning.
• You see the current slot on Reboot from the main screen.
• Now that everything is allegedly installed, let's reboot system. 20:50:00 (approx). Boot animation plays. At 170sec it paused, then resumed. For 10min. Back to squre 1, letting it bootloop overnight.
• Next plan will be to reinstall minus Magisk, and wait for a fix to eventually appear.
• 30min on charge brought the battery to 36% (according to TWRP, once I got it booted). 50% is recommended, but 36% should be plenty.
• fastboot boot ../twrp-3.3.1-5-pioneer.img
• adb sideload lineage-17.1-20200425-nightly-pioneer-signed.zip
• Back to main screen - Reboot - to Bootloader. Current slot: A.
• fastboot boot ../twrp-3.3.1-5-pioneer.img
• adb sideload open_gapps-arm64-10.0-nano-20200422.zip
• Back to main screen - Reboot - to Bootloader. Current slot: B. I could have rebooted from the sideload screen, but I wanted to see the current slot. Battery is down to 34%.
• Rebooting: 10:03:45. Boot animation starts. !@#$%^* after 220sec it paused for about 20sec, then boot animation resumed. Same behavior as before.
• The one difference is, I didn't wipe the Dalvik cache (although the 220sec boot time suggests that it was not populated yet). When I get control of the machine again, I'll try doing factory reset, then reboot again.
• When rebooting it showed Android for under 1sec, then rebooted again. LOS Recovery said, Can't load Android system, your data may be corrupted. I took its option to factory data rest again. Boot animation appears. 14:48:55 ... 51:45 (170sec), finished booting, running initial setup.
• Diffs in initial setup: It actually found updates (to apps I assume) and installed them. (That's the only difference I saw.)
• Next big job: to install a zillion apps. What I really need is a big list of URLs for installation. I should restore from backup.
• How to import my home directory:
  • Make a tar file on Diamond of jimc and htdocs. (40.5Mb compressed)
  • Use AndFTP to import it to the root of internal storage.
  • Use the local terminal in JuiceSSH and un-tar it.
  • It worked.
• Set a pattern, set up fingerprints. Settings - Security.

Android Settings: http://localhost/~jimc/hardware/sam-galaxy-s5/settings.shtml In addition to the steps in the above URL, or supplementing them, I set these:

• About Phone - Emergency Contacts
• System - Developer Options - Automatic system updates (turn off)
• System - Developer Options - Android Debugging (turn on)
• System - Developer Options - Rooted Debugging (turn on)
• System - Developer Options - Device Hostname (set to selen)
• System - Developer Options - Mobile Data Always Active (turn off)
• Battery - Battery Status Style (submenu) - Battery Status Style (change to Circle); Battery Percentage (change to Inside the Icon)
• Apps & Notifications - Advanced - Permission Manager - Microphone - change status, Google (to denied), Messaging (leave enabled), leave the rest; when other apps are installed, add to this list.

Apps in the pristine state, from LOS and Google Apps (there are a lot more in /system/apps).

• Assistant (like Siri®)
• AudioFX
• Browser (similar to Google's Chromium)
• Calculator
• Calendar
• Camera
• Clock
• Contacts
• Email
• Files
• FM Radio
• Gallery
• Google
• Messaging
• Music
• Phone
• Play Store
• Recorder
• Settings

Tidbit: Announcement of LineageOS-17.0 for Pioneer (OP LuK1337, 2019-10-08). See particularly the Reporting Bugs section. Specifically you can turn off SELinux by running setenforce 0 from a root shell, and enforcement actions are logged in /data/misc/audit/audit.log . Later in the discussion, LuK1337 says he tried the then-prevailing Magisk and it worked for him.

Selection

Testing

Setup

Apps

More Info

Top

Photo Credit