Raspberry Pi® Logo
Valid HTML 4.01 Transitional

Raspberry Pi 3B
Gentoo64 on Raspberry Pi

Jim Carter, 2018-02-06

Table of Contents

Introduction and Issues

I have an issue in which 3D GPU accelerated graphics locks up on OpenSuSE Tumbleweed for aarch64, coincident with steadily declining free CMA memory. Is the problem specific to OpenSuSE, which has unique features such as EFI booting, or is it independent of the distro? That determines in which forum or bugzilla I ask for help. To make that decision I need to test GPU acceleration in another distro, preferably as different from OpenSuSE as possible.

According to Gavinmc42 replying to my forum post about turning on GPU acceleration, Gentoo has this working with a 64bit kernel.

Diversion about Gentoo: It's about as far from OpenSuSE as you can get. While all normal distros distribute precompiled packages, Gentoo's biggest unique feature is that, as a matter of policy, each end user is expected to download the source code and compile it on the spot. There are, of course, tools to automate this process, which have proven to be quite reliable. Portage is the name of the whole package management suite, and emerge is the program that the sysadmin invokes to compile and install a package, analogous to zypper on SuSE. Precompiled packages are used occasionally for packages that are too complex, touchy, or resource intensive for local compilation to be practical; Mozilla Firefox is a prime example (but is discontinued as of about 2018-08-xx). For deployment on multiple machines they have tools and documentation so packages can be compiled once on a server and turned into binary packages, and these can then be installed on leaf nodes.

Besides political correctness, this being an open source distro, one big advantage of local compilation is that the packages installed on each leaf node can take into account features and optimizations appropriate for that host, even accomodating different architectures, rather than requiring cooperation from the distro, which can only make space for a few precompiled package variants such as i586 vs. i686 vs. x86_64.

Web resources for installing Gentoo:

The first trial was to just steal Gentoo's start.elf file. The foreign start.elf was able to start u-boot, which execed Grub2, which loaded the specified kernel and initrd, but the kernel did not run. There were no useful error messages, but I would bet that the SuSE kernel tried and failed to retrieve the device tree that start.elf would have left somewhere for it.

For further progress I am going to have to actually install Gentoo. There is a chicken and egg issue here: there is no prebuilt installer for aarch64, so you have to compile the kernel and basic packages on an existing working host, which for me will have Intel architecture (x86_64, or as they call it, amd64). This means cross-compiling. The installation guide shows all the steps (cross fingers). But the packages for the cross-compile toolchain are for Gentoo, meaning I will first have to do a full installation of Gentoo for x86_64.

Hostnames: Rather than using phrases like the Raspberry Pi I'm using in this writeup the machines' hostnames on my net.

In this writeup I often abbreviate Raspberry Pi as RPi.

Petra's resources are:

Outline of the steps:

Downloading the Media

The following procedure mostly follows the Gentoo Handbook; page transitions are noted. The Handbook has a lot of explanation and alternatives, and it should be read concurrently. Much of the procedure is executed 2.5 times, first to go from the minimal installer to a working cross-compiler setup on Petra, then to cross-compile an arm64 kernel, install it on Orion's disc, and do as much setup as Petra can handle, and finally to boot up Orion and finish the setup steps that require running arm64 code on the Raspberry Pi.

Handbook page Choosing the Media

Before setting up Petra's Gentoo disc I downloaded the minimal installer ISO and the stage3 tarball for amd64 (Petra) from the (recommended) download page, dated 2018-10-04. Plus the DIGESTS.asc file; follow the link under Details. Plus Stage3 for arm64 (no signature, hiss, boo). I'm doing this now so the procedure to populate the disc wouldn't be interrupted by the download. With my dates, the full filenames are install-amd64-minimal-20181004T214502Z.iso and stage3-amd64-20181004T214502Z.tar.xz . Use the most recent versions, not these back-version examples.

Verification procedure:

Both machines need their proper host keys, ssh_config and sshd_config, and an authorized_keys file. Also your regular /etc/hosts. On the VM host I made a directory called keys.petra or keys.orion with the needed content, root-relative, e.g. when preparing it: rsync -a /root/.ssh/known_hosts keys.petra/ , which copies into keys.petra/root/.ssh/known_hosts , creating the intermediate directories.

For the ssh configuration files I used the ones I set up for OpenSuSE. But there was a problem of version skew. OpenSuSE has openssh-7.7p1, OpenSSL1.1.0h-fips whereas Gentoo has OpenSSH_7.7p1, OpenSSL 1.0.2p, and there are some features that SuSE has and Gentoo doesn't. Comment these out in the conf files that will go onto the Gentoo machines.

Partition Discs on the Host

Handbook page Preparing the Disks

We just finished Choosing the Media and we're now skipping to Preparing the Disks. I'm following the Howto with improvements. This procedure applies to both Petra and Orion, and can be done ahead of time on the host, for both in parallel.

Booting Petra from the Minimal Installer

Booting Petra from the minimal installer: In the Handbook this is two pages back, Choosing the Media, section 4, Booting.

Begin Installation

In music this would be a repeat mark: I'm going to first install Gentoo for x86_64 on Petra, then come back here and install Gentoo for arm64 on Orion's disc, then a switch to Orion itself, with most but not all of the steps identical.

Partitioning: I already partitioned both discs on the host, but if you didn't, now is the time to do it on the guest. See above for details of the partition sizes and the MBR (not GPT) partition table. You can do both discs in parallel the first time through, but be sure to get the correct disc. Do e.g. parted /dev/vda print (or /dev/vdb) and distinguish the discs by their total size: 17Gb (16Gib) for Petra and 32Gb for Orion.

Follow the Handbook's instructions to format each partition with the appropriate filesystem type. Be sure to get the correct disc and the correct partition on it. I assign a label to each partition and mount that way. Although the device pathnames (like /dev/vdb3) for the SD card are pretty stable, until you actually put it in a RPi, in general you can't rely on consistent major device numbers, so you always should mount by label or by UUID, and I think UUIDs are prolix, meaningless and ugly. Here are the filesystem creation commands, assuming you have the same major number I got. The suffix -02 or -04 refers to the number I marked on the SD card, to distinguish various ones. This is important because (as in this case) you may have similarly named partitions from multiple discs on the same machine. Here, Petra's devices and label suffix are shown.

The mini-installer for Petra could use some swap space and the Handbook line about that, in the section Activating the Swap Partition, is easy to miss: swapon /dev/vda2, or probably safer, swapon /dev/disk/by-label/SWAP-02. When you come back here to populate Orion's disc, Petra will already have its swap activated.

The system clock needs to be correct, and the VM imports it from the host. But the mini-installer only has UTC. Leave the timezone alone for now and convert times to the local timezone in your head if needed.

In the coming sections you install Gentoo on Petra's or Orion's disc, so you need to mount one or the other now. Petra's label suffixes are shown.

Handbook page Installing Stage 3 (Installation Files)

Deposit on the disc the stage3 that you previously downloaded, like this (on the host):
scp /s1/kvm/petra/gentoo/stage3-amd64-20181023T214502Z.tar.xz petra:/mnt/gentoo/
(give the root password.) For Orion get the arm64 file, but it still goes onto petra:/mnt/gentoo/ .

Now un-tar the stage3 tarball. (amd64 for Petra, arm64 for Orion.) Mind the special options. Since the Gentoo files use standard extensions to represent the compression algorithm, this need not be specified on the command line. The Gentoo Handbook specifies verbose output (a list of files copied, the 'v' option), and you will want to capture the output in a file for easy perusal. The instructions in the Handbook assume that you will have downloaded just one tar file in /mnt/gentoo, so ./stage3* will find it (check with ls).

On both discs, now that the new disc has some population, edit /mnt/gentoo/etc/fstab to reflect the partitions to be mounted at boot, as they will be when you reboot the machine.

Here's Orion's /etc/fstab . Petra's is the same except for the label suffix (-02).

# device                mountpoint      type            opts      dump/pass
LABEL=ROOT-04           /               ext4            noatime         0 1
LABEL=BOOT-04           /boot           vfat            noatime		1 2
LABEL=SWAP-04           none            swap            defaults	0 0
#/dev/cdrom             /mnt/cdrom      auto            noauto,ro       0 0

Preset some configuration for the net, on both discs. This is also needed when you jump into the chroot jail. If you forgot to do it and have to do it on the booted-up Orion, paths beginning with ./ are relative to Orion's root.

On the mini-installer you should already be using ssh, which means you will already have changed the root password. If not, do it now: passwd root and set your usual root password. Now:
cp -p /etc/shadow /mnt/gentoo/etc/shadow
Do this for both discs, so root can log in to both Petra and Orion.

I tweaked /etc/portage/make.conf with these additions; see also the next paragraph where GENTOO_MIRRORS is also appended to make.conf. When you install Petra this is the installer's make.conf, but I made the same changes in Petra's and Orion's instance, /mnt/gentoo/etc/portage/make.conf . (Don't mess with the cross-compiler's make.conf.)

Handbook page Installing the Gentoo Base System

I used mirrorselect -i to point at two geographically nearby mirrors: http://gentoo.osuosl.org/ (Oregon State Open Source Lab) and http://gentoo.cs.utah.edu/ (University of Utah Computer Science Department). Use up and down arrow to pick a mirror. Spacebar selects the highlighted mirror; you can select more than one; Enter (when OK is highlighted) appends the environment variable setting to /etc/portage/make.conf (the one in the minimal installer's ramdisc). You will want to append the same setting to /mnt/gentoo/etc/portage/make.conf as well so it will be used when you compile toolchain packages. And similarly on Orion's disc. The list is space separated and quoted.
GENTOO_MIRRORS="http://gentoo.osuosl.org/ http://gentoo.cs.utah.edu/"

mkdir ./etc/portage/repos.conf # That's in /mnt/gentoo
cp ./usr/share/portage/config/repos.conf ./etc/portage/repos.conf/gentoo.conf
Just copied, no changes.

Both machines need their proper host keys, ssh_config and sshd_config, an authorized_keys file, and /etc/hosts with local addresses (if any), which I packed up in advance in the host's keys.petra and keys.orion directories. There are several ways to get the keys onto the new disc. It's possible to shut down Petra, mount the root partition on the host using a loop device, and just copy, but it's a pain to get the correct offset and so I don't recommend using the loop device. Probably the least confusing method is:

Go to Jail

The following steps are done in Petra's chroot jail. Except as noted, the same steps are needed on Orion, but you need to defer them until the disc image has been copied to Orion's SD card and Orion has been booted up, because the disc contains arm64 (aarch64) software that needs a RPi to run it. So jump to Cross Compiling, picking up after your backward jump to set up Orion. Follow the provided link back here after you have cross-compiled the kernel etc. and booted up Orion.

When setting up Petra (not on a booted-up Orion), you mount these installer objects. The Handbook's command lines for --make-rslave are for installing systemd which I'm not doing.

Now Petra makes a suicide leap into the chroot jail:
chroot /mnt/gentoo /bin/bash
source /etc/profile
export PS1="(chroot) ${PS1}"

Diversion on how to install packages, called emerge in Gentoo:

Next the ebuild repository snapshot (Portage package index) has to be downloaded. The command is emerge-webrsync, no command line arguments. Afterward do eselect news read |& less to read news back to 2013-09-27 (only 13 items, none of which you need to respond to).

Do eselect profile list |& less; the currently (default) selected profile is marked with a '*'. This is default/linux/amd64/17.0 (stable), and I did not change it. A systemd profile is available, and on OpenSuSE I use systemd, but I'm not going to turn it on to avoid waking sleeping dragons. Gentoo utils have a tendency to barf a lot on the screen, in this case 54 choices. Piping through less is a good idea. [Update: later on Orion, not Petra, I switched to default/linux/arm64/17.0/desktop which needed 162 more packages and 24.7 more hours to compile.]

@world will now be updated. (Orion too, once running.) This is the set of keystone packages, that is, packages that are picked by the user versus being required by other packages. Your profile provides most of these packages, but any package that you emerge explicitly goes into @world unless you give an option to keep it out, or remove it later. Command:
emerge --ask --verbose --update --deep --newuse @world
Took 9 mins on Petra, 2 hours on Orion. If I had gone directly to the desktop profile on Orion, it would have taken 48 hours.

After @world is updated, it's recommended to do emerge --ask --depclean to get rid of unneeded packages. The first time through, answer no and peruse the proposed deletions (unmerge) carefully. For some reason, app-editors/nano is provided by the mini-installer but is not a keystone package in the default profile, so it would be deleted. To select something as a keystone package (which depclean has a compulsion to preserve), do e.g. emerge --noreplace app-editors/nano, and it will be added to the @world set but not recompiled.

Now you have timezones beyond UTC and can set one (Petra and Orion).

Next set the locale (Petra and Orion). Do these steps:

Handbook page Configuring the Kernel

This is only for Petra; on Orion you have already cross-compiled and installed it. Do these steps:

In the following step on Orion, the X-Server turned out to depend (at build time I think) on sys-devel/llvm, which is the core of the clang 'C' compiler. It's an interesting concept, but very resoure-intensive when it's being emerged. It repeatedly ran the RPi out of memory. I finally figured out an effective intervention. I had provided 0.5Gb of swap space, which is plenty for audio-video performance (which is spoiled if swap has to be used), but which overflowed with llvm stuff. The solution which should have been obvious is to provide more swap space. I created a 2.5Gb file on Orion and activated it (swapon /var/spool/swapfile2). Actually I think 1Gb would have been enough but I wasn't able to capture exactly the amount of swap used. Success at last! The swap file can be deactivated and deleted when the compilation is over. Well, maybe not: suppose llvm gets updated? It would have been more sanitary if I had provided a bigger swap partition in the beginning. [Update: llvm was not the only package killed by skimpy swap space. And llvm was updated when I switched to the desktop profile, taking 7.8 hours for that package alone.]

I emerged these utility programs that I'm going to need or that I like. We're compiling now for the x86_64 virtual machine (Petra); later these will have to be done again on the RPi (Orion). To save retyping and hassle I made a script with the emerge command line and package list. I provided the options --noreplace --keep-going. The first one avoids rebuilding packages that have already been installed, in case the process dies and I have to do it again. The second means, if an individual emerge fails, to compile the rest of them, excluding packages that require the one that failed.

Compiling all this stuff on the RPi, 354 packages counting dependencies, took 22:52 (almost 23 hours). These keystone packages could not be emerged:

Handbook page Configuring the System

We've already edited /etc/fstab and set the root password. To start services (network, sshd) at boot, do this. On Petra this works in the chroot jail, but Orion has to be already booted.

If you've just booted Orion, jump back to the section after the chroot jail and do the steps that require executing on the RPi. You will work through to here again. Of course you don't have to re-enable services, nor configure the bootloader; your final step would be to create your ordinary user account (below).

Handbook page Configuring the Bootloader

Petra needs Grub as its bootloader; Orion uses the native booter (start.elf), not Grub. Do this in Petra's chroot jail:

Handbook page Finalizing

We already set the root password, and already tossed /stage3-….tgz . On both machines I added my own ordinary user account. Command lines for reference:

For Orion running on the RPi, it is supposed to have all its packages and to be ready to run some tests. For the results, jump ahead to Testing Orion.

Rebooting Petra:

If you have just rebooted Petra, your next job is to set up Orion's new disc. Pop back to Begin Installation. There you will mount Orion's disc on (Petra's) /mnt/gentoo and /mnt/gentoo/boot. cd /mnt/gentoo. Stuff them with the arm64 stage3 file. Continue until Petra would have entered the chroot jail, which Orion's disc can't support, then come back here and cross-compile the kernel.

Cross-Compiling

At this point, you're executing on the real Petra (not the mini-installer) and are going to emerge the cross-compiler, and then to compile the kernel and other ARM64 packages for Orion.

Web Resources:

Crossdev runs on one architecture (x86_64) and compiles for another (aarch64 in this case). Here is how to create the cross-compiler toolchain:

Now I have my toolchain to build aarch64 programs. One strategy is to build just the kernel using Sakaki's companion guide for 64bit kernel building. The other is to do a complete installation of Gentoo. The latter is what I planned to do, since Petra can compile bigger packages and can do it faster than the RPi, but I was not able to get it to work.

The 64bit installation writeup recommends doing the actual compilation as an ordinary user, which is always wise if it's practical.

See above for how to put partitions and filesystem formats on Orion's disc. If you didn't do it earlier, do it now and mount the appropriate partitions on /mnt/gentoo and /mnt/gentoo/boot .

This is going to be nasty: the Gentoo64 Howto says to install a Portage snapshot by following the Handbook instructions, which tell some preparatory steps (listed above) and then you chroot into /mnt/gentoo followed by execing /bin/bash for ARM on your Intel processor. Fortunately this is not required to boot Orion (the machine on the SD card), and I will have to defer syncing Portage until then.

Installing the kernel: Copy arm64 boot files and the kernel onto Orion's disc.

Additional configuration.

I wanted to cross-compile a bunch of packages, because Petra has a faster CPU, and also more memory which is important for sys-devel/llvm. It looks like the normal strategy is to set up distcc. See discussion here. The package is built on the target host (Orion) except the compilation itself is handed off to the remote server (Petra). I thought this sounded like too much work and complication, so I tried to cross-compile binary packages, the idea being to copy them to Orion (or deposit them on Orion's disc, which is mounted), and install them once Orion is booted. Dreamer! Emerge --buildpkgonly does not build dependent packages. If you build them by hand they are recognized, but in some cases the requiring package wants to run a program it provides, which of course is impossible. I punted: I'm going to get a result a lot faster by compiling natively on Orion, assuming I can work around the memory issue for sys-devel/llvm. See in the emerge section for how I handled that.

Supposedly Orion is ready to boot.

Testing Orion

The first step in testing Orion is to compile X-Windows software. Back in the setup area I emerged x11-base/xorg-server, xfce-base/xfce4-meta and quite a lot of others. (For proper organization, that stuff belongs here, but I'm leaving it in setup to reflect how I actually did the installation.)

Next, the test procedure involves extensive exercises for Mozilla Firefox (www-client/firefox). So I emerged it. 79 hours later… Really, the RPi needed 78hr 54min 10sec to compile 28 packages, including the clang 'C' compiler and rustc for the Rust object oriented language, both of which are used extensively in Firefox and both of which are giants. Firefox alone took 26.8 hours to compile.

Positive control: Without KMS, default CMA memory is 8Mib with 1644kb free. I started up the CMA free memory monitor, which every 30 secs prints the date and grep CmaFree /proc/meminfo. Now starting a XFCE session, and going through 1.16Mb of text/html on Firefox. Firefox started. Billable CPU timet:

Firefox Web Content X-Server
CPU Time (sec) 405.9 54.3 96.4
%CPU 114% 23% 16%
%memory 50% (all together)

While Firefox was being started, CMA memory went as low as 0. During the test it went up and down, as high as 2966kb, returned to 0 once, end at 908kb. Conclusion: Firefox is operating normally and can do the test if the video driver is not flaky.

Rebooted with dtoverlay=vc4-fkms-v3d. Default CMA memory is 256Mib (262144 kB really kib), with 237092 kB free. vc4.ko is loaded and we do have /dev/dri/card0. The X-Server did not attempt to load /usr/lib64/dri/vc4_dri.so, rather tried glamoregl which failed to initialize. AIGLX says screen 0 is not DRI2 capable, and loaded swrast instead, successfully. vc4_dri.so would have been used on OpenSuSE.

Repeating the test.

Firefox Web Content X-Server
CPU Time (sec) 272.5 150.5 297.5
%CPU 80.1% 33.8% 91.7%
%memory 60.3% (all together)

Wnen the test started, free CMA memory was 228972 kB (down slightly from 237092 kB). At the finish, 228472 kB; the lowest value seen was 226696 kB but some was freed. It's interesting that different component programs had significantly different CPU usage.

Continuing on to show the big photo essay. At the end, CMA memory was 219356 kB, slightly but not alarmingly reduced. (18:27) Conclusion: with software rendering (swrast_dri.so), the memory leak does not occur and the test can be finished successfully.

Results of emerge -s raspberrypi.

media-libs/raspberrypi-userland version 9999

Userspace tools and libraries. The working directory for the ebuild is /var/tmp/portage/media-libs/raspberrypi-userland-9999/ . In the prepare phase, failed patch raspberrypi-userland-9999-gentoo.patch (in ./files, which is a symlink to /usr/portage/media-libs/raspberrypi-userland/files, which contains 2 patches, a udev rule, and 4 others that I didn't look at closely). See ./temp/raspberrypi-userland-9999-gentoo.patch.out . Collecting files for a bug report.

The patch was attempted with -p0 through -p4, but the file to be patched was missing. $PWD is /var/tmp/portage/media-libs/raspberrypi-userland-9999/work/raspberrypi-userland-9999 . The target file in the patch is raspberrypi-userland-9999/host_applications/linux/apps/dtoverlay/CMakeLists.txt A plausible existing target file is ./work/raspberrypi-userland-9999/host_applications/linux/apps/dtoverlay/CMakeLists.txt . In other words, either work/ should be prepended or it should have cd'd to that directory. If I change dir, the target file is found but the hunk isn't. The context lines are present, but the material to be removed is not there, as if the patch had already been applied.

The other patch looks like it should be OK, cross fingers. I turned raspberrypi-userland-9999-gentoo.patch into a symlink to /dev/null. ebuild /usr/portage/media-libs/raspberrypi-userland/raspberrypi-userland-9999.ebuild prepare — Digest verification failed (duh). Try --skip-manifest . That helped, but now Cannot find $EPATCH_SOURCE! (the patch file). I got through that idiocy and the patch was non-applied. This turns out to be a duplicate of Gentoo bug 641046; I appended a comment to that effect.

Now I'm going to do ebuild file.ebuild merge. Gakk, /var/tmp/portage/media-libs/raspberrypi-userland-9999/work/raspberrypi-userland-9999/interface/khronos/common/khrn_int_hash_asm.s:36: Error: unknown architecture `armv6' And it goes downhill from there. Collecting files for another bug report.

media-libs/raspberrypi-userland-bin version 1.20170303

Mtime is Jul 30 2017. This is possibly a binary package from the above one. The source URL is http://gentoo.osuosl.org/distfiles/raspberrypi-firmware-1.20170303.tar.gz , 113Mb (oink). Build directory size is 209Mib, installed size 8.3Mib. Stuff ended up in opt/bin and opt/vc/lib64. Compilation phase took zero time. It provides e.g. libEGL.so and libOpenVG.so and libGLESv2.so, but not /usr/lib64/dri/vc4_dri.so . When it's installed, GPU acceleration did not happen.

media-video/raspberrypi-omxplayer version 9999

Command line media player for the Raspberry Pi. Irrelevant.

sys-boot/raspberrypi-firmware version 9999

Raspberry PI boot loader and firmware. Does not have to be installed on the execution host.

sys-kernel/raspberrypi-image version 9999

Raspberry PI boot loader and firmware, so it says. Is this a precompiled kernel?

sys-kernel/raspberrypi-sources version 4.14.9999

Raspberry PI kernel sources. We cross-compile instead.

sys-boot/raspberrypi-mkimage version 0_p20120201

Raspberry Pi kernel mangling tool mkimage/imagetool-uncompressed.py . I'm not sure what this is, but I doubt that I need it.

Now trying: ebuild /usr/portage/media-libs/raspberrypi-userland/raspberrypi-userland-9999.ebuild unpack . The failed patch is raspberrypi-userland-9999-gentoo.patch . The first argument has to be an ebuild file. These saved files appear relevant: /usr/portage/media-libs/raspberrypi-userland-bin/raspberrypi-userland-bin-1.20170303.ebuild Mtime is Jul 30 2017. This is possibly a binary package. The source URL is http://gentoo.osuosl.org/distfiles/raspberrypi-firmware-1.20170303.tar.gz , 113Mb (oink). Build directory size is 209Mib, installed size 8.3Mib. Stuff ended up in opt/bin and opt/vc/lib64. Compilation phase took zero time. It provides e.g. libEGL.so and libOpenVG.so and libGLESv2.so, but not /usr/lib64/dri/vc4_dri.so . /usr/portage/media-libs/raspberrypi-userland/raspberrypi-userland-9999.ebuild Dated Feb 28 2017, has a lot less cleanup interventions than the above. Trying media-libs/raspberrypi-userland-bin . Before emerge, /usr/lib64/dri/vc4_dri.so ctime = Nov 9 10:47. After emerge, same. The binary package did not provide /usr/lib64/dri/vc4_dri.so . /var/log/Xorg.0.log was identical: did not load vc4_dri.so and tried to init glamoregl, failing.

Conclusion: I have not been able to turn on GPU acceleration in Gentoo-64, and so I am not able to test whether the graphics freezeups are fixed in Gentoo.

Miscellaneous Points

To cross-compile a package

To cross-compile a package when you're populating Orion's new disc: Initially, see Cross Compiling with Portage and in particular, you need to do this on Petra (once):
emerge-wrapper --target aarch64-unknown-linux-gnu --init

Now to produce a binary package, do one of these alternatives:
emerge-wrapper --target aarch64-unknown-linux-gnu --ask --buildpkgonly category/package
emerge-aarch64-unknown-linux-gnu --ask --buildpkgonly category/package

But it doesn't do dependencies automatically, and often pkg-config will check on the version of an unknown library, failing. You then have to locate the library's name and category, and build it. Then retry the failed package — but that doesn't help if it's trying to actually execute what the package provides.

The normal way to produce binary packages is as a byproduct of a normal emerge, with the --buildpkg option. Then dependent packages would also be saved, autoconf tests would be satisfied from the already installed dependent packages, and if a compilation command line used an executable program from a build-time dependent package (like rustc), it would already be available.

When you run dispatch_conf you need to set this environment variable: CONFIG_PROTECT="/usr/aarch64-unknown-linux-gnu/etc"

The default PKGDIR=/usr/portage/packages . Cross compiled packages end up in /usr/aarch64-unknown-linux-gnu/packages/category/pkg.tbz2 . Copy the package onto the target host (Orion), then install it by: emerge pkg.tbz2.

I was never able to actually produce a useful binary package and install it.

distcc, Distributed Compilation Server

When Gentoo is deployed on a group of hosts with similar architecture, a common strategy is to run distcc on one or more powerful hosts, and the less powerful ones hand off compilation to the servers. It would appear (I'm not sure of this) that the 'C' preprocessor is run on the client, where header files are among the unpacked source code, to produce something that the server can run through gcc or g++, sending back the object file. There is also documentation for cross-compilation with distcc.

Initially I bypassed distcc because it looked like a lot of work to set up, given that cross-compiling was needed. But subsequent experience shows that the investment of time would have been worthwhile. Except for one minor detail: distcc has to be handcrafted (by the developers) to use each compiler, and the lion's share of compilation time, for my installation of XFCE and Firefox, went to programs compiled by rustc (for the Rust object-oriented language) and clang (a new and improved 'C' and C++ implementation; see also llvm which was the first component to give me trouble). None of these jobs could be done on the server. So it turned out that I was correct: distcc would not turn a profit for me.

Jiggering a package's USE list

How to jigger the USE variable for a particular package: The XFCE wiki page shows an example. Create /etc/portage/package.use/xfce with the content app-text/poppler -qt5 meaning when building package xfce, get rid of the qt5 support in poppler which is a pig. The minus sign means to suppress that USE flag; without the minus you're turning on the flag.

equery is provided byapp-portage/gentoolkit . To find out what USE flags a requiring package sets: equery w sys-fs/lvm2

What I would really like to know is what USE flags the required package is sensitive to.

More about dispatch-conf

When a file is auto updated in /etc and certain other dirs, the new version is named ._cfg$NUMBER_original_name. dispatch-conf looks for such files and helps you update them. There is a session log in /var/log/dispatch-conf.log . Need to set up /etc/dispatch-conf.conf and create listed archive directory(s) (default is /etc/config-archive ). Here is a complete list of dispatch-conf's action letters:

Unscrambling the big dependency tangle

There is a dependency tangle; a long list of programs require dev-libs/glib. Another long list require dev-util/gdbus-codegen. They need to be of the same version. dev-libs/glib-2.56.2 is the most recent version. But dev-util/gdbus-codegen-2.56.2-r1 is not recognized as satisfying the dependency of the requiring programs and matching glib in version. I broke the tangle by emerging it explicitly, and the dependency resolver accepted it.

emerge --ask "=dev-util/gdbus-codegen-2.56.2-r1" -- It actually proposes to do the emerge, including upgrading glib and glib-utils. Compilation succeeded. I got this from a search on Google with these keywords: gentoo emerge arm64 dev-libs/glib-2.56.2 dev-util/gdbus-codegen-2.52.3. The useful result was a Gentoo Browse instance which gives, per package, the available versions, their keywords, USE flags, dependencies, inverse dependent packages, and changelogs. See also https://packages.gentoo.org/packages/dev-lang/spidermonkey for a package index.

Temperature limiter is working. Often all 4 cores are running compilations. /sys/devices/virtual/thermal/thermal_zone0/temp says 82C to 84C and there's a thermometer symbol overlaying the display. /sys/devices/system/cpu/cpufreq/policy0/cpuinfo_cur_freq = 1.2GHz but evidently this is being throttled. It looks like the threshold for the GPU taking action is 80C; the thermometer is off at 79C.

It finished 81 packages in 2:41:24 elapsed = 161min, then failed to compile dev-lang/spidermonkey-52.9.1_pre1::gentoo . Lots of complaints about Unable to trace static ELF: /sbin/ldconfig: /sbin/ldconfig -p. There were a lot of warnings about coding practices that someone should look at. The real error message was aarch64-unknown-linux-gnu-g++: internal compiler error: Killed (program cc1plus). Rebuilding just this package separately. Fails equally. Cutting a long story short: I suppressed parallel make and added more swap space (both discussed extensively previously), and dev-lang/spidermonkey compiled successfully.

What Happened to Thunar?

When I switched the profile to default/linux/arm64/17.0/desktop, it decided to compile Thunar, XFCE's file manager. This failed in the following dependency chain:

So why do I care about thin-provisioning-tools? This forum post on Google Groups about thin-provisioning-tools (OP Andrew Lowe, 2017-09-15) gives some insight. LVM handles enlarging (or shrinking) partitions at runtime and fusing partitions on different discs into one logical partition. One possibiity is to defer allocating underlying space until data is actually written to it, called thin provisioning. It is very unlikely that a Raspberry Pi will ever use either LVM or thin provisioning.

Respondent Neil Bothwick says: Add "sys-fs/lvm2 -thin" to /etc/portage/package.use and thin-provisioning-tools will go away. In my case /etc/portage/package.use is already a directory (all files therein are effectively concatenated), and I created a file called sys-fs-lvm2 with just that content (and a #comment). Indeed, sys-block/thin-provisioning-tools was not emerged and Thunar was.

Other respondents discussed why you might keep or toss these packages, and where the thin USE flag came from. It is not a global USE setting; sys-fs/lvm2 specifically has it configured. The explicit -thin suppresses lvm2's default.

Gentoo Package for GLMark2

According to the Gentoo wiki article on Benchmarking, app-benchmarks/glmark2 used to be available but it no longer builds. The source code is available on Github.

Raspberry Pi® Logo
Photo and Image Credit