User:0xdc/substrate/20210501

Time for a new set of builds and a new set of release notes!

Catalyst
One recent dependency update has removed some functionality from catalyst. -0.9.6 removed snakeoil.osutils.FsLock, which catalyst did use. I've patched that functionality out for this build and going forward, but if you can't patch it, downgrade back to 0.9.5 or earlier.

importd signature verification
I've added signature verification to my upload script.

You'll need to put my GPG public key into /etc/systemd/import-pubring.gpg

This means for publicly available builds (i.e. https://builds.roflmao.space/index.html), you can now run  ("yes" is the default value of the verify flag).

Note that importd is is very picky about ETag caching, and unfortunately my hosting provider (OVHcloud OpenStack Swift) doesn't do proper ETags (there is a Swift pipeline that can fix ETags, but it doesn't appear to be enabled).

Pulling a tarball stage will work... once; if you try it again systemd will keep downloading the file again but will error when creating the read-only image. A patch for systemd can be found here.

OpenStack controller re-enabled
The issue with the OpenStack controller stage should be resolved. I was getting Segmentation Faults when loading a specific python library which appears to have a C component.

Since the C compiler for the system python (gcc) and this C library (tcc) in the stage are different, I think this produced an incompatibility. Fortunately the developers of the wrapt package which was causing it seemed to protect against this with an environment variable that can disable the C extension, this has been added to the stage and running a controller node is now possible again.

Dropping overlayfs
I've dropped "overlayfs" from the kernel command line of the livecds. This was added because the default livecd setup used to mount the squashfs image directly as the rootfs, hence it was unwritable.

Somewhere along the way, that behaviour has changed. The genkernel initramfs copies the necessary files into a read-write tmpfs and uses that as the backing store for the rootfs.

I also found out that the overlayfs handling disables docache, meaning that you cannot eject and remove the CD. Removing overlayfs from the cmdline now makes the docache boot option do its expected task.

Overlayfs is still useful in extreme memory exhausted environments (<1GB, min 256MB) where the system's RAM is not large enough to store the contents in a tmpfs.

Installation profiles
I have included two sets of systemd-repart definition files. The first does a rather traditional non-volatile installation. The other handles a volatile/stateless installation.

Both are pretty fiddly and need more documentation, and you can still do a Handbook install from the live environment anyway.

Some examples (to be run in a virtual machine, change /dev/vda as necessary):

Merged /usr
I've added some stages that are built with USE=-split-usr i.e. merged /usr. This facilitates fully volatile boots with. Primarily, the minimal livecd is the only consumer of these stages.

amd64
upstream stage3 seed: 20210428T214503Z (2021-04-28 21:45:03 +00:00)

armv7a
upstream stage3 seed: 20210428T171149Z (2021-04-28 17:11:49 +00:00)

bindist xorg stage4 build is still failing; I believe it's probably related to bug#761298