User:Maffblaster/Gentoo ideas
This page lists various improvement ideas I have for Gentoo (as a distribution).
Portkey - Cool name for software.
Quick update system
Recently I worked on a Gentoo system that was not updated since 2020.10. Updating via traditional systems were not working due to Portage not supporting a new EAPI (EAPI 8). Attempting to follow Sam's Portage rescue steps worked up until trying to perform a world update. Sometimes there are too many sticky points to update an old system, so it be easier to capture the world file, profile setting, and generate a whole new base-system. The goal would be to generate an up-to-date version of the old image. Look into Project:RelEng GRS?
This is also useful to submit a job to a powerful host and have that host return a disk image file to overwrite the entire drive. This would imply the image is smaller than the target drive, and may require a run-once partition expander script to run (like GenPi64 does on first boot).
Certain packages may have been removed from ::gentoo, and these will need to be hand-reconciled manually.
Apport
Apport is program that catches crash dumps and renames them according to some qualifying parameters.
- Homepage: https://wiki.ubuntu.com/Apport
- Source: https://code.launchpad.net/~apport-hackers/apport/trunk
- Releases: http://archive.ubuntu.com/ubuntu/pool/main/a/apport/
Also look into Whoopsie.
Kernel
Fake slots with Gentoo-sources
The idea
The Mozilla project uses a concept called fakeslots in order to select a release branch (esr or rapid) of Firefox in order to keep a specific branched version of Firefox updated.
This same concept applied to kernel sources on Gentoo would make a lot of sense, and - similar to the reasons for fake slot usage for Firefox - would enable system administrators from having to p.accept_keywords or p.mask around certain versions in order to keep the package manager on a specific branched version supported by the kernel's upstream project.
The problem it solves
Presently when a system administrator installs a kernel source, without any SLOTs specified, Portage will pull down the latest stable or unstable version of the package. This is expected and typical. However, because newer kernels often result in certain bugs, and for general system stability, it is wise to 'lock in' to a supported LTS branch and only update between the patched versions of the same lts branch.
Something like the following would then be possible, which would then update only new versions of the latest current LTS branch (v6.1).
root #emerge --ask sys-kernel/gentoo-sources:lts-v6.1EAPI
A place to record various ideas that may correspond to the ebuild API.
USE flag sets
It would be nice to have a something similar to package sets, but for USE flags. Lets call them "USE sets" for the purpose of recording this idea. Something like this idea was proposed in GLEP 29! USE sets would likely need to be standardized in a per-ebuild repository basis; described with metadata (in the package's associated metadata.xml file?).
USE flags sets could be categorized based on MIME format, or functionality such as: ebook formats, compression algorithm formats, multimedia container formats, audio coding formats, video coding formats, graphics stuff, file system formats, wireless communication protocols (bluetooth).
For example, a user has chosen to select the default/linux/amd64/17.1/desktop/plasma/systemd (stable) profile and would like to be able to quickly and easily add support for all compression algorithms that are supported in the main ebuild repository, including, but not limited to, 7zip, bzip2, lzma, zip, zlib, zstd, etc...
How could a user accomplish this task in Gentoo currently?
The would likely add each individual USE flag into the system's make.conf file, and add a comment to describe what the added flags do. For example:
/etc/portage/make.confAdding all compression algorithms, and ebook file formats supported in ::gentoo# Multimedia container formats
USE="bluray heif matroska vdp" #...
# Compression algorithm formats
USE="${USE} 7zip bzip2 lzma zip zlib zstd" #...
# Ebook file formats
USE="${USE} epub mobi markdown pdf" #...
How a user could accomplish this in the future?
Similar to license sets...
/etc/portage/make.confAdding multimedia container formats, compression algorithms, and ebook file formats supported in ::gentoo# Add all compression algorithm formats, multimedia formats, video coders, and audio coders
USE="${USE} @COMPRESSION_ALGORITHMS @MULTIMEDIA_CONTAINERS @VIDEO_CODERS @AUDIO_CODERS" #...
searching for USE sets
eix, emerge --search, and other tools support...
packages.g.o integration
Displaying USE flag sets packages.g.o
Testing
Tests that will need to run if a new USE flag is added into the set.
Implementing an idea into Gentoo
License groups were implemented in GLEP 23 and owned by the License project. Implementing this idea would like take a similar path: Discuss at length with a RFC to see if the idea would fly with the Gentoo project a large, (re)write a GLEP to capture the idea, then work with projects to determine how ownership would work...
Something like this idea was already proposed in GLEP 29!
Portage
repo.presync.d/* hooks
Presently Portage has a repos.postsync.d/ hook directory in /etc/portage for post-repository syncs. This enables custom scripts to act on various repositories after a sync. The primary purpose for repos.postsync.d/ scripts is to generate package metadata, which in turn speeds up Portage's operations.
Similarly, the repos.presync.d/ directory could enable pre-sync hooks to be executed before Portage syncs the repositories. There may be little benefit to having this option, however some users may which to do things like "snapshot" the repository before syncing.
I guess the same thing could be done with a cron job or timer.
Btrfs snapshot of rootfs before @world update
Create a script (emerge wrapper? pre/post hook?) to issue a rootfs snapshot before @world updates. Safely remove created snapshot post update.
New options
- Output (
--list?) the packages in a set, display a count of the total installed packages in the set. Example:
root #emerge --list @world- This can presently be performed with equery:
user $equery list @world
- Add a
--verifyoption to verify the integrity of installed packages. The digests seem to already exist; Portage just needs to reference them. Would be similar to the qcheck command from app-portage/portage-utils, but accepts sets/package atoms/globs. Examples:
root #emerge --verify @profileroot #emerge --verify @systemroot #emerge --verify @worldroot #emerge --verify www-client/chromium
Better progress indicators
Portage could have a progress bar for update when updating a predefined set; particularly when using the the --jobs option. This would also be more useful if systems are configured to use binary packages, since compilation has already occurred, and therefore less information needs to display to standard output in general.
Something like:
# Demo code
Can be enabled/disabled via the FEATURE variable (disabled by default for now).
/etc/portage/make.confProgress bar for PortageFEATURES="progress-bar"
End-to-end PGP signed hash verification from source upstream to installation target
Since the Gentoo ebuild repository contains PGP signed hashes from all Gentoo developers as part of the contemporary development standard, there is an implicit trust relationship between package consumers (endpoint system administrators/users) and the Gentoo developer community. This proposition is an effort support the next level of hardening against supply chain attack, which has surfaced as a concern in the modern software development lifecycle.
More details on how this works technically can be found at https://devmanual.gentoo.org/eclass-reference/verify-sig.eclass/index.html and https://mgorny.pl/articles/verify-sig-by-example.html
Current supply chain assurance model
Portage has built-in safety checks between the Gentoo developer community and endpoint. Post-download, it will (properly!) fail if a package source hash does not calculate correctly. This is to protect against man-in-the-middle attacks, since ::gentoo package sources are mirrored by sponsored and volunteer infrastructure.
This can be verified using the following command:
root #emerge --fetchonly --emptytree @worldEnhanced supply chain assurance model
When supported by upstream projects, the next level of assurance for end-to-end trust is for the Gentoo developer to include the package originator signed hashes for (supported) package releases alongside traditional package sources. These "originator hashes" may or may not be PGP signed, which offers the additional benefit of identity validation and would be an additional assurance check. Including support for such an enhancement would increase the trust model by creating an auditable chain back to package source originator (development team/project).
In summary, two possible assurance enhancements could be provided (in particular for core software, such as packages within the @system set):
- Support for package originator hash validation.
- Support for package originator PGP signed hash validation.
Workflow
The Gentoo developer would need to hash and sign the hash (or PGP signed hash) provided by the package originator in addition to the tarball they receive from the package originator.
Testing
Currently system administrators have the option of downloading the source tarball directly from upstream's SRC_URL values via ebuild's RESTRICT="mirror" or Portage's FEATURE="mirror" feature.
When using the above features, Portage would verify only the hash (and possible PGP signature)
Except for 'custom packages' which have been created by the Gentoo developer and other similar edge cases, it should be possible to always result in valid hashes from the package source originator.
Summary
For package originators (upstream projects) that offer hashes or PGP signed hashes, it would be of high value to have Portage automatically ensure the sources provided by upstream have not been compromised through the entire supply chain: upstream project's developer/project team > Gentoo development community > Gentoo mirrors > Endpoint systems.
Implementation ideas
This would likely need to be an EAPI improvement, and obviously could only be supported if the package originator release hashes and/or PGP signed hashes alongside their packages.
binpkg tarballs
Right now Portage creates tbz2 tarballs for binpkgs even when other forms of compression are used to create the files (xz, gz, lz, etc.). Portage should generate binpkgs with a corresponding extension whenever formatting is adjusted in make.conf:
- gz = tgz
- lz = tlz
- xz = txz
Output
#emerge --sync(Could take a couple of minutes if you have a lot of binary packages.)
.='update pass' *='binary update' #='/var/db update' @='/var/db move' s='/var/db SLOT move' %='binary move' S='binary SLOT move' p='update /etc/portage/package.*'/usr/portage/profiles/updates/1Q-2017......................................
Userspace hardening
- Disable udev (new device creation/management) when the screen is locked. This way devices that can harm the system cannot be created unless there is a known user at the keyboard.
- If you have to replace/re-plug the keyboard you'll be in trouble, however it would avoid things like PoisonTap, which is a newer example of this.
Zorin OS
It would be nice to see Zorin OS shell themes available in Gentoo.
Subgraph OS
Make use of the user space application firewall present in Subgraph OS.
Upstream dead? No releases since 2017. Built on Debian Linux, Subgraph OS (GitHub) had some nifty isolation and segregation techniques to enforce controls between components of the operation system.
Ares Operating System
Future experiments could include attempting Portage and subsequently Gentoo's environment to run on a micro-kernel, such as Ares OS.
This would be similar to the Gentoo Alt project of the past, wherein Gentoo was running on the FreeBSD-based operating systems.
Virtual workspaces in VR
Get a VR headset. Get an open source display engine. Create a virtual workspace. Configure the display however you want. Display would forward windows directly to the canvas in VR. Could show as physical monitors in a virtual space, or as a 180 or 360 degree curved environment.
I imagine this will not be more useful until VR headsets attain higher resolution, since the primary mode in this case would be viewing text.
eclean improvements
eclean -d distfiles should remove old items that are no longer listed in the main repo. It should at least ask if the user would like to clean them.
user $eclean -d distfiles * Building file list for distfiles cleaning...
* Your distfiles directory was already clean.
The following unavailable installed packages were found
app-crypt/gnupg-2.1.15-r1
app-shells/bash-4.4-r1
app-text/hunspell-1.4.1-r1
dev-libs/libical-2.0.0-r1
dev-python/cryptography-1.5.3
dev-qt/qtcore-5.6.2
media-gfx/imagemagick-6.9.6.4
media-libs/libsndfile-1.0.27
sys-apps/openrc-0.22.3
sys-libs/readline-7.0-r1
virtual/perl-CPAN-Meta-2.150.5 ...distfile name(s) not known/saved
virtual/perl-CPAN-Meta-Requirements-2.140.0 ...distfile name(s) not known/saved
virtual/perl-CPAN-Meta-YAML-0.18.0 ...distfile name(s) not known/saved
virtual/perl-Carp-1.400.0 ...distfile name(s) not known/saved
virtual/perl-Compress-Raw-Zlib-2.69.0 ...distfile name(s) not known/saved
virtual/perl-Data-Dumper-2.160.0 ...distfile name(s) not known/saved
virtual/perl-Digest-1.170.0-r6 ...distfile name(s) not known/saved
virtual/perl-Exporter-5.720.0-r1 ...distfile name(s) not known/saved
virtual/perl-ExtUtils-CBuilder-0.280.225 ...distfile name(s) not known/saved
virtual/perl-ExtUtils-Install-2.40.0-r1 ...distfile name(s) not known/saved
virtual/perl-ExtUtils-MakeMaker-7.100.100_rc ...distfile name(s) not known/saved
virtual/perl-ExtUtils-Manifest-1.700.0-r2 ...distfile name(s) not known/saved
virtual/perl-ExtUtils-ParseXS-3.310.0 ...distfile name(s) not known/saved
virtual/perl-File-Path-2.120.100_rc ...distfile name(s) not known/saved
virtual/perl-File-Spec-3.630.0 ...distfile name(s) not known/saved
virtual/perl-File-Temp-0.230.400-r4 ...distfile name(s) not known/saved
virtual/perl-Getopt-Long-2.480.0 ...distfile name(s) not known/saved
virtual/perl-IO-1.360.0 ...distfile name(s) not known/saved
virtual/perl-JSON-PP-2.273.0-r1 ...distfile name(s) not known/saved
virtual/perl-MIME-Base64-3.150.0-r1 ...distfile name(s) not known/saved
virtual/perl-Module-Metadata-1.0.31 ...distfile name(s) not known/saved
virtual/perl-Parse-CPAN-Meta-1.441.700 ...distfile name(s) not known/saved
virtual/perl-Perl-OSType-1.9.0 ...distfile name(s) not known/saved
virtual/perl-Pod-Simple-3.320.0 ...distfile name(s) not known/saved
virtual/perl-Scalar-List-Utils-1.420.200_rc ...distfile name(s) not known/saved
virtual/perl-Storable-2.560.0 ...distfile name(s) not known/saved
virtual/perl-Test-Harness-3.360.0 ...distfile name(s) not known/saved
virtual/perl-Text-ParseWords-3.300.0-r1 ...distfile name(s) not known/saved
virtual/perl-Time-Local-1.230.0-r3 ...distfile name(s) not known/saved
virtual/perl-parent-0.234.0 ...distfile name(s) not known/saved
virtual/perl-podlators-4.70.0 ...distfile name(s) not known/saved
virtual/perl-version-0.991.600 ...distfile name(s) not known/saved
virtual/python-ipaddress-1.0 ...distfile name(s) not known/saved
www-client/chromium-55.0.2883.35
www-client/firefox-49.0
www-client/google-chrome-54.0.2840.90
x11-drivers/xf86-video-intel-2.99.917_p20161103
x11-libs/libdrm-2.4.71
xfce-extra/xfce4-battery-plugin-1.0.99
xfce-extra/xfce4-datetime-plugin-0.6.99
xfce-extra/xfce4-netload-plugin-1.2.99
xfce-extra/xfce4-smartbookmark-plugin-0.4.99