User:Veremit/Catalyst upgrades

Catalyst upgrades
A few ideas have been circulating about some improvements to the catalyst code. This page attempts to document progress on these ideas.

Add 'compression_level' option to spec files and config

 * arm/sparc 32-bit ul have problems with high xz compression options. We can override the defaults for these platforms with this option, either in spec files, or set sane defaults in the arch configs.
 * this requires the extra functionality adding to the PyDecomp module, then the appropriate calls can be added to catalyst

Add a new 'step' to perform eclean-pkg

 * Currently there is no means to wipe old packages from the cache; save blitzing the folder, or manually running a 'find' command. There is already a tool to perform this function, but we currently have no means of automatically doing it within the catalyst process 'run'.

Remove custom config parsing code

 * Catalyst currently uses some custom code to parse the /etc/catalyst/catalyst.conf file. Replace this with built-in python config parser code

Convert arch 'modules' to config files

 * arches have custom configurations set by means of an 'arch.py' file which sets some predefined values - make these config files, change their install location and they can be customised by users more easily.

Fix snapshot/snapcache locking

 * snapshot file should be locked when being read and written, to prevent a scheduled snapshot creation from breaking a scheduled catalyst run which unluckily tries to unpack the snapshot at the same time
 * snapcache should probably be readonly _and_locking_removed_ to allow multiple catalyst simultaneous catalyst instances to share the snapcache. If locking is still desired then only use exclusive locking for write operations and normal locks for reading to prevent the snapshot cache being updated while it is being read.

Use /usr/portage as portage snapshot source

 * whilst snapshot tars and their caches work for builder machines, why can't we use the /usr/portage 'tree' for local builds?

Catalyst documentation
The suggestion was made to utilise Sphinx for Portage on IRC. It could be a great opportunity to gather some more documentation about how catalyst works together, and make it available in a convenient, readable form!