Project:Portage/Proposals

This is where we will flesh out ideas for the direction to take areas of portage's code.

Portage sync
This is the proposed layout of a totally revamped sync system. It uses the plug-in manager already in portage.

portage.sync.pdf


 * Advantages


 * 1) Flexible
 * 2) Easily extended
 * 3) Less maintenance, core sync manager does not need changes for new modules
 * 4) Add on modules can be user installed for special types

Proposed To Do

 * 1) Determine the information the module_spec needs to provide to the sync manager and clients  <== InProgress
 * 2) Move the plug-in manager code out of emaint's namespace, into portage's
 * 3) Put a freeze on the current sync code to reduce the rebasing needed for the merge on completion  <== Done
 * 4) Create a new feature branch to use for this change   <== Done, plugin-sync
 * 5) Code the sync manager code  <== Preliminary coding done.
 * 6) Determine the method to be able to do concurrent repo syncs
 * 7) Possibly move the TaskHandler class into the SyncManager class or subclass it
 * 8) Make the sync manager provide any common services functions such as dropping from root down to user privileges     <== Done
 * 9) Move the sync code to a new rsync module  <== Done
 * 10) Move the git code to it's own module      <== Done
 * 11) re-write the code in action_sync.       <== minimally migrated to the new SyncManager class
 * 12) Begin testing, debugging                  <== Rsync, git tested working.  CVS untested
 * 13) Add to module_spec for repos.conf variables it accepts/requires
 * 14) Add repos.conf validation checks functions
 * 15) Determine the location/directory for 3rd party sync modules to be installed to.
 * 16) Re-factor the code to optimize for the new structure.
 * 17) Move the web-rsync code from emerge into a module
 * 18) Modify emerge-webrsync cli to use the new webrsync module
 * 19) MORE testing, debugging.
 * 20) merge the branch back into master

Examples
Example Proposed extensions

{{CodeBox|title=__init__.py for a new extended git plug-in module|lang=python|1=
 * 1) Copyright 2014 Gentoo Foundation
 * 2) Distributed under the terms of the GNU General Public License v2

"""Rsync plug-in module for portage.  Performs a git operations on repositories """

module_spec = { 'name': 'git', 'description': __doc__, 'provides':{ 'module1': { 'name': "git", 'class': "GitSync", 'description': __doc__, 'functions': ['sync', 'install', 'validate',], 'func_desc': { 'sync': 'Performs a git pull on the repository', 'install': 'Installs a new git clone of the repository', 'validate': 'performs validation checks on the repos.conf ' \ 'entries for the repository', },                        'conf': { 'location': 'Repo location eg: /var/lib/gentoo/repos/gentoo', 'sync-type': 'The sync module type associated with the repo. eg: "git"', 'sync-uri': 'the uri to use for initial installation', {,			}		}	} }}

Discussion
portage-dev mail list threads


 * topic: New proposed modular sync system
 * 1)  gmane thread

To be discussed at the next portage team meeting, Feb. 9, 2014 @ 19:00 UTC in #gentoo-meetings

Introduction
Portage is getting overcomplex with growing number of build-env quirks and FEATURES. Would be good to figure out a nice plugin API to handle them.

Potential build-env plugins
The following features could be split into separate plugins:
 * ccache,
 * distcc,
 * fakeroot,
 * fuse-sandbox (in TODO),
 * ipc-sandbox,
 * network-sandbox,
 * sandbox,
 * userpriv.

Required facilities
Based on the potential plugins, the plugin system needs to provide at least:
 * entry/exit hooks for phase execution,
 * environment manipulation,
 * convenient wrapper creation and PATH manipulation,
 * ordering facilities — i.e. plugins requiring root privileges would have to be ordered before userpriv.