Project:Portage/Repoman-Module-specs
Repoman has three plugin module systems:
- vcs: various vcs specific code modules, eg: git, hg (mercurial), svn (subversion), ...
- scan: Holds the primary checks performed by repoman
- linechecks: An ebuild module submodule plugin system for performing the checks in the multi-check operations of the ebuild module. These checks are run on the various lines of the ebuild itself. Looking for common errors, invalid data, etc...
Common Module specs
- Each module is contained in a directory of the same name and should reflect the type of module operations it performs.
- The directory requires and __init__.py. This file contains specifics about the module classes it provides and the info required by those classes.
- The module can contain an arbitrary amount of files to perform the tasks. However, the details of the exported classes must be listed in the module_spec dictionary in the __init__.py. It is only this file and data which is imported and loaded upon startup. The classes and functions the module defines are only imported and initialized when needed.
- Each module should not require information from another module, nor import code from any other module. They should be self contained other than getting information from the parent process.
VCS Modules
The vcs module is required to supply two classes:
* A status class, various functions or methods to supply status information repoman requires. * A changes class, a method to scan the vcs for changes needing to be committed.
# Copyright 2014-2015 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 doc = """Git (git) plug-in module for portage. Performs variaous git actions and checks on repositories.""" __doc__ = doc[:] module_spec = { 'name': 'git', 'description': doc, 'provides':{ 'git-module': { 'name': "git_status", 'sourcefile': "status", 'class': "Status", 'description': doc, 'functions': ['check', 'supports_gpg_sign', 'detect_conflicts'], 'func_desc': { }, 'vcs_preserves_mtime': False, 'needs_keyword_expansion': False, }, 'git-changes': { 'name': "git_changes", 'sourcefile': "changes", 'class': "Changes", 'description': doc, 'functions': ['scan'], 'func_desc': { }, }, } }
The required exported functions for the status module are: ['check', 'supports_gpg_sign', 'detect_conflicts'] If not relevant to the vcs, then dummy functions must be provided to return correct return values to repoman.
The required exported functions for the changes module are: ['scan']
Scan Modules
# Copyright 2015-2016 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 doc = """Ebuild plug-in module for repoman. Performs an IsEbuild check on ebuilds.""" __doc__ = doc[:] module_spec = { 'name': 'manifest', 'description': doc, 'provides':{ 'manifest-module': { 'name': "manifests", 'sourcefile': "manifests", 'class': "Manifests", 'description': doc, 'functions': ['check', 'create_manifest', 'digest_check'], 'func_desc': { }, 'mod_kwargs': ['options', 'portdb', 'qatracker', 'repo_settings', ], 'func_kwargs': { 'checkdir': (None, None), 'xpkg': (None, None), }, 'module_runsIn': ['pkgs'], }, }, 'version': 1, }
A detailed explanation of the above is to be added in future... (TODO)
# -*- coding:utf-8 -*- class ScanBase(object): '''Skeleton class for performing a scan for one or more items to check in a pkg directory or ebuild.''' def __init__(self, **kwargs): '''Class init @param kwargs: an optional dictionary of common repository wide parameters that may be required. ''' # Since no two checks are identicle as to what kwargs are needed, # this does not define any from it here. super(ScanBase, self).__init__() """ # sample check def check_foo(self, **kwargs): '''Class check skeleton function. Define this for a specific check to perform. @param kwargs: an optional dictionary of dynamic package and or ebuild specific data that may be required. Dynamic data can vary depending what checks have run before it. So execution order can be important. ''' # Insert the code for the check here # It should return a dictionary of at least {'continue': False} # The continue attribute will default to False if not returned. # This will allow the loop to continue with the next check in the list. # Include any additional dynamic data that needs to be added or updated. return False # used as a continue True/False value """ @property def runInPkgs(self): '''Package level scans''' # default no run (False) and empty list of functions to run # override this method to define a function or # functions to run in this process loop # return a tuple of a boolean or boolean result and an ordered list # of functions to run. ie: return (True, [self.check_foo]) # in this way, it can be dynamically determined at run time, if # later stage scans are to be run. # This class instance is maintaned for all stages, so data can be # carried over from stage to stage # next stage is runInEbuilds return (False, []) @property def runInEbuilds(self): '''Ebuild level scans''' # default empty list of functions to run # override this method to define a function or # functions to run in this process loop # return a tuple of a boolean or boolean result and an ordered list # of functions to run. ie: return (True, [self.check_bar]) # in this way, it can be dynamically determined at run time, if # later stage scans are to be run. # This class instance is maintaned for all stages, so data can be # carried over from stage to stage # next stage is runInFinal return (False, []) @property def runInFinal(self): '''Final scans at the package level''' # default empty list of functions to run # override this method to define a function or # functions to run in this process loop # return a tuple of a boolean or boolean result and an ordered list # of functions to run. ie: return (True, [self.check_baz]) # in this way, it can be dynamically determined at run time, if # later stage scans are to be run. # This class instance is maintaned for all stages, so data can be # carried over from stage to stage # runInFinal is currently the last stage of scans performed. return (False, [])
A scan module should sublclass the above ScanBase class, override the functions as needed. A scan module has up to three options to run code at certain points in the full scan cycle.
- runInPkgs: This is for scans to be run at the package level (ie: all ebuilds, metadat.xml, etc.) List all functions to be run at this point in the cycle.
- runInEbuilds: This is for scans to be run on a specific ebuild.
- runInFinal: This is to for primarily for checks to be performed at the end of the scan cycle. (ie: summarize data from functions in the previous two levels, pass or fail accordingly)
Linechecks Modules
# Copyright 2015-2016 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 doc = """Eapi plug-in module for repoman LineChecks. Performs eapi dependant checks on ebuilds.""" __doc__ = doc[:] module_spec = { 'name': 'eapi', 'description': doc, 'provides':{ 'definition-check': { 'name': "definition", 'sourcefile': "definition", 'class': "EapiDefinition", 'description': doc, }, 'srcprepare-check': { 'name': "srcprepare", 'sourcefile': "checks", 'class': "UndefinedSrcPrepareSrcConfigurePhases", 'description': doc, }, 'eapi3deprecated-check': { 'name': "eapi3deprecated", 'sourcefile': "checks", 'class': "Eapi3DeprecatedFuncs", 'description': doc, }, 'pkgpretend-check': { 'name': "pkgpretend", 'sourcefile': "checks", 'class': "UndefinedPkgPretendPhase", 'description': doc, }, 'eapi4incompatible-check': { 'name': "eapi4incompatible", 'sourcefile': "checks", 'class': "Eapi4IncompatibleFuncs", 'description': doc, }, 'eapi4gonevars-check': { 'name': "eapi4gonevars", 'sourcefile': "checks", 'class': "Eapi4GoneVars", 'description': doc, }, }, 'version': 1, }
Configuration
The following is the format of the repository level configuration files. The files are stacked according to the layout.conf inheritance order. In this way a repository can inherit another repositories repoman configuration, then extend or limit the actions it will perform on the specific repositrory.
Currently the repository level configuration system is restricted to an `--experimental-repository-modules=y` option for now until the file format has been approved and accepted into the gentoo repository. At this time, the files used are installed as part of the repoman installation and reside in {EPREFIX}/usr/share/repoman/repository/ directory. These are the same files that currently reside in the gentoo repo. But the files are to be considered experimental until their format is finalized and approved. The --experimental-repository-modules=y can be added to a REPOMAN_DEFAULT_OPTS variable in /etc/portage/make.conf
qa_data.yml
--- # This yaml syntax file holds various configuration data for # the Quality-Assurance checks performed. # no random drive-by commits please # Please obtain authorization from the portage team # # Overlay maintainers override/add/negate checks at your discression # but support for third party module will be limited to the plugin API # # Repoman API version (do not edit) version: 1 # minimum repoman_version: 2.3.3 allowed_filename_chars: "a-zA-Z0-9._-+:" max_description_length: 80 # missingvars check: Mandatory (non-defaulted) ebuild variables # list missingvars: - KEYWORDS - LICENSE - DESCRIPTION - HOMEPAGE # file.executable check, non executable files # list no_exec_files: - Manifest - ChangeLog - metadata.xml # qawarnings: Non-fatal warnings, # all values in here MUST have a corresponding qahelp entry # list qawarnings: - changelog.missing - changelog.notadded - dependency.unknown - dependency.badmasked - dependency.badindev - dependency.badmaskedindev - dependency.badtilde - dependency.missingslot - dependency.perlcore - DESCRIPTION.toolong - digest.assumed - digest.unused - EAPI.deprecated - ebuild.notadded - ebuild.nesteddie - ebuild.absdosym - ebuild.minorsyn - ebuild.badheader - ebuild.patches - file.empty - file.size - HOMEPAGE.virtual - inherit.unused - inherit.deprecated - IUSE.rubydeprecated - java.eclassesnotused - KEYWORDS.dropped - KEYWORDS.stupid - KEYWORDS.missing - LICENSE.deprecated - LICENSE.virtual - metadata.warning - PDEPEND.suspect - portage.internal - RDEPEND.implicit - RDEPEND.suspect - repo.eapi-deprecated - RESTRICT.invalid - usage.obsolete - upstream.workaround - uri.https - virtual.suspect - wxwidgets.eclassnotused # ruby_deprecated: Deprecated ruby targets # list ruby_deprecated: - ruby_targets_ruby18 - ruby_targets_ruby19 - ruby_targets_ruby20 # suspect_rdepend: Common build only Dependencies # not usually run time dependencies # list suspect_rdepend: - app-arch/cabextract - app-arch/rpm2targz - app-doc/doxygen - dev-lang/nasm - dev-lang/swig - dev-lang/yasm - dev-perl/extutils-pkgconfig - dev-qt/linguist-tools - dev-util/byacc - dev-util/cmake - dev-util/ftjam - dev-util/gperf - dev-util/gtk-doc - dev-util/gtk-doc-am - dev-util/intltool - dev-util/jam - dev-util/pkg-config-lite - dev-util/pkgconf - dev-util/pkgconfig - dev-util/pkgconfig-openbsd - dev-util/scons - dev-util/unifdef - dev-util/yacc - media-gfx/ebdftopcf - sys-apps/help2man - sys-devel/autoconf - sys-devel/automake - sys-devel/bin86 - sys-devel/bison - sys-devel/dev86 - sys-devel/flex - sys-devel/m4 - sys-devel/pmake - virtual/linux-sources - virtual/linuxtv-dvb-headers - virtual/os-headers - virtual/pkgconfig - x11-misc/bdftopcf - x11-misc/imake # suspect_virtual: Dependencies that should usually be made to the virtual # Not to the final target library # dictionary suspect_virtual: dev-libs/libusb: virtual/libusb dev-libs/libusb-compat: virtual/libusb dev-libs/libusbx: virtual/libusb dev-util/pkg-config-lite: virtual/pkgconfig dev-util/pkgconf: virtual/pkgconfig dev-util/pkgconfig: virtual/pkgconfig dev-util/pkgconfig-openbsd: virtual/pkgconfig # valid_restrict: ??? # list valid_restrict: - binchecks - bindist - fetch - installsources - mirror - preserve-libs - primaryuri - splitdebug - strip - test - userpriv
repository.yml
--- # repository-modules.yaml # # This is the repository configuration file for repoman modules # # no random drive-by commits please # Please obtain authorization from the portage team # # Overlay maintainers override/add/negate checks at your discression # but support for third party module will be limited to the plugin API # # Repoman API version (do not edit) version: 1 # minimum repoman_version: 2.3.3 # NOTE: for non-gentoo repos, any custom modules added will need their # module names to the modules list in order for them to run. # These are the non-mandatory modules that can be disabled/enabled. # use -foo notation to disable, just like use flags # Add custom modules to enable them too scan_modules: description eapi ebuild_metadata fetches files keywords live manifests multicheck pkgmetadata profile restrict ruby linechecks_modules: assignment eapi3assignment implicitdepend hasq useq preservelib bindnow inherit dosym definition srcprepare eapi3deprecated pkgpretend eapi4incompatible eapi4gonevars paralleldisabled autodefault gentooheader nooffset nesteddie patches emakeparallel srccompileeconf srcunpackpatches portageinternal portageinternalvariableassignment quote quoteda httpsuri builtwith uselesscds uselessdodoc whitespace blankline addpredict noasneeded
linechecks.yml
--- # linecheck.yaml # configuration file for the LineCheck plugins run via the multicheck # scan module # no random drive-by commits please # Please obtain authorization from the portage team # # Overlay maintainers override/add/negate checks at your discression # but support for third party module will be limited to the plugin API # # Repoman API version (do not edit) version: 1 # minimum repoman_version: 2.3.3 eclass_export_functions: - ant-tasks - apache-2 - apache-module - aspell-dict - autotools-utils - base - bsdmk - cannadic - clutter - cmake-utils - db - distutils - elisp - embassy - emboss - emul-linux-x86 - enlightenment - font-ebdftopcf - font - fox - freebsd - freedict - games - games-ggz - games-mods - gdesklets - gems - gkrellm-plugin - gnatbuild - gnat - gnome2 - gnome-python-common - gnustep-base - go-mono - gpe - gst-plugins-bad - gst-plugins-base - gst-plugins-good - gst-plugins-ugly - gtk-sharp-module - haskell-cabal - horde - java-ant-2 - java-pkg-2 - java-pkg-simple - java-virtuals-2 - kde4-base - kde4-meta - kernel-2 - latex-package - linux-mod - mozlinguas - myspell - myspell-r2 - mysql - mysql-v2 - mythtv-plugins - oasis - obs-service - office-ext - perl-app - perl-module - php-ext-base-r1 - php-ext-pecl-r2 - php-ext-source-r2 - php-lib-r1 - php-pear-lib-r1 - php-pear-r1 - python-distutils-ng - python - qt4-build - qt4-r2 - rox-0install - rox - ruby - ruby-ng - scsh - selinux-policy-2 - sgml-catalog - stardict - sword-module - tetex-3 - tetex - texlive-module - toolchain-binutils - toolchain - twisted - vdr-plugin-2 - vdr-plugin - vim - vim-plugin - vim-spell - virtuoso - vmware - vmware-mod - waf-utils - webapp - xemacs-elisp - xemacs-packages - xfconf - x-modular - xorg-2 - zproduct eclass_info_experimental_inherit: autotools: funcs: - eaclocal - eautoconf - eautoheader - eautomake - eautoreconf - _elibtoolize - eautopoint comprehensive: true # Exempt eclasses: # git - An EGIT_BOOTSTRAP variable may be used to call one of # the autotools functions. # subversion - An ESVN_BOOTSTRAP variable may be used to call one of # the autotools functions. exempt_eclasses: - git - git-2 - subversion - autotools-utils eutils: funcs: - estack_push - estack_pop - eshopts_push - eshopts_pop - eumask_push - eumask_pop - epatch - epatch_user - emktemp - edos2unix - in_iuse - use_if_iuse - usex comprehensive: false flag-o-matic: funcs: - 'filter-(ld)?flags' - 'strip-flags' - 'strip-unsupported-flags' - 'append-((ld|c(pp|xx)?))?flags' - 'append-libs' comprehensive: false libtool: funcs: - elibtoolize comprehensive: true exempt_eclasses: - autotools multilib: funcs: - get_libdir # These are "eclasses are the whole ebuild" type thing. exempt_eclasses: - autotools - libtool - multilib-minimal comprehensive: false multiprocessing: funcs: - makeopts_jobs comprehensive: false prefix: funcs: - eprefixify comprehensive: true toolchain-funcs: funcs: - gen_usr_ldscript comprehensive: false user: funcs: - enewuser - enewgroup - egetent - egethome - egetshell - esethome comprehensive: true # non experimental_inherit eclass_info: autotools: funcs: - eaclocal - eautoconf - eautoheader - eautomake - eautoreconf - _elibtoolize - eautopoint comprehensive: true ignore_missing: true # Exempt eclasses: # git - An EGIT_BOOTSTRAP variable may be used to call one of # the autotools functions. # subversion - An ESVN_BOOTSTRAP variable may be used to call one of # the autotools functions. exempt_eclasses: - git - git-2 - subversion - autotools-utils prefix: funcs: - eprefixify comprehensive: true usex_supported_eapis: - "0" - "1" - "2" - "3" - "4" - "4-python" - "4-slot-abi" in_iuse_supported_eapis: - "0" - "\1" - "2" - "3" - "4" - "4-python" - "4-slot-abi" - "5" - "5-hdepend" - "5-progress"