Project:Portage/Repoman-Module-specs

From Gentoo Wiki
Jump to:navigation Jump to:search
The information in this article is representative of former times and has been archived. It can be used for reference, but is most likely not appropriate for current usage. Generally archived articles should not be edited.

Repoman has three plugin module systems:

  • vcs: various vcs specific code modules, eg: git, hg (mercurial), svn (subversion), ...
  • scan: Holds the primary checks performed by repoman
  • linechecks: An ebuild module submodule plugin system for performing the checks in the multi-check operations of the ebuild module. These checks are run on the various lines of the ebuild itself. Looking for common errors, invalid data, etc...

Common Module specs

  1. Each module is contained in a directory of the same name and should reflect the type of module operations it performs.
  2. The directory requires and __init__.py. This file contains specifics about the module classes it provides and the info required by those classes.
  3. The module can contain an arbitrary amount of files to perform the tasks. However, the details of the exported classes must be listed in the module_spec dictionary in the __init__.py. It is only this file and data which is imported and loaded upon startup. The classes and functions the module defines are only imported and initialized when needed.
  4. Each module should not require information from another module, nor import code from any other module. They should be self contained other than getting information from the parent process.

VCS Modules

The vcs module is required to supply two classes: 

   * A status class, various functions or methods to supply status information repoman requires.
   * A changes class, a method to scan the vcs for changes needing to be committed.
CODE Example __init__.py
# Copyright 2014-2015 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

doc = """Git (git) plug-in module for portage.
Performs variaous git actions and checks on repositories."""
__doc__ = doc[:]


module_spec = {
	'name': 'git',
	'description': doc,
	'provides':{
		'git-module': {
			'name': "git_status",
			'sourcefile': "status",
			'class': "Status",
			'description': doc,
			'functions': ['check', 'supports_gpg_sign', 'detect_conflicts'],
			'func_desc': {
			},
			'vcs_preserves_mtime': False,
			'needs_keyword_expansion': False,
		},
		'git-changes': {
			'name': "git_changes",
			'sourcefile': "changes",
			'class': "Changes",
			'description': doc,
			'functions': ['scan'],
			'func_desc': {
			},
		},
	}
}

The required exported functions for the status module are: ['check', 'supports_gpg_sign', 'detect_conflicts'] If not relevant to the vcs, then dummy functions must be provided to return correct return values to repoman.

The required exported functions for the changes module are: ['scan']

Scan Modules

CODE Example __init__.py
# Copyright 2015-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

doc = """Ebuild plug-in module for repoman.
Performs an IsEbuild check on ebuilds."""
__doc__ = doc[:]


module_spec = {
	'name': 'manifest',
	'description': doc,
	'provides':{
		'manifest-module': {
			'name': "manifests",
			'sourcefile': "manifests",
			'class': "Manifests",
			'description': doc,
			'functions': ['check', 'create_manifest', 'digest_check'],
			'func_desc': {
			},
			'mod_kwargs': ['options', 'portdb', 'qatracker', 'repo_settings',
			],
			'func_kwargs': {
				'checkdir': (None, None),
				'xpkg': (None, None),
			},
			'module_runsIn': ['pkgs'],
		},
	},
	'version': 1,
}

A detailed explanation of the above is to be added in future... (TODO)

CODE ScanBase Class (repoman/modules/scan/scanbase.py)
# -*- coding:utf-8 -*-


class ScanBase(object):
	'''Skeleton class for performing a scan for one or more items
	to check in a pkg directory or ebuild.'''

	def __init__(self, **kwargs):
		'''Class init

		@param kwargs: an optional dictionary of common repository
						wide parameters that may be required.
		'''
		# Since no two checks are identicle as to what kwargs are needed,
		# this does not define any from it here.
		super(ScanBase, self).__init__()

	""" # sample check
	def check_foo(self, **kwargs):
		'''Class check skeleton function. Define this for a
		specific check to perform.

		@param kwargs: an optional dictionary of dynamic package and or ebuild
						specific data that may be required.  Dynamic data can
						vary depending what checks have run before it.
						So execution order can be important.
		'''
		# Insert the code for the check here
		# It should return a dictionary of at least {'continue': False}
		# The continue attribute will default to False if not returned.
		# This will allow the loop to continue with the next check in the list.
		# Include any additional dynamic data that needs to be added or updated.
		return False  # used as a continue True/False value
	"""

	@property
	def runInPkgs(self):
		'''Package level scans'''
		# default no run (False) and empty list of functions to run
		# override this method to define a function or
		# functions to run in this process loop
		# return a tuple of a boolean or boolean result and an ordered list
		# of functions to run.  ie: return (True, [self.check_foo])
		# in this way, it can be dynamically determined at run time, if
		# later stage scans are to be run.
		# This class instance is maintaned for all stages, so data can be
		# carried over from stage to stage
		# next stage is runInEbuilds
		return (False, [])

	@property
	def runInEbuilds(self):
		'''Ebuild level scans'''
		# default empty list of functions to run
		# override this method to define a function or
		# functions to run in this process loop
		# return a tuple of a boolean or boolean result and an ordered list
		# of functions to run.  ie: return (True, [self.check_bar])
		# in this way, it can be dynamically determined at run time, if
		# later stage scans are to be run.
		# This class instance is maintaned for all stages, so data can be
		# carried over from stage to stage
		# next stage is runInFinal
		return (False, [])

	@property
	def runInFinal(self):
		'''Final scans at the package level'''
		# default empty list of functions to run
		# override this method to define a function or
		# functions to run in this process loop
		# return a tuple of a boolean or boolean result and an ordered list
		# of functions to run.  ie: return (True, [self.check_baz])
		# in this way, it can be dynamically determined at run time, if
		# later stage scans are to be run.
		# This class instance is maintaned for all stages, so data can be
		# carried over from stage to stage
		# runInFinal is currently the last stage of scans performed.
		return (False, [])

A scan module should sublclass the above ScanBase class, override the functions as needed. A scan module has up to three options to run code at certain points in the full scan cycle.

  1. runInPkgs: This is for scans to be run at the package level (ie: all ebuilds, metadat.xml, etc.) List all functions to be run at this point in the cycle.
  2. runInEbuilds: This is for scans to be run on a specific ebuild.
  3. runInFinal: This is to for primarily for checks to be performed at the end of the scan cycle. (ie: summarize data from functions in the previous two levels, pass or fail accordingly)

Linechecks Modules

CODE Example __init__.py (repoman/modules/linechecks/eapi)
# Copyright 2015-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

doc = """Eapi plug-in module for repoman LineChecks.
Performs eapi dependant checks on ebuilds."""
__doc__ = doc[:]


module_spec = {
	'name': 'eapi',
	'description': doc,
	'provides':{
		'definition-check': {
			'name': "definition",
			'sourcefile': "definition",
			'class': "EapiDefinition",
			'description': doc,
		},
		'srcprepare-check': {
			'name': "srcprepare",
			'sourcefile': "checks",
			'class': "UndefinedSrcPrepareSrcConfigurePhases",
			'description': doc,
		},
		'eapi3deprecated-check': {
			'name': "eapi3deprecated",
			'sourcefile': "checks",
			'class': "Eapi3DeprecatedFuncs",
			'description': doc,
		},
		'pkgpretend-check': {
			'name': "pkgpretend",
			'sourcefile': "checks",
			'class': "UndefinedPkgPretendPhase",
			'description': doc,
		},
		'eapi4incompatible-check': {
			'name': "eapi4incompatible",
			'sourcefile': "checks",
			'class': "Eapi4IncompatibleFuncs",
			'description': doc,
		},
		'eapi4gonevars-check': {
			'name': "eapi4gonevars",
			'sourcefile': "checks",
			'class': "Eapi4GoneVars",
			'description': doc,
		},
	},
	'version': 1,
}


Configuration

The following is the format of the repository level configuration files. The files are stacked according to the layout.conf inheritance order. In this way a repository can inherit another repositories repoman configuration, then extend or limit the actions it will perform on the specific repositrory.

Note
Currently the repository level configuration system is restricted to an `--experimental-repository-modules=y` option for now until the file format has been approved and accepted into the gentoo repository. At this time, the files used are installed as part of the repoman installation and reside in {EPREFIX}/usr/share/repoman/repository/ directory. These are the same files that currently reside in the gentoo repo. But the files are to be considered experimental until their format is finalized and approved. The --experimental-repository-modules=y can be added to a REPOMAN_DEFAULT_OPTS variable in /etc/portage/make.conf


qa_data.yml

CODE qa_data.yml File
---
# This yaml syntax file holds various configuration data for
# the Quality-Assurance checks performed.

# no random drive-by commits please
# Please obtain authorization from the portage team
#
# Overlay maintainers override/add/negate checks at your discression
# but support for third party module will be limited to the plugin API
#

# Repoman API version (do not edit)
version: 1
# minimum
repoman_version: 2.3.3


allowed_filename_chars: "a-zA-Z0-9._-+:"
max_description_length: 80

# missingvars check: Mandatory (non-defaulted) ebuild variables
# list
missingvars:
    - KEYWORDS
    - LICENSE
    - DESCRIPTION
    - HOMEPAGE

# file.executable check, non executable files
# list
no_exec_files:
    - Manifest
    - ChangeLog
    - metadata.xml

# qawarnings: Non-fatal warnings,
#             all values in here MUST have a corresponding qahelp entry
# list
qawarnings:
    - changelog.missing
    - changelog.notadded
    - dependency.unknown
    - dependency.badmasked
    - dependency.badindev
    - dependency.badmaskedindev
    - dependency.badtilde
    - dependency.missingslot
    - dependency.perlcore
    - DESCRIPTION.toolong
    - digest.assumed
    - digest.unused
    - EAPI.deprecated
    - ebuild.notadded
    - ebuild.nesteddie
    - ebuild.absdosym
    - ebuild.minorsyn
    - ebuild.badheader
    - ebuild.patches
    - file.empty
    - file.size
    - HOMEPAGE.virtual
    - inherit.unused
    - inherit.deprecated
    - IUSE.rubydeprecated
    - java.eclassesnotused
    - KEYWORDS.dropped
    - KEYWORDS.stupid
    - KEYWORDS.missing
    - LICENSE.deprecated
    - LICENSE.virtual
    - metadata.warning
    - PDEPEND.suspect
    - portage.internal
    - RDEPEND.implicit
    - RDEPEND.suspect
    - repo.eapi-deprecated
    - RESTRICT.invalid
    - usage.obsolete
    - upstream.workaround
    - uri.https
    - virtual.suspect
    - wxwidgets.eclassnotused

# ruby_deprecated: Deprecated ruby targets
# list
ruby_deprecated:
    - ruby_targets_ruby18
    - ruby_targets_ruby19
    - ruby_targets_ruby20

# suspect_rdepend: Common build only Dependencies
#                  not usually run time dependencies
# list
suspect_rdepend:
  - app-arch/cabextract
  - app-arch/rpm2targz
  - app-doc/doxygen
  - dev-lang/nasm
  - dev-lang/swig
  - dev-lang/yasm
  - dev-perl/extutils-pkgconfig
  - dev-qt/linguist-tools
  - dev-util/byacc
  - dev-util/cmake
  - dev-util/ftjam
  - dev-util/gperf
  - dev-util/gtk-doc
  - dev-util/gtk-doc-am
  - dev-util/intltool
  - dev-util/jam
  - dev-util/pkg-config-lite
  - dev-util/pkgconf
  - dev-util/pkgconfig
  - dev-util/pkgconfig-openbsd
  - dev-util/scons
  - dev-util/unifdef
  - dev-util/yacc
  - media-gfx/ebdftopcf
  - sys-apps/help2man
  - sys-devel/autoconf
  - sys-devel/automake
  - sys-devel/bin86
  - sys-devel/bison
  - sys-devel/dev86
  - sys-devel/flex
  - sys-devel/m4
  - sys-devel/pmake
  - virtual/linux-sources
  - virtual/linuxtv-dvb-headers
  - virtual/os-headers
  - virtual/pkgconfig
  - x11-misc/bdftopcf
  - x11-misc/imake

# suspect_virtual: Dependencies that should usually be made to the virtual
#                  Not to the final target library
# dictionary
suspect_virtual:
  dev-libs/libusb: virtual/libusb
  dev-libs/libusb-compat: virtual/libusb
  dev-libs/libusbx: virtual/libusb
  dev-util/pkg-config-lite: virtual/pkgconfig
  dev-util/pkgconf: virtual/pkgconfig
  dev-util/pkgconfig: virtual/pkgconfig
  dev-util/pkgconfig-openbsd: virtual/pkgconfig

# valid_restrict: ???
# list
valid_restrict:
    - binchecks
    - bindist
    - fetch
    - installsources
    - mirror
    - preserve-libs
    - primaryuri
    - splitdebug
    - strip
    - test
    - userpriv

repository.yml

CODE repository.yml
---
# repository-modules.yaml
#
# This is the repository configuration file for repoman modules
#
# no random drive-by commits please
# Please obtain authorization from the portage team
#
# Overlay maintainers override/add/negate checks at your discression
# but support for third party module will be limited to the plugin API
#

# Repoman API version (do not edit)
version: 1
# minimum
repoman_version: 2.3.3

# NOTE: for non-gentoo repos, any custom modules added will need their
# module names to the modules list in order for them to run.

# These are the non-mandatory modules that can be disabled/enabled.
# use -foo notation to disable, just like use flags
# Add custom modules to enable them too
scan_modules:
    description
    eapi
    ebuild_metadata
    fetches
    files
    keywords
    live
    manifests
    multicheck
    pkgmetadata
    profile
    restrict
    ruby

linechecks_modules:
    assignment
    eapi3assignment
    implicitdepend
    hasq
    useq
    preservelib
    bindnow
    inherit
    dosym
    definition
    srcprepare
    eapi3deprecated
    pkgpretend
    eapi4incompatible
    eapi4gonevars
    paralleldisabled
    autodefault
    gentooheader
    nooffset
    nesteddie
    patches
    emakeparallel
    srccompileeconf
    srcunpackpatches
    portageinternal
    portageinternalvariableassignment
    quote
    quoteda
    httpsuri
    builtwith
    uselesscds
    uselessdodoc
    whitespace
    blankline
    addpredict
    noasneeded

linechecks.yml

CODE linechecks.yml
---
# linecheck.yaml

# configuration file for the LineCheck plugins run via the multicheck
# scan module
# no random drive-by commits please
# Please obtain authorization from the portage team
#
# Overlay maintainers override/add/negate checks at your discression
# but support for third party module will be limited to the plugin API
#

# Repoman API version (do not edit)
version: 1
# minimum
repoman_version: 2.3.3

eclass_export_functions:
    - ant-tasks
    - apache-2
    - apache-module
    - aspell-dict
    - autotools-utils
    - base
    - bsdmk
    - cannadic
    - clutter
    - cmake-utils
    - db
    - distutils
    - elisp
    - embassy
    - emboss
    - emul-linux-x86
    - enlightenment
    - font-ebdftopcf
    - font
    - fox
    - freebsd
    - freedict
    - games
    - games-ggz
    - games-mods
    - gdesklets
    - gems
    - gkrellm-plugin
    - gnatbuild
    - gnat
    - gnome2
    - gnome-python-common
    - gnustep-base
    - go-mono
    - gpe
    - gst-plugins-bad
    - gst-plugins-base
    - gst-plugins-good
    - gst-plugins-ugly
    - gtk-sharp-module
    - haskell-cabal
    - horde
    - java-ant-2
    - java-pkg-2
    - java-pkg-simple
    - java-virtuals-2
    - kde4-base
    - kde4-meta
    - kernel-2
    - latex-package
    - linux-mod
    - mozlinguas
    - myspell
    - myspell-r2
    - mysql
    - mysql-v2
    - mythtv-plugins
    - oasis
    - obs-service
    - office-ext
    - perl-app
    - perl-module
    - php-ext-base-r1
    - php-ext-pecl-r2
    - php-ext-source-r2
    - php-lib-r1
    - php-pear-lib-r1
    - php-pear-r1
    - python-distutils-ng
    - python
    - qt4-build
    - qt4-r2
    - rox-0install
    - rox
    - ruby
    - ruby-ng
    - scsh
    - selinux-policy-2
    - sgml-catalog
    - stardict
    - sword-module
    - tetex-3
    - tetex
    - texlive-module
    - toolchain-binutils
    - toolchain
    - twisted
    - vdr-plugin-2
    - vdr-plugin
    - vim
    - vim-plugin
    - vim-spell
    - virtuoso
    - vmware
    - vmware-mod
    - waf-utils
    - webapp
    - xemacs-elisp
    - xemacs-packages
    - xfconf
    - x-modular
    - xorg-2
    - zproduct

eclass_info_experimental_inherit:
    autotools:
        funcs:
            - eaclocal
            - eautoconf
            - eautoheader
            - eautomake
            - eautoreconf
            - _elibtoolize
            - eautopoint
        comprehensive: true
        # Exempt eclasses:
        # git - An EGIT_BOOTSTRAP variable may be used to call one of
        #       the autotools functions.
        # subversion - An ESVN_BOOTSTRAP variable may be used to call one of
        #       the autotools functions.
        exempt_eclasses:
            - git
            - git-2
            - subversion
            - autotools-utils
    eutils:
        funcs:
            - estack_push
            - estack_pop
            - eshopts_push
            - eshopts_pop
            - eumask_push
            - eumask_pop
            - epatch
            - epatch_user
            - emktemp
            - edos2unix
            - in_iuse
            - use_if_iuse
            - usex
        comprehensive: false
    flag-o-matic:
        funcs:
            - 'filter-(ld)?flags'
            - 'strip-flags'
            - 'strip-unsupported-flags'
            - 'append-((ld|c(pp|xx)?))?flags'
            - 'append-libs'
        comprehensive: false
    libtool:
        funcs:
            - elibtoolize
        comprehensive: true
        exempt_eclasses:
            - autotools
    multilib:
        funcs:
            - get_libdir
        # These are "eclasses are the whole ebuild" type thing.
        exempt_eclasses:
            - autotools
            - libtool
            - multilib-minimal
        comprehensive: false
    multiprocessing:
        funcs:
            - makeopts_jobs
        comprehensive: false
    prefix:
        funcs:
            - eprefixify
        comprehensive: true
    toolchain-funcs:
        funcs:
            - gen_usr_ldscript
        comprehensive: false
    user:
        funcs:
            - enewuser
            - enewgroup
            - egetent
            - egethome
            - egetshell
            - esethome
        comprehensive: true

# non experimental_inherit
eclass_info:
    autotools:
        funcs:
            - eaclocal
            - eautoconf
            - eautoheader
            - eautomake
            - eautoreconf
            - _elibtoolize
            - eautopoint
        comprehensive: true
        ignore_missing: true
        # Exempt eclasses:
        # git - An EGIT_BOOTSTRAP variable may be used to call one of
        #       the autotools functions.
        # subversion - An ESVN_BOOTSTRAP variable may be used to call one of
        #       the autotools functions.
        exempt_eclasses:
            - git
            - git-2
            - subversion
            - autotools-utils
    prefix:
        funcs:
            - eprefixify
        comprehensive: true

usex_supported_eapis:
    - "0"
    - "1"
    - "2"
    - "3"
    - "4"
    - "4-python"
    - "4-slot-abi"

in_iuse_supported_eapis:
    - "0"
    - "\1"
    - "2"
    - "3"
    - "4"
    - "4-python"
    - "4-slot-abi"
    - "5"
    - "5-hdepend"
    - "5-progress"
Retrieved from "https://wiki.gentoo.org/index.php?title=Project:Portage/Repoman-Module-specs&oldid=1160285"