Project:Security/GLSA Coordinator Guide

This document contains procedures, tips and tricks applying to the GLSA Coordinator job.

Accounts
A certain number of accounts must be established before working as a GLSA coordinator. To draft GLSAs you must get a GLSAMaker account. To manage security bugs you need to have a Bugzilla account, which will be upgraded to  privileges. To send GLSA announcements you need to have a yourname@gentoo.org address (i.e. to be a Gentoo developer). This address should be allowed to send to gentoo-announce. To commit XML GLSAs you need your developer account to be upgraded with commit access to the GLSA directory in the  CVS repository. Finally you need an IRC name. You will be required to hang out in the #gentoo-security channel whenever you're available online.

GPG key
You must create a GPG key for your yourname@gentoo.org email address. You can either create a specific key or add the gentoo.org address to an existing key. The key ID should be set in the LDAP, and you should check that your name and key ID appears on the developer list. It is very important that the key is published at least on the subkeys.pgp.net keyserver. It can also be submitted on other keyservers.

Environment
You must setup a CVS environment on your home machine to be able to commit your GLSAs to the tree. This is done by checking out a part of the  CVS repository to a given directory (for example ~/gentoo_cvs):

Setup CVS environment

Mailing-list subscriptions
As a security developer, you will be added to the gentoo-core list and security@gentoo.org alias. You should also subscribe to gentoo-announce, gentoo-dev and gentoo-security.

Security bug types and Bugzilla components
All security bugs are located in the  Bugzilla product. If you find a security bug which does not have the correct product name, please fix it immediately. There are several different types of security bugs, and each has its own component.

Vulnerability
Vulnerabilities are security bugs in the upstream software or introduced in the Gentoo ebuild packaging. These bugs result in GLSAs. Kernel bugs have their own category and should not filed as.

Kernel
Kernel vulnerabilities are treated using a separate procedure. To easily distinguish them from the other bugs, they are filed under the  category. Kernel bugs do not result in GLSAs.

Default config
Gentoo packages should be as secure by default as possible. Default configuration bugs are filed when the default configuration shipped with the package can be improved in terms of security. bugs do not generate GLSAs.

Auditing
Vulnerabilities that are found by Gentoo developers should be double-checked before going out in the open (to upstream developers or security lists). They are filed as Confidential Bugs (see below) and with the  component. When the finding has been verified they are switched to.

Restricted bugs
Sometimes a bug is communicated to us under the promise we'll keep it secret until a public release, usually known as the embargo date or coordinated release date. Restricted bugs have the "Gentoo Security" checkbox checked and therefore can only be accessed by Gentoo Security Team members. External people (package maintainers, arch testers) may be added on a per-name basis, aliases should never be used (because they are too wide and won't allow bug comments).

There are three different types of restricted bugs. The first (and most secret) ones are the  bugs. A bug is classified when it contains information that should never be released. This includes quotes of personal emails sent to restricted mailing-lists or non-public intermediary patches. Classified bugs are identified by the  keyword in their Status Whiteboard. Once CLASSIFIED, a bug cannot go back to unclassified status unless at least two security managers agree to declassify it. bugs should never be opened (unrestricted).

The second type of restricted bugs is  bugs. These are bugs that contain information that should be kept secret until an agreed-upon coordinated release date. No part of the bug (affected package name, description, proposed patch or whatever) should ever leak outside the bug. Patches must NOT be committed to portage CVS.

The third (and less secret) type of restricted bugs is the  bugs. Semi-public bugs should be kept secret, but patches may be committed to portage. This is generally when the vulnerability is not known to the general public but could be accessed by anyone (patch in upstream CVS for example).

Bug summary rules
The bug summary has two formats depending on the whiteboard status.

When no fixed version is in the Portage tree (whiteboard is [upstream] or [ebuild]), versions should be omitted until a stable candidate has been identified. In this instance, the summary syntax will be:

When a stable candidate or fixed version is available in Portage (whiteboard is [stable], [glsa], or [cleanup]), the bug title is changed to indicate which versions of the package are vulnerable. In this instance, the bug summary syntax will be:

Multiple packages with similar names can be grouped in the bug summary using braces. This is commonly used with binary packages. For example, Mozilla products are also available as a binary package. This is listed in the bug summary as:

Another example is the Oracle JRE and Oracle JDK packages, where the middle of the package name is different:

Multiple CVE references in a single bug can be grouped in the summary by year. For example, the Oracle JRE and Oracle JDK January 2014 Critical Patch Update included CVE references from 2013 and 2014. These can be grouped in the bug summary as follows:

Multiple fixed slots for a single package can be grouped in the bug summary after the package name. For example, Wireshark has 1.8.x and 1.10.x stable branches. When a fixed version of each slot is in the Portage tree, it can be noted in the bug summary as follows:

See the next section for more about status whiteboard rules.

Status whiteboard rules
The status whiteboard in Bugzilla lets us keep track of the security bug resolution steps. It should be following this pattern: "RATING [status] coordinator", where:

The following statuses are accepted:

The following extra information is admitted:

Examples:

Determining original bug status
When the fix is not available from the upstream developer and/or there is no official patch to solve the problem, the bug is in  status. If a fix is available, the bug is in  status. If a fixed ebuild is put in the Portage tree, the next step is stabilizing, so the bug is in  status. When the fixed ebuild is in Portage with all required keywords set and if the issue is severe enough to warrant a GLSA, the bug is in  status. Is the bug not severe enough, GLSA voting is next, so the status would be.

Bugs in [upstream] status
In  status, we wait for the fixed version or a decent patch to appear. This status requires to regularly check upstream for information: development and announce mailing lists, websites, bugs database, VCS when available, are all other relevant sources of information. Unofficial patches must be taken into account only if the upstream developer does not react to the vulnerability, and they must be thoroughly double-checked to ensure they are not evil. When a fix version is announced or a patch is found, the bug enters  status.

If there is no reaction from upstream and no patch, we enter  status. The only option here is to security-mask the vulnerable package and issue a temporary GLSA if needed. See the policy for more details on the security masking approval procedure. The status whiteboard should then be flagged with masked and/or tempglsa keywords and bug severity set to.

Bugs in [ebuild] status
In  status, we must identify the maintainer of the package, and summon him to commit a fixed ebuild. Sources to identify the herd or developer responsible for maintaining a package are the metadata.xml file in the directory of the package in portage and the Changelog file showing who did the latest version bumps. You should Cc: the herd and maintainer responsible for the package on the bug and ask him to provide an ebuild for the fixed version. You should periodically check that an ebuild wasn't committed to Portage without any comment on the bug (it happens). When the ebuild appears, the bug enters  status.

If the maintainer doesn't show up, we enter  status. In case a fixed version is available, you should test if a simple version bump (renaming the ebuild to the needed/fixed version and emerging it) works. If only a patch is available, you should test if it applies cleanly. Then you should find a Security Team member with Portage tree commit rights to do the bump and mark the ebuild  for testing.

If the bump is not easy and/or any problem is detected with the bumped ebuild, the only option is to security-mask the unmaintained package and issue a temporary GLSA if needed. See the policy for more details on the security masking approval procedure. The status whiteboard should then be flagged with  and/or   keywords and bug severity set to.

Bugs in [stable] status
In  status, you have to determine what KEYWORDS the provided ebuild needs before the GLSA can go out. This can be tricky. You have to consider every version currently in portage so that the fix ebuild has equally or more stable keywords than any other packages in Portage. Here is an example:

Once you have determined (and noted for reference on the bug) the needed KEYWORDS, you should Cc: arch teams and ask them to mark the ebuild stable or testing accordingly. To make sure that the arch teams will pick the bug up, don't forget to add "STABLEREQ" to the bug's "Keywords" field. The arches alias are archname@gentoo.org (x86@gentoo.org, ppc@gentoo.org...). All arches (including "unsupported" arches) must be called. But note that only "supported" arches (as defined in the policy) are needed before the bug can advance to  status You should periodically check for new keywords in the ebuild, as sometimes they are changed without a comment in the bug. As soon as the required KEYWORDS are in the ebuild for all supported arches, the bug enters  status.

If the arch teams take too much time testing and changing the KEYWORDS, or they refuse to mark stable a package due to outstanding problems, the bug enters  status. We must track down arch maintainers to have them mark stable or help testing. You should also convince the arches that if they discover a bug in the ebuild that already was in previous "stable" versions, the ebuild should be marked "stable" anyway, even if it's not really stable, as specified in the policy. If the KEYWORDS can't be met, the only other option is to security-mask the package: there is no acceptable unaffected version, so it is like if no acceptable upstream fix was provided.

Bugs in [glsa] status
After stabilization, if the security bug is corrected and the severity of the bug requires a GLSA, the status becomes. A GLSA request should be filed in GLSAMaker. The bug remains open until the GLSA is published.

There are also cases where a GLSA vote must take place to decide if a GLSA is needed (  status). If at least two Security Team members vote for "no GLSA", then no GLSA should be issued, and the bug status will become. Any vulnerable versions should be removed from Portage, then the bug can be closed as FIXED without releasing a GLSA.

For bugs where the severity is trivial, the bug status after stabilization can become. Any vulnerable versions should be removed from Portage, then the bug can be closed as FIXED without releasing a GLSA.

When a GLSA is needed but nothing was sent, the bug enters  status. This is the case when there are delays during drafting, reviewing or sending the advisory. Any member of the Security Team should take the lead and finalize the GLSA.

Bugs in [cleanup] status
This status is not in order like the others. It is used to denote that the maintainer needs to remove vulnerable ebuild versions in the tree.

For example, the foo package has a vulnerability in versions earlier than 1.23. The version 1.24 was added to Portage and is stable already (i.e. the stabilization was done before the security impact of the release was known), only old ebuilds need to be removed. The next status in this case would be  or   depending on severity.

Another use case might be when a package was updated, and all steps are done (i.e. a GLSA was sent or the team decided against sending one), but the vulnerable ebuilds should really be removed to avoid users accidentally installing them, the bug could be left open and set to the  state. This is usually not needed tough, but may be used on a case-to-case base when the Security Team has an elevated interest in vulnerable ebuilds getting removed for whatever reasons.

Status whiteboard rules
Confidential bugs should be following this pattern: , where:

The following extra statuses are accepted in confidential bugs:

Handling confidential bugs
Semi-public bugs should be handled as public bugs, except that no herd or arch alias should be CC-ed on the bug but rather specific developers.

Confidential and classified bugs need extra care. The ebuild and files fixing the vulnerability should NOT be committed to the Portage tree, and no part of those bugs should ever be discussed in public. Patches or Portage overlay tarballs can be attached to the bug, though. Testers will have to setup their own overlay to test it. The idea with those bugs is to prepare the ebuild and have it tested for the coordinated release date, so that it can be released with all needed KEYWORDS together with the GLSA at the same time or shortly after the issue goes public.

General rules
The GLSA should use the affected software name with proper capitalization, not the Gentoo package name. For example, you should use "MySQL" and not "mysql". Lowercase names should be used only if the software has an all-lowercase name or if the software is identified by its command name ("traceroute").

You should not copy any part of any existing advisory, but you may use them as sources of information for our GLSA. If you copy, for example a software description, from a website, please use a citation and quotes.

Title, Synopsis, Keywords
The title must be short (< 60 characters in most cases) and have the application name (not category) in front. It should clearly identify the impact, without getting into any details. Version should be left out, except in rare cases where it allows to identify the package more clearly. Multiple affected packages should be separated by a comma. Examples include:

Synopsis is a short (< 160 characters), but complete, description of the vulnerability. People reading only the Synopsis should get a good idea what the vulnerability is and what impact it has. Examples include:

The GLSA keyword category is typically set to "Ebuild" and should contain the name of the main software vulnerable (for example, "MySQL"). Other keyword types include "Portage" (for portage bugs) and "Infrastructure" (for server compromises).

Access, Severity
Access should be "Local" or "Remote". Local vulnerabilities can be carried out only by a local user of the system. For example it implies to run a specific executable to elevate privileges, or access to a temporary directory to run a symlink attack. Remote vulnerabilities are those that can be exploited by an attacker with or without an account on the system. Remote vulnerabilities can either be active (exploiting a listening server by sending malicious queries) or passive (enticing a user to connect client software to a malicious server or to open malicious files).

Severity is an indication of how deep the vulnerability is. It is defined in the Vulnerability Treatment Policy, in the table "Evaluate the vulnerability type". Note that it depends only on the maximum risk, not on the commonality of the configuration option at risk. A buffer overflow allowing arbitrary code execution in a rare client software, only when it uses a specific configuration option, theoretically remains a "High" severity. A denial of service on all configurations of Apache theoretically remains a "Normal" severity. In rare cases the severity may be adjusted, when several members of the Security Team agree, to a more relevant level. For example, a vulnerability allowing defacement of web sites on Apache and all configurations should probably be "High" rather than "Normal".

Unaffected, Vulnerable packages
This section provides information about versions of packages that are unaffected or vulnerable to the vulnerability described in this advisory. You should pay special attention to those numbers as they are one of the few areas where mistakes imply an errata publication.

There are the different fields composing an version entry. The package name field must list the category and package name ("net-mail/exim"). Regarding the "Architectures" field, you should put "*" in it when the version description applies to all the arches marked in the ebuild. You should use multiple entries for different arches if the version description is different from arch to arch. The "Auto" field must be set to true if the package is upgradeable by emerge.

For version fields, there are multiple cases:

The simple case is when the vulnerability is present in all old versions, and is fixed in all versions newer than a specific fix version. In this case, you should use ">= first fixed version" as unaffected and "< first fixed version" as vulnerable. You should double-check that there was no ebuild between the last affected version and the first fixed version.

A more complex case is when the vulnerability is present only in a few recent versions. Let's take the example of a package where 1.2.8-r2 was not affected, 1.2.9, 1.2.9-r1 and 1.2.9-r2 were affected, and 1.2.10 is fixed. In this case there are two possibilities:

Another complex case is when the vulnerability is present in multiple slots for a package. For example, a package may have versions 1.5.x as SLOT="0" and versions 1.2.x as SLOT="1.2". If a vulnerability exists in 1.5.8 and 1.2.46 and was fixed in 1.5.9 and 1.2.47, the unaffected and vulnerable versions would look like this:

Finally, when the package has no fixed version, you should omit the "Unaffected" entry for that package and set "Auto" to "no".

Background, Description, Impact
The Background section gives information about the package at risk. It describes briefly what it is and gives any information that is helpful in understanding the part of the package vulnerable. The Background section should be shorter than the Description section or the Impact section. Good examples include:

The Description section details what the vulnerability is, without telling what happens when it is exploited. It should be understandable by people without too much security background. Sometimes there is no information about the vulnerability at all, in which cases you should let the description short. Good examples include:

The Impact section describes the global impact of the vulnerabilities described in the Description section, when exploited. It should focus on the maximum risk. Multiple impacts should be listed from highest to lowest severity. Good examples:

Workaround, Resolution
The workaround describes if there is any simple way (other than upgrading to the fix version) of avoiding to be vulnerable. Good examples include:

The Resolution section explains in human-readable terms what you have to do to be fully protected against the vulnerability. This section must look like this:

{{Code|Resolution example| All Heimdal users should upgrade to the latest stable version: OpenOffice.org users on the PPC architecture should: on the GLSA to download the XML version, and save it into your local gentoo_cvs/gentoo/xml/htdocs/security/en/glsa/ directory. Then you should add and commit the XML file:

{{Code|Commit the GLSA| you@home gentoo_cvs $ cd gentoo/xml/htdocs/security/en/glsa you@home glsa $ cvs add glsa-YYYYMM-NN.xml you@home glsa $ cvs commit -m "GLSA YYYYMM-NN" glsa-YYYYMM-NN.xml }}

E-mail announcement
Click on the Txt download to have a text version of the GLSA. Check that the affected/unaffected section looks good, then prepare an email with the following rules:


 * From and Return-Path must be set to yourname@gentoo.org
 * To and Cc should be set according to policy
 * Subject must be "[ GLSA XXXXYY-ZZ ] Your vulnerability here" (you should copy/paste from the title in the Txt file)
 * the body of the mail should be the content of the Txt file, from the "Gentoo Linux Security Advisory" header to the end of the file
 * email must be signed by the GPG key for your yourname@gentoo.org address

Bug closing
You should check that the email got through to gentoo-announce, then you can close the related bug(s), by setting their status to RESOLVED/FIXED, with a comment pointing to the GLSA number.

Errata/Update publication
An erratum is published when we made a mistake otherwise it is an update. When policy warrants a republication these guidelines should be followed:


 * Revision should be correctly set in XML. This means that revision might need to be manually corrected in GLSAMaker data directory, if you do multiple changes using GLSAMaker ( ie. September 10, 2004: 02 )
 * If there is no vulnerability no affected packages should be listed in the XML
 * Title can change ( ie. GLSA 200409-14 for Samba and GLSA 200411-01 for ppp)
 * Not all Errata need republication. Policy details when republication is needed.
 * For the text version GLSAmaker can add the relevant information to updates and erratas. Manually follow the instructions provided by GLSAmaker.
 * Text version must contain an Errata or Update section (example shown below) and AFTER THAT only the sections changed (XML version does not have extra sections)

For two complete errata email examples see ERRATA: [ GLSA 200409-14 Samba: Remote printing non-vulnerability] (where there were no real vulnerability) and ERRATA: [ GLSA 200801-09 X.Org X server and Xfont library: Multiple vulnerabilities] (where the problem was not correctly fixed in the initial version). For an update example see UPDATE: [ GLSA 200410-30 GPdf, KPDF, KOffice: Vulnerabilities in included xpdf] (where the fix introduced another vulnerability).

Otherwise normal GLSA publication guidelines should be followed.

Information gathering

 * packageview is a tool that will open packages.gentoo.org and Gentoo ViewCVS at the right place for a given category and package name. It helps to determine what keywords are needed and to track changes to a package.

GLSA publication helpers

 * glsacommit is a bash function handling GLSA commit. It features ssh-agent keyadding, glsa-check conformity doublecheck and has "Are you sure" functions. Edit it to suit your needs and directory locations.

Security Subversion repository

 * The Security Subversion repository contains several tools to collaboratively assess whether we are affected by new CVE identifiers, and tools to determine target keywords. Most tools directly interact with Bugzilla, making manual copy-pasting unnecessary.

Acknowledgements
We would like to thank the following authors and editors for their contributions to this guide:


 * Thierry Carrez
 * Sune Kloppenborg Jeppesen
 * Matthias Geerdsen
 * Robert Buchholz
 * Alex Legler