User:MGorny/GLEP:42

Abstract
This GLEP proposes a new way of informing users about important updates and news related to the tree.

Motivation
Although most package updates are clean and require little user action, occasionally an upgrade requires user intervention. Recent examples of the latter include the  stabilisation on   and the   database format changes.

There are currently several ways of delivering important news items to our users, none of them particularly effective:


 * Gentoo Weekly News
 * The,   and   mailing lists
 * The Gentoo Forums
 * The main Gentoo website
 * RSS feeds of Gentoo news
 * and  messages in   or

A more reliable way of getting news of critical updates out to users is required to avoid repeats of various prior upgrade debacles. This GLEP proposes a solution based around pushing news items out to the user via the  tree.

Requirements
An adequate solution must meet all of the following requirements:


 * Preemptive
 * Users should be told of changes *before* they break a system, not after the damage has already been done. Ideally, the system administrator would be given ample warning to plan difficult upgrades and changes, rather than only being told just before action is necessary.


 * No user subscription required
 * It has already been demonstrated that many users do not read the  mailing list or   feeds. A solution that requires subscription has no advantage over current methods.


 * No user monitoring required
 * It has already been demonstrated that many users do not read news items posted to the Gentoo website, or do not read news items until it is too late. A solution that relies upon active monitoring of a particular source has no advantage over current methods.


 * Relevant
 * System administrators who do not use a particular package should not have to read news items which affect purely that package. Some news items may be of relevance to most or all users, but those that are not should not be forced upon users unnecessarily.


 * Lightweight
 * It is not reasonable to expect all users to have an MTA, web browser, email client, cron daemon or text processing suite available on their system. Users must not be forced to install unreasonable extra software to be able to read news items.


 * No privacy violations
 * Users of the solution should not be required to provide information about their systems (for example, IP addresses or installed packages).


 * Multiple delivery method support
 * Some users may wish to view news items via email, some via a terminal and some via a web browser. A solution should either support all of these methods or (better still) make it simple to write clients for displaying news items in different ways.

The following characteristics would be desirable:


 * Internationalisable
 * Being able to provide messages in multiple languages may be beneficial.


 * Quality control
 * There should be some way to ensure that badly written or irrelevant messages are not sent out, for example by inexperienced developers or those whose English language skills are below par.


 * Simple for developers
 * Posting news items should be as simple as is reasonably possible.


 * Simple for users
 * Reading relevant news items should be as simple as is reasonably possible.


 * Compatibility with existing and future news sources
 * A news system would ideally be able to be integrated with existing news sources (for example, Forums, GWN, the main Gentoo website) without excessive difficulty. Similarly, easy interoperation with any future news sources should not be precluded.

Overview
News items are published and delivered to users as follows:


 * 1) A news item is written. The format to be used is described below.
 * 2) The news item is reviewed, following the process described in News Item Quality Control.
 * 3) The news item is committed to a CVS (or Subversion ) repository. From here, it is merged with the rsync tree. This is described in News Item Distribution.
 * 4) Users fetch the news item when they sync. This ensures that the news items in question are pushed to the user before the user accidentally makes an unwanted change. No changes to the existing rsync process are required by this GLEP.
 * 5) The package manager filters the news item and, if it is relevant, marks the news item for reading. The package manager should also display a notice informing the user that there are unread news items.
 * 6) The news item is handled by the user's choice of news item reader. See News Item Clients.

Required Portage Enhancements
The following extensions to Portage are required:


 * Every repository (including overlays) will require a unique identifier. It is assumed that an identifier will be a string consisting of characters from  to ,   to  ,   to  ,   (plus),   (hyphen),   (underscore).


 * Portage must provide a way for external programs to obtain a list of all repository identifiers for a given system. It is assumed that this will be in the form of a  command (e.g.  ).


 * Portage must provide a way for external programs to obtain the base path for a repository with a given ID. It is assumed that this will be in the form of a  command (e.g.  ).


 * Portage must extend  to support restrictions to a given repository ID.


 * Portage must extend  to implement a command which returns whether or not the profile used for a given repository ID is exactly the given profile (e.g.  ).

These extensions are assumed during the following specification.

News Item Identities
Each news item will have a unique identifier. This identifier will be in the form, where   is the year (e.g.  ), is the month ( through  ) and dd is the day of the month (  through  ). The  is a very short name describing the news item (e.g.  ), consisting only of the characters ,  ,   (plus),   (hyphen) and   (underscore).

News Item Directories
Each news item will be represented by a directory whose name is the same as the news item's identifier.

The directory will contain a file named, which contains the text of the news item, in English, in the format described below.

If a news item is translated, other files named  (where   is the ISO 639 two letter language code, and the date remains the same as the original news item) will also be provided. However, only the English version of a news item is authoritative. This anglocentricity is justified by precedent.

News Item Files
A news item file is a text file, encoded using UTF-8 for compatibility with and for the same reasons as existing Gentoo documentation and the tree.

News items must be signed with a detached GPG signature:

gpg --armour --detach-sign ????-??-??-*.??.txt

This GLEP does not specify the type or strength of signature to be used, nor does it discuss how, if at all, a centralised keychain will be provided. These issues should be handled as part of the signing policy discussions.

A news item file's content will consist of an RFC 822 style header followed by the main body of the message as plain text. This GLEP defines various optional and mandatory headers. Future GLEPs may propose new headers — tools handling these news items must ignore any unrecognised header.

News Item Headers
The following headers describe the purpose and format of the news item:


 * A short (maximum 44 characters) descriptive title. Mandatory.
 * A short (maximum 44 characters) descriptive title. Mandatory.


 * Author's name and email address, in the form . Mandatory; multiple author headers may be specified if appropriate.
 * Author's name and email address, in the form . Mandatory; multiple author headers may be specified if appropriate.


 * For translated news items, the translator's name and email address. Multiple translator headers may be specified if appropriate.
 * For translated news items, the translator's name and email address. Multiple translator headers may be specified if appropriate.


 * Must be . Mandatory.
 * Must be . Mandatory.


 * Date of posting, in  format (e.g. 2005-12-18) for compatibility with GLEP 45 . Translations should use the date of the original news item. Mandatory.
 * Date of posting, in  format (e.g. 2005-12-18) for compatibility with GLEP 45 . Translations should use the date of the original news item. Mandatory.


 * Initially 1. Should be incremented every time a change is made to the news item. Changes that require a re-read of the news item (i.e., most changes that are not spelling or formatting related) should instead use a new news item. Mandatory.
 * Initially 1. Should be incremented every time a change is made to the news item. Changes that require a re-read of the news item (i.e., most changes that are not spelling or formatting related) should instead use a new news item. Mandatory.


 * Must be . Future revisions to the format may increment the minor number for backwards-compatible changes, or the major number for major changes.
 * Must be . Future revisions to the format may increment the minor number for backwards-compatible changes, or the major number for major changes.

The following headers are used for filtering:


 * A dependency atom (for example,  or  ). If the user has the package specified installed from the repository from which the news item was obtained, the news item should be displayed.
 * A dependency atom (for example,  or  ). If the user has the package specified installed from the repository from which the news item was obtained, the news item should be displayed.


 * A keyword name, for example  or  . If the user is on the keyword in question, the news item should be displayed.
 * A keyword name, for example  or  . If the user is on the keyword in question, the news item should be displayed.


 * A profile path, for example . If the user is using the exact profile in question, the news item should be displayed. This header may be used to replace   files in the future.
 * A profile path, for example . If the user is using the exact profile in question, the news item should be displayed. This header may be used to replace   files in the future.

The algorithm used to determine whether a news item is 'relevant' is as follows:


 * For each  header type which occurs at least once:
 * The news item is not relevant if none of the headers of this type are successfully matched.
 * Otherwise the news item is relevant.

In particular, if no  header is specified, a news item will be relevant for all users.

This algorithm was chosen because it makes conditions like "display this news item for  users who are on   or   relatively simple to specify — it is believed that these kinds of condition are far more likely to occur than "display this news item for people using , or for people on   or  " or "display these news items for people who use   and who are on both   and  ".

News Item Body
The header section must be followed by a blank line, then the main body of the text.

The text body should be wrapped at 72 characters. No fancy formatting or tab characters should be used — the news item may be being displayed directly to a terminal. Paragraphs should be separated by a blank line.

Hyperlinks may be used to refer to further information (for example, an upgrade guide). However, the main body of the news item should be descriptive and not simply a "read this link" text. It is assumed that the user will have access to a web browser somewhere, but not necessarily on the box which is being administrated — this will be the case on many servers and routers, for example.

Example News Item
This hypothetical news item could be used for an upgrade to the  database format which breaks forward compatibility.

News Item Quality Control
There have been complaints regarding the comprehensibility of some upgrade notices and news items in the past. This is understandable — not every Gentoo developer speaks English as a first language. However, for the sake of clarity, professionalism and avoiding making us look like prats, it is important that any language problems be corrected before inflicting a news item upon end users.

Thus, at least 72 hours before a proposed news item is committed, it must be posted to the  mailing list and  ed to   (exceptions may be made in exceptional circumstances). Any complaints — for example regarding wording, clarity or accuracy — must be addressed before the news item goes live.

News items must only be for important changes that may cause serious upgrade or compatibility problems. Ordinary upgrade messages and non-critical news items should remain in  notices. The importance of the message to its intended audience should be justified with the proposal.

Server Side
News items are to be made available via the standard rsync tree. This removes any need for polling of a remote source.

A new repository will be created for news items. The type (CVS or Subversion), location and access controls on this repository are beyond the scope of this GLEP.

This repository will contain directories named, where   is the current year. This separation will help keep news items more manageable.

The contents of this repository will automatically be merged with the main rsync tree, placing the items in a  directory. The method used for merging these items and the frequency at which it will occur is beyond the scope of this GLEP; a similar setup is already used for merging GLSAs into the rsync tree.

The main rsync tree will not use the  subdirectory layout. The news item directories will all be immediately under the directory.

Client Side
Whenever relevant unread news items are found, the package manager will create a file named  (if it does not already exist) and append the news item identifier (eg  ) on a new line.

All news item related files should be root owned and in the  group with the group write (and, for directories, execute) bits set. News files should be world readable.

Notification that new relevant news items will be displayed via the  tool in a similar way to the existing "configuration files need updating" messages:

* Important: there are 5 unread news items. * Type emerge --help news to learn how to read news files.

Checks for new news messages should be displayed:


 * After an
 * After an
 * Before an  (which may also include a red warning message)

The package manager does not need to know how to launch the user's choice of news client. This is consistent with the way configuration file updates are handled.

The package manager may use a timestamp check file to avoid having to process news items unnecessarily.

The package manager must keep track of news items that have already been added to the unread list to avoid repeatedly marking a deleted news item. This could be handled via a  file containing the IDs of news items that have already been added to a   file, but this method is not required by this GLEP.

Users who really don't care about news items can use  to filter out the   directory.

News Item Clients
Once a news item is marked for reading, third party tools (or traditional core Unix tools) can be used to display and view the news files.

When a news item is read, its name should be removed from the  file. If a news client acts as an interactive reader rather than a gateway, it should then add the name to a  file in the same directory with the same file format.

An  module shall be created as the 'suggested' display tool; other display tools (for example, a news to email forwarder, which would be ideal for users who sync on a ) are left as options for those who desire them.

News Item Removal
News items can be removed (by removing the news file from the main tree) when they are no longer relevant, if they are made obsolete by a future news item or after a long period of time. This is the same as the method used for  entries.

Integration with Existing Systems
It would be simple to convert these news items into the format used for news items on the Gentoo website or posts for the  mailing list.

There is an existing automated tool for posting GLSAs to the forums. A similar tool can be used for these news items.

Backwards Compatibility
Backwards compatibility is not a concern here. Existing tools will simply ignore the  directory.

Reference Implementation
A reference implementation of the required package manager support can be found in Paludis, along with a reference newsreader implemented as an eselect module.

Credits
The idea behind notifying users of news updates via Portage comes from Stuart Herbert.

Thanks to Lance Albertson, Stephen Bennett, Donnie Berkholz, Grant Goodyear, Brian Harring, Marius Mauch, Dan Meltzer, Jason Stubbs, Paul de Vrieze and Alec Warner for input. Some of the ideas presented here are theirs, others go completely against their suggestions.

Example Files
example-news-item.txt An example news item.

Copyright
This work is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/.