Upstream repository shutdowns/Google code

This page intends to organize required actions to prepare the announced shutdown of Google code. 

Urgency
A wrong URL in an ebuild is a bug. It is necessary to prevent a flood of manually written tickets in bugzilla. We do not have the manpower and the tools to manage 500 new, manually written bugs plus duplicates efficiently. Hence it is important that the ebuilds are fixed soon. "These tarballs will be available throughout the rest of 2016."

List of packages
or use the md5 cache

Output (googlecode-shutdown.txt) of the command as run on 2016-11-05.

Who maintains how many broken packages and what are their names?

 * Who maintains how many broken packages? owner_histogram.txt (NumberOfPackages Contact)
 * Who maintains which broken package? packages_by_owner.txt (PackageName Contact)

Security
Many projects got forked several times during the shut down procedure and it is not always easy to find out which fork is the right one for the Gentoo ebuild.

While many packages need an update to a new URL it is easy to plant silently a link to malicious source code instead of the original source code. It is also possible that a Gentoo developer uses a wrong fork by accident.

Brainstorming

 * First step: just 'clone' all SRC_URI to a random web server, and fix the current one to point at it.
 * Second step: figure out if the project has moved or has been frozen.

Mirrors
Q: Are there sources on googlecode that we must not mirror? A: No:

cd /usr/portage/metadata/md5-cache grep "URI=.*googlecode\.com" -R -l | xargs grep "RESTRICT"

returns all RESTRICT variables of the googlecode ebuilds. They do not use restrictions that would forbid mirroring in general.


 * https://devmanual.gentoo.org/general-concepts/mirrors/

Statistics
In the case of BerliOS it took about one year to fix 50% of the ebuilds. Perhaps it is interesting to keep some statistics.

Command to collect date and number of packages with googlecode in the *URI variable:

A graph and the raw data is available on http://www.akhuettel.de/~huettel/plots/googlecode.php

Fixing a package

 * If there is no HOMEPAGE https://wiki.gentoo.org/wiki/No_homepage

Worklist

 * DONE 2015-03-22 create tracker bug on bugzilla (jonasstein)
 * DONE 2016-10-28 Find out which packages need to be fixed exactly. Need bash scripts. (jonasstein)
 * DONE 2016-10-28 Create statistics on the progress. (dilfridge,jonasstein)
 * DONE 2016-11-05 send mail to gentoo-dev (jonasstein)
 * DONE 2016-11-05 create script to generate maintainer:package lists (kentnl)
 * DONE 2016-11-12 create the statistics on some historical dates for the plot. (dilfridge)
 * DONE 2016-11-24 send the mail to maintainers of remaining broken ebuilds (jonasstein)
 * TODO after 2016-12-01 create bug tickets for remaining broken ebuilds
 * TODO find out, which packages do not allow a mirror in the license (treecleaning candidate)
 * TODO verify, that all ebuilds are mirrored. Mirror all source files (tar balls...), add link here, add link to log which packages failed to mirror.
 * TODO find out how to proceed with mirrors if upstream is dead. Do we have a policy on that?
 * TODO a google code repository makes repoman really sad, so repoman should tell the user about its feelings ;-)

Other repositories that were shut down
To learn from history:


 * BerliOS (70 ebuilds affected)
 * gitorious.org (26 ebuilds affected)