Upstream repository shutdowns/bitbucket.org

This page intends to organize required actions to prepare the BitBucket retires support for mercurial repositories (bug #737896).

List of packages

Use script and grep hackery:

user $

cd /usr/portage ; dirname $(grep -r '//bitbucket.org' --exclude-dir=metadata --include=*.ebuild  -l) | sort -u | python query.py

FILE query.py

import fileinput
import portage
import re
import requests


def check_uri(uri):
    if re.search(r'//bitbucket\.org', uri):
        response = requests.get(uri)
        if response.status_code > 399:
            # print("URI failed:", uri)
            return True
    return False


p = portage.db[portage.root]["porttree"].dbapi

for package in fileinput.input():
    src_uri_affected = False
    homepage_affected = False

    pkg_info = p.cp_list(package.strip())
    ebuild_info = p.aux_get(pkg_info[-1], ["SRC_URI", "HOMEPAGE"])

    for src_uri in ebuild_info[0].split(" "):
        src_uri_affected = src_uri_affected or check_uri(src_uri)
    for homepage in ebuild_info[1].split(" "):
        homepage_affected = homepage_affected or check_uri(homepage)

    if src_uri_affected or homepage_affected:
        print(package.strip(), src_uri_affected, homepage_affected)

Howto fix

Find and replace all URLs contains https://bitbucket.org in HOMEPAGE, SRC_URI and HG_URI_REPO (for live ebuilds). Most of packages already moved their repositories to another sites (like Github or Heptapod).

If there no active project mirror available, you can try https://bitbucket-archive.softwareheritage.org/ tor download & mirror tarballs in mirror.gentoo.org. If there no tarballs, you can make them from repository snapshot from bitbucket-archive site as last resort.

Worklist

DONE 2020-08-19 bug #737896 Tracker for activity (Winterheart (talk))
DONE 2020-08-20 List of affected packages (Winterheart (talk))
DONE 2020-08-20 Send message to gentoo-dev (Winterheart (talk))
TODO after 2020-??-?? Pmask
TODO verify, that all ebuilds are mirrored. Mirror all source files (tarballs...), add link here, add link to log which packages failed to mirror.
TODO Create statistics on the progress.
TODO send the mail to maintainers of remaining broken ebuilds

Packages to fix (2021-06-25)

# name src_uri_fetch_error homepage_fetch_error
dev-haskell/fdo-notify False True
dev-python/tempita True False
media-libs/x265 False True
media-video/atomicparsley-wez True False
media-video/transcode False True