Upstream repository shutdowns/bitbucket.org

This page intends to organize required actions to prepare the BitBucket retires support for mercurial repositories (bug #737896).

List of packages

Use script and grep hackery:

user $cd /usr/portage ; dirname $(grep -r '//bitbucket.org' --exclude-dir=metadata --include=*.ebuild -l) | sort -u | python query.py

FILE query.py

import fileinput
import portage
import re
import requests


def check_uri(uri):
    if re.search(r'//bitbucket\.org', uri):
        response = requests.get(uri)
        if response.status_code > 399:
            # print("URI failed:", uri)
            return True
    return False


p = portage.db[portage.root]["porttree"].dbapi

for package in fileinput.input():
    src_uri_affected = False
    homepage_affected = False

    pkg_info = p.cp_list(package.strip())
    ebuild_info = p.aux_get(pkg_info[-1], ["SRC_URI", "HOMEPAGE"])

    for src_uri in ebuild_info[0].split(" "):
        src_uri_affected = src_uri_affected or check_uri(src_uri)
    for homepage in ebuild_info[1].split(" "):
        homepage_affected = homepage_affected or check_uri(homepage)

    if src_uri_affected or homepage_affected:
        print(package.strip(), src_uri_affected, homepage_affected)

Howto fix

Find and replace all URLs contains https://bitbucket.org in HOMEPAGE, SRC_URI and HG_URI_REPO (for live ebuilds). Most of packages already moved their repositories to another sites (like Github or Heptapod).

If there no active project mirror available, you can try https://bitbucket-archive.softwareheritage.org/ tor download & mirror tarballs in mirror.gentoo.org. If there no tarballs, you can make them from repository snapshot from bitbucket-archive site as last resort.

Worklist

• DONE 2020-08-19 bug #737896 Tracker for activity (Winterheart (talk))
• DONE 2020-08-20 List of affected packages (Winterheart (talk))
• DONE 2020-08-20 Send message to gentoo-dev (Winterheart (talk))
• TODO after 2020-??-?? Pmask
• TODO verify, that all ebuilds are mirrored. Mirror all source files (tarballs...), add link here, add link to log which packages failed to mirror.
• TODO Create statistics on the progress.
• TODO send the mail to maintainers of remaining broken ebuilds

Packages to fix (2020-09-22)

# name src_uri_fetch_error homepage_fetch_error
app-text/csvfix True True
app-vim/frawor True True
app-vim/gundo True False
app-vim/splice True False
dev-python/django-auth-ldap False True
dev-python/tempita True False
games-action/lugaru False True
media-gfx/qiv True True
media-libs/x265 False True
media-video/atomicparsley-wez True False
media-video/transcode False True
net-misc/connect True True
net-misc/qtm True False
net-misc/sks True True
sys-power/ncpufreqd False True
x11-misc/openbox-menu True False
x11-plugins/wmcpuwatch True True