User:Kangie/Zero to Gentoo PR Hero

This article provides a guide to configuring a GitHub account and local environment to submit Pull Requests to the Gentoo GitHub mirror.

In this example we will perform a simple version bump of a package, using "dev-libs/foo" as an example.

As this package does not exist some common-sense substitution will need to be made in order to follow along.

Prerequisites

While it is not required that the reader have an understanding of Git or GitHub this is desirable knowledge; those unfamiliar should read the following resources:

Git Basics - Git Workflow - A basic introduction to git and GitHub
GitHub Getting Started - A basic introduction to GitHub

It is assumed that the reader either has existing SSH and GPG keys to use or is capable of following the instructions to create them.

Note
It is possible to authenticate painlessly over HTTPS with a Personal Access Token (PAT) while using a git credentials manager to ensure that credentials do not need to be entered each time. As SSH is the better option this is what has been documented.

Preparation

Note
While the email addresses used in SSH and GPG are arbitrary, they will be publicly visible on commits and for the purposes of GitHub should match the verified email address for the account.

GitHub

In this step we will prepare the GitHub account and local environment to submit Pull Requests.

First, set up a GitHub Account to use for Gentoo pull requests; use of an existing account is encouraged but not required.

Navigate to GitHub and follow the bouncing ball.

Next, configure this account for SSH access. Authenticating via SSH enables users to avoid the use of a password (deprecated) or Personal Access Token (PAT) when pushing changes to GitHub.

Reference the following official GitHub Documentation

Important
Keep private keys secure. These instructions provide a basic level of security but generate and store the private key on the same workstation as is used for development. A Hardware Security Key is far more secure.

TL;DR:

1. Generate a new SSH keypair, if required:

user $ssh-keygen -t ed25519 -C "Larry.the.C0w@gentoo.zip"

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/larry/.ssh/id_ed25519):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/larry/.ssh/id_ed25519
Your public key has been saved in /home/larry/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:FM9FZ2US8YiT72WziwsgqzrbHIRFv6mC4x2SGyeuNkg Larry.the.C0w@gentoo.zip
The key's randomart image is:
+--[ED25519 256]--+
|     .  .  .o *+o|
|    . .  + . = = |
|     . .. o + . .|
|    o  .o    o   |
|   . . +S.    ..o|
| Eo . . o .  . oo|
|oB + o .   .  .. |
|+oO.= o     . . .|
|++.+++       o.. |
+----[SHA256]-----+

2. Add the key to the ssh agent:

user $eval "$(ssh-agent -s)"

user $ssh-add ~/.ssh/id_ed25519

3. Add the public key to the GitHub account:

user $cat ~/.ssh/id_ed25519.pub

user $# copy the output to the clipboard

a. In the upper-right corner of any GitHub page, click the profile photo, then click Settings.

b. In the "Access" section of the sidebar, click SSH and GPG keys.

c. Click New SSH key or Add SSH key.

d. In the Title field, add a descriptive label for the key.

e. Select authentication; this will be the default - commits will be signed by GPG later.

f. In the "Key" field, paste the public key.

g. Click Add SSH key.

Tools and Configuration

Emerge the following tools and configure them:

Important
Read and understand the Gentoo Copyright Policy. The following steps will instruct Git and Pkgdev to automatically sign off on a Certificate of Origin. This is a requirement for all commits to the Gentoo repository.

root #emerge --ask app-crypt/gnupg dev-vcs/git dev-util/pkgcheck dev-util/pkgdev

Configure the tools as follows, substituting appropriate values.

GNU Privacy Guard

Note
See the main wiki article for background and in-depth configurationg guidance.

FILE ~/.gnupg/gpg.confReference GLEP63 configuration

# Assume that command line arguments are given as UTF8 strings.
utf8-strings

# when outputting certificates, view user IDs distinctly from keys:
fixed-list-mode

# long keyids are more collision-resistant than short keyids (it's trivial to make a key
# with any desired short keyid)
# NOTE: this breaks kmail gnupg support!
keyid-format 0xlong

# when multiple digests are supported by all recipients, choose the strongest one:
personal-digest-preferences SHA512 SHA384 SHA256 SHA224

# preferences chosen for new keys should prioritize stronger algorithms:
default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 BZIP2 ZLIB ZIP Uncompressed

# You should always know at a glance which User IDs GPG thinks are legitimately bound to
# the keys in the keyring:
verify-options show-uid-validity
list-options show-uid-validity

# include an unambiguous indicator of which key made a signature:
# (see http://thread.gmane.org/gmane.mail.notmuch.general/3721/focus=7234)
# (and http://www.ietf.org/mail-archive/web/openpgp/current/msg00405.html)
sig-notation issuer-fpr@notations.openpgp.fifthhorseman.net=%g

# when making an OpenPGP certification, use a stronger digest than the default SHA1:
cert-digest-algo SHA512
s2k-cipher-algo AES256
s2k-digest-algo SHA512

Pinentry

app-crypt/pinentry is a helper application that gpg-agent uses to request the passphrase in a graphical window. It comes in many flavors, including: gtk3, qt5, tty, and curses.

Note
Building pinentry-curses is recommended, as it is typically used as a fallback pinentry.

If app-crypt/pinentry was installed with more than one frontend, it is possible to choose between them with the eselect pinentry command:

root #eselect pinentry list

Available pinentry binary implementations:
  [1]   pinentry-gnome3
  [2]   pinentry-qt5
  [3]   pinentry-curses
  [4]   pinentry-tty *

Select the appropriate pinentry:

root #eselect pinentry set pinentry-qt5

Key Generation

Note
This presents a very brief, GitHub-focused overview of key generation. For more information see the GnuPG.

This example generates a key with a 4096-bit RSA key and SHA512 digest. This is a relatively common configuration and does not hurt anything.

For the purposes of this documentation the GPG key is configured not to expire; users should consider setting an expiration date and rotating keys.

user $gpg --full-generate-key

gpg (GnuPG) 2.4.3; Copyright (C) 2023 g10 Code GmbH
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: directory '/home/larry/.gnupg' created
gpg: keybox '/home/larry/.gnupg/pubring.kbx' created
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
  (14) Existing key from card
Your selection?
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (3072) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0)
Key does not expire at all
Is this correct? (y/N) Y

GnuPG needs to construct a user ID to identify your key.

Real name: Larry the Cow
Email address: Larry.the.C0w@gentoo.zip
Comment: I am the bovine king
You selected this USER-ID:
    "Larry the Cow (I am the bovine king) <Larry.the.C0w@gentoo.zip>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

At this point the user will be prompted to enter a secure passphrase for the key using their selected pinentry mechanism. The following output will be received (with a different key ID):

user $gpg --list-secret-keys --keyid-format LONG

. . .
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: directory '/home/larry/.gnupg/openpgp-revocs.d' created
gpg: revocation certificate stored as '/home/larry/.gnupg/openpgp-revocs.d/D86ECD222504284306FBD326B724D03F46DCC26B.rev'
public and secret key created and signed.

pub   rsa4096 2023-11-05 [SC]
      D86ECD222504284306FBD326B724D03F46DCC26B
uid                      Larry the Cow (I am the bovine king) <Larry.the.C0w@gentoo.zip>
sub   rsa4096 2023-11-05 [E]

Validate this key is present:

user $gpg --list-secret-keys --keyid-format LONG

gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   2  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 2u
/home/larry/.gnupg/pubring.kbx
-------------------------------
sec   rsa4096/B724D03F46DCC26B 2023-11-05 [SC]
      D86ECD222504284306FBD326B724D03F46DCC26B
uid                 [ultimate] Larry the Cow (I am the bovine king) <Larry.the.C0w@gentoo.zip>
ssb   rsa4096/BC30BADED363CE5B 2023-11-05 [E]

Take note of the GPG Key ID - it will be used to tell Git what to use in the next step. In this example it is B724D03F46DCC26B.

Note
Add the GPG key to your GitHub account to get the cool verified badge on commits; this is not a required step however.

Git

Warning
GitHub does not support signed pushes, which is a key difference between contributing here and on Gentoo infrastructure. Sign commits, not pushes!

Note
Configuration with the --global flag applies to the current user in all repositories—it has a counterpart --local which applies only to the current repository.

user $git config --global user.name "Larry the Cow"

user $git config --global user.email "Larry.the.C0w@gentoo.zip"

Note
The following values are optional: commit.gpgsign, if set to true will be sign all commits with the default GPG key. This is a relatively common configuration and does not hurt anything. push.autoSetupRemote ensures that users do not manually need to set upstreams for branches.

user $git config --global commit.gpgsign true

user $git config --global user.signingkey B724D03F46DCC26B

user $git config --global push.autoSetupRemote true

If setting a subkey include the ! suffix.

Pkgcheck and Pkgdev

Add the SIGNED_OFF_BY line to make.conf

FILE /etc/portage/make.conf

SIGNED_OFF_BY="Larry the Cow <Larry.the.C0w@gentoo.zip>"

Configure pkgdev to automatically sign off on commits and scan for common mistakes. We will push manually.

FILE /etc/pkgdev/pkgdev.conf

[DEFAULT]
commit.signoff = true
commit.scan = true

Instruct pkgcheck to perform internet-based checks (e.g. is the HOMEPAGE reachable); add a 30s timeout:

FILE /etc/pkgcheck/pkgcheck.conf

[DEFAULT]
net =
timeout = 15

Forking and configuring the Gentoo GitHub mirror

In this section the Gentoo GitHub mirror will be forked and configured in preparation for submitting a Pull Request.

Note
This step (and the previous ones) only need to be performed once to configure the environment - each PR will use this configuration.

1. Navigate to the Gentoo GitHub mirror and click the 'Fork' button. https://github.com/gentoo/gentoo

2. Clone this new fork using the SSH endpoint to create a local working copy:

user $git clone git@github.com:Larry/gentoo.git

3. Add the Gentoo GitHub as an upstream so that the repositories can be synced

user $git remote add upstream https://github.com/gentoo/gentoo.git

2. Sync the fork. This likely will not capture any changes now but will be important to the Git workflow later!

user $git checkout master

user $git fetch upstream

user $git merge upstream/master

user $

git push

remote: Enumerating objects: 2757, done.
remote: Counting objects: 100% (2754/2754), done.
remote: Compressing objects: 100% (763/763), done.
remote: Total 2757 (delta 2004), reused 2719 (delta 1991), pack-reused 3
Receiving objects: 100% (2757/2757), 1.12 MiB | 4.34 MiB/s, done.
Resolving deltas: 100% (2004/2004), completed with 58 local objects.
From https://github.com/gentoo/gentoo
   a12ff80e3f89..65e367ff8bd8  master     -> upstream/master
Successfully rebased and updated refs/heads/master.
Enumerating objects: 3329, done.
Counting objects: 100% (3328/3328), done.
Delta compression using up to 32 threads
Compressing objects: 100% (801/801), done.
Writing objects: 100% (2804/2804), 527.72 KiB | 5.03 MiB/s, done.
Total 2804 (delta 2395), reused 2402 (delta 2003), pack-reused 0
remote: Resolving deltas: 100% (2395/2395), completed with 421 local objects.
To github.com:Larry/gentoo.git
   a12ff80e3f89..65e367ff8bd8  master -> master

Making changes

A proper git(hub) workflow is beyond the scope of this document. However, we will cover the basics.

All PRs must be submitted from a branch other than master. There are several reasons to follow the branch-and-pull-request workflow, including:

It is easier to keep track of changes
It is easier to revert changes
It is easier to collaborate with others
Changes for one package will not interact with changes for another package
It is possible to create a new branch from a clean master and work on something else at any time

It is possible to have many branches (and simultaneous open Pull Requests) while maintaining a clean master branch.

1. Create a branch for the changes:

user $git checkout -b bump-foo

Switched to a new branch 'bump-foo'

2. Navigate to the foo directory. If there is a live ebuild template use it, otherwise use the previous version as a basis for the new ebuild:

user $cd dev-libs/foo

user $cp dev-libs/foo/foo-4.0.0.ebuild dev-libs/foo/foo-4.0.1.ebuild

3. Edit the ebuild as required. For the sake of this example we will pretend that we're updating the SRC_URI to point to GitHub, too:

Note
Ensure that KEYWORDS is set appropriately.

4. Add the changes to the manifest:

root #pkgdev manifest

* generating manifest: dev-libs/foo::gentoo

Note
For a more complex example involving patching the ebuild souces see Musl patching notes

5. Use the ebuild command to build and test the software. Seek guidance to fix any issues that are encountered - buggy software is not accepted into the Gentoo repository!

root #ebuild foo-4.0.1.ebuild manifest clean test [merge]

Appending /data/development/gentoo to PORTDIR_OVERLAY...
Forcing test.
. . .
TESTDONE: 1648 tests were considered during 31 seconds.
TESTDONE: 1320 tests out of 1320 reported OK: 100%
make[1]: Leaving directory '/var/tmp/portage/dev-libs/foo/work/foo-4.0.1-abi_x86_64.amd64/tests'
>>> Completed testing dev-libs/foo-4.0.1

5. Assuming the tests pass with no issues, stage the changes to be committed:

user $git add .

6. Use pkgdev to commit the changes:

user $

pkgdev commit

dev-libs/foo
  UnstableOnly: for arch: [ amd64 ], all versions are unstable: [ 4.0.0, 4.0.1 ]
[bump-foo 3b2c598ce75e] dev-libs/foo: add 4.0.1
 2 files changed, 804 insertions(+)
 create mode 100644 dev-libs/foo/foo-4.0.1.ebuild

Note
For a trivial version bump pkgdev will typically get the commit message right without issue. Use git commit --amend to amend the commit message if this is not the case.

7. Push the changes

user $git push

Enumerating objects: 20, done.
Counting objects: 100% (20/20), done.
Delta compression using up to 32 threads
Compressing objects: 100% (18/18), done.
Writing objects: 100% (18/18), 12.31 KiB | 1.54 MiB/s, done.
Total 18 (delta 6), reused 7 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (6/6), completed with 2 local objects.
remote:
remote: Create a pull request for 'bump-foo' on GitHub by visiting:
remote:      https://github.com/Larry/gentoo/pull/new/bump-foo
remote:
To github.com:Larry/gentoo.git
 * [new branch]                bump-foo -> bump-foo
branch 'bump-foo' set up to track 'origin/bump-foo'.

GitHub will return a URI in the response - use this to submit a Pull Request using the WebUI.

Troubleshooting

gpg: «task» failed: Inappropriate ioctl for device

Immediate band-aid fix:

user $export GPG_TTY=$(tty)

Long-term fix:

Don't use an elevated shell
Make sure the right pinentry is selected