Gentoo git workflow

Commit policy

 * Atomic commits (one logical change)
 * All commits must be repoman-valid (exceptions only on right-hand branches)
 * Commits may span across multiple ebuilds/directories if it's one logical change
 * Every commit on the left-most line of the history (that is, all the commits following the first parent of each commit) must be gpg signed by a gentoo dev
 * repoman must be run from all related ebuild directories (or related category directories or top-level directory) on the tip of the local master branch (as in: right before you push and also after resolving push-conflicts)

Atomicity

 * Commits in git are cheap and local, so use them often
 * Don't do version bumps and ebuild cleanup in one commit (makes reverting ebuild removals more difficult)
 * It is ok to break atomicity in order to make a commit repoman-valid (or you should reconsider the order you commit in, e.g. license first, then the new ebuild)

Commit message format

 * All lines max 70-75 chars
 * First line brief explanation
 * Second line always empty
 * Optional detailed multiline explanation must start at the third line
 * Commits that affect primarily a particular subsystem should prepend the following code to the first line of the commit message:
 * Single package ->
 * Profile directory ->
 * Eclass directory ->
 * Licenses directory ->
 * Metadata directory ->
 * A whole category ->
 * If CATEGORY/PN is very long and you can't reasonable stick to the 75 char limit, just exceed it in that case
 * If the change affects multiple directories, but is mostly related to a particular subsystem, then prepend the subsystem which best reflects the intention (e.g. you add a new license, but also modify profiles/license_groups)
 * it is also encouraged to use formats such as,   and so on, also see the kernel patch guideline

Example
app-misc/foo: version bump to 0.5

This does also fix the security bug 93829 and introduces the new USE flag 'bar'.

Acked-by: Hans Wurst  Reported-by: Alice Wonderland 

Branching model

 * The primary production-ready branch is master (users will pull from here), there are no non-fast-forward pushes allowed
 * There may be developer-specific, task-specific, project-specific branches etc

Naming convention

 * Developer branches: dev/
 * Project branches: project/
 * If in doubt or if the branch could be useful to others, discuss the naming on-list beforehand

About rebasing

 * Primary use case: in case of a non-fast-forward push conflict to remote master, try  first; if that yields complicated conflicts, abort the rebase and continue with a regular merge (if the conflicts are trivial or even expected, e.g. arch teams keywording/stabilizing stuff, then stick to the rebase)
 * To preserve merges during a rebase use  (if appropriate, e.g. for user branches)
 * Don't use --preserve-merges if you do an interactive rebase (see BUGS in git-rebase manpage)
 * Commits that are not on the remote master branch yet may be rewritten/squashed/splitted etc via interactive rebase, however the rebase must never span beyond those commits
 * Never rebase on already pushed commits
 * There are no particular rules for rebasing on non-master remote branches, but be aware that others might base their work on them
 * There are no particular rules for rebasing non-remote branches, as long as they don't clutter the history when merged back into master
 * Don't do complicated rebases to avoid a merge commit at all cost (it may even cause losing information, e.g. user signatures)

About merging

 * Do not ever commit implicit merges done by git pull. You may want to set  to avoid git implicitly creating those
 * If a rebase fails or is too complicated, do a regular merge instead
 * Do a merge if the information is useful (e.g. pulled from a foreign remote user branch or merged a non-trivial eclass conversion back into master) and force a merge commit (non-fast-forward merge via --no-ff)
 * To avoid a merge commit when merging local branches back to master (e.g. information is not useful), you may try to force a fast-forward merge by first rebasing the local branch against master and then merging it into master, see here
 * Extend merge commit messages with useful information, e.g. how conflicts were solved
 * Keep in mind that all commits of the first parent of the history must be gpg signed by a gentoo dev, so you may want to force merge commits especially for user branches

Remote model
We have a main developer repo where developers work & commit (every developer has direct push access). For every push into developer repo, automated magic thingie merges stuff into user sync repo and updates the metadata cache there.

User sync repo is for power users than want to fetch via git. It's quite fast and efficient for frequent updates, and also saves space by being free of ChangeLogs.

On top of user sync repo rsync is propagated. The rsync tree is populated with all old ChangeLogs copied from CVS (stored in 30M git repo), new ChangeLogs are generated from git logs and Manifests are expanded.

Best practices

 * before starting work on your local master, it's good to first pull the latest changeset (if any) from remote master
 * it might be a good idea for projects/developers to accumulate changes either in their own branch or a separate repository and only push to remote master in intervals (that decreases the push rate and potential conflicts)

Cloning
Clone the repository. This will make a shallow clone and speed up clone time:

If you want the full history, just omit  from the above command.

Configuration
All developers should at least have the following configuration settings in their local dev repository. These setting will be written to and can also be edited manually. Run these from within the repository you've just cloned in the step above:

First, to get your gpg key run this command. It should be the top line (starting with pub). If you have more than one key with the uid you will need to select the correct key yourself (from the list of returned keys).

Now write the actual config.

In addition, you need to tell repoman which key to use, in case you use repoman for committing.

Workflow walkthrough
These are just examples and people may choose different local workflows (especially in terms of when to stage/commit) as long as the end result works and is repoman-checked. These examples try to be very safe, but basic.

Before doing anything, make sure your git tree is in a correct state and you are in the branch you actually want to work on.

Common ebuild work

 * 1) Pull in the latest changeset, before starting your work:
 * 2) Do the work (including removing files)
 * 3) Make sure you are in the ebuild directory
 * 4) Create the manifest:
 * 5) Stage files (including removed ones), if any:
 * 6) Check for errors:
 * 7) If errors occur, fix them and continue from point 4
 * 8) Commit the files:
 * 9) Push to the dev repository:
 * 10) If updates were rejected because of non-fast-forward push, try:  first, then run:  and continue from point 8.
 * 11) If the rebase fails, but the conflicts are trivial and don't contain useful information (such as keyword stabilization), fix the conflicts and finish the rebase via:  and continue from point 4
 * 12) If the rebase fails and the conflicts are complicated or you think the information is useful, continue with a regular merge:
 * 13) If merge conflicts occur, fix them via:  and continue from point 4
 * 14) If no merge conflicts occur, run:  and continue from point 8 above.

Pull requests from users

 * 1) Identify the remote URL and the branch to be merged.
 * 2) Add a new remote:
 * 3) Fetch the changes:
 * 4) You may review the changes manually first:
 * 5) Checkout the remote branch:  (you are now in detached HEAD mode)
 * 6) Test the ebuilds and run repoman:
 * 7) If everything is fine, switch back to master:
 * 8) Now merge the changes with an enforced merge commit (both for the info and for gpg signature checking to work):
 * 9) Push to the dev repository (same as in the previous section)

Pull requests from developers
Pull requests from developers can be treated in the same way as user pull requests. However, they may also be cherry-picked (might be easier, especially for a single commit), rebased when necessary or applied in other ways, since there will be a valid gentoo developer signature anyway.

If you want to cherry-pick, you do the same as for user pull requests until the merge and then may do the following instead:


 * If it's just one commit, do:
 * If there are multiple commits in one consistent range, do:

repoman: "Malformed CVS Header" warning
The former  header is no more and has been substitued with a   header during the migration. Hence, you'll often run into this warning:

For the time being, you can safely ignore it and go ahead with your changes. A patch has already been sent out and discussed on the portage-dev mailing list to match the new header and silence this warning. See the full discussion.

Working with git
Especially staging files can be tedious on the CLI. So you may want to use the graphical clients (for browsing history) and  (for staging/unstaging changes etc.). You have to enable the  USE flag for.

Using the gentoo git checkout as your local tree
If you want to use your git development checkout as your local tree, you have to do two things:
 * 1) make sure the directory has the correct access rights
 * 2) generate/get metadata-cache, dtd, glsa, herds.xml and news yourself

For the latter, there are example hooks for portage and paludis. You should probably skip the files /etc/portage/repos.conf/gentoo.conf and /etc/paludis/repositories/gentoo.conf respectively, so that portage/paludis doesn't mess up your checkout (as in: disable auto-sync).

Github PRs made easy
Disclaimer: Chewi showed me this trick (me = monsieurp). All credits go to him. If you knew it before, well, too bad, you had to document it first! :p

With this out of the way, here's the magic trick to make merging PRs a cinch. First things first, cd into your gentoo Git repository and add these lines to the repo's .git/config file:

[remote "github"] url = git@github.com:gentoo/gentoo.git fetch = +refs/heads/*:refs/remotes/github/* fetch = +refs/pull/*/head:refs/remotes/github/pr/*

Now type:

git should go and fetch all PRs, and list them as branches. Each PR will now be displayed as a branch name, each with a different name such as "remotes/github/pr/123". For instance if you wish to fetch changes brought by https://github.com/gentoo/gentoo/pull/105, simply type:

to change to the branch containing those changes. Easy, isn't it? :)

Grafting Gentoo History Onto the Active Repo
To graft the historical Gentoo repository onto your current one simply run:

The general syntax of the last command is:

(This may be useful if you're not using the official versions of the repositories.)

Once you've merged the history, git will behave as if it is one big repo, and you should still be able to push from that to Gentoo as the head is untouched. Merging the history will remove the gpg signature on the initial gentoo commit (since it is being modified to do the graft).

External resources

 * Git Tutorial by Lars Vogel
 * Tips and Tricks on gitready
 * kernel patch guideline