Writing Rust ebuilds

This is a short reference, intended to be read alongside Basic guide to write Gentoo Ebuilds and the cargo.eclass documentation.

cargo.eclass

Rust has its own package manager Cargo, which Gentoo's cargo.eclass utilizes. This is very convenient for software developers. However it comes with some caveats for maintainers.

Rust programs are usually statically bound. This makes it so that their runtime dependencies are easy to figure out compared to C programs, but their build dependencies are harder to get right. The good thing is that upstream needs to figure that out for packagers. Packagers only need to translate what upstream already figured out into the cargo.eclass' way of handling dependency programs.

Rusts dependency mechanism

Rust lingo for program is 'crate'. Crates can be libraries or runnable programs. As package the term is not defined strictly, so basically every Rust program that is provided in binary or source form can be considered a crate. In In the context of packaging, crates are often dependency libraries of a package. Rust developers upload their crates along with documentation to crates.io. Other Rust developers can use the uploaded crates in their program by stating its name and version in a configuration file for Rusts build system Cargo. If a dependency is only declared using the crate name, Cargo assumes it should download that dependency from crates.io. But developers can also declare dependencies with additional information like a URL to a Git Repository and a git tag to get dependency crates from elsewhere.

Making a versioned ebuild

Gentoo has its own programs to make ebuilds out of Rust projects automatically: dev-util/cargo-ebuild and app-portage/pycargoebuild.

using cargo ebuild

To use cargo ebuild, the repository of the Rust program that is about to be packaged is needed. This is usually done by cloning the git repository of the program. After obtaining the sources

user $cargo ebuild

in the sources repository can be run to generate an ebuild. Cargo ebuild automatically checks dependencies for vulnerabilities, which takes a while. One can run

user $cargo ebuild --noaudit

instead to speed up this process.

using pycargoebuild

To use pycargoebuild to generate an ebuild, run pycargoebuild in the repository of the program:

user $pycargoebuild ./

If everything goes well, cargo ebuild (or pycargoebuild) saves an ebuild into the Rust programs repository folder, which can then be moved into an ebuild repository like ::gentoo or the ::guru.

However, this will only give a blank frame to work with. The homepage and description needs to be added and if the Rust program is not on crates.io the SRC_URI as well.

Licenses

Cargo ebuild and pycargoebuild also generates a LICENSE variable in the ebuild, but it can not be guaranteed that these are correct. The command cargo license in the Rust repository generates a list of licenses. Packagers must be aware that cargo license will give the licenses in SPDX format, which Gentoo does not always use. So some effort needs to be put into 'translating' them. Because the code of the dependency libraries is compiled into the finished binary (statically linked), all licenses of every dependency crate used in a package must be stated in the LICENSE variable.

Common problems with dependencies in versioned ebuilds

crates.io dependencies

If all dependencies are fetched from crates.io, the automatically generated ebuilds from cargo ebuild or pycargoebuild often work from scratch. Dependencies that are fetched from crates.io are declared in the CRATES variable in the ebuild the same way they are in cargo.toml of the Rust project:

FILE cargo.toml

[dependencies]
itertools = "0.10.3"

'Cargo ebuild' and 'pycargoebuild' will put them into the CRATES variable and they are put into the SRC_URI variables via $(cargo_crate_uris) which is automatically added to SRC_URI by 'cargo ebuild' and 'pycargoebuild'. They are then automatically fetched and unpacked into the right place via cargo.eclass' cargo_src_unpack.

github/ gitlab etc. dependencies

Sometimes upstream developers are not happy with the version of a crate on crates.io or a crate is not available there and they want to get it from some git repository instead. Then they declare a dependency in cargo.toml like this:

FILE cargo.toml

tree-sitter-haxe = { git = "https://github.com/vantreeseba/tree-sitter-haxe", version = "0.2.2", optional = true }

Which means that the release 0.2.2 from the tree-sitter-haxe github repo should be checked out. We can not clone a git repository without git-r3.eclass but we can simulate this in the ebuild. (If the cargo.toml of a Rust program has a line like tree-sitter-c-sharp = { git = "https://github.com/tree-sitter/tree-sitter-c-sharp", branch = "master" } in its dependencies, it wants to fetch the latest commit on the master branch of that Github repository. This essentially forces packagers to make a live ebuild instead of a versioned ebuild.)

For this example the lines

declare -A GIT_CRATES=(
       [tree-sitter-haxe]="https://github.com/vantreeseba/tree-sitter-haxe;32f6bda9b568ae47c89678096de9b4d0cbd450b8"
)

should be put into the ebuild. The commit hash is obtained by browsing the files of that release (v0.2.2) on Github and copying it from the URL. Also it can often be found in the Cargo.toml and Cargo.lock files of the upstream repository. Grep the repository for the crates name to find that.

If cargo.toml is not found in $crate_name-$commithash

Usually the cargo.eclass looks for the Cargo.toml file in the folder $WORKDIR/$crate_name-$commit-hash. (So in the example above $WORKDIR/tree-sitter-haxe-32f6bda9b568ae47c89678096de9b4d0cbd450b8. However some crates have that file in a different place or have a different name in Cargo.toml as their Github repository name. In this case the path to Cargo.toml can be defined using

declare -A GIT_CRATES=(
       [tree-sitter-haxe]="https://github.com/vantreeseba/tree-sitter-haxe;32f6bda9b568ae47c89678096de9b4d0cbd450b8;tree-sitter-haxe-%commit%/mypath/to/cargotoml"
)

Where %commit% is automatically replaced with the commit hash specified before. If this last part of that line is not provided, the cargo.eclass composes that path out of the crate name (the part in []) and the commit hash. The path is a relative path that starts from $WORKDIR. The part in [] usually needs to be the name of the package in the Cargo.toml of that package.

Writing live ebuilds

Writing live ebuilds for Rust packages is way easier than writing non-live ones. The first frame of the ebuild can be created via cargo ebuild or pycargoebuild as well. Then, set the ebuilds version in the file name to 9999.

Remove the unnecessary CRATES variable from Live ebuilds. Also, SRC_URI is not used. Sources are often fetched via git-r3.eclass or similar.

In src_unpack() put cargo_live_src_unpack. This will fetch the needed dependency crates using cargo.

Problems with live ebuilds

can't update a git repository in the offline mode

When a Rust project uses unpinned dependencies in their Cargo.toml, e.g.

CODE

tree-sitter-c-sharp = { git = "https://github.com/tree-sitter/tree-sitter-c-sharp", branch = "master" }

cargo will always want to check if that dependency is up do date in later stages of the ebuild. This will cause the error

can't update a git repository in the offline mode

In this case, add cargo_src_configure --frozen into src_configure(). This will stop cargo from checking what it just fetched is up to date.

USE flags

USE flags for rust are usually added via the cargo.eclass:

CODE

src_configure() {
 local myfeatures=(
        barfeature
        $(usev foo)
 )
        cargo_src_configure
 }

Switch off default features

Often Rust programs do not allow to switch off single features. So in order to make it possible to choose certain features, disable all default with cargo_src_configure --no-default-features and then enable only the features necessary with myfeatures like mentioned above.

Using a vendor tarball like in Go ebuilds

The go-module.eclass and the cargo.eclass are very similar regarding the functionality of loading statically linked dependencies. In a Go ebuild a packager defines the variable EGO_SUM with a list of dependencies from upstream, in a Rust ebuild the CRATES variable has the same functionality. The eclasses then proceed to download those dependencies and put them in the correct places in the working directory.

While it is an ongoing and controversial discussion whether the EGO_SUM-functionality should be deprecated or not in the go-module.eclass, the cargo.eclass is built with the CRATES variable in mind. The go.eclass offers the possibility to use a vendor-tarball instead of the EGO_SUM-functionality. That is a tar archive that contains all the statically linked dependencies. While Cargo also offers the possibility to create such a vendor-tarball as easily as Go, the cargo.eclass does not offer to use such a vendor-tarball without doing some extra steps.

Using a vendor-tarball instead of using the CRATES variable has some benefits. For example you don't need to recreate the list of dependencies (e.g. by using pycargoebuild in the upstream repo) and copy it into the new ebuild when bumping the version of that ebuild. If the vendor tarball is hosted right, bumping a Rust-ebuild is as easy as renaming the ebuilds filename with the new version.

Creating a vendor tarball

Creating a vendor tarball can be done by running cargo vendor in the upstream repository. It will create a vendor folder, which can be put into a tar archive with tar -czvf vendor.tar.gz vendor.

Using a vendor tarball in an ebuild

Because the cargo.eclass is somewhat built with the CRATES variable in mind, it will complain when this variable is not set when the eclass in inherited. To work around this, we can define the variable as CRATES=" ". The cargo.eclass automatically sets up the build to look in $CARGO_HOME/gentoo to find dependencies. This folder is the folder cargo_home in $WORKDIR. A packager can either set up the ebuild to unpack the vendor-tarball into that directory, (by not calling default or cargo_src_unpack in src_unpack() and writing src_unpack() with own instructions.) or let portage do its default thing and let it unpack the vendor tarball into $WORKDIR, then do ln -s "${WORKDIR}/vendor/"* "${CARGO_HOME}/gentoo/". (Depending on how exactly that tarball is created it could be that the path here needs to be altered slightly.)

See also

• Basic guide to write Gentoo Ebuilds — getting started writing ebuilds, to harness the power of Portage, to install and manage even more software.
• User:Schievel/autocreate_rust_sources

External resources

• Documentation of cargo.eclass