Kotlin/Package Maintainer Guide

This guide contains information for people interested in creating and maintaining ebuilds for packages that use Kotlin. Maintainers of Kotlin library packages such as dev-java/kotlin-stdlib and dev-java/kotlin-reflect should also consult the Kotlin library package maintainer guide.

The Kotlin ecosystem
Many Kotlin projects use Gradle as the build system, which is the first obstacle to build packages for those projects with Portage. As of February 2022, Gradle is not integrated with Portage yet, and there is not an eclass that can be used to build and package artifacts for a Gradle project easily. Invoking Gradle directly in an ebuild is not feasible because Gradle downloads project dependencies from the Internet on its own, requiring  to be disabled in Portage's FEATURES and preventing any dependencies installed by Portage to be reused, so it is against the best practices.

Yet it is still possible to build most Gradle-based Kotlin projects without Gradle thanks to the feasibility of reverse engineering and the standalone Kotlin command-line compiler. Those Gradle-based Kotlin projects use the Kotlin Gradle plugin, which calls the Kotlin compiler to compile Kotlin source files. The plugin prints the Kotlin compiler arguments it would use to Gradle debug logs controlled by, which are standard Kotlin compiler options recognized by the standalone compiler. In general, invoking the standalone compiler using the same set of compiler arguments the Kotlin Gradle plugin would use produces the same results.

Gentoo packages for the standalone compiler and Kotlin eclasses are provided to support this approach of building Kotlin packages within Portage. The eclasses provide an easy-to-use interface to the standalone compiler. By following this guide, the set of Kotlin compiler arguments required to build a Kotlin project can be discovered quickly, and an ebuild for the project can thus be created.

The current framework for Kotlin support on Gentoo focuses on the Kotlin/JVM platform, which is the most widely used and the most mature target platform for computer applications among all supported platforms. When a Kotlin project is built for the Kotlin/JVM platform, it can be well-integrated with other Java packages and JVM to augment the Java ecosystem on Gentoo. A Java package on Gentoo can use a Kotlin package as a dependency like if it were a normal Java library.

Eclasses
The following eclasses are available for packages that use Kotlin:


 * 
 * The eclass that provides the backbone for any eclasses and ebuilds that need to use the Kotlin compiler. It does not export any phases; rather, it provides various helper functions for decomposed steps to build a Kotlin package.
 * This eclass is suitable for packages whose Kotlin part is small and not the major part, like Reactor Core 3.4.14, which is a Java package with 415 Java source files and only 4 Kotlin source files.


 * 
 * The eclass for packages written entirely or mostly in Kotlin. It exports various   phase functions that call the helper functions in, so by inheriting this eclass, ebuilds need not call those helper functions on their own and do not need to define   phase functions at all if the default behaviors suffice.
 * This eclass adds dependency on the Kotlin compiler by default. If the Kotlin compiler is not always needed,  should be used instead.

These eclasses all inherit because they use the same approach as  for building a package targeting JVM: the build system employed by the upstream is circumvented if it is not integrated with Portage, and command-line tools are invoked directly to simulate the build steps performed by the build system.

There are some other files that are generally not needed by normal Kotlin packages; those eclasses are mainly for Kotlin library and compiler packages.

Obtaining Kotlin compiler arguments for a package
The most important step for creating a new Kotlin package is to find out the Kotlin compiler arguments that the Kotlin Gradle plugin would use to build the project. After the arguments have been collected, an ebuild can be written fairly quickly.

To summarize, the method of collecting the Kotlin compiler arguments is to build the project using Gradle with the  option and search for the arguments in the output. As Gradle debug logs are very verbose, finding the arguments in the output is not a trivial task. This section will introduce strategies that make this task easier.

An example of creating a package for a real-world Kotlin project will be presented below to demonstrate the entire procedure. The package being created in the example is kotlinx-cli 0.3.4, which uses Gradle as the build system.

Prepare the project's source tree
To build the project, its entire source tree, including the Gradle build script, needs to be downloaded. The source tree can either be unpacked from an archive or cloned using a version control system. In the latter case, make sure the version being packaged is checked out, unless a live ebuild is to be created.

If the project uses the Gradle Wrapper, using the wrapper is recommended so the exact Gradle version the upstream uses will be run. To find out if the Gradle Wrapper is used, check if the source tree has a script in its root directory. The project being packaged in this example uses the Gradle Wrapper, thus it will be invoked in the following steps.

Find out the Gradle compilation task for Kotlin/JVM
Gradle usually runs dozens of tasks to compile all sources of the project for the  lifecycle task, which causes the Gradle debug logs to contain too much information. Usually, only one of the tasks will invoke the Kotlin compiler to generate classes for Kotlin/JVM. By running only this task with the  option, the amount of debug log messages to search for is greatly reduced.

Projects that target multiple platforms (including the project being packaged in this example) have multiple tasks that will invoke the Kotlin compiler, and each of them generates the classes for only one platform. In this case, only the task for the Kotlin/JVM platform should be run; tasks for Kotlin/JS, Kotlin/Native, Kotlin for Android etc. should be ignored.

To check the compilation task Kotlin/JVM, run to get a list of all tasks that would be executed for the   lifecycle task, and then find out the proper task by using task names as clues.

As its name suggests, the  task shown in the example output is the compilation task for Kotlin/JVM.

Run the task with debug level logging enabled
After knowing the correct compilation task, the debug logs can be collected from the output of.

As of Kotlin 1.6, the Gradle Kotlin plugin prints the compiler arguments in a message that contains the string. There will also be two more messages before the message that are relevant to the compiler invocation:
 * The first message contains the Kotlin compiler class called. For Kotlin/JVM, the class should be org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.


 * The second message contains the classpath with which the Kotlin compiler was called. This message can usually be ignored.

The following is a truncated example of messages printed for a compiler invocation:

2022-02-01T18:50:22.341-0800 [DEBUG] [org.gradle.api.Task] [KOTLIN] Kotlin compiler class: org.jetbrains.kotlin.cli.jvm.K2JVMCompiler 2022-02-01T18:50:22.341-0800 [DEBUG] [org.gradle.api.Task] [KOTLIN] Kotlin compiler classpath: /home/leo/.nobackup/gradle/.gradle/caches/modules-2/files-2.1/... 2022-02-01T18:50:22.341-0800 [DEBUG] [org.gradle.api.Task] [KOTLIN] :kotlinx-cli:compileKotlinJvm Kotlin compiler args: -Xallow-no-source-files -classpath /home/leo/.nobackup/gradle/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.6.0/baf82c475e9372c25407f3d132439e4aa803b8b8/kotlin-stdlib-jdk8-1.6.0.jar:...

The output may still be too verbose to inspect. To make searching easier, any of the following strategies may be used:


 * Use a pager like for convenient scrolling and searching:


 * Write the output to a file for further inspection and print it to the console at the same time:


 * Filter out irrelevant lines and print only log messages for Kotlin compiler invocation:


 * If the output has been saved to a file, it can be filtered in a similar manner:

It is recommended that the message containing the compiler arguments is saved to a file to ease the inspection on it later on.

Make a human-readable list of compiler arguments
The length of Kotlin compiler arguments can make inspections on them difficult. To decompose the arguments into lines of individual options and values, can be used to replace spaces with newline characters. If the arguments have been saved to a file, the following command can be used to complete this:

It is recommended that this output is saved to a file too since it still contains some very long arguments that can be decomposed, such as the value of, and. For example, the following command saves the output to and prints it to the console at the same time:

Kotlin eclasses primer
This section contains fundamental information regarding the Kotlin eclasses.

Eclass variables to be set before inherit
This subsection covers  variables, which are variables that should be defined before a Kotlin eclass is inherited.

Compatible Kotlin feature releases
A Kotlin project might be compatible with multiple Kotlin feature releases, even if it declares only one version in its Gradle build script. To allow an ebuild to use any compatible Kotlin feature release to build the package instead of pin a specific feature release, the Kotlin eclasses require a KOTLIN_COMPAT variable to be set before  for declaring all compatible Kotlin feature release versions.

KOTLIN_COMPAT is similar to PYTHON_COMPAT and LUA_COMPAT : it must be declared as an array, and it should contain a list of identifiers for the Kotlin feature releases supported by the ebuild. The identifier format is  for Kotlin 1. x.

The Kotlin eclasses will generate KOTLIN_SINGLE_TARGET USE_EXPAND flags for the ebuild, which work like PYTHON_SINGLE_TARGET and LUA_SINGLE_TARGET. Those USE_EXPAND flags may be utilized by users to select the version of Kotlin compiler they would like to use to build the package before it is merged.

If a new package is being created, it is recommended that only a single identifier is added to KOTLIN_COMPAT first. More versions can be added later after compatibility tests.

Assuming the feature release of the Kotlin version declared in the Gradle build script is 1. x, the initial identifier added to KOTLIN_COMPAT is suggested to be for the first feature release in the following list (from top to bottom) that has a supported KOTLIN_SINGLE_TARGET :
 * 1. x
 * 1.( x + 1)
 * 1.( x + 2)
 * 1.( x - 1)
 * 1.( x - 2) or lower, which are unlikely to be compatible

For example, kotlinx-cli 0.3.4 declares Kotlin 1.6.0 in under its source tree, which corresponds to feature release 1.6. At the time this example was created,  was not a supported KOTLIN_SINGLE_TARGET, nor were   and. was available, making Kotlin 1.5 the first version in the list that had a supported KOTLIN_SINGLE_TARGET. So, the initial value of KOTLIN_COMPAT should be.

Declaration of special USE flags
Similar to JAVA_PKG_IUSE in, the Kotlin eclasses recognize a KOTLIN_IUSE variable set before  that can be used to enable extra eclass functionality controlled by USE flags. Supported USE flags include:
 * 
 * Install an archive of the package's source files so they can be viewed from some IDEs.
 * Every ebuild should include this USE flag in KOTLIN_IUSE, unless there are no source files to archive (for instance, the ebuild is for a binary package or a metapackage).


 * 
 * Enable dependencies and/or preparations necessary to run the package's tests, which are controlled by.
 * If this USE flag is included, another eclass variable KOTLIN_TESTING_FRAMEWORKS should also be defined to include identifiers for testing frameworks to run in .  This variable is analogous to JAVA_TESTING_FRAMEWORKS in .  Supported values include:
 * : Run tests with JUnit 4
 * : Compare JAVA_JAR_FILENAME with JAVA_BINJAR_FILENAME using and  on available architectures

Dependencies
A Kotlin package's dependencies pertaining to Java usually can be categorized into the following types:
 * Kotlin compiler
 * A provider of virtual/kotlin.


 * Dependencies for KOTLIN_IUSE
 * Tools needed to support USE flags defined in KOTLIN_IUSE.


 * Kotlin libraries
 * Libraries that are a part of the Kotlin programming language itself, such as dev-java/kotlin-stdlib, dev-java/kotlin-stdlib-jdk8, dev-java/kotlin-test-junit, dev-java/kotlin-reflect, etc.


 * JDK
 * A provider of.


 * JRE
 * A provider of.


 * Classpath dependencies
 * Third-party Kotlin libraries, Java libraries, and other JVM libraries that should be present in the classpath for both during build time and JVM during runtime.


 * Build-only JVM libraries
 * JVM libraries that are needed only during build time.


 * Runtime-only JVM libraries
 * JVM libraries that are needed only during runtime.

This subsection will cover how each type of dependencies should be handled in a Kotlin package's ebuild.

Kotlin compiler
automatically inserts dependency specifications for virtual/kotlin into DEPEND, so the ebuild does not need to include them.

does not automatically insert virtual/kotlin into DEPEND. If the package needs to use the Kotlin compiler, the dependency needs to be added by the ebuild. A KOTLIN_UTILS_DEPS output variable is provided by for conveniently inserting dependency specifications for virtual/kotlin that are conditional upon KOTLIN_SINGLE_TARGET  USE_EXPAND flags, so the correct feature release of Kotlin compiler respecting KOTLIN_SINGLE_TARGET can be pulled.

Dependencies for KOTLIN_IUSE
If KOTLIN_IUSE contains any recognized USE flags, then some extra build dependencies are needed to support them.

automatically inserts specifications for those dependencies, so the ebuild does not need to include them.

does not automatically insert the dependency specifications. If the package defines any recognized USE flags in KOTLIN_IUSE, the dependency needs to be added by the ebuild. A  function is provided by  for conveniently inserting those dependency specifications into DEPEND.

Kotlin libraries
None of the Kotlin eclasses automatically insert dependency specifications for any Kotlin libraries into the dependency classes, thus ebuilds must declare Kotlin libraries they need themselves.

Same as the Kotlin compiler, Kotlin library packages are slotted based on the feature releases, so it is important to ensure that the correct slot is defined in the dependency specifications. It is recommended that Kotlin libraries and the Kotlin compiler are all in the equal slot for the same feature release. This is because the versions of Kotlin libraries and compiler pulled and used by the Kotlin Gradle plugin are matched by default. Using Kotlin libraries for a feature release different from the compiler might not be supported by JetBrains or the package upstream.

A  function is provided to allow the correct slot of Kotlin libraries that matches the feature release of Kotlin compiler selected via KOTLIN_SINGLE_TARGET to be used. It takes a set of dependencies as input and echoes a series of dependency specifications for them that are conditional upon KOTLIN_SINGLE_TARGET USE_EXPAND flags. Instances of string  in the input will be replaced by the proper slot.

CP_DEPEND
recognizes a special, undocumented CP_DEPEND variable for dependencies that should be present in the classpath for both during build time and JVM during runtime. By using CP_DEPEND, there is no need to repeat the same dependencies in JAVA_GENTOO_CLASSPATH for adding them to the classpath.

The output of  cannot be used in CP_DEPEND .    echoes a dependency specification containing USE -conditional operators, which will be treated as invalid tokens by. For instance, the following dependency declarations do not work:

This will lead to an error like the following when the package is being compiled:

>>> Compiling source in /var/tmp/portage/dev-java/kotlinx-cli-0.3.4/work/kotlinx-cli-0.3.4 ... * ERROR: dev-java/kotlinx-cli-0.3.4::spark-overlay failed (compile phase): *  Invalid CP_DEPEND atom kotlin_single_target_kotlin1-4?, ensure a SLOT is included *  * Call stack: *    ebuild.sh, line  127:  Called src_compile *  environment, line 2733:  Called kotlin_src_compile *  environment, line 2681:  Called kotlin-utils_src_compile *  environment, line 2565:  Called java-pkg_gen-cp 'JAVA_GENTOO_CLASSPATH' *  environment, line 1595:  Called die * The specific snippet of code: *              die "Invalid CP_DEPEND atom ${atom}, ensure a SLOT is included";

Therefore, to add dependencies included in the argument to  to the classpath, they must be declared in JAVA_GENTOO_CLASSPATH instead.

The format of dependency specifications recognized in JAVA_GENTOO_CLASSPATH is different from that of the ebuild dependency classes:  should be written as   if SLOT equals   or   otherwise. A  function is provided to make the format conversion easier.

Like  (whose name ends in  ),   (whose name ends in  ) replaces instances of   with the slot of Kotlin libraries that the package links to when it is compiled. However,  must be called from a phase function because the slot depends on which KOTLIN_SINGLE_TARGET  USE_EXPAND flag is enabled, so it can only be determined while the package is being built.

Other dependencies
Other non-Kotlin dependencies and tools that are not pulled as dependencies by the Kotlin eclasses can be declared normally as if they were for a normal Java package.


 * JDK
 * should be put into DEPEND regardless of which EAPI is used.


 * JRE
 * should be put into RDEPEND.


 * Classpath dependencies
 * These dependencies can be put into CP_DEPEND .   should be added to both DEPEND and RDEPEND if it is not empty, regardless of which EAPI is used.


 * Build-only JVM libraries
 * These libraries should be put into DEPEND regardless of which EAPI is used.


 * Runtime-only JVM libraries
 * These libraries should be put into RDEPEND.

Converting Kotlin compiler arguments to an ebuild
After collecting the Kotlin compiler arguments needed to build a Kotlin package and knowing the basics about Kotlin eclasses, an installable ebuild for the Kotlin package is only a few variables away. provides some variables for conveniently setting common Kotlin compiler options; it automatically uses the values of these variables to assemble the complete Kotlin compiler command and invokes it to run the Kotlin compiler.

Each of the following subsection contains instructions to convert the compiler option described by the subsection title to an eclass variable's value. When applicable, the example from will continue to be used for demonstration.

The  option for the Kotlin compiler is identical to the option with the same name for : it includes paths to JARs and directories containing classes that may be needed by the sources being compiled.

Kotlin libraries whose JAR appears in the classpath should be handled with instructions in ; other JARs' providers should be put into CP_DEPEND.

If the value for the  option in the Kotlin compiler arguments is too long to read, the following command may be used to make it more readable:

This may be converted to the following dependency declaration:

This option usually contains a set of Kotlin source files and is thus very long. The following command can be used to produce a more readable list of files:

Although the number of files might look overwhelming, it is common that the files all reside in only one or two directories. Please find the smallest set of directories that contain all the files. Subdirectories may be omitted as long as it would not cause extra files to be included in the set. In this example, all the source files are under the directory in the project's source tree, so this directory alone constitutes the smallest set of directories containing all the files.

Then, the smallest set of directories may be defined inside an array and assigned to the KOTLIN_COMMON_SOURCES_DIR variable.

To verify if the smallest set of directories is correct, run the following commands, and compare their output with the list of files in the value of the  option. Replace the value of KOTLIN_COMMON_SOURCES_DIR with what would be defined in the ebuild, and replace with the path to the project's source tree's root (a trailing slash   is needed):

source files
The file paths that are standalone arguments are the Kotlin source files to be compiled. For these files, please also come up with the smallest set of directories containing all of them, and define them in an array assigned to the KOTLIN_SRC_DIR variable. The smallest set of directories can be verified similarly.

This option's value can be set in the ebuild using the KOTLIN_MODULE_NAME variable. This variable's default value is, so if the option's value in the Kotlin compiler arguments is the same as  , then the variable can be omitted.

and
These options do not always appear in the Kotlin compiler arguments for a package. If they do, and they have the same value, the value can be set in the ebuild using the KOTLIN_WANT_TARGET variable. If their values are not the same, or only one of the options is specified, then the arguments need to be added to the KOTLIN_KOTLINC_ARGS array.

If this option appears in the Kotlin compiler arguments, then the project contains Java source files in addition to Kotlin source files, and the Java source files are located in the directories specified in the value of this option. These directories should be added to an array assigned to the KOTLIN_JAVA_SOURCE_ROOTS variable.

Options to ignore
The following options in the Kotlin compiler argument for a package should be ignored and not added to the ebuild.


 * This option enables Kotlin compiler plugins at the specified paths. This is only required when the Kotlin compiler is called by the Kotlin Gradle plugin; the standalone Kotlin command-line compiler comes with all default Kotlin compiler plugins installed and enabled, so in general, there is no need to include any extra plugins using this option.
 * However, if the Kotlin project uses an extension plugin, a custom plugin or a third-party plugin, then this option should not be ignored and should be added to KOTLIN_KOTLINC_ARGS.
 * Here are some ways to check whether a Kotlin project uses a non-default Kotlin compiler plugin:
 * Look at the project's Gradle build script to see if any Kotlin compiler plugin artifact is pulled as a dependency
 * Go through the list of JARs in the value of this option, and see if there is any JAR that has a file name does not contain  and does not exist under the  directory under the Kotlin compiler installation path
 * Go through the list of JARs in the value of this option, and see if there is any JAR that has a file name does not contain  and does not exist under the  directory under the Kotlin compiler installation path


 * This option can be ignored if the  option is set because   implies.
 * This option can be ignored if the  option is set because   implies.


 * This option can be ignored unless a different JDK from the one pointed by JAVA_HOME is intended to be used.
 * This option can be ignored unless a different JDK from the one pointed by JAVA_HOME is intended to be used.


 * This option merely specifies the output path where classes built by the Kotlin compiler should be stored, thus it is expected to be different for different builds. The Kotlin eclasses will set a proper output path using this option, so ebuilds need not include it.
 * This option merely specifies the output path where classes built by the Kotlin compiler should be stored, thus it is expected to be different for different builds. The Kotlin eclasses will set a proper output path using this option, so ebuilds need not include it.


 * This option can be ignored if the extra verbose output of the Kotlin compiler is not desired.
 * This option can be ignored if the extra verbose output of the Kotlin compiler is not desired.


 * This option can be ignored if the Kotlin compiler version information printed to standard output when this option is enabled is not desired.
 * This option can be ignored if the Kotlin compiler version information printed to standard output when this option is enabled is not desired.

Other options
All other options that neither have a dedicated subsection above nor have been mentioned in should be added to the array assigned to the KOTLIN_KOTLINC_ARGS variable.

Options changed between Kotlin feature releases
The command-line interface of the Kotlin compiler usually changes between feature releases. If the feature release of Kotlin compiler used to build the Kotlin project within Portage is different from the feature release of the Kotlin Gradle plugin used by the project, those changes might cause package build failures. Fortunately, some failures can be fixed quickly and easily.

For the example used throughout this guide, the instructions in the previous sections should lead to production of the ebuild below. As per the criteria listed in,  was selected as the initial and only element for KOTLIN_COMPAT despite the fact that the Kotlin Gradle plugin's version is 1.6.0.

When the package is being installed, the following error will be reported because  is a new Kotlin compiler option added in Kotlin 1.6 and is not available on Kotlin compiler 1.5:

* kotlinx-cli-0.3.4.tar.gz BLAKE2B SHA512 size ;-) ...                 [ ok ]  * Using: openjdk-8  * Using Kotlin compiler package: kotlin-bin-1.5 >>> Unpacking source... >>> Unpacking kotlinx-cli-0.3.4.tar.gz to /var/tmp/portage/dev-java/kotlinx-cli-0.3.4/work >>> Source unpacked in /var/tmp/portage/dev-java/kotlinx-cli-0.3.4/work >>> Preparing source in /var/tmp/portage/dev-java/kotlinx-cli-0.3.4/work/kotlinx-cli-0.3.4 ... >>> Source prepared. >>> Configuring source in /var/tmp/portage/dev-java/kotlinx-cli-0.3.4/work/kotlinx-cli-0.3.4 ... >>> Source configured. >>> Compiling source in /var/tmp/portage/dev-java/kotlinx-cli-0.3.4/work/kotlinx-cli-0.3.4 ...  * Compiling ... error: invalid argument: -opt-in=kotlin.Experimental info: use -help for more information  * ERROR: dev-java/kotlinx-cli-0.3.4::spark-overlay failed (compile phase):  *   kotlin-utils_kotlinc failed

Removing the  options from KOTLIN_KOTLINC_ARGS will produce the following warnings instead:

core/commonMain/src/ArgParser.kt:136:6: warning: this class can only be used with the compiler argument '-Xopt-in=kotlin.RequiresOptIn' @OptIn(ExperimentalCli::class) ^ core/commonMain/src/ArgParser.kt:177:33: warning: this API is experimental. It may be changed in the future without notice. private var usedSubcommand: Subcommand? = null ^

The first warning message implies that there is a  option available on Kotlin compiler 1.5, which should be equivalent to Kotlin compiler 1.6's   option judging from the similarity between their names. In fact, all errors and warnings on Kotlin compiler 1.5 can be eliminated by replacing  with   in KOTLIN_KOTLINC_ARGS.

Pruning the list of dependencies
The Kotlin Gradle plugin often includes JARs that are unnecessary in the  option's value for the Kotlin compiler. It is recommended that the dependencies providing those redundant JARs are removed from the ebuild's dependency classes, so users will not be forced to install unnecessary dependency packages.

One method of checking for unnecessary dependencies is to use the tool, which is shipped with JDK, to inspect the Java packages needed by the Kotlin project's JAR artifact.

Building the Kotlin project's JAR using Gradle
The JAR artifact of the Kotlin project built by Gradle is preferred to the JAR generated by the ebuild because Gradle can certainly reproduce an exact upstream build of the project. To build the project's JAR artifact for the Kotlin/JVM platform using Gradle, use the same method in to get a list of all tasks that would be run for the   lifecycle task, then find the task whose name suggests that it builds the JAR for Kotlin/JVM. For the example used throughout this guide, the task is.

Once the correct task is found, run it to build the JAR:

If the task is the correct one, a JAR will be created somewhere under the project's source tree. It can be found using :

If the project uses Gradle wrapper, there will be a in the output, which can be ignored. The (other) JAR in the output,, is then the JAR artifact of the Kotlin project for Kotlin/JVM.