User:Leo3418/Hamcrest 2.x migration

This page explains the proposal to migrate packages in the Gentoo repository that use Hamcrest 1.x to Hamcrest 2.x.

Problem statement
Hamcrest released version 2.1 to provide the org.hamcrest JPMS module (referred to as "module" below), which other Java projects can use to integrate with the JPMS.

When another Java project depends on the org.hamcrest module, building the project requires Hamcrest 2.x, and only 2.x, to be in the class path.


 * Using Hamcrest 1.x in place of 2.x would trigger an error like the following:

{{GenericCmd|output= error: the unnamed module reads package org.hamcrest.internal from both hamcrest.core and org.hamcrest assertj-core/src/main/java9/module-info.java:13: error: module org.assertj.core reads package org.hamcrest.internal from both org.hamcrest and hamcrest.core module org.assertj.core { ^ }}
 * Hamcrest 1.x must not be in the class path. Having both 1.x and 2.x in the class path would result in the following errors:

Under the context of Gentoo Java packages, it is easy to ensure that Hamcrest 2.x is in the class path when a package needs it, but it is difficult to avoid having both 1.x and 2.x in the class path. A real example is AssertJ Core 3.24.2, which depends on both JUnit 4.13.2 and Hamcrest 2.2. JUnit 4.13.2 depends on Hamcrest 1.3. Therefore, the class path will contain both version 1.3 and version 2.2, leading to the aforementioned errors.

To resolve this problem, packages that depend on Hamcrest 1.x should be migrated to 2.x to satisfy the requirement that 2.x must be the exclusive Hamcrest version in the class path.

Feasibility
According to the Hamcrest upstream, the API of Hamcrest 2.x "has not changed since Hamcrest 1.3", so Hamcrest 2.x is supposed to be a compatible drop-in replacement for 1.3, or maybe even 1.x.

One exception, as the upstream noted, is the org.hamcrest.Factory annotation's removal. The upstream said that the annotation "should not be used in client code", so ideally, this should not be a problem. In reality, at least one known example of a Java project that uses the annotation exists, which is JUnit 4.13.2. Compiling JUnit 4.13.2 with Hamcrest 2.2 fails due to these errors:

Therefore, the migration would require some Gentoo downstream changes to either Hamcrest 2.x or Java projects that depend on Hamcrest 1.x, so the issues pertaining to org.hamcrest.Factory can be resolved.

Other parties' reactions
This section covers the following things, which potentially affect whether the migration should be performed, and how the migration should be executed:
 * Whether the Hamcrest upstream has any plans to reinstate the org.hamcrest.Factory annotation
 * Whether those Java projects that depend on Hamcrest 1.x plan to upgrade to 2.x
 * Other distributions' resolution to this problem

Hamcrest upstream
The Hamcrest upstream consciously removed the org.hamcrest.Factory annotation before releasing 2.1 despite a related user inquiry, so they are unlikely to add the annotation back. If someone had requested Hamcrest to add the annotation back, it is reasonable to infer that the upstream would disregard the request and respond that the Java packages that use Hamcrest was the party that is responsible to resolve this issue and should eliminate any uses of the annotation.


 * https://github.com/hamcrest/JavaHamcrest/issues/224#issuecomment-446984317: A user commented that org.hamcrest.Factory was no longer available.
 * https://github.com/hamcrest/JavaHamcrest/issues/224#issuecomment-446989079: A Hamcrest project member replied with regards to the user's comment, saying "you shouldn't need it".

With regards to JUnit 4.13.2, the Hamcrest upstream provided an upgrade guide for Gradle and Maven projects that use JUnit 4.13.2. The guide is not applicable to Gentoo as Gentoo Java packages typically do not use Gradle or Maven to build a project, even if the project's build system is Gradle or Maven.

JUnit upstream
The JUnit 4 project issue tracker has a ticket for Hamcrest 2.2 upgrade, but the JUnit upstream has not made any visible progress on resolving the ticket yet.

Debian
Debian builds JUnit 4.13.2 with Hamcrest 2.2. They patch both JUnit and Hamcrest to fix build issues. In particular, the Hamcrest patch adds back the org.hamcrest.Factory annotation to Hamcrest 2.2.

Possible solutions
This section presents and evaluates different solutions to this problem that are applicable to Gentoo.

Add Auto-Module-Name to dev-java/hamcrest-core
Based on my experiment, declaring  in a Hamcrest 1.x JAR file allows the JAR file to co-exist with Hamcrest 2.x in the class path without triggering the "the unnamed module reads package from both hamcrest.core and org.hamcrest" error. To declare the, add this line to the  ebuild:

The benefit of this solution is that it just requires the minimal change and does not apply any patches to any packages, so:
 * It should not require changing other Java packages that depend on Hamcrest 1.x.
 * It does not create extra maintenance burden due to any additional patches.

These are the possible sources of risk in this solution, though the likelihood and potential negative impact of each risk is still to be studied:
 * Setting Automatic-Module-Name for a JAR file, while the upstream does not set it
 * Having two JAR files that both bear the same Automatic-Module-Name and contain a lot of classes with the same name in the class path

Create a standalone package for org.hamcrest.Factory
This solution involves creating an ebuild that installs a JAR file with only the org.hamcrest.Factory annotation. When a Java package that needs the annotation is migrated to Hamcrest 2.x, it will depend on both Hamcrest 2.x and this ebuild.

The benefits of this solution are:
 * It does not require patching Hamcrest 2.x, so it does not have any potential side effect on packages that already depend on Hamcrest 2.x.
 * It might be possible to clean up Hamcrest 1.x from the Gentoo repository if the Hamcrest upstream's statement that Hamcrest 2.x does not have any API changes except org.hamcrest.Factory is accurate: this additional ebuild will enable migration of all existing Gentoo packages that depend on Hamcrest 1.x to 2.x because it adds back the only deleted part of the Hamcrest API, so no packages need to depend specifically on Hamcrest 1.x anymore.
 * It helps avoid the case where two class files for the same class name exist in the class path. The Automatic-Module-Name for dev-java/hamcrest-core approach does nothing with regards to this: it rather enables Hamcrest 1.x and 2.x to be in the class path at the same time.  That said, the actual negative impact of having two class files with the same class name is still to be studied though.

The drawbacks of this solution are:
 * It still does not eliminate the need to patch packages that depend on Hamcrest 1.x. For instance, Debian still had to patch JUnit 4.13.2 itself to migrate it to Hamcrest 2.x.
 * The new ebuild still needs to set Automatic-Module-Name for its JAR file to allow the JAR file to co-exist with Hamcrest 2.x in the class path, so it still shares the same potential risks pertaining to Automatic-Module-Name with the previous solution.

Add org.hamcrest.Factory to Hamcrest 2.x
This is the same as Debian's Hamcrest 2.x migration strategy: patch both Hamcrest 2.x to add back the org.hamcrest.Factory annotation and packages that depend on Hamcrest 1.x as needed to migrate them to Hamcrest 2.x.

The benefits of this solution are:
 * Debian has already tested it since they have been using it for a while.
 * It would not trigger any new issues pertaining to Automatic-Module-Name because it does not involve adding Automatic-Module-Name to any package at all.

The drawbacks of this solution are:
 * It still does not eliminate the need to patch packages that depend on Hamcrest 1.x – same as the previous solution.
 * The Hamcrest 2.x version that Gentoo ships will no longer be the vanilla Hamcrest 2.x built from the verbatim upstream source.