Project:Toolchain/libcrypt implementation

glibc provides many interfaces and there's a steady process to deprecate some of these built-in implementations to allow other providers on the system to better fulfill their purpose. Eventually, glibc will remove its libcrypt implementation, so this migration is unavoidable.

Gentoo has already done this before with:


 * SunRPC to libtirpc
 * NIS to libnsl

This time, we're working on replacing 's libcrypt with. This effort is being tracked in.

TL;DR

 * Most people won't have to do anything special at all, just sync and do a world upgrade!
 * Check out Troubleshooting if you hit a problem.
 * The most common issue is needing to use temporarily, then doing world again without that special   option.
 * The second most common issue is needing to upgrade Python to not incorrectly include crypt.h and link against libcrypt (which may break anything trying to use libpython): should be sufficient on an already up to date system, but use this only if truly necessary. Recommend running  first.

Preliminaries
Don't skip these!


 * 1) Ensure   (default) has been set for now !
 * 2) If a user's password was set before the year 2008, the password may need changed in order to force it to be protected using newer mechanisms than e.g. md5crypt. A new  version has been added to avoid this issue  although issues are being investigated.
 * 3) Sync your copy of the repository, especially to avoid issues like.
 * 4) Fully upgrade the system ( or similar). This is critical to mitigate possible Portage issues.
 * 5) Depclean.

/etc/portage changes
This section is now obsolete for ~arch users since the changes have now been made in the Gentoo ebuild repository. It is still relevant if you're using stable.

Upgrade system
Perform a full world upgrade again. If all is going well, there should be many rebuilds caused by.

Troubleshooting
General tips before getting into specific situations:


 * Use more backtracking if needed (e.g. )
 * Disable binpkgs temporarily

error: crypt.h: No such file or directory
It is possible to hit an error such as: /usr/include/python3.9/Python.h:44:10: fatal error: crypt.h: No such file or directory 44 | #include      |          ^

Such errors are likely to be caused by a Portage scheduling issue -. A workaround for this has been introduced into  which temporarily preserves the header (removed because of the confusion with collisions).

Newer versions of each Python in Gentoo no longer unnecessarily link against libcrypt or try to include crypt.h:
 * 1)  (and python:3.10 if you have that installed already)

Perl fails when trying to emerge libxcrypt
If you have an newer version of glibc installed already with -crypt, you'll need to employ a workaround in order to trigger the preservation mechanism again:

Now delete the aforementioned entries just created. Then:

Please only do this if you are unable to merge libxcrypt itself with a Perl error.

Python can't be re-emerged
Re-emerging Python may rarely give an error like:

In this case, please run: 1. 2. only if your existing Python wasn't too old (make sure you already had e.g. Python 3.9). Otherwise, skip straight to step 3. 3. Run a world upgrade/try again with your world upgrade.

Conflicts: Nudge the virtual
In some cases, confusing conflicts have appeared.

If this happens, it is recommended to nudge the virtual first:

Then complete a full world upgrade.

Conflicts: static-libs conflicts
Even with the nudge, you may still get some conflicts. It's possible they're to do with.

Make sure that static-libs consistently enabled on both and  if a package e.g.  or  requires it:

Then try the nudge again, then a full world upgrade.

Conflicts: General (try with explicit reverse dependencies)
Where the above  and   did not help rebuilding all reverse dependencies of, handing that list to portage explicitly has been proven to work in some cases. All it takes is either (of ) or  (from ) to gather the list and glueing things together in shell:

For the variant using :

For the variant using :

What this does is collect reverse dependencies, take the first word of each line, prepend "=" to each line, and forward the resulting lines to.

This works because we're forcing Portage to consider rebuilding both libxcrypt and any packages which must be rebuilt together with it simultaneously.

Circular dependencies
A Portage (arguably PMS) bug (and its many duplicates) mean that the   eclass pulls in the latest version of Python even if it is not listed in PYTHON_TARGETS.

See Circular_dependencies#Solution_2 for more details / fixes!

This can manifest as:

In this case, please try.

Collisions

 * Collisions on just libcrypt.h where libcrypt.h is not owned by any package was possible for a time although the workaround has since been removed. We recommend using FEATURES="unmerge-orphans" and not FEATURES="collision-protect".


 * Collisions between glibc and libxcrypt are the result of a possible Portage bug. Work around this with.

Blockers
On one occasion, a nonsensical blocker occurred (on a more outdated system):

Do NOT attempt to unmerge.

In this case, was sufficient to install the new   and allow a subsequent world upgrade to be cleanly calculated (even if it looks scary):

Masking previous subslot
In some cases, it's helped to mask the older subslot of when faced with conflicts that cannot be solved by the other methods described here. A high  value is likely to be needed still!

Last resort
If the system is in a state where part of it has been rebuilt against libxcrypt and part has not, emerge output will appear similar to:

Note that in the previous output, some packages expected, while some expected. The initial investigation should be focused on trying to packages stuck on   to investigate why  cannot consider for them for upgrades.

If such investigation fails, it may be necessary to use a sledgehammer approach. Avoid this unless completely necessary:

Then proceed to attempt a world upgrade without the  option. If it fails, please refer to Conflicts: try with explicit reverse dependencies.

Expected problems

 * Largely should be smooth when building applications
 * Fedora switched in 2018 and have hit few issues, upstreaming patches where they did (rarely) hit something
 * Missing dependencies on (or missing subslot operator :=)
 * Note that while it's possible (not verified) we could avoid some rebuilds, this is both not guaranteed and doing the work now allows us to migrate to -compat on libxcrypt in future with a revbumped virtual (not yet possible due to so many pre-built binaries requiring compat).

Developer information
gyakovlev has implemented a QA check to ease finding missing dependencies.


 * If your package links against, you need to explicitly depend on   in DEPEND and RDEPEND.


 * Please do revision bump (straight-to-stable) your package (recommended: ) when adding a either the dependency or a subslot operator for it. Revbump all versions in tree to avoid the need for stabilisation later and to apply the change consistently.


 * Note that false positives can occur with as-needed, as ever. Please see Project:Quality_Assurance/As-needed and.


 * If your package doesn't link against  but uses it otherwise (e.g. runtime dlopen), please ensure you RDEPEND on   (no := required for rebuilds).


 * If your package doesn't use, no action is required.

TODO

 * 1) Get sufficient testing on tinderboxes (ago, toralf, ...)
 * 2) * sam is going to switch to libxcrypt for arch testing too
 * 3) Ensure we're happy with the virtuals
 * 4) Discuss plan for migration (draft news item, ...)
 * 5) Achieve full set of keywords
 * 6) Finalise keywording in (as of writing, only ~m68k remains)
 * 7) Stabilise a version of libxcrypt on all architectures where we have stable glibc, ongoing in

Migration plan

 * 1) Revbump  with IUSE="crypt" (not +crypt)
 * 2) Revbump sys-libs/libxcrypt with IUSE="+system"?
 * 3) Unmask the virtual
 * 4) Migrate masks to musl + uclibc-ng profiles? (... and include a force on the m68k prices until they keyword it?)

Future work
We can look into building with -compat in the distant future (and even then, not guaranteed), but this is not doable for now given the large number of binaries which expect 's libcrypt.

We can consider making another version of the virtual which is indefinitely masked for this. Please note that it's not possible to migrate back to 's libcrypt if you disable the compatibility interface in.

=See also=


 * Gentoo's news item on the migration
 * Fedora's notes
 * A set of notes/instructions by sam on migrating systems manually
 * A forum thread on troubleshooting the migration