Project:TeX/TeX Live Bump Story
Summary and intended audience
This document aims at describing the process involved in bumping a new TeX Live release.
It corresponds to the notes I have taken while bumping TeX Live 2014 and 2015 (expanded by zlogene to 2019 and 2020 accordingly). Intended audience is myself so that I do not forget anything next year and fellow developers that would like to help (improving this document is also welcome!). If you are not a Gentoo developer and wish to help, I would strongly suggest you to find something easier: 8 years, thus 8 releases, after the initial commit of TeX Live to the Gentoo portage tree I still happen to make mistakes or oversee some problems. Back in 2007 it took roughly 6 months (not full time though) to get something acceptable for the Gentoo portage tree. Nowadays, it is much simpler since most of the process is done by scripts and eclasses, and this document aims at explaining how everything works.
Upstream source code (texlive-core)
Upstream source code is pointed from https://www.tug.org/texlive/acquire-tar.html. We use a svn checkout for this (see below).
This gives us for TeX Live 2019: ftp://tug.org/historic/systems/texlive/2019/texlive-20190410-source.tar.xz
Generating dev-texlive/* ebuilds
Use branchYYYY if the branch for the current year has been cut, otherwise trunk.
Get a texlive svn checkout from https://www.tug.org/texlive/svn/, it is huge (adapt the year as needed, don't use trunk unless the branch hasn't been cut yet):
user $svn co svn://tug.org/texlive/branches/branch2023And the Gentoo scripts:
user $git clone https://gitweb.gentoo.org/proj/tex.gitThe latter was initially thought as a portage overlay with scripts to auto-generate what could be. Back in 2007, most of the testing was done in this overlay. Nowadays, it merely serves the purpose of a script repository to generate the dev-texlive ebuilds.
Adjust paths in scripts/texlive/common/Makefile.common
Probably only TL_TRUNK and TEXLIVE_OVERLAY_LOCATION needs to be changed.
The "work" directory to be used is within the Gentoo scripts repository cloned earlier, in this case scripts/texlive/tlpsrc. Ebuild generation is done from there.
Check if there are new collections in upstream tlpsrc
user $ls ~/texlive/branch2023/Master/tlpkg/tlpsrc/collection-*Upstream collections are mapped 1-1 to dev-texlive/* ebuilds. We ignore collection-texworks and collection-wintools which are not useful on Linux.
We sometimes remove stuff from them because they have their own ebuilds but that's all.
For example, in TeX Live 2013 -> 2014, there seems to be new collections:
- collection-langchinese
- collection-langkorean
- collection-langjapanese
Add them to Makefile.def
Final adjustments
Adjust TEXLIVE_VERSION to 2021 in scripts/texlive/tlpsrc/Makefile.common.
make commands
Ebuilds are generated by a Makefile and a lot of scripting. make handles dependencies very well, which makes it the perfect candidate since that is all we need.
Here is a quick description of the most important commands:
make dist: Downloads all dev-texlive/* distfiles from CTAN and version them properly (this is where TEXLIVE_VERSION is used). Note that packages on CTAN are not versioned per tarball name, thus what you get is the current Tex Live release updated to the date you run this. This takes a while. We use some upstream perl scripts, so maybe you'll need to install additional perl packages.
make ebuilds: Creates the ebuilds. This also takes a while since it downloads the tex catalogue to fill ebuild with licenses corresponding to shipped files. cataloguelicense2gentoolicense.sh is the license mapping script; it adds wrong licenses if it doesn't know the tex catalogue license, so check it didn't: grep mapped *.ebuild. Nothing, good.
make ebuilds-install: Installs the ebuilds to TEXLIVE_OVERLAY_LOCATION.
After having bumped locally binary packages (texlive-core & friends), update app-text/texlive meta ebuild and try to upgrade. There will likely be collisions: Add blockers to your new dev-texlive ebuilds and let portage solve it. Blockers can be added to the generation scripts: see *.extradeps. Sometimes some files will collide with app-text/texlive-core: This usually means that a script which comes from said package, not texlive-core; it should be removed from texlive-core: see 010_all_installedscripts.patch applied to app-text/texlive-core.
make commands (examples)
After adjusting the needed paths in tex/scripts/texlive/tlpsrc/Makefile.common and tex/scripts/texlive/common/Makefile.common:
- make -C ~/git/tex/scripts/texlive/tlpsrc -j$(nproc)
- make -C ~/git/tex/scripts/texlive/tlpsrc dist -j$(nproc)
You may need to run the latter repeatedly to find issues with missing Perl modules.
app-text/texlive-core and friends
Packages split out of texlive-core
Some packages have been split from texlive-core and are built separately from sources provided by texlive-core while disabled in texlive-core ebuild: This allows for more unitary upgrades or security fixes. To date, those are:
Versioning convention is ${PN}-${VERSION}_p${TLTARBALL_VERSION} where: ${VERSION} is the package version, e.g. for ptexenc the one defined in texk/ptexenc/version.ac and TLTARBALL_VERSION is the version of texlive sources, 20140525 for TeX Live 2014, 20150521 for 2015.
Some of them can be bumped with the old texlive: ptexenc is usually one of these. You need to scp texlive source tarball to a developer's webspace on dev.gentoo.org (links are hardcoded in the texlive-module.eclass, check it out before uploading) and bumping it makes the distfile permanent. Note that most of them require some tarball you obtain by running make dist above.
kpathsea
dev-libs/kpathsea is a bit special since it is core to the whole TeX installation. It contains scripts to configure the TeX installation and, most importantly here, the main texmf.cnf file.
Grab texmf.cnf from TeX Live svn trunk; we split it for easier management, split files are hosted in cvs, e.g. : gentoo/src/patchsets/texlive/2015/texmf.d
texmf.cnf from TeX Live svn trunk is made for a sandboxed install (e.g. TeX Live DVD); since we install on the system, we need to make a few changes. Port needed changes by diffing with previous year releases. Most important non-standard changes are: texmf-site and GLOBALVARTEXFONTS additions. Once done, make a tarball out of it and use it in kpathsea ebuild. You can use ../mktexmfd.sh 7 from gentoo/src/patchsets/texlive/2015 to create kpathsea-texmf.d-7.tar.xz that you can cp to $DISTDIR and scp to pecker.
For TeX Live 2014 it was possible to bump kpathsea alone: Paths remained roughly the same and TeX Live 2013 could work on top of kpathsea from 2014. This is usually not the case. For texlive 2015, kpathsea started to install generated headers under $libdir, requiring packages using it to use the kpathsea.pc pkgconfig file: This meant all its reverse dependencies should be fixed before bumping.
texlive-core
Check if there are new packages in collection-binextra, add them if needed to the ebuild (see TL_CORE_BINEXTRA* variables in texlive-core ebuilds). Port the patches. Make adjustments: blockers, new configure options, etc.
Note on system poppler support (deprecated since tl-2021)
TexLive also bundles (almost) all needed files to be compat with different poppler versions, typically the file names come in the format of filename-poppler${version}.cc (for example pdftoepdf-poppler0.76.0.cc or pdftosrc-poppler0.76.0.cc, see texk/web2c/pdftexdir for more info), you just need to make a patch from these files to make texlive-core work with system poppler lib.
We're ready!
Once everything is done: do a few up/downgrade paths to catch collisions you might have not seen before. Test it. Once happy, it's good and can be committed!