Project:Perl/Dot-In-INC-Removal

Perl 5.26 makes some significant changes to the Perl Environment, which is expected to affect many users, and at the time of writing this, still has hundreds of bugs open for on various packages, and is expected to need exceptional care from Gentoo Maintainers and end users alike.

Perl Module Loading
Perl has 4 main mechanisms for loading code from external libraries:



All of the above syntax imply looking in an array called, which contains a list of paths to search for the module/file in question to load, in a similar mechanism as how calling commands from the bash shell traverses the contents of the   environment variable searching for an applicable binary.

Prior to Perl 5.26,  had a default last entry of , which behaved similar to how appending the string   to   might work: it allowed Perl to assume that, if the library in question was not available from any of the official installed paths, that the developer who wrote the code might intend to have their code in  , so it falls back to check there also.

This technique was very useful, and was widespread in places where assumptions could be safely made about where the command was being invoked from.

For instance, it was incredibly common to have a project laid out like such:

/ /Makefile.PL  /inc/Module/Install.pm

Where  contained:

use inc::Module::Install;

And due to an established standard of what  would be when calling   it mostly just worked.

It was also incredibly common to have a layout like

/t/ /t/foo.t /t/lib/Bar.pm

Where  contained:

use t::lib::Bar

And this was also reliable.

Similarly, lots of private projects likely utilize this convenience for similar reasons, and it is especially useful for any tool that wants to load extensions/configuration from Perl code by assuming a working directory.

The problem with
Unfortunately, the documentation for  here wasn't entirely clear, and lead the interpretation that

do "File.pm"

Was logically equivalent to

eval qx[cat File.pm];

Wherein it only gave the impression that was the case due to the presence of  in.

Subsequently, a substantial number of people used  under the assumption it had the same semantics as , where the path can always assumed to be relative to

Motivation for change: CVE-2016-1238
04 April 2016, Debian Report a Security issue with perl, which became eventually known as CVE-2016-1238 Perl RT Ticket

Naturally, assuming  in   has its downsides: Especially when the user executing the code is in a   that is not under their control, ( or was under an attackers control at some previous time ).

is subsequently a great example of a location where  falls under an attackers control, and so any code that  's to   as part of its build process, and then later proceeds to load modules, becomes an arbitrary-code execution attack surface.

And this attack surface is amplified by the presence of "Optional Dependencies", as an optional dependency is not required for the application to function normally ( exhibiting no failures in a non-attack location ), but can still be loaded in an attack location.

Pre-5.26 Mitigations for CVE
As this security issue is multi-layered, and naturally, affected code will be still vulnerable older Perls that lack language-level mitigation, the first line of defence was for all vulnerable uses of this feature to change their code to avoid the vulnerable condition.

End user applications and scripts that anticipate these kinds of problems being possible ( even if unlikely ) were encouraged to be fortified against this, by explicitly dropping  from   early in the applications life cycle.

Perl 5.22.3 and Perl 5.24.1 shipped fixes of this sort of nature in a few of its modules and  scripts to avoid loading any dependencies that could not be conceivably expected to load from somewhere other than a proper Perl5 Install Path, but the language level mechanism was preserved in-tact, and the breakage fallout was effectively nil.

5.26 CVE Mitigation
Perl 5.26 introduced new changes to the long established mechanics:


 * Perl has a new configure tunable, which when enabled (the default), omits the tailing   from the default
 * Perl has a new environment variable  that becomes active under

This means in practice with the stock configuration, that  is no longer in , and any code that previously tried to load from   will start to fail, and become fatal conditions for  ( unless trapped by  )

, due to its default mechanism of not failing upon module load, has special mechanisms added to induce warnings indicating that it would normally find a file, but no longer does.

Not Security, Just Improved Defaults
It should be stated at this point that these CVE Mitigations aren't really security solutions, they make specific classes of security defects harder to fall into by accident under default configuration, but by no means make Perl significantly more secure, and you should pay little attention to the word "safe" used in.


 * 1) It is still entirely possible for any Perl code to add literally any path into
 * 2) Literally every path in   ( including those added by other code ) can be subject to the 3rd party unauthorized privileged arbitrary execution problem.
 * 3)   is by no means universally insecure, and as long as the person who controls   in entirety is the same person who runs the script in question, then there is no security threat.

For instance, if somebody were to do:

use lib "/tmp";

You would have the same security vulnerability, albeit more nefarious and not requiring anyone to. And then a would-be attacker can place custom versions of modules 'd in "/tmp" and execute arbitrary code.

This is because the real vulnerability is the ability to load executable code from arbitrary paths, and if you have any of those paths under the control of an attacker, then they may take precedence over the intended ones.

More over, if the administrator of your computer is careless, and leaves any of the stock Perl  paths such that they are writeable by anyone ( e.g: attacker in the group and the directories/files are , or the directories/files are   ), or, were at any time writeable by the same threat audience, then the same security threats apply.

Consequences
Unfortunately, a vast number of packages relied on the old mechanic, and without failure counter-balances, amount to a failure rate of between 10% and 30% of all Perl Software, both in the wild, and on CPAN.

It is expected this problem is much worse in private codebases, where there is no safety net of the CPAN Testers, and no way for dedicated individuals to sit down and find all the affected private code prior to an affected Perl being released. And the CPAN Testers infrastructure also doesn't test critical opensource tools like Autoconf and Automake, both of which have bugs now related to newer Perl releases.

The Majority of the failures seen on CPAN appear in the  or   paths, and in C  as mentioned above, with the majority of   failures being due to the use of.

Unfortunately, one of the leading users of  is , who has many prolific and heavily used modules, of which, many are shipped in Gentoo. But  hasn't been active in recent years, leaving those modules in a state of desperate need of maintainership.

Counter Balance
As a result of these significant problems, Perl upstream has instituted a workaround to limit the fallout, by pushing that CPAN installers like  and   set   as follows, to restore the legacy   long enough to get the package to install.

$ENV{PERL_USE_UNSAFE_INC} = 1 unless exists $ENV{PERL_USE_UNSAFE_INC}

This approach is also employed in, so that even outside a CPAN installer, performing   will retain legacy environment semantics by default.

Hazards of Counter-Balance
Unfortunately, this counter-balance, while avoiding bugs in  and tests, also means that the runtime code that the tests target, is also regressed into legacy behaviour.

Which can mean in practice, that setting  will make tests pass, and then give a false sense of security, only to reveal broken code when used at runtime, where the workaround ceases to be active.

Maintainers have also frequently overlooked this mechanic, and tested their fixes for  problems, only to find they've published code which is in fact, still broken.

Fixing the problem at its source
The ideal solution is to fix the problem properly, and this typically means changing your  invocations.

Ideally, these changes should be made upstream, and Gentoo will replicate them once fixed. But depending on time, Gentoo maintainers may start applying some of these fixes themselves.

Convert implied relative paths to explicit paths
Where the required source is specified as a path already, the best approach is to use an explicit relative or absolute path, ie: one that starts with either a  or , as follows:

do "./Foo.pm" do "/absolute/path/to/Foo.pm" require "./Foo.pm" require "/absolute/path/to/Foo.pm"

Such paths are special cased in both  and   at all. By taking advantage of that, you can avoid tweaking, which minimises the risk of hiding other real errors.

Use a custom directory for your resources
In tests, and in some  cases, it can make sense to relocate assets you know you need into a custom location. For instance, if you previously used:

use t::Foo;

It might be more sensible to create, relocate   to  , and rename the packages inside from   to  , and then change the load code into:

use lib "t/testlib"; # add t/testlib to @INC use Foo;            # load Foo from t/testlib via @INC

Restore from Perl
This option should be considered as a last choice, because reinserting  in either configuration scripts, or tests, is likely to mask real issues in the runtime code, and adding this code in any modules runtime source is likely to restore the vulnerability problem, so it should only really be used when desperate.

However, for some things, ( like  ) there's no real alternative, due to the code making strong assumptions and dependence on the code layout and load method.

use lib '.';              # restore '.' so inc::Module::Install loads from ./inc/Module/Install.pm   use inc::Module::Install;

If you find its possible for you to temporarily restore  to load a single dependency and its children, you *might* get away with doing:

BEGIN { local @INC = @INC; unshift @INC, '.'; require Foo; }

Which will reset  after the scope ends. But this comes with a caveat, in that you must know that none of the things you require are expected to augment  themselves, or those augments will also be lost at the end of the scope.

Applying Patches to Gentoo Packages
All Gentoo Perl Packages in  and similar support gentoo user-patches.

So if you can formulate one of the above fixes yourself, you can have it applied at install time by saving the relative fix in:

/etc/portage/patches/${CATEGORY}/${PN}-${PV}/0001-your-name-here.patch

And all entries in  will be applied in lexicographic order during

However, there may be plenty of packages affected by this problem outside the Gentoo Perl direct ecosystem, which may not yet have these provisions, and you may need to temporarily maintain your own variation of the, including: putting the relevant file in   and somehow getting it applied by manually tweaking your copies   function.

But this technique is probably too technical for most people, even though it gives the most resilient result.

Restoring temporarily from ENV on a per-package basis
This may be slightly more convenient than fixing the package itself for many users, at the price of potentially installing broken packages.


 * 1) Create a file in   called
 * 2) Put the following in it: PERL_USE_UNSAFE_INC=1
 * 3) Create a file like
 * 4) For each affected package, add a line like: =dev-perl/foo-1.0-r1  perl-dot-in-inc

This will ensure the work-around is only applied for packages that expressly need it, so you can keep tabs on the failures and report them.

At build time, portage will read the environment file, and tell Perl to restore, and that part of the build will work the same as it did on 5.24

Restoring temporarily from ENV for all packages
This is the least desirable option from the standpoint of security/reliability, but the easiest.

Simply add the line:

PERL_USE_UNSAFE_INC=1

to

And then everything that does anything with Perl will see  in   like it did on 5.24

Restoring in Perl itself
This is suggested as an absolute last option for end users who are stuck in a situation where the number of failures pertaining to  in Gentoo Packages is too high and they have custom code and infrastructure outside Portage that relies on the old mechanic, and they have no time to fix the real problems, ... or they don't have the time to properly assess their production software for this issue, and want to avoid the risk of accidental failures.

This solution should be considered temporary at best as this option is likely to evaporate upstream in the near future, and this option globally reintroduces the security risk stated above.


 * 1) Create a file in   called
 * 2) Put the following in it: EXTRA_ECONF="-Udefault_inc_excludes_dot
 * 3) Create a file like
 * 4) Add a line like: dev-lang/perl  perl-5.26-always-dot

And this will ensure that EXTRA_ECONF is set in the environment for every version of perl that gets built, and this tells perl configure to "unset" the  option, which means   will always be in , just like it was in Perl 5.24.0

Links

 * 1) https://bugs.gentoo.org/showdependencytree.cgi?id=613764&hide_resolved=0 - Tree of bugs for Perl 5.26
 * 2) https://metacpan.org/pod/release/XSAWYERX/perl-5.26.0-RC2/pod/perldelta.pod#Removal-of-the-current-directory-%28%22.%22%29-from-@INC - Official upstream documentation on   in   with a lot more writing targeted at people using vanilla installations and upstream authors.
 * 3) https://rt.perl.org/Ticket/Display.html?id=127834#txn-1394453 - CVE 2016-1238 Perl Bug