Portage performance notes
I've noticed some odd things with regard to Portage performance. I'm documenting them here so I have a public way to reference them.
cpython vs pypy
mgorny has noticed that pypy3 is much faster for dependency calculations (for him). In his blog post, he notes that many people were not able to reproduce his results. I surmise that much of this is due to few people realizing that
USE=native-extensions drastically affects pypy performance. With that flag on, Portage compiles C extensions, while pypy.org's performance page notes, "Any C extension module recompiled with PyPy takes a very large hit in performance. PyPy supports C extension modules solely to provide basic functionality."
Once I turn that flag off, I can indeed produce a similar speedup:
$ hyperfine "EPYTHON=pypy3 emerge -pvuDN @world" "EPYTHON=python3.10 emerge -pvuDN @world" Benchmark 1: EPYTHON=pypy3 emerge -pvuDN @world Time (mean ± σ): 33.539 s ± 1.081 s [User: 32.602 s, System: 0.642 s] Range (min … max): 32.102 s … 35.993 s 10 runs Benchmark 2: EPYTHON=python3.10 emerge -pvuDN @world Time (mean ± σ): 104.236 s ± 1.403 s [User: 102.861 s, System: 0.694 s] Range (min … max): 102.330 s … 106.934 s 10 runs Summary 'EPYTHON=pypy3 emerge -pvuDN @world' ran 3.11 ± 0.11 times faster than 'EPYTHON=python3.10 emerge -pvuDN @world'
However, even with this change, some people still report no significant change between cpython and pypy.
Anecdotally, I've noticed that package merges are much slower with pypy3 (as opposed to just dependency calculations as above). I haven't found a package with which I can reproducibly demonstrate the performance difference.