User:Vazhnov/Dirty hacks

Article description::Maybe it will help to anybody who wants to do some digging.

The problem
When I tried to emerge Firefox, I've received system hang because of out of memory: User:Vazhnov/Knowledge Base:emerge out of memory. My swap made a lot of writes and reduced SSD lifetime.

I want to see "Maximum resident set size" for ebuild compile, to get some memory usage info for problem User:Vazhnov/Knowledge Base:emerge out of memory. I use "/usr/bin/time -v" (GNU time) for this.

How compile process looks: $ ps -eF --forest … root    30385 25074 27 37937 144860  0 19:39 pts/4    00:00:12  |           \_ /usr/bin/python3.6 -b /usr/lib/python-exec/python3.6/emerge -n gnome-extra/gnome-calculator root     1776 30385  0  3928  9696   1 19:40 pts/4    00:00:00  |               \_ /usr/bin/python3.6m /usr/lib/portage/python3.6/pid-ns-init 1777 root     1777  1776  0  3928  9784   4 19:40 pts/6    00:00:00  |                   \_ /usr/bin/python3.6m /usr/lib/portage/python3.6/pid-ns-init 250 250 250 18 0,1,2 /usr/bi portage  1780  1777  0   593  1692   5 19:40 pts/6    00:00:00  |                       \_ [x11-libs/gtksourceview-4.2.0] sandbox /usr/lib/portage/python3.6/ebuild.sh compile portage  1781  1780  0  3217  7612   3 19:40 pts/6    00:00:00  |                           \_ /bin/bash /usr/lib/portage/python3.6/ebuild.sh compile portage  1803  1781  0  3219  6728   2 19:40 pts/6    00:00:00  |                               \_ /bin/bash /usr/lib/portage/python3.6/ebuild.sh compile portage  1808  1803  0  2395  4112   0 19:40 pts/6    00:00:00  |                                   \_ /bin/bash /usr/lib/portage/python3.6/ebuild-helpers/emake portage  1810  1808  0  1972  2684   1 19:40 pts/6    00:00:00  |                                       \_ make -j6 portage  1813  1810  0  1970  2724   4 19:40 pts/6    00:00:00  |                                           \_ make all-recursive

/usr/bin/time example output
$ free -m total       used        free      shared  buff/cache   available Mem:          3840        3484          55          20         300         278 Swap:         4999         759        4240 $ ps -eF --forest … lenin    6853     1  0  2864  1516   1 15:56 ? 00:00:04 tmux lenin    6854  6853  0  2703  1708   1 15:56 pts/1    00:00:00  \_ -zsh lenin    6859  6854  0   562    60   0 15:56 pts/1    00:00:00      \_ /usr/bin/time -v sudo emerge -n virtual/rust root     6860  6859  0  2148   536   1 15:56 pts/1    00:00:00          \_ sudo emerge -n virtual/rust root     6861  6860  0 25128 65568   1 15:56 pts/1    00:00:24              \_ /usr/bin/python3.7 -b /usr/lib/python-exec/python3.7/emerge -n virtual/rust root    18393  6861  0  3843  1092   1 15:59 pts/1    00:00:00                  \_ /usr/bin/python3.7m /usr/lib/portage/python3.7/pid-ns-init 18394 root    18394 18393  0  3844  1236   0 15:59 pts/2    00:00:00                      \_ /usr/bin/python3.7m /usr/lib/portage/python3.7/pid-ns-init 250 250 250 18 0,1,2 portage 18397 18394  0   572   456   3 15:59 pts/2    00:00:00                          \_ [dev-lang/rust-1.41.1] sandbox /usr/lib/portage/python3.7/ebuild.sh compile portage 18398 18397  0  3264   492   3 15:59 pts/2    00:00:00                              \_ /bin/bash /usr/lib/portage/python3.7/ebuild.sh compile portage 18414 18398  0  3263   448   0 15:59 pts/2    00:00:00                                  \_ /bin/bash /usr/lib/portage/python3.7/ebuild.sh compile portage 18424 18414  0  5274   848   2 15:59 pts/2    00:00:00                                      \_ python3.6 ./x.py build -vv --config=/var/tmp/notmpfs/portage/dev portage 19486 18424  0  5215  7456   1 16:00 pts/2    00:00:01                                          \_ /var/tmp/notmpfs/portage/dev-lang/rust-1.41.1/work/rustc-1.4 portage  3114 19486  0 176938 28556  1 17:42 pts/2    00:00:04                                              \_ /var/tmp/notmpfs/portage/dev-lang/rust-1.41.1/work/rust- portage  6273  3114 86 691472 1777400 1 17:46 pts/2   00:15:34                                                  \_ /var/tmp/notmpfs/portage/dev-lang/rust-1.41.1/work/r portage  6678  3114 95 427741 1207816 3 17:56 pts/2   00:07:43                                                  \_ /var/tmp/notmpfs/portage/dev-lang/rust-1.41.1/work/r portage  6914  3114 98 166234 349452 3 18:02 pts/2    00:02:17                                                  \_ /var/tmp/notmpfs/portage/dev-lang/rust-1.41.1/work/r portage  7017  3114 12 58113 56464   1 18:04 pts/2    00:00:00                                                  \_ /var/tmp/notmpfs/portage/dev-lang/rust-1.41.1/work/r
 * Command was:
 * in
 * 4GB RAM is not enough for this, so swap was used:

The output is: Command being timed: "sudo emerge -n virtual/rust" User time (seconds): 38526.59 System time (seconds): 824.97 Percent of CPU this job got: 368% Elapsed (wall clock) time (h:mm:ss or m:ss): 2:58:05 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 2246920 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 106085 Minor (reclaiming a frame) page faults: 157744830 Voluntary context switches: 2173051 Involuntary context switches: 1865943 Swaps: 0 File system inputs: 7552472 File system outputs: 22477776 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0

Iteration one: /usr/bin/ebuild
First I tried to replace which is symlink to.

I can't replace original /usr/lib/python-exec/python-exec2 and call it from bash wrapper because of error: /usr/lib/python-exec/python-exec2: python-exec2 is a wrapper, it must not be run directly.

It needs to be pointed by symlink, but not by bash script wrapper.

I did it successfully by:

But then I realized that not uses.

Fail!

uses some python code to run :

Iteration two: /usr/lib/portage/python3.7/ebuild.sh
Save original file:

Create file:

Set permissions:

I tried to save information I need to, but unfortunately it is sandboxed (I had to add rules into ):

* ACCESS DENIED: open_wr:      /var/log/emerge-time.log /usr/lib/portage/python3.6/ebuild.sh: line 8: /var/log/emerge-time.log: Permission denied * ACCESS DENIED: fopen_wr:     /var/log/emerge-time.log /usr/bin/time: /var/log/emerge-time.log: Permission denied * ACCESS DENIED: open_wr:      /var/log/emerge-time.log /usr/lib/portage/python3.6/ebuild.sh: line 10: /var/log/emerge-time.log: Permission denied * --- ACCESS VIOLATION SUMMARY --- * LOG FILE: "/var/tmp/portage/gnome-extra/gnome-calculator-3.32.2/temp/sandbox.log"

So I can write into.

Then I found PORTAGE_LOGDIR allows to save build logs, with all STDOUT and STDERR output of.

Result
Final output example : >>> Compiling source in /var/tmp/portage/gnome-extra/gnome-calculator-3.32.2/work/gnome-calculator-3.32.2 ... ninja -v -j6 -l0 -C /var/tmp/portage/gnome-extra/gnome-calculator-3.32.2/work/gnome-calculator-3.32.2-build … >>> Source compiled. Command being timed: "/usr/lib/portage/python3.6/ebuild.sh.ORIG compile" User time (seconds): 29.38 System time (seconds): 2.63 Percent of CPU this job got: 403% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:07.93 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 145368 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 735962 Voluntary context switches: 2626 Involuntary context switches: 9964 Swaps: 0 File system inputs: 0 File system outputs: 20312 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0