GCC ICE reporting guide

Resources

Bugs (upstream)

A guide to debugging GCC Internal Compiler Errors (ICEs).

This page helps with GCC crashing when building a Gentoo package (or any software) with internal compiler error: Segmentation fault.

To extract useful information from the crash to report a bug requires a few steps. This article lists the steps and provides a few real-world debugging sessions.

Quick guide

Tl;DR

1. Extract self-contained preprocessed file (extracted with -save-temps)
2. Verify gcc still crashes when ran on preprocessed file
3. Expand -march=native to flags not dependent on your machine
4. File a bug on https://bugs.gentoo.org with the following details:
   1. Provide a preprocessed file (or a minimized file)
   2. Provide gcc -v output
   3. Provide emerge --info
   4. Provide -march=native expansion
5. [optional] Minimize amount of flags needed to trigger the error
6. [optional] Minimize self-contained example
7. [optional] File a bug on https://gcc.gnu.org/bugzilla

If you don't know how to all of the above, read on to the detailed guide below!

Detailed guide

Steps to track an ICE (internal compiler errors) down

Check if an ICE is reproducible

Sometimes gcc crashes due to external reasons not directly related to gcc:

• Out-of-memory condition (check dmesg)
• Bad RAM modules (sys-apps/memtest86+)
• Unstable overclock of CPU or RAM

Make sure the above is not your case!

Look at build.log with the ICE and try to resume the build to see if the same command causes the same ICE. Usually the command looks something like make --jobs=8 --load-average=8 CPPFLAGS= CFLAGS= LDFLAGS=.

Go to the Working directory (specified in build.log as well) and rerun the build command.

You should see the exact command and the failure.

Our running example will be an ICE on dev-lang/python-3.6.5:

user $cd '/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5'

user $LANG=C make

running build
running build_ext
INFO: Can't locate Tcl/Tk libs and/or headers
building 'cmath' extension
gcc-7.3.0 -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -march=prescott -mfpmath=sse -O2 -pipe -fomit-frame-pointer -fwrapv -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I./Include -I. -I/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Include -I/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5 -c /var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.c -o build/temp.linux-i686-3.6/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.o
In file included from /var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.c:11:0:
/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/clinic/cmathmodule.c.h: In function 'cmath_acos':
/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/clinic/cmathmodule.c.h:45:1: internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://bugs.gentoo.org/> for instructions.

The build fails every time LANG=C make is rerun. Yay!

Extract exact command

If the build system already prints exact command, just re-execute it (make sure you are in the correct directory). If the build system does not print the command, you will have to use other methods of getting command like running strace or passing verbose flags manually.

In the case of Python, the command is printed as-is:

user $LANG=C gcc-7.3.0 -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -march=prescott -mfpmath=sse -O2 -pipe -fomit-frame-pointer -fwrapv -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I./Include -I. -I/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Include -I/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5 -c /var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.c -o build/temp.linux-i686-3.6/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.o

In file included from /var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.c:11:0:
/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/clinic/cmathmodule.c.h: In function 'cmath_acos':
/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/clinic/cmathmodule.c.h:45:1: internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://bugs.gentoo.org/> for instructions.

Extract self-contained source (use -save-temps)

Add -save-temps to the gcc command on the previous step to extract a preprocessed sample. foo.c will create foo.i (or foo.ii for c++).

Obligatory Python example:

user $LANG=C gcc-7.3.0 -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -march=prescott -mfpmath=sse -O2 -pipe -fomit-frame-pointer -fwrapv -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I./Include -I. -I/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Include -I/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5 -c /var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.c -o build/temp.linux-i686-3.6/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.o -save-temps

... same failure ...

The preprocessed file is in cmathmodule.i now. You can get more tips at https://gcc.gnu.org/bugs/.

Expand -march=native, exact gcc version and other system-specific options

Some compiler options like -march=native are dependent on the environment.

You need to resolve them into exact options. One way to do it is to query gcc for expansion. Pick your arch instead of arch=sandybridge to avoid diff on march= value. That will minimize the diff.

user $arch=sandybridge; for t in param target; do cmd="gcc -Q -O2 -march=$arch --help=$t"; diff -U0 <(LANG=C $cmd) <(LANG=C $cmd -march=native); done

--- /dev/fd/63	2020-06-10 22:26:45.832792880 +0100
+++ /dev/fd/62	2020-06-10 22:26:45.832792880 +0100
@@ -70,3 +70,3 @@
-  --param=l1-cache-line-size= 		32
-  --param=l1-cache-size=      		64
-  --param=l2-cache-size=      		512
+  --param=l1-cache-line-size= 		64
+  --param=l1-cache-size=      		32
+  --param=l2-cache-size=      		8192
--- /dev/fd/61	2020-06-10 22:26:45.856792856 +0100
+++ /dev/fd/60	2020-06-10 22:26:45.856792856 +0100
@@ -16 +16 @@
-  -maes                       		[disabled]
+  -maes                       		[enabled]
@@ -26 +26 @@
-  -mavx                       		[disabled]
+  -mavx                       		[enabled]
...

Here -march=native is substituted for -march=sandybridge --param=l1-cache-line-size=64 --param=l1-cache-size=32 --param=l2-cache-size=8192 -maes -mavx ....

Make sure bug is reproducible after substitution as well!

Report bug on bugs.gentoo.org

File a bug at https://bugs.gentoo.org/ against toolchain@gentoo.org and provide a few details:

• gcc -v output
• Attach .i reproducer

[bonus] minimize needed flags to reproduce failure

You can do similar expansion for optimizer options as well: LANG=C gcc -O2 -Q --help=optimizers

To expand -O3 into -O2 + extra, you can use diff -U0 <(LANG=C gcc -O2 -Q --help=optimizers) <(LANG=C gcc -O3 -Q --help=optimizers)

Try to find minimal set of flags needed to trigger the crash by removing some expanded options.

[bonus] minimize self-contained source using cvise

Now to the fun part! Let's shrink .i down to a manageable size with help of dev-util/cvise.

1. root #emerge --ask dev-util/cvise
2. Write a test.sh tester:
   FILE test.sh

   LANG=C gcc-7.3.0 -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -march=prescott -mfpmath=sse -O2 -pipe -fomit-frame-pointer -fwrapv -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I./Include -I. -I/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Include -I/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5 -c cmathmodule.i -o cmathmodule.o \
   >gcc_out.txt 2>&1
   grep "internal compiler error" gcc_out.txt >/dev/null 2>&1
   

3. Validate tester
   user $./test.sh

   user $echo $?

   0

4. Reduce!

user $cvise test.sh cmathmodule.i

===< 2430 >===
running 4 interestingness tests in parallel
===< pass_includes :: 0 >===
===< pass_unifdef :: 0 >===
===< pass_comments :: 0 >===
===< pass_blank :: 0 >===
(1.2 %, 733906 bytes)
(27.4 %, 539234 bytes)
===< pass_clang_binsrch :: replace-function-def-with-decl >===
(29.5 %, 523834 bytes)
(30.2 %, 518466 bytes)
(32.0 %, 505350 bytes)
...
          ******** cmathmodule.i ********

typedef struct {
  double a
} b;
b c;
d, e, f;
g() {
  _setjmp();
  b h;
  if (e)
    h.a = copysign(f, d);
  c = h;
}

Done!

cmathmodule.i contains the same shrunk example.

You can also pass the tester command directly to cvise without creating a shell script, e.g.:

user $LANG=C cvise --commands "gcc-7.3.0 -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -march=prescott -mfpmath=sse -O2 -pipe -fomit-frame-pointer -fwrapv -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I./Include -I. -I/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Include -I/var/tmp/portage/dev-lang/python-3.6.5/work/Python-3.6.5 -c cmathmodule.i -o cmathmodule.o |& fgrep \"internal compiler error\"" cmathmodule.i

[bonus] Check if bug still exists on vanilla gcc master

Building gcc locally could look like:

user $git clone git://gcc.gnu.org/git/gcc.git

user $mkdir -p gcc-build && cd gcc-build

user $../gcc/configure --enable-languages=c,c++ --disable-bootstrap --prefix="$(pwd)/../gcc-installed" --disable-nls CFLAGS="-O1 -ggdb" CXXFLAGS="-O1 -ggdb"

user $make && make install

Look at configure options of Gentoo's gcc -v to get more options to add to defaults. Be careful: some configure options like --enable-default-pie --enable-default-ssp can change code generation enough to trigger or hide a bug.

[bonus] Report bug on gcc.gnu.org/bugzilla

File the bug at https://gcc.gnu.org/bugzilla/ with attached minimal reproducer and gcc -v detals.

[bonus] Extract gcc backtrace

Example session would be:

user $gdb --quiet --args gcc-7.3.0 -fPIC -march=prescott -mfpmath=sse -O1 -c cmathmodule.c -o cmathmodule.o

(gdb) set follow-fork-mode child 
(gdb) break internal_error
(gdb) run
...
Thread 2.1 "cc1" received signal SIGSEGV, Segmentation fault.
[Switching to process 17286]
0x085375d4 in lra_eliminate_reg_if_possible(rtx_def**) ()
(gdb) bt
#0  0x085375d4 in lra_eliminate_reg_if_possible(rtx_def**) ()
...

[bonus] Fix gcc bug

By now you have a source tree and a command for how to build it!

A few starting points for you:

• https://gcc.gnu.org/contribute.html for general notes on coding tips and guidelines
• https://gcc.gnu.org/onlinedocs/

Good luck!

LTO example

LTO bugs are harder to track down because there's an extra step (a few) between the compiler and the final binary.

GCC's wiki touches on this.

PR105600 is an example of such a GCC bug.

Summary

Warning
TODO

Extract exact command

In this case, CMake was being used with its Makefile generator, so needed to run make VERBOSE=1 ....

This gave:

user $/usr/bin/c++ -O2 -fsanitize=undefined -O3 -DNDEBUG -Wl,-z,relro -Wl,-z,now CMakeFiles/TestSettingsJSONSerialization.dir/TestSettingsJSONSerialization_autogen/mocs_compilation.cpp.o CMakeFiles/TestSettingsJSONSerialization.dir/TestSettingsJSONSerialization.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/mumble_client_object_lib_autogen/mocs_compilation.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/About.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/ACLEditor.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/API_v_1_0_x.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/AudioConfigDialog.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Audio.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/AudioOutputCache.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/AudioInput.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/AudioOutput.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/AudioOutputSample.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/AudioOutputSpeech.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/AudioOutputUser.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/AudioStats.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/AudioWizard.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/BanEditor.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/CELTCodec.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Cert.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/ClientUser.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/ConfigDialog.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/ConfigWidget.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/ConnectDialog.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/CustomElements.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Database.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/DeveloperConsole.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/EchoCancelOption.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/EnumStringConversions.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Global.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/GlobalShortcut.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/GlobalShortcutButtons.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/JSONSerialization.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/LCD.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/LegacyPlugin.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/ListenerLocalVolumeDialog.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Log.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/LookConfig.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/MainWindow.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Markdown.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Messages.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/MumbleApplication.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/NetworkConfig.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/OpusCodec.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/PluginConfig.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Plugin.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/PluginInstaller.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/PluginManager.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/PluginManifest.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/PluginUpdater.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/PositionalAudioViewer.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/PositionalData.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/PTTButtonWidget.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/QtWidgetUtils.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/RichTextEditor.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Screen.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/SearchDialog.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/ServerHandler.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/ServerInformation.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/SettingsKeys.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Settings.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/SharedMemory.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/SocketRPC.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/SvgIcon.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/TalkingUI.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/TalkingUIContainer.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/TalkingUIEntry.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/TalkingUISelection.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/TextMessage.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/ThemeInfo.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Themes.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Tokens.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Translations.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Usage.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/UserEdit.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/UserInformation.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/UserListModel.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/UserLocalNicknameDialog.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/UserLocalVolumeDialog.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/UserModel.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/UserView.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/VersionCheck.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/ViewCert.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/VoiceRecorder.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/VoiceRecorderDialog.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/WebFetch.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/XMLTools.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/widgets/CompletablePage.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/widgets/MUComboBox.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/widgets/MultiStyleWidgetWrapper.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/widgets/RichTextItemDelegate.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/widgets/SearchDialogItemDelegate.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/widgets/SearchDialogTree.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/__/ACL.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/__/Channel.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/__/ChannelListenerManager.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/__/Connection.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/__/Group.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/__/User.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/__/__/3rdparty/smallft/smallft.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/SharedMemory_unix.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/GlobalShortcut_unix.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/Log_unix.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/os_unix.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/TextToSpeech_unix.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/mumble_client_object_lib_autogen/EWIEGA46WW/qrc_mumble.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/mumble_client_object_lib_autogen/2IKLLRAJFH/qrc_mumble_flags_0.cpp.o ../../mumble/CMakeFiles/mumble_client_object_lib.dir/mumble_client_object_lib_autogen/6RJHKKPRHE/qrc_DefaultTheme.cpp.o -o ../../../tests/TestSettingsJSONSerialization /usr/lib64/libQt5Test.so.5.15.3 /usr/lib64/libPocoXML.so.82 /usr/lib64/libexpat.so /usr/lib64/libPocoZip.so.82 /usr/lib64/libPocoFoundation.so.82 /usr/lib64/libpcre.so /usr/lib64/libz.so -lpthread -lrt /usr/lib64/libsndfile.so ../../libshared.a /usr/lib64/libprotobuf.so /usr/lib64/libssl.so /usr/lib64/libcrypto.so /usr/lib64/libQt5Network.so.5.15.3 /usr/lib64/libQt5Xml.so.5.15.3 ../../tracy/libTracyClient.a -ldl /usr/lib64/libQt5Concurrent.so.5.15.3 /usr/lib64/libQt5Sql.so.5.15.3 /usr/lib64/libQt5Svg.so.5.15.3 /usr/lib64/libQt5Widgets.so.5.15.3 /usr/lib64/libQt5Gui.so.5.15.3 /usr/lib64/librt.a /usr/lib64/libXext.so /usr/lib64/libX11.so /usr/lib64/libspeex.so /usr/lib64/libspeexdsp.so ../../mumble/rnnoise/librnnoise.a /usr/lib64/libQt5Core.so.5.15.3

during IPA pass: icf
lto1: internal compiler error: Segmentation fault
0x16bd85b internal_error(char const*, ...)
        ???:0
0x15b8f53 ipa_icf::sem_variable::equals(ipa_icf::sem_item*, hash_map<symtab_node*, ipa_icf::sem_item*, simple_hashmap_traits<default_hash_traits<symtab_node*>, ipa_icf::sem_item*> >&)
        ???:0
0x15b9f67 ipa_icf::sem_item_optimizer::subdivide_classes_by_equality(bool)
        ???:0
0x15c0683 ipa_icf::sem_item_optimizer::execute()
        ???:0
0x15c1d6f ipa_icf::pass_ipa_icf::execute(function*)
        ???:0
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <https://bugs.gentoo.org/> for instructions.
lto-wrapper: fatal error: /usr/bin/c++ returned 1 exit status
compilation terminated.
/usr/lib/gcc/aarch64-unknown-linux-gnu/12.1.0/../../../../aarch64-unknown-linux-gnu/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[2]: *** [src/tests/TestSettingsJSONSerialization/CMakeFiles/TestSettingsJSONSerialization.dir/build.make:356: tests/TestSettingsJSONSerialization] Error 1
make[2]: Leaving directory '/root/mumble/build'
make[1]: *** [CMakeFiles/Makefile2:1503: src/tests/TestSettingsJSONSerialization/CMakeFiles/TestSettingsJSONSerialization.dir/all] Error 2
make[1]: Leaving directory '/root/mumble/build'
make: *** [Makefile:146: all] Error 2

Minimize responsible objects

The failure is in lto1 which by its nature is considering multiple objects. To narrow down the failure, the full command being passed must be considered.

To minimize the first set of objects, convert the command line given previously (copy/paste the full list of objects from the e.g. g++ call):

user $echo "CMakeFiles/TestSettingsJSONSerialization.dir/TestSettingsJSONSerialization_autogen/mocs_compilation.cpp.o CMakeFiles/TestSettingsJSONSerialization.dir/TestSettingsJSONSerialization.cpp.o [...]" >> /tmp/full_object_list

user $cat /tmp/full_object_list | tr ' ' '\n' | sed -e "s:^:$(pwd)../..//:" > /tmp/full_object_list_fixed

Notes:

• At this point, the author fixed up the remaining entries manually, given there were only a handful in this case, but a complete sed transformation would skip any absolute library paths like /usr/lib64/libcrypto.so.
• Remember to drop the single '-o' and next immediate line.

user $cp /tmp/full_object_list_fixed bad

Create a test script for cvise-delta:

FILE check.sh

#!/bin/bash
# Reproduces the ICE by calling g++ on the two bad object files (within @bad)
files=$(<bad)
if [[ -z "${files[@]}" ]] ; then
        exit 1
fi

# The flags here are dependent on what was breaking earlier!
# The key part is: -o /dev/null @bad ...
/usr/bin/c++ -v -Wl,-debug -flto=99 -o /dev/null -O2 -fsanitize=undefined -O3 -DNDEBUG -Wl,-z,relro -Wl,-z,now @bad 2>&1 | grep -q "internal compiler error"

Unleash cvise-delta:

user $chmod +x check.sh

user $cvise-delta ./check.sh bad

Now bad should contain only a small number of bad objects:

user $cat bad

/root/mumble/build/src/tests/TestSettingsJSONSerialization/CMakeFiles/TestSettingsJSONSerialization.dir/TestSettingsJSONSerialization.cpp.o
/root/mumble/build/src/tests/TestSettingsJSONSerialization/../../mumble/CMakeFiles/mumble_client_object_lib.dir/JSONSerialization.cpp.o

Running either a modified (to comment out grep and possibly debug arguments, if desired, only temporarily) check.sh with ./check.sh or running manually, one sees the failure happen with the new reduced set of objects:

user $/usr/bin/c++ -v -flto=99 -o /dev/null -O2 -fsanitize=undefined -O3 -DNDEBUG -Wl,-z,relro -Wl,-z,now @bad 2>&1

during IPA pass: icf
lto1: internal compiler error: Segmentation fault
0x16bd85b internal_error(char const*, ...)
        ???:0
0x15b8f53 ipa_icf::sem_variable::equals(ipa_icf::sem_item*, hash_map<symtab_node*, ipa_icf::sem_item*, simple_hashmap_traits<default_hash_traits<symtab_node*>, ipa_icf::sem_item*> >&)
        ???:0
0x15b9f67 ipa_icf::sem_item_optimizer::subdivide_classes_by_equality(bool)
        ???:0
0x15c0683 ipa_icf::sem_item_optimizer::execute()
        ???:0
0x15c1d6f ipa_icf::pass_ipa_icf::execute(function*)
        ???:0
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <https://bugs.gentoo.org/> for instructions.
lto-wrapper: fatal error: /usr/bin/c++ returned 1 exit status
compilation terminated.
/usr/lib/gcc/aarch64-unknown-linux-gnu/12.1.0/../../../../aarch64-unknown-linux-gnu/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status

The point being: the same failure now occurs with far fewer arguments because cvise-delta reduced the list down to those responsible.

Warning
TODO: Finish the example!

See also

• Bisecting_with_live_ebuilds#GCC_example
• Bugzilla/Bug_report_guide — explains how to report bugs on the Gentoo Bugzilla
• Bugzilla/Guide — covers the recommended method of reporting bugs for Gentoo.