The purpose of this page is to provide users, in particular new ones, with a set of techniques and tools to allow them to troubleshoot and fix problems with their Gentoo setups on their own. In addition, for those problems which are more complicated, this article seeks to provide users with the skills to collect information that will aid them and their supporters in resolving their issues with a greater degree of expediency.
This article assumes that the Gentoo Handbook has been read and that there is a basic understanding of using Gentoo.
Where to get help
Gentoo has quite a few wikis all of which have a wealth of information. Be wary that some articles may contain old or outdated information, but hands down, reading wiki articles is always a critical step when using Gentoo.
Since IRC channels require active presence, some users might prefer leaving a message at a forum or mailing list which continues to exist after closing the browser. When an answer pops up, a notification (usually through an e-mail) is sent back.
When all else fails, and it appears something is broken with Gentoo, Gentoo's bug tracker is a great resource to see if something has already been reported, or to create new bugs if need be. This is a very effective way to reach the Gentoo Developers, which can then use the given feedback to improve the problematic situation and solve the problem.
This is a collection of tools that are highly recommended:
|app-portage/gentoolkit||provides a collection of tools for interacting with Gentoo; it includes the valuable equery, eclean, euse and eshowkw, see their man pages for more information.|
|app-portage/eix||provides eix, a tool for querying portage for packages|
|app-portage/pfl||provides e-file, a tool for querying what package contains a given file; it also works for packages that aren't installed since it uses an online lookup.|
|app-portage/genlop||provides genlop, a tool for parsing emerge logs; handy to figure out when certain packages were installed, updated and to see how long they took to compile.|
|app-portage/elogv||provides elogv, a ncurses based interface for browsing emerge logs.|
|sys-apps/pciutils||provides lspci, a tool for gathering information about PCI devices.|
|sys-apps/usbutils||provides lsusb, a tool for gathering information about USB devices.|
|sys-process/htop||provides htop, a tool for monitoring processes. Like top, but more advanced.|
|sys-process/iotop||provides iotop, a top-like tool for monitoring IO activity by process.|
|net-analyzer/nettop||provides nettop, a top-like tool for examining network traffic by protocol, port, and process.|
|app-text/wgetpaste||provides wgetpaste, a tool for uploading text directly to a pastebin.|
|net-irc/weechat||provides weechat, a simple text based IRC client.|
In order to shape a problem into a solution you first need to understand the problem well enough. If the problem is not clear then it will be very hard, if not impossible to come up with a solution. An accurate, detailed description of the problem is more likely to yield a solution that best fits the problem.
Identifying the problem
This part may seem like it's a no-brainer, but without it, any troubleshooting will be next to impossible.
Take note of a few things:
- Is this problem related to hardware or software?
- What was done or has been changed recently that may have been the cause of this issue?
- Can additional information about the problem be collected?
Storing any form of detail, logs, other people experiencing it and more can be valuable and gives a good overview.
Driver issues for hardware is one of the more common issues that are reported on IRC.
First and foremost, identify what the hardware is that is causing problems. lspci and lsusb are great tools for this purpose. For demonstration purposes, we'll be looking at an Ethernet card as the troublesome device.
Determining the proper driver
First look with lspci and find the device information:
03:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8071 PCI-E Gigabit Ethernet Controller (rev 16)
It is also possible to use the
-n option to give a short-hand notation (which might be easier to search for on the Internet):
03:00.0 0200: 11ab:436b (rev 16)
Verifying the driver is loaded
Getting the driver is less than half of the battle. Most of the trouble with drivers is making sure that they are loaded and operating correctly.
Relying on our friend lspci again, run:
03:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8071 PCI-E Gigabit Ethernet Controller (rev 16) Subsystem: Acer Incorporated [ALI] Device 014e Kernel driver in use: sky2
Note that here, an indicator is given that a driver is in use. If not, then no driver has claimed responsibility over this hardware. Assuming the driver was built as a module rather than built-in, try modprobe <modulename> to attempt loading the right module. If the driver was built-in, consider making it a module, because modules can be loaded with modprobe and unloaded with modprobe -r which saves on number reboots while debugging the issue.
Troubleshooting a driver
Compile the driver as a module, then capture the output it generates when the module is loaded. Make sure to first remove all modules that depend on the driver (see lsmod).
The following command will compare the dmesg output after removing the module (in this example,
r8169) with the output after adding the module again, effectively showing the messages that were added (lines starting with +).
diff -u <(modprobe -r r8169; dmesg) <(sleep 1; modprobe r8169; dmesg) | grep ^+
The most common issue is missing firmware. If a message shows up that informs the administrator about missing firmware, grab either sys-kernel/linux-firmware or a specific firmware package from Portage and install it. After the firmware installation, unload and load the kernel module again to see if this fixes the situation.
Dependency graph slot conflicts
Occasionally, messages like this occur when trying to emerge a package:
!!! Multiple package instances within a single package slot have been pulled !!! into the dependency graph, resulting in a slot conflict:
Generally, this means that the packages listed have multiple requested versions in the dependency graph. And fortunately most, if not all, cases allow to resolve it by manually upgrading to the highest requested version. So assuming that the following message was received:
app-emulation/emul-linux-x86-xlibs:0 (app-emulation/emul-linux-x86-xlibs-20120127::gentoo, installed) pulled in by ~app-emulation/emul-linux-x86-xlibs-20120127 required by (app-emulation/emul-linux-x86-medialibs-20120127::gentoo, installed) (and 2 more with the same problem) (app-emulation/emul-linux-x86-xlibs-20120520::gentoo, ebuild scheduled for merge) pulled in by >=app-emulation/emul-linux-x86-xlibs-20120520 required by (net-im/skype-220.127.116.11-r99::gentoo, ebuild scheduled for merge) ~app-emulation/emul-linux-x86-xlibs-20120520 required by (app-emulation/emul-linux-x86-medialibs-20120520::gentoo, ebuild scheduled for merge)
The most recent version desired is 20120520, so:
emerge --ask --oneshot =app-emulation/emul-linux-x86-xlibs-20120520
Often, it's advantageous to one-shot (
-1) all of the conflicts at once because they sometimes depend on one another (see also World set).
Here is another example of a block:
app-text/poppler:0 (app-text/poppler-0.24.5::gentoo, installed) pulled in by app-text/poppler:0/44=[xpdf-headers(+)] required by (dev-tex/luatex-0.76.0::gentoo, installed) app-text/poppler:0/44=[cxx,jpeg,lcms,tiff,xpdf-headers(+)] required by (net-print/cups-filters-1.0.43::gentoo, installed) (app-text/poppler-0.24.3::gentoo, ebuild scheduled for merge) pulled in by >=app-text/poppler-0.12.3-r3:0/43= required by (app-text/texlive-core-2013-r1::gentoo, installed) poppler:0/43
There should be no problem changing poppler versions. Note that app-text/texlive-core does not require a specific version of poppler, but it needs to be rebuild with whatever version is used:
emerge --ask --oneshot =app-text/poppler-0.24.5 app-text/texlive-core
Here is another example that could have resulted in a blockage:
emerge --ask --verbose --update --newuse --deep @world
Portage warns about an update that can not be performed, however, there is no issue because this update is not required by a package. More precisely, the update must be skipped because mate-base/mate-applets-1.12.1-r4 requires a version of sys-power/cpupower package lower than 4.7.
Collecting additional information
When seeking help, more times than never additional information will be asked, such as the output of a command. Some of these produce dozens or hundreds of lines of text - too much to be suitable for pasting into IRC chat rooms, such as the Gentoo support channel on Freenode. That's where wgetpaste comes in handy - it allows users to post short links that contain long output.
Make sure that app-text/wgetpaste is installed:
emerge --ask wgetpaste
For instance, to show the contents of /etc/conf.d/net, using wgetpaste is as simple as:
Or let's say that someone needs the detailed output of an lspci command is:
wgetpaste -c 'lspci -nnk'
Sometimes it is necessary to redirect stderr to stdout so that error messages can be pasted as well. That can be done like this:
wgetpaste -c 'emerge -pv <package>'
These return a link to a pastebin site page that can be easily shared with someone to get help.
Things to consider pasting:
- In case of hardware problems or kernel issues:
- In case of portage issues:
emerge --verbose --info | wgetpaste
- In case of package installation failures:
- In case of Xorg issues:
- In case all installed packages on the system needs to be provided:
eix-update && eix-installed all
Wrapping things together
Below is a shell function that can be used to have several commands executed and their output processed by a single wgetpaste command so that only one URL needs to be provided to the user or developer that wants to help out:
wgetpaste -i -c "lspci -nnk" -c "lsusb" -c "cat /usr/src/*$(uname -r)/.config"\ -c "cat /var/log/Xorg.0.log" -c "dmesg" -c "eix-update" -c "eix-installed all"
Please do not use pastebin services on the gentoo-user mailing list
Most pastebin services expire old pastes periodically (not to mention that pastebin services are hardly profitable, and close down all the time) whereas mailing list messages are archived in multiple places and are referenced for years. If the relevant output is short, or can be trimmed, then copy and paste it into the body of the email - longer output can be directed into a text file and added as a plain attachment.
Forming a solution
With a good description of the problem, a complete history of actions and all sorts of debugging data (problem details, hardware / software information, logs, backtraces and more) a good set of useful data is available to look for a solution.
In general, repeat the following steps to come to a solution:
- Get an idea where the problem might be, think about possible causes.
- If there are no obvious areas to look into, become more acquainted with the related areas or ask an expert more about it.
- Sometimes it is necessary to make assumptions to proceed. When assumptions are made, put it to the test: false assumptions should not make users and developers blind of other causes.
- Obtain additional debugging information in the area of a problem, or perform tests.
Often this will lead to multiple possible causes. It is important to test them (to verify it is the actual cause or not) and therefore test them individually; this is also known as "divide and conquer".
Let's say a problem like "my browser sometimes displays white pages when I boot" comes up and a lot of details are already known about it, then there may be number of possible causes:
- Is this because the browser was improperly closed when the system rebooted?
- Verify this by killing the process and pulling the plug several times, each time checking if the pages are white after booting again. If this is a cause, is it the actual cause or are there other causes?
- Is this the result of a specific browser version being broken?
- Try older or newer versions for a few days or reboots and see if they are broken as well.
And so on...
With the additional information, dare to think about more specific causes:
- Does a certain pinned tab X that loads plugin Y break the system?
- Unpin the tab so it doesn't open on launch anymore or disable the plugin, see if this keeps the problem away.
- Graphical issues were reported as well, might this maybe mean this is caused by the video drivers?
- Try a different version for those drivers, or alternative drivers if available.
As each possible cause is investigated, the actual cause of the problem might come closer and closer.
And in the end, if the cause is still not found, the Where to get help resources at the top of this article yield a multitude of places with people that are looking forward to help; make them happy with a careful problem description and mentioning what has already been tried so far, it'll help them help out.