Handbook Talk:AMD64/Installation/Stage

From Gentoo Wiki
Jump to:navigation Jump to:search
This is a talk page. Please add newer comments below older ones, and sign your comments using four tildes (~~~~). When adding a new section (at the bottom of the page), please mark it as "open for discussion" by using {{talk|open}} so it will show up in the list of open discussions.

Removing stage3 after invoking tar xpf?

Talk status
This discussion is done as of October 10, 2018.

This is bloatware police, you ware caught leaving thrash behind, please fix your unemergefull behavior. •`_´•

Remove stage-3 after `tar xpf`is invoked and remove it from the finalizing.

--Kreyren (talk) 16:39, 15 September 2018 (UTC)

Read the handbook.... All the way through: Handbook:AMD64/Installation/Finalizing#Removing_tarballs. :P --Maffblaster (talk) 16:01, 10 October 2018 (UTC)


Talk status
This discussion is done.

Maybe a pointer to use ntpdate (a la `ntpdate -s time.nist.gov`) should show up instead of just date? Not everybody has an accurate clock handy :) Hlzr (talk) 05:05, 7 August 2016 (UTC)

Valid point, although this presumes the system as a connection to the internet, which I guess we're presuming in the first place since readers are instructed to download the stage 3 file. I'll consider adding an alternative here. Thanks for the input! --Maffblaster (talk) 19:39, 3 October 2016 (UTC)

Extended attributes while untarring

Talk status
This discussion is done.

The --xattr option when untarring the stage3 tarball is ignored when using the minimal install iso for amd64 built on 20150709.--Bamapookie (talk) 21:57, 26 July 2015 (UTC)

Grab a newer version of the ISO containing a newer version of tar. Should be fine at this point. --Maffblaster (talk) 19:36, 3 October 2016 (UTC)

Choosing a stage tarball

Talk status
This discussion is done.

No-multilib (pure 64-bit)

 Selecting a no-multilib tarball to be base of the new system will provides a complete 64-bit operating system environment.This effectively renders the ability able to switch to multilib profiles improbable (although it is not impossible). 

A better (from grammar and context perspective) sentence would be :

 Selecting a no-multilib tarball as base of the new system will provide only a complete 64-bit operating system environment.  This effecti-vely renders the ability to switch to multilib profiles improbable ...

(Georgios Doumas)

Georgios, I'll take a look and try to make some grammatical improvements. You can sign your messages on here by clicking the "Signature and timestamp" button in the format box above. :) --Maffblaster (talk) 20:34, 10 March 2016 (UTC)
What about having a short mention in this section about the overview on the download page?--Charles17 (talk) 06:58, 11 March 2016 (UTC)
Subpages open a bug on Bugzilla if you'd like. Main www is outside wiki scope. :) Kind regards, --Maffblaster (talk) 19:36, 3 October 2016 (UTC)

Portage and stage3 security recommendations

Talk status
This discussion is still ongoing as of Oct 30 2016.

As outlined at bugs #597804 and #597800 portage does not operate securely by default. Changes that seem to be pending include:

  • stage3 images will include cryptographic keys for the automated establishment of trust
  • stage3 images will include a gentoo key management utility

While these are promising improvements, they have been pending for 4 years already and may not be completed soon, and are not enough to fully secure portage's operation, which currently requires manual processes.

Therefore, currently it would seem useful to add a pointer right here in the installation instructions to the secure portage configuration information already documented at Working with Gentoo / Portage Features / Fetching Files / Validated Portage tree snapshots. It might be better to move it here, rather than keep it there, since it's now critical.

Note that the secure sync only works for emerge-webrsync and no security is possible with traditional rsync. Walter (talk) 23:02, 29 October 2016 (UTC)

Note that securing portage is pointless if the stage3 image downloaded has been compromised. Therefore I would suggest as a related change taking the current text regarding validation of stage3 and promoting it to its own subheading: Validating the stage tarball. In addition, the current text does not explain the problems with man in the middle attacks (in recent years well documented as utilized by state actors) that cannot be resolved with the current recommended process (ie. download stage3 and digests at same time over same network link from an official gentoo mirror - none of which are encrypted under any protocol = both digest and stage3 are compromised at the same time, therefore cannot be trusted). The current text is misleading. Suggested order of content:
  • Big fat warning box saying that while the step is optional, if you skip this step there is absolutely no guarantee that you will ever have a secure system and it is highly recommended to bother.
Rationale: Importance currently understated. New users perhaps unfamiliar with significance.
  • Method of obtaining the Gentoo keys on a non-Gentoo host system being used as an install platform should be described. This uses HTTPS to obtain the key IDs, followed by the HKP GPG keyserver protocol (an unencrypted protocol based upon HTTP) to obtain the keys. Probably the keys themselves should be provided via HTTPS.
Note: The URL for the keys is apparently the Wiki page over here.
Rationale: Required for subsequent steps.
  • Info box on additional high-assurance step of double-validating — re-fetching the same keys from another device/network connection or proxy server, preferably from a different mirror, eg. via smartphone with mobile data, Tor, a secure proxy, or an ssh tunnel (bold underlined super-obvious highlight that critically this must be run from outside the chroot - otherwise you are running potentially compromised code and giving it your remote server credentials!) to a network-geographically disparate server.
Rationale: This protects against failures in SSL (eg. state-level attackers able to forge certificates to enable SSL MITM), locally compromised SSL certificate chains, MITM attacks on the current HKP (= HTTP = unencrypted) based GPG key acquisition process, and compromised mirrors.
  • GPG validation of the stage3.
Note: Text currently present.
Rationale: Requirement for subsequent steps.
  • Digest validation.
Note: Currently before the above, and pointless from a security standpoint without first doing the above.
Rationale: Validates downloaded binary.
  • Immediate transition to a new top-level heading (between Installing a stage tarball and Configuring compile option) called Securing portage, with existing content from over here, and a big fat warning box that it is required to maintain system integrity (as per original point).
-- Walter (talk) 23:18, 29 October 2016 (UTC)
I saw your comment on bug #597800 pointing here. And I think you should also add your points as a new first chapter in the Security_Handbook even before the Pre installation concerns section.

--Charles17 (talk) 09:56, 28 December 2016 (UTC)

Your suggestions and concerns did not go unnoticed. I did see them but have no gotten around to them at the moment. As for the present, it would probably be a good idea to capture the practical steps to integrate these changes in wiki markup in the Security Handbook as Charles17 suggested. I will review them and make determinations on each of them over the course of the next few days. Kind regards, --Maffblaster (talk) 21:53, 29 December 2016 (UTC)

Ping-back to 'Verifying the install ISO' when 'installing stage3'

Talk status
This discussion is still ongoing as of 2017-02-20.

I thought it might be handy to have a link in the Para. that starts "Just like with the ISO file..." back to the Installation/Media stage [1] for people like myself, who don't always download a fresh ISO image every install (we likely have one lying around!) but will always download an up-to-date clean stage3, and may not remember the GPG commands to download the Gentoo RelEng keys, and verify. There is a infobox for the verify command, alternatively we could repeat the 'downloading gentoo key' box into this para. if that was deemed appropriate instead.

-- veremit (talk) 21:43, 20 February 2017 (UTC)


Talk status
This discussion is done as of 2017-04-16.

Please replace http://gcc.gnu.org/onlinedocs/ with https. Fturco (talk) 15:22, 16 April 2017 (UTC)

This was completed some time ago. Perhaps I didn't see or forgot to close this discussion. Thanks, Francesco Turco (Fturco) . --Maffblaster (talk) 18:16, 29 December 2017 (UTC)


Talk status
This discussion is done as of May 15, 2017.

In the NOTE of section CFLAGS and CXXFLAGS, please add a link to Safe CFLAGS.

The GCC optimization guide has more information on how the various compilation options can affect a system.

Thanks. Luttztfz (talk) 12:43, 14 May 2017 (UTC)

I still think it is good to also link to the GCC optimization guide, but I will add the additional reference to the Safe CFLAGs article and perhaps mention that Safe CFLAGs may be a more practical place for beginners to start. Thank you! --Maffblaster (talk) 17:06, 15 May 2017 (UTC)

Tarballs compressed via xz now

Talk status
This discussion is done as of 2018-11-07.

Starting this week, at least amd64 stage3's are now compressed via xz, not bz2. I'd adjust the tarball wildcard to stage3-*.tar.* or similar

Iamben (talk) 15:53, 29 December 2017 (UTC)

Thanks, Ben! I have applyed a change that should work. Let me know if I missed something. --Maffblaster (talk) 20:55, 29 December 2017 (UTC)
The command uses a wrong parameter because of the brace expansion (see "tar not work porperly"). --Feng (talk) 13:36, 31 December 2017 (UTC)
I can second Feng's comment. Brace expansion and pathname expansion are two different things entirely. One should not use the former where the latter is intended. The present usage of brace expansion in the handbook constitutes a dangerous anti-pattern, as well as setting a bad example for users unfamiliar with shell mechanics. Consider the following code:
# mkdir empty && cd empty && touch foo.bz2 foo.bz2.DIGESTS
# printf '<%s>' foo.{bz2,xz}  # this is not a glob and cares not for existing pathnames
# printf '<%s>' foo.*         # this is a glob
# shopt -s extglob
# printf '<%s>' foo.@(bz2|xz) # this is a glob (of the extended variety)
# printf '<%s>' foo.+([^.])   # and so is this
In short, have the user enable the extglob shell option in order to exploit a relatively safe generic approach e.g. tar xpf stage3-*.tar.@(bz2|xz). At present, the handbook contains no valid example of a command where the characteristics of brace expansion are appropriate. Also, the commands are not consistently generic. For example, the tar command exploits * as a globbing character, whereas the user is expected to fill in <release> manually in other commands. EDIT: I just realised that Feng made a similar suggestion in Maffblaster's page. Indeed, his suggestion of using stage3-*.tar.?(bz2|xz) is superior, as it virtually guarantees that one pathname, at most, is matched in the common case. --kerframil (talk) 04:23, 21 July 2018 (UTC)
This has been added. Thanks. --Grknight (talk) 14:29, 7 November 2018 (UTC)

The commands in the "Verifying and validating" chapter could be further improved IMHO. First, if extended globbing is to be used then a command `shopt -s extglob' should be added.

Second, the use of '<' and '>' causes problems if the command is just copypasted to a console window without editing. Why not replace <release> with a single asterisk, as already used for the `tar xvpf' command?

Finally, the `gpg --verify' command did not work for me. I got the complaint "gpg: not a detached signature" and the command exited with status 2 (the sha512 and whirlpool digests were OK though).

--Rafo (talk) 21:50, 1 January 2019 (UTC)

Properly preserve all xattrs like filecaps

Talk status
This discussion is done as of 2017-12-29.

For filecaps to be preserved on unpack (like for USE=filecaps on iputils so ping isn't setuid), --xattrs isnt quite enough, that won't preserve all xattr namespaces. --xattrs-include="*.*" should be enough, this will catch the user.* stuff for filecaps, security.* stuff for pax markings, and more.

Releng is currently disabling filecaps on affected pkgs for stage3 but would like to turn this caps on eventually. We need to get the handbook adjusted before that can (safely) happen.

Iamben (talk) 15:53, 29 December 2017 (UTC)

Nice catch! Thank you for this information. I'll make the change now. --Maffblaster (talk) 20:57, 29 December 2017 (UTC)

Easy verify checksums via rhash

Talk status
This discussion is still ongoing as of June 23, 2018.

I propose add another way to check integrity of downloaded files. There is tool app-crypt/rhash which can automatically detect hash method from comments in DIGESTS files, so verifying downloaded files with SHA512 and WHIRLPOOL is simple as:

rhash -c stage3-*.tar.xz.DIGESTS

I found this as the most simple and universal method.

--Thaaxaco (talk) 20:07, 23 June 2018 (UTC)

Change command for stage3 to actual compression method

Talk status
This discussion is done.

Currently, the command shown to extract the stage3 use the tar.bz2, but the stage3 is actually a tar.xz, so the instructions is outdated. — The preceding unsigned comment was added by Noamcore (talkcontribs) 19:08, 4 October 2018‎

Commands have been updated. --Grknight (talk) 14:27, 7 November 2018 (UTC)

It is 7DEC2019 and the commands in the handbook for AMD64 still show bz2 commands instead of -xvJpf for .tar.xz files.

RFC: Add big red beep about systemd stages

Talk status
This discussion is still ongoing.

During years people keep asking "why do we have blocks installng gentoo with systemd???"

If you dig up you will find they are using "standard" openrc stages installing systemd-based systems. While eudev vs udev and openrc vs systemd blocks can be solved manually it looks missinformative for new users and they get confused. So I would like we add a note about systemd stages existence as well as this allows us to reduce amount of further questions. — The preceding unsigned comment was added by Zlogene (talkcontribs)

Missed format update for stage3 unpacking

Talk status
This discussion is still ongoing.

All commands dealing with the stage 3 now correctly end in ?(bz2|xz) , except the actual extraction. As the up-to-date amd64 images at least are now all .xz this causes issues for people. Ideally add ?(bz2|xz) instead of bz2, or replace bz2 with xz.

Chymæra (talk) 02:10, 18 December 2018 (UTC)

The commands are still for bz2 extraction and not for xz.

Incomprehensible sentence: 'Depending on the installation medium, the only tool ...'

Talk status
This discussion is still ongoing as of 2019-11-02.

The following sentence doesn't make sense: 'Depending on the installation medium, the only tool necessary to download a stage tarball is a web browser.'

What does this sentence mean? Maybe something like: 'In order to download a stage tarball, you will need a web browser.'?

--Mike155 (talk) 20:46, 2 November 2019 (UTC)

Suggestions for Improvement

Talk status
This discussion is still ongoing as of 24 July 2020.

Quoted material is from "Installing the Gentoo installation files".

"Official Gentoo installation media includes the ntpd command ..."

Ungrammatical. A plural subject demands a plural verb.

Official Gentoo installation media include the ntpd command ...

"Official media includes a configuration file ..."

The same error. Media are plural.

Official media include a configuration file ...

"The date command can also be used to perform a manual set on the system clock."

Very clumsy style. I suggest

The date command can also be used to set the system clock.

"Depending on the installation medium, the only tool necessary to download a stage tarball is a web browser."

This is a non sequitur. I suggest either

Irrespective of the installation medium, the only tool necessary to download a stage tarball is a web browser.

or (and this is even better)

The only tool necessary to download a stage tarball is a web browser.

"To optimize Gentoo, it is possible to set a couple of variables which impacts the behavior of Portage ..."

Once again, a subject does not agree with its verb ("couple" is the implicit subject of the subordinate clause beginning "which ...").

... set a couple of variables which impact the behavior of Portage ...

"...too much optimization can make programs behave bad ..."

Adjectives are never verb modifiers, in the English language.

... too much optimization can make programs behave bad[ly] ...

Or, better yet,

... too much optimization can make programs misbehave ...

"The MAKEOPTS variable defines how many parallel compilations should occur when installing a package. A good choice is the number of CPUs (or CPU cores) in the system plus one, but this guideline isn't always perfect."

Linus Torvalds recommends setting this variable equal to twice the number of processors in the system. That way there is always one more compile job waiting to run when the previous one in that thread wraps up. It sort of makes sense, since Linux starts a separate thread running for each CPU core it finds fairly early during bootup.

user $ps ux | grep cpuhp
root        13  0.0  0.0      0     0 ?        S    08:59   0:00 [cpuhp/0]
root        14  0.0  0.0      0     0 ?        S    08:59   0:00 [cpuhp/1]
root        19  0.0  0.0      0     0 ?        S    08:59   0:00 [cpuhp/2]
root        24  0.0  0.0      0     0 ?        S    08:59   0:00 [cpuhp/3]
root        29  0.0  0.0      0     0 ?        S    08:59   0:00 [cpuhp/4]
root        34  0.0  0.0      0     0 ?        S    08:59   0:00 [cpuhp/5]
root        39  0.0  0.0      0     0 ?        S    08:59   0:00 [cpuhp/6]
root        44  0.0  0.0      0     0 ?        S    08:59   0:00 [cpuhp/7]

"Update the /mnt/gentoo/etc/portage/make.conf file to match personal preference and save (nano users would hit Ctrl+x)."

Users -- especially novice users -- probably ought to say Ctrl+o before they exit the program. Yes, I know that nano will prompt me to save the changes. But a guy can make mistakes that way. It's always safer to explicitly save your changes, then say "exit". I've learned that the hard way. --Davidbryant (talk) 16:48, 24 July 2020 (UTC)

Default download directory for stage3 tarball

Talk status
This discussion is still ongoing.

The default location for downloading stage3 is /var/tmp/catalyst/builds/default it is written on page Catalyst I propose to mention this location on page and give link to the catalyst page and tool. Einstok Fair (talk) 03:17, 13 September 2020 (UTC)

Most users are not going to use Catalyst particularly on their first install. Instead they will often download it from the gentoo.org website. --Grknight (talk) 13:13, 14 September 2020 (UTC)

Proposition to split stage3 into rstage3 and bstage3

Talk status
This discussion is done.

For some systems only runtime dependencies are necessary. And for other systems full toolchains are necessary https://bugs.gentoo.org/742251 Einstok Fair (talk) 03:19, 13 September 2020 (UTC)

This has nothing to do with improving the Handbook so not the correct place to discuss such sweeping changes. --Grknight (talk) 13:16, 14 September 2020 (UTC)
Talk status
This discussion is still ongoing.

links browser

With the new admincd, using the browser "links" to view https://gentoo.org/wiki/Handbook:AMD64/Installation/Stage#Introduction

suggests: "Verify the current date and time by running the date command:

root #date"

Rjc (talk) 00:16, 19 October 2020 (UTC)