Handbook Talk:AMD64/Installation/Stage

Removing stage3 after invoking tar xpf?
This is bloatware police, you ware caught leaving thrash behind, please fix your unemergefull behavior. •`_´•

Remove stage-3 after `tar xpf`is invoked and remove it from the finalizing.

--Kreyren (talk) 16:39, 15 September 2018 (UTC)

Read the handbook.... All the way through : Handbook:AMD64/Installation/Finalizing. :P --Maffblaster (talk) 16:01, 10 October 2018 (UTC)

NTP
Maybe a pointer to use ntpdate (a la `ntpdate -s time.nist.gov`) should show up instead of just date? Not everybody has an accurate clock handy :) Hlzr (talk) 05:05, 7 August 2016 (UTC)


 * Valid point, although this presumes the system as a connection to the internet, which I guess we're presuming in the first place since readers are instructed to download the stage 3 file. I'll consider adding an alternative here. Thanks for the input! --Maffblaster (talk) 19:39, 3 October 2016 (UTC)

Extended attributes while untarring
The --xattr option when untarring the stage3 tarball is ignored when using the minimal install iso for amd64 built on 20150709.--Bamapookie (talk) 21:57, 26 July 2015 (UTC)


 * Grab a newer version of the ISO containing a newer version of . Should be fine at this point. --Maffblaster (talk) 19:36, 3 October 2016 (UTC)

Choosing a stage tarball
No-multilib (pure 64-bit) Selecting a no-multilib tarball to be base of the new system will provides a complete 64-bit operating system environment.This effectively renders the ability able to switch to multilib profiles improbable (although it is not impossible). A better (from grammar and context perspective) sentence would be : Selecting a no-multilib tarball as base of the new system will provide only a complete 64-bit operating system environment. This effecti-vely renders the ability to switch to multilib profiles improbable ... (Georgios Doumas)


 * Georgios, I'll take a look and try to make some grammatical improvements. You can sign your messages on here by clicking the "Signature and timestamp" button in the format box above. :) --Maffblaster (talk) 20:34, 10 March 2016 (UTC)


 * What about having a short mention in this section about the overview on the download page?--Charles17 (talk) 06:58, 11 March 2016 (UTC)


 * open a bug on Bugzilla if you'd like. Main www is outside wiki scope. :) Kind regards, --Maffblaster (talk) 19:36, 3 October 2016 (UTC)

Portage and stage3 security recommendations
As outlined at bugs #597804 and #597800 portage does not operate securely by default. Changes that seem to be pending include:
 * stage3 images will include cryptographic keys for the automated establishment of trust
 * stage3 images will include a gentoo key management utility

While these are promising improvements, they have been pending for 4 years already and may not be completed soon, and are not enough to fully secure portage's operation, which currently requires manual processes.

Therefore, currently it would seem useful to add a pointer right here in the installation instructions to the secure portage configuration information already documented at Working with Gentoo / Portage Features / Fetching Files / Validated Portage tree snapshots. It might be better to move it here, rather than keep it there, since it's now critical.

Note that the secure sync only works for emerge-webrsync and no security is possible with traditional rsync. Walter (talk) 23:02, 29 October 2016 (UTC)


 * Note that securing portage is pointless if the stage3 image downloaded has been compromised. Therefore I would suggest as a related change taking the current text regarding validation of stage3 and promoting it to its own subheading: Validating the stage tarball. In addition, the current text does not explain the problems with man in the middle attacks (in recent years well documented as utilized by state actors) that cannot be resolved with the current recommended process (ie. download stage3 and digests at same time over same network link from an official gentoo mirror - none of which are encrypted under any protocol = both digest and stage3 are compromised at the same time, therefore cannot be trusted). The current text is misleading. Suggested order of content:
 * Big fat warning box saying that while the step is optional, if you skip this step there is absolutely no guarantee that you will ever have a secure system and it is highly recommended to bother.
 *  Rationale : Importance currently understated. New users perhaps unfamiliar with significance.
 * Method of obtaining the Gentoo keys on a non-Gentoo host system being used as an install platform should be described. This uses HTTPS to obtain the key IDs, followed by the HKP GPG keyserver protocol (an unencrypted protocol based upon HTTP) to obtain the keys.  Probably the keys themselves should be provided via HTTPS. 
 *  Note : The URL for the keys is apparently the Wiki page over here.
 *  Rationale : Required for subsequent steps.
 * Info box on additional high-assurance step of double-validating &mdash; re-fetching the same keys from another device/network connection or proxy server, preferably from a different mirror, eg. via smartphone with mobile data, Tor, a secure proxy, or an ssh tunnel ( bold underlined super-obvious highlight  that critically this must be run from outside the chroot - otherwise you are running potentially compromised code and giving it your remote server credentials!) to a network-geographically disparate server.
 *  Rationale : This protects against failures in SSL (eg. state-level attackers able to forge certificates to enable SSL MITM), locally compromised SSL certificate chains, MITM attacks on the current HKP (= HTTP = unencrypted) based GPG key acquisition process, and compromised mirrors.
 * GPG validation of the stage3.
 *  Note : Text currently present.
 *  Rationale : Requirement for subsequent steps.
 * Digest validation.
 *  Note : Currently before the above, and pointless from a security standpoint without first doing the above.
 *  Rationale : Validates downloaded binary.
 * Immediate transition to a new top-level heading (between Installing a stage tarball and Configuring compile option) called Securing portage, with existing content from over here, and a big fat warning box that it is required to maintain system integrity (as per original point).
 * -- Walter (talk) 23:18, 29 October 2016 (UTC)


 * I saw your comment on pointing here.  And I think you should also add your points as a new first chapter in the Security_Handbook even before the Pre installation concerns section.

--Charles17 (talk) 09:56, 28 December 2016 (UTC)


 * Your suggestions and concerns did not go unnoticed. I did see them but have no gotten around to them at the moment. As for the present, it would probably be a good idea to capture the practical steps to integrate these changes in wiki markup in the Security Handbook as suggested. I will review them and make determinations on each of them over the course of the next few days. Kind regards, --Maffblaster (talk) 21:53, 29 December 2016 (UTC)

Ping-back to 'Verifying the install ISO' when 'installing stage3'
I thought it might be handy to have a link in the Para. that starts "Just like with the ISO file..." back to the Installation/Media stage for people like myself, who don't always download a fresh ISO image every install (we likely have one lying around!) but will always download an up-to-date clean stage3, and may not remember the GPG commands to download the Gentoo RelEng keys, and verify. There is a infobox for the verify command, alternatively we could repeat the 'downloading gentoo key' box into this para. if that was deemed appropriate instead.

-- veremit (talk) 21:43, 20 February 2017 (UTC)

HTTP to HTTPS
Please replace http://gcc.gnu.org/onlinedocs/ with https. Fturco (talk) 15:22, 16 April 2017 (UTC)


 * This was completed some time ago. Perhaps I didn't see or forgot to close this discussion. Thanks, . --Maffblaster (talk) 18:16, 29 December 2017 (UTC)

CFLAGS and CXXFLAGS
In the NOTE of section CFLAGS and CXXFLAGS, please add a link to Safe CFLAGS. Note The GCC optimization guide has more information on how the various compilation options can affect a system. Thanks. Luttztfz (talk) 12:43, 14 May 2017 (UTC)


 * I still think it is good to also link to the GCC optimization guide, but I will add the additional reference to the Safe CFLAGs article and perhaps mention that Safe CFLAGs may be a more practical place for beginners to start. Thank you! --Maffblaster (talk) 17:06, 15 May 2017 (UTC)

Tarballs compressed via xz now
Starting this week, at least amd64 stage3's are now compressed via xz, not bz2. I'd adjust the tarball wildcard to stage3-*.tar.* or similar

Iamben (talk) 15:53, 29 December 2017 (UTC)


 * Thanks, Ben! I have applyed a change that should work. Let me know if I missed something. --Maffblaster (talk) 20:55, 29 December 2017 (UTC)


 * The command uses a wrong parameter because of the brace expansion (see  "tar not work porperly"). --Feng (talk) 13:36, 31 December 2017 (UTC)


 * I can second Feng's comment. Brace expansion and pathname expansion are two different things entirely. One should not use the former where the latter is intended. The present usage of brace expansion in the handbook constitutes a dangerous anti-pattern, as well as setting a bad example for users unfamiliar with shell mechanics. Consider the following code:

   
 * 1) mkdir empty && cd empty && touch foo.bz2 foo.bz2.DIGESTS
 * 2) printf '<%s>' foo.{bz2,xz}  # this is not a glob and cares not for existing pathnames
 * 1) printf '<%s>' foo.*         # this is a glob
 * 1) shopt -s extglob
 * 2) printf '<%s>' foo.@(bz2|xz) # this is a glob (of the extended variety)
 * 1) printf '<%s>' foo.+([^.])   # and so is this


 * In short, have the user enable the extglob shell option in order to exploit a relatively safe generic approach e.g. . At present, the handbook contains no valid example of a command where the characteristics of brace expansion are appropriate. Also, the commands are not consistently generic. For example, the tar command exploits * as a globbing character, whereas the user is expected to fill in   manually in other commands. EDIT: I just realised that Feng made a similar suggestion in Maffblaster's page. Indeed, his suggestion of using   is superior, as it virtually guarantees that one pathname, at most, is matched in the common case. --kerframil (talk) 04:23, 21 July 2018 (UTC)


 * This has been added. Thanks. --Grknight (talk) 14:29, 7 November 2018 (UTC)

The commands in the "Verifying and validating" chapter could be further improved IMHO. First, if extended globbing is to be used then a command `shopt -s extglob' should be added.

Second, the use of '<' and '>' causes problems if the command is just copypasted to a console window without editing. Why not replace with a single asterisk, as already used for the `tar xvpf' command?

Finally, the `gpg --verify' command did not work for me. I got the complaint "gpg: not a detached signature" and the command exited with status 2 (the sha512 and whirlpool digests were OK though).

--Rafo (talk) 21:50, 1 January 2019 (UTC)

Properly preserve all xattrs like filecaps
For filecaps to be preserved on unpack (like for USE=filecaps on iputils so ping isn't setuid), --xattrs isnt quite enough, that won't preserve all xattr namespaces. --xattrs-include="*.*" should be enough, this will catch the user.* stuff for filecaps, security.* stuff for pax markings, and more.

Releng is currently disabling filecaps on affected pkgs for stage3 but would like to turn this caps on eventually. We need to get the handbook adjusted before that can (safely) happen.

Iamben (talk) 15:53, 29 December 2017 (UTC)


 * Nice catch! Thank you for this information. I'll make the change now. --Maffblaster (talk) 20:57, 29 December 2017 (UTC)

Easy verify checksums via rhash
I propose add another way to check integrity of downloaded files. There is tool which can automatically detect hash method from comments in DIGESTS files, so verifying downloaded files with SHA512 and WHIRLPOOL is simple as:

I found this as the most simple and universal method.

--Thaaxaco (talk) 20:07, 23 June 2018 (UTC)


 * It's good to mention the tool, however it is important to note we can only support tools available in the official live environment, which does not include . On another note, we try to avoiding mentioning alternative tools to the verification and extraction process since the handbook provides instructions using the official minimal live environment. You're welcome to add rhash to another area of the wiki (perhaps the Complete Handbook), but we will not be adding it here. Thank you! --Maffblaster (talk) 20:39, 5 November 2021 (UTC)

Change command for stage3 to actual compression method
Currently, the command shown to extract the stage3 use the tar.bz2, but the stage3 is actually a tar.xz, so the instructions is outdated.


 * Commands have been updated. --Grknight (talk) 14:27, 7 November 2018 (UTC)

It is 7DEC2019 and the commands in the handbook for AMD64 still show bz2 commands instead of -xvJpf for .tar.xz files.

RFC: Add big red beep about systemd stages
During years people keep asking "why do we have blocks installing Gentoo with systemd???"

If you dig up you will find they are using "standard" openrc stages installing systemd-based systems. While eudev vs udev and openrc vs systemd blocks can be solved manually it looks miss-informative for new users and they get confused. So I would like we add a note about systemd stages existence as well as this allows us to reduce amount of further questions.


 * We have added sections regarding systemd and that the relevant stage tarballs should be used. See this edit Special:Diff/1028319/1028325. Thanks to the work of, the www.g.o/downloads page also clearly delineates between the two with very nice 'tags'. This should really aid our users in downloading the correct stages for the init system target. Thank you for reporting! --Maffblaster (talk) 21:27, 5 November 2021 (UTC)

Missed format update for stage3 unpacking
All commands dealing with the stage 3 now correctly end in ?(bz2|xz), except the actual extraction. As the up-to-date amd64 images at least are now all .xz this causes issues for people. Ideally add ?(bz2|xz) instead of bz2, or replace bz2 with xz.

Chymæra (talk) 02:10, 18 December 2018 (UTC)


 * As of now stage archives are zipped using .xz. This issue has been corrected in the handbook a while ago. Thank you for reporting. --Maffblaster (talk) 21:04, 5 November 2021 (UTC)

Incomprehensible sentence: 'Depending on the installation medium, the only tool ...'
The following sentence doesn't make sense: 'Depending on the installation medium, the only tool necessary to download a stage tarball is a web browser.'

What does this sentence mean? Maybe something like: 'In order to download a stage tarball, you will need a web browser.'?

--Mike155 (talk) 20:46, 2 November 2019 (UTC)


 * This sentence has been removed for better clarity. Thank you for reporting. See Special:Diff/1028328/1028331. --Maffblaster (talk) 21:21, 5 November 2021 (UTC)

Suggestions for Improvement
Quoted material is from "Installing the Gentoo installation files".

- "Official Gentoo installation media includes the  command ..."

Ungrammatical. A plural subject demands a plural verb.

Official Gentoo installation media include the  command ...

- "Official media includes a configuration file ..."

The same error. Media are plural.

Official media include a configuration file ...

- "The  command can also be used to perform a manual set on the system clock."

Very clumsy style. I suggest

The  command can also be used to set the system clock.

- "Depending on the installation medium, the only tool necessary to download a stage tarball is a web browser."

This is a non sequitur. I suggest either

Irrespective of the installation medium, the only tool necessary to download a stage tarball is a web browser.

or (and this is even better)

The only tool necessary to download a stage tarball is a web browser.

- "To optimize Gentoo, it is possible to set a couple of variables which impacts the behavior of Portage ..."

Once again, a subject does not agree with its verb ("couple" is the implicit subject of the subordinate clause beginning "which ...").

... set a couple of variables which impact the behavior of Portage ...

- "...too much optimization can make programs behave bad ..."

Adjectives are never verb modifiers, in the English language.

... too much optimization can make programs behave bad[ly] ...

Or, better yet,

... too much optimization can make programs misbehave ...

- "The MAKEOPTS variable defines how many parallel compilations should occur when installing a package. A good choice is the number of CPUs (or CPU cores) in the system plus one, but this guideline isn't always perfect."

Linus Torvalds recommends setting this variable equal to twice the number of processors in the system. That way there is always one more compile job waiting to run when the previous one in that thread wraps up. It sort of makes sense, since Linux starts a separate thread running for each CPU core it finds fairly early during bootup.

-

"Update the  file to match personal preference and save (nano users would hit  + )."

Users -- especially novice users -- probably ought to say +  before they exit the program. Yes, I know that nano will prompt me to save the changes. But a guy can make mistakes that way. It's always safer to explicitly save your changes, then say "exit". I've learned that the hard way. --Davidbryant (talk) 16:48, 24 July 2020 (UTC)

Default download directory for stage3 tarball
The default location for downloading stage3 is /var/tmp/catalyst/builds/default it is written on page Catalyst I propose to mention this location on page and give link to the catalyst page and tool. Einstok Fair (talk) 03:17, 13 September 2020 (UTC)


 * Most users are not going to use Catalyst particularly on their first install. Instead they will often download it from the gentoo.org website. --Grknight (talk) 13:13, 14 September 2020 (UTC)


 * I agree with, catalyst is still a bit of a mystery many, including myself! We will not be linking to catalyst in the handbook at this time. --Maffblaster (talk) 21:29, 5 November 2021 (UTC)

Proposition to split stage3 into rstage3 and bstage3
For some systems only runtime dependencies are necessary. And for other systems full toolchains are necessary https://bugs.gentoo.org/742251 Einstok Fair (talk) 03:19, 13 September 2020 (UTC)


 * This has nothing to do with improving the Handbook so not the correct place to discuss such sweeping changes. --Grknight (talk) 13:16, 14 September 2020 (UTC)

links browser
With the new admincd, using the browser "links" to view https://gentoo.org/wiki/Handbook:AMD64/Installation/Stage#Introduction

suggests: "Verify the current date and time by running the date command: root #date"

Rjc (talk) 00:16, 19 October 2020 (UTC)


 * Not sure what you are saying here... Are you saying is not included in the admin live environment? You can see it is included in the .spec file. Since I'm very confused I will close this discussion for now. Please feel free to reopen if you want to discuss more in the future. --Maffblaster (talk) 21:42, 5 November 2021 (UTC)

Updates accordingly the signature verification process
In section Verifying and validating, the provided instructions don’t work for the verification of the DIGEST file and its (supposedly) detached signature.

Same with the DIGEST file:

However, it looks like this is a signature itself is valid, and holds the content of the DIGEST file, but doesn’t allow to verify the DIGEST file itself:

Two possible outcomes: When using gpg(1), the correction option to create a detachable signature is "--detach-sign", as follow (with --armor, for only ASCII content):
 * 1) Create a real detachable signature (to Infrastructure team, I guess):

On its own, GnuPG will create an "asc" suffixed file (with --armor, it’s ".sig").

Currently, with this issue, we can’t check the content of the DIGEST file itself, using the "--verify" option.
 * 1) Update the instructions as follow:

We can fix the instruction and provide a way to see the signed content thanks to "--decrypt" option; however, "decrypt" is not an obvious way to verify a signature, but it allows to check its content when disregarding stderr (same as above):

For me, the fix is the solution 1; it’s not a wiki issue, it looks like a signed process fault, and I guess this discussion is not at the right place.

But I might be wrong ;-)

By the way, I might have found a link to this issue: bug #587486

Thican (talk) 00:42, 17 January 2021 (UTC)

MAKEOPTS
If  is not set in , it now defaults to. Should this be mentioned in the section at the bottom of this page? Or perhaps that section could just be removed? Matthews (talk) 15:22, 7 August 2021 (UTC)