Raspberry Pi 3 64 bit Install

History
Its been about a year since the Raspberry Pi3 with a 64bit capable CPU became available. At the outset, it was difficult to install Gentoo on the Pi 3 in 64 bit mode. A lot of work by a lot of people has almost brought a 64 bit Gentoo install on the Pi 3 down to almost a standard handbook install.

The Gentoo arm64 offering is experimental at the time of writing. That means that very little is marked stable. Expect to use ACCEPT_KEYWORDS="~arm64" and then use package.accept_keywords too.

What Works
All the Pi 3 hardware is supported in 64 bit mode. That's WiFi USB Ethernet (needs USB) Bluetooth Hardware Video Acceleration Sound over HDMI

Untested
PAL/NTSC Video Output Analogue Sound Output

Whats Required
Gentoo Install on a PC microSD card reader for the PC Raspberry Pi 3 microSD card > 8G USB Keyboard USB Mouse HDMI Display The content of the microSD card will be wiped during the install.

Installation Overview

 * Install crossdev on the PC
 * Fetch the Raspberry Pi firmware
 * Fetch the Raspberry Pi kernel
 * Partition the microSD card
 * Fetch the Gentoo bits of the install
 * Cross compile and install your kernel
 * Setup
 * Boot the Pi to Test

Install Crossdev on the PC
There are no prebuilt kernel images for the Pi3 in its arm64 mode. Until you have a arm64 kernel, you can't boot it in 64 bit mode. Its a little chicken and egg. Once the Pi is running, it can build its own kernels.

crossdev is Gentoos' tool for building cross compiler tool chains. Once its installed, we will use it to build the arm64 kernel on the Gentoo PC. will install the crossdev tool.

Using Crossdev to Build A Cross Compiler
You may need to nominate an overlay to store the cross ebuilds, crossdev will let you know.

There are other parameters you can pass to crossdev too.

Convert files as crossdev asks e.g. error: please convert /etc/portage/package.env to a directory by appending _file to the existing filename

making the directory

then moving package.env_file into the directory.

Rinse and repeat until crossdev is happy.

crossdev will take a while. Its building binutils:             binutils-[latest] gcc:                  gcc-[latest] headers:              linux-headers-[latest] libc:                 glibc-[latest]

When crossdev completes you will have a cross toolchain

It will also create an arm64 target root in /usr/aarch64-unknown-linux-gnu/ This is used by cross emerge.

Pure cross compiling, other than the kernel, is out of scope of this guide.

Fetch the Raspberry Pi Firmware
The Raspberry Pi Firmware is maintained in a git repository. You will need to install git if you don't have it.

Don't do anything you don't need to do as root.

As your normal user, make some space in the home directory for raspberry pi will do nicely.

This will be for the Raspberry Pi Firmware and kernel

will fetch the Pi firmware into a directory called firmware

It will actually fetch all the history too. If you are short of space or bandwidth, a shallow clone is all that's needed. A full clone is 6.3G and growing, we only need the /boot directory out of the master branch.

There is nothing to build. ~/raspberrypi/firmware/boot is used as is.

Fetch the Raspberry Pi Kernel
Stay in raspberrypi and

will fetch the kernel into a directory called linux.

With absolutely no fanfare at all, 64 bit support was added to this kernel tree late in 2016. No more searching for odd patches.

Not everything has been accepted by the mainline kernel yet but its getting closer. Feel free to test for yourself.

The master branch may be broken as its commit by commit as it happens. Test if you want to. Its better to use an identified tag. At the time of writing that is rpi-4.10.y. The 4.10 mainline kernel is still at -rc status.

That's the kernel source tree in place, ready for configuring and cross compiling.

Configure The Kernel
As your user

The bcmrpi3_defconfig is almost right as it stands. It defaults to the powersave CPU governor, which runs the Pi at 600MHz. All the governors are there. Ondemand is recommended.

Either use menuconfig to change the default CPU governor, add something to the kernel command line, or change it after booting.

Search for CPU_FREQ_DEFAULT_GOV go to that location and set the default to Ondemand. Exit menuconfig, saving the change.

.config - Linux/arm64 4.10.0-rc6 Kernel Configuration > CPU Power Management > CPU Frequency scaling ────────────────────────────────── ┌────────────────────────── CPU Frequency scaling ───────────────────────────┐ │ Arrow keys navigate the menu.  selects submenus ---> (or empty   │    │  submenus ). Highlighted letters are hotkeys. Pressing  includes, │ │  excludes,  modularizes features. Press  to exit, <?>   │ │ for Help,  for Search. Legend: [*] built-in [ ] excluded   module │ │ ┌────────────────────────────────────────────────────────────────────────┐ │   │ │    [*] CPU Frequency scaling                                           │ │ │ │   [*]   CPU frequency transition statistics                           │ │ │ │   [ ]     CPU frequency transition statistics details                 │ │ │ │         Default CPUFreq governor (powersave)  --->                    │ │ │ │   <*>   'performance' governor                                        │ │ │ │   -*-   'powersave' governor                                          │ │ │ │   <*>   'userspace' governor for userspace frequency scaling          │ │ │ │   <*>   'ondemand' cpufreq policy governor                            │ │

The kernel .config contains lots of support for hardware you don't have and possibly have never heard of. More confident readers may be tempted to trim things out now. A word of advice - don't, at least, not until the system boots.

Cross Compiling The Kernel
The kernel will not link with the gold linker. It ends with aarch64-unknown-linux-gnu-ld: fatal error: -shared and -pie are incompatible If you don't know what the gold linker is, you are not using it.

The build is conventional, other than telling the build system to build for arm64 and use the cross compiler.

Change X to the number of parallel MAKE jobs you want to run. Convention is cores+1.

Background
The Pi does not need a boot loader. There is no firmware for the arm CPU, like there is in a PC. Instead, the GPU manages loading software for the CPU to execute, while the CPU is held reset. Its the GPU that has the 'BIOS' that gets everything started.

This makes it impossible to brick the Pi, since at worst, the microSD card needs to be reloaded.

This arrangement does impose some constraints on the microSD card. The partition table must be MSDOS The first partition (/boot) must be vfat.

The bootcode.bin file has some useful defaults that make setup easier, which we will take advantage of later.

Partitioning
Depending on how the microSD card is connected to you PC, it my be /dev/sdX or /dev/mmcblkY

In the example below, its /dev/sdk. Using the partitioning tool of your choice, make three partitions on your microSD card. boot 128Mb swap 2G root (/) the rest.

Using fdisk, and your microSD card block device, not my /dev/sdk

Make The Partition Table and Add Partitions
Make a new MSDOS disklable Command (m for help): o

Add a new partition - this will be 128Mb for /boot Command (m for help): n Partition type p  primary (0 primary, 0 extended, 4 free) e  extended (container for logical partitions) Select (default p): p Partition number (1-4, default 1): 1 First sector (2048-15523839, default 2048): Last sector, +sectors or +size{K,M,G,T,P} (2048-15523839, default 15523839): +128M Has created a new partition 1 of type 'Linux' and of size 128 MiB.

Add a 2G partition 2 for swap.

Put the remaining space in partition 3 for root. Check with Command (m for help): p Using an 8G microSD card it should show Device    Boot   Start      End  Sectors  Size Id Type /dev/sdk1         2048   264191   262144  128M 83 Linux /dev/sdk2       264192  4458495  4194304    2G 83 Linux /dev/sdk3      4458496 15523839 11065344  5.3G 83 Linux

Set Flags and Partition Types
Toggle the bootable flag on partition 1 Command (m for help): a Partition number (1-3, default 3): 1 The bootable flag on partition 1 is enabled now.

Mark partition 1 as FAT Command (m for help): t Partition number (1-3, default 3): 1 Partition type (type L to list all types): b

Changed type of partition 'Linux' to 'W95 FAT32'.

Mark partition 2 as swap Command (m for help): t Partition number (1-3, default 3): 2 Partition type (type L to list all types): 82

Changed type of partition 'Linux swap / Solaris' to 'Linux swap / Solaris'.

Check again. Device    Boot   Start      End  Sectors  Size Id Type /dev/sdk1 *       2048   264191   262144  128M  b W95 FAT32 /dev/sdk2       264192  4458495  4194304    2G 82 Linux swap / Solaris /dev/sdk3      4458496 15523839 11065344  5.3G 83 Linux

Save The New Partiton Table
Exit fdisk with either w or q  w   write table to disk and exit q  quit without saving changes

Make Filesystems
You know the block devices from the partitioning step above. Fill in the ... to match your system.

vfat for /boot

mkswap for swap

ext4 for root

The i-node count cannot be changed after filesystem creation and can limit the number of files on a files system. The Gentoo repository alone needs over 17,000 i-nodes.

Fetch the Gentoo bits of the install
To make it easy to cross refer to the Gentoo_Handbook

Mount the microSD card root filesystem at /mnt/gentoo

Install the arm64 Stage 3
Following the Gentoo_Handbook fetch the arm64 stage3 and untar it to /mnt/gentoo in the normal way.

/mnt/gentoo/tmp should be empty. Clear it now or the the Pi will do it at boot. That may take a long time.

Before and after clearing /mnt/gentoo/tmp Filesystem               1K-blocks      Used Available Use% Mounted on /dev/sdk3                   5380212   1205968   3881228  24% /mnt/gentoo /dev/sdk3                  5380212    874636   4212560  18% /mnt/gentoo

Install a Portage Snapshot
This step is not actually needed to boot the Pi but emerge won't work without it

Following the Gentoo_Handbook fetch and unpack a portage shapshot in the normal way.

Careful readers can copy their host /usr/portage as long as ./packages and ./distfiles are omitted.

The Raspberry Pi Foundation Provided Files
Mount the microSD card boot at /mnt/gentoo/boot. (The microSD card root should still be mounted at /mnt/gentoo)

As root, copy the content of your normal users ~/raspberrypi/firmware/boot to /mnt/gentoo/boot.

This will generate a warning cp: failed to preserve ownership for ' ': Operation not permitted for every file. vfat does not support permissions, so the warning are expected and can be ignored.

You should end up with files in /mnt/gentoo/boot, not a directory called boot.

Not all the files there are required. Its a one size fits all for all models of Raspberry Pi operating in 32 bit mode.

Install the Kernel to the microSD Card
The kernel was built above, now to install it.

The kernel is in three parts kernel binary kernel modules the device tree

Install The Kernel Binary
As root, copy the kernel binary from the build location

It is possible to use other kernel file names by adding entries to /boot/config.txt

Install The Device Tree
The device tree binary (.dtb) describes the hardware to the kernel. This avoids having all the existing hardware configurations ard coded into the kernel.

Due to the way the Raspberry Pi 64 bit kernel support has been added, there are going to be two different device trees with the same file name. A 32 bit version and a 64 bit version. They are not interchangeable. Move the 32 bit version out of the way. That's renamed the 32 bit version.

Copy the dtb from the build location

It is possible to use other device tree file names by adding entries to /boot/config.txt

Install The Kernel Modules
From the top of the kernel tree, install the kernel modules.

The INSTALL_MOD_PATH is the root of the filesystem the modules are to be installed into. Due to kernel naming its unlikely you will have a 4.10.0-rc6-v8+ so omitting INSTALL_MOD_PATH is probably harmless to your build host install.

Checking The Kernel Install
COPYING.linux          bcm2708-rpi-cm.dtb      bcm2710-rpi-cm3.dtb  fixup_db.dat  kernel8.img   start_db.elf LICENCE.broadcom       bcm2709-rpi-2-b.dtb     bootcode.bin         fixup_x.dat   overlays      start_x.elf bcm2708-rpi-b-plus.dtb bcm2710-rpi-3-b.dtb     fixup.dat            kernel.img    start.elf bcm2708-rpi-b.dtb      bcm2710-rpi-3-b.dtb_32  fixup_cd.dat         kernel7.img   start_cd.elf

Notice the two bcm2710-rpi-3-b.dtb device tree binaries and the kernel8.img.

4.10.0-rc6-v8+ Shows that the kernel modules were installed to the correct location.

Install WiFi Firmware
The on board wifi needs firmware. The files brcmfmac43430-sdio.txt and brcmfmac43430-sdio.bin are both required to be present in /mnt/gentoo/lib/firmware/brcm/

They can be downloaded via wireless.wiki.kernel.org

Install Bluetooth Firmware
The on board bluetooth needs the firmware files bcm43xx-0.fw and bcm43xx_hdr-0.fw in /mnt/gentoo/lib/firmware.

It also needs a startup script in /mnt/gentoo/etc/local.d/bluetooth.start

/usr/bin/hciattach /dev/ttyAMA0 bcm43xx 921600 noflow -
 * 1) !/bin/bash
 * 2) Bluetooth is attached to /dev/ttyAMA0 so attach it and
 * 3) load the firmware

Setup
Much of the setup prior to booting a new Gentoo install is done in a chroot. Chrooting into an arm64 install from some other arch is beyond the scope of this guide. Instead, only the bare minimum setup, from outside the chroot is covered.

Root Password
There are several ways to generate a password hash for /etc/shadow I usually copy the hash from another system. My Pi uses an /etc/shadow root entry root:$6$xxPVR/Td5iP$/7Asdgq0ux2sgNkklnndcG4g3493kUYfrrdenBXjxBxEsoLneJpDAwOyX/kkpFB4pU5dlhHEyN0SK4eh/WpmO0::0:99999:7::: The matching password is raspberry.

Feel free to use that line to replace the root entry in /mnt/gentoo/etc/shadow

You can change the password once you are logged in.

/etc/fstab
On the Pi, the microSD card will be /dev/mmcblk0 with partitions /dev/mmcblk0p1, /dev/mmcblk0p2 and /dev/mmcblk0p3

Edit /mnt/gentoo/etc/fstab to match.

/dev/mmcblk0p1         /boot           vfat            noauto,noatime  1 2 /dev/mmcblk0p2         none            swap            sw              0 0 /dev/mmcblk0p3         /               ext4            noatime         0 1

/boot/config.txt
This file will not exist until its created. Create it with the following content.

/boot/cmdline.txt
This file will not exist until its created. Create it with the following content.

This boots the Pi with a kernel command line of 8250.nr_uarts=0 cma=256M@256M dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1920 bcm2708_fb.fbheight=1080 bcm2709.boardrev=0xa02082 bcm2709.serial=0x8e2830fe smsc95xx.macaddr=B8:27:EB:28:30:FE bcm2708_fb.fbswap=1 bcm2709.uart_clock=48000000 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000 root=/dev/mmcblk0p3 rootfstype=ext4 rootwait

It really is one long line.

Setting the Console Keymap
This step is optional if you can log in using the default keymap="us"

Set the keymap to something you use, e.g. keymap="dvorak-uk"

Boot the Pi to Test
Unmount the microSD card.

When the prompt returns, move the microSD card to the Raspberry Pi and power on.

For 10 seconds (it seems much longer) you should see the GPU 'Rainbow' test pattern, then the familiar boot messages.

Log in at the Pi console. Nothing was added to any runlevels during the install, so networking was not started, nor anything that depends on networking, like ntpd and sshd.

The Pi does not have a hardware real time clock. Its time will be Jan 1, 1970.

What Next
As always with Gentoo, if it booted, that's the hard bit done.

All The setup steps in the Gentoo Handbook Fix the MAC address or use a static IP Allow root logins via ssh Add a crond, a logger and other things the handbook does before the reboot. Add Kernel Sources (or at least the .config)

WiFi and Bluetooth
Its unlikely that WiFi or Bluetooth will work at first boot. Expect to add some control tools for both.

CFLAGS
CFLAGS="-march=armv8-a+crc -mtune=cortex-a53 -ftree-vectorize -O2 -pipe -fomit-frame-pointer"

gcc-6.x allows the use of -march=native but that will prevent the use of distcc. The above is the same as gcc-6.3 would set for -march=native anyway.

ACCEPT_KEYWORDS
Outside of the @system set, arm64 is either testing or keyword masked. Set ACCEPT_KEYWORDS="~arm64" and expect to use package.accept_keywords too.

Upgrade gcc then rebuild all of the installed C++ software.

MAKEOPTS
With only 1G RAM, and four cores, the conventional MAKEOPTS="-j5" is a bit aggressive for building larger things. It will force swapping or even appear to lock up the Pi completely, to the point where it won't even respond to the console.

Use files in /etc/portage/env/ and entries in /etc/portage/package.env to set MAKEOPTS on a per package basis.

Networking
dhcpcd is not in the stage 3, nor is eth0 in the default runlevel. will bring up eth0

sshd
The default configuration for sshd will not allow password based root logins. add your ssh public key for root make a normal user in the wheel group edit /etc/ssh/sshd_config to allow password based root logins

Where to Get Help
On Internet Relay Chat irc.freenode.net#gentoo-arm irc.freenode.net#gentoo-embedded

On the Gentoo Forums, start a new topic in the Gentoo on Alternative Architectures forum.

I don't mind a PM on the forums with a link to your post. I don't do one to one help via email or the forums PM system. You will either get no response at all or a request to make a public post. That way others may learn from your misfortune.

Acknowledgements
Everyone contributing to the arm64 software base.

Especially Sakaki, who showed the way on the final steps.