User:NeddySeagoon/Pi3

From Gentoo Wiki
Jump to: navigation, search

Neddys Notes to self to get Gentoo on a Raspberry Pi 3 in 64 bit mode

Warning
This page is hopelessly out of date

It can be regarded as the 'hair shirt' method of running 64 bit Gentoo on a Raspberry Pi 3. One year after the release of the 64 bit Raspberry Pi, its all a lot easier to have arm64 Gentoo on your Raspberry Pi. So much so, it needs a whole Raspberry_Pi_3_64_bit_Install new guide.

Follow that guide if you want to do it yourself.

Skip to the end of the document if you just want to download ready made filesystems. These are not updated on a regular basis.

There is also see Sakakis' image

The cross compile bits are still valid but there is a DAS-U-Boot, a kernel tree and .dts that just works, so getting /boot right is not nearly as difficult as described here.

Background

Its been 4 years since the Raspberry Pi B was launched' with 256Mb RAM and a single core, 700MHz, armv6j CPU.

Installing Gentoo was pretty much a matter of following the handbook and adding in cross distcc to speed things up.

The Pi3 is provided with 32 bit software. Its a four core armv7a CPU on the surface. The tried and tested install methods should still work.

If you just want to drive your Ferrari to the corner shop, that's well documented on this wiki

The article is about running Gentoo on the Pi 3 in 64 bit mode, as an armv8a. That's what the CPU is capable of. There is no handbook nor any stage files for that yet.

The Raspberry Pi 3 is a bit of a dog on the I/O front. The four USB ports and the Ethernet port are all on a singe USB2 interface from the CPU. That's unchanged is the last four years. That gives 400Bit/sec for everything connected there,

The new WiFi is on the second SD card interface, and the Bluetooth is on a UART, so they do not contend for the available USB port bandwidth.

Overview

As there is no stage files, the conventional chroot install is out of the question. There is nothing to chroot into. The workaround for that is to build some armv8a code on some other system in a process known as cross compiling.

Warning
Cross compiling has difficulties all of its own. If you have not made and used a cross compiler, practice in arm7a mode first.

Cross compiling is the process of building packages on one architecture, (the build host) that are intended to run an another architecture (the target). I will be building on amd64 for armv8a.

Important
Its not possible to jump a chasm in several small jumps. The 64 bit kernel and userland needs to JustWork

That's a little bit of an exaggeration. If your 64 bit cross compiled kernel can get as far as a console, you may get kernel error messages.

Warning
Not much will work. You will be doing well to get the serial console

Serial Console

A 3.3v serial port adaptor is required. You will need a USB to serial port adaptor as PC serial ports if you have one, uses either +/- 5v or +/-12v, depending on the age of your PC. Either will destroy a Pi serial port and may be more.

Warning
A USB to 5v serial port adaptor will destroy something too, so do check

Das U-Boot

The Raspberry Pi does not normally use a boot loader. The boot files are loaded by the firmware. However, the CPU needs to be at the correct privilege level when control is passed to the kernel, or the kernel won't run. Das U-Boot is loaded in the normal way. It then loads the kernel and passes control to it with the correct privilege level for the kernel to run.

We need a 64 bit arm8a Das U-Boot to get things started.

Kernel

To run 64 bit code we need a 64 bit armv8a kernel. Once the CPU is switched into 64 bit mode, before even Das U-Boot is loaded, the 32 bit kernel will not run. Even if it did, it would not run 64 bit code.

Cross Compiled Userspace

Once the kernel runs successfully, it will try to run /sbin/init, or whatever init= points to.

Systemd users are on there own. This author does not use systemd and is not about to start while bringing up 64 bit Gentoo on a Raspberry Pi3.

We need a cross tool chain and a target root filesystem to put all the armv8a code into. Its a really bad idea to emerge armv8a code to your build host.

Getting Started

On your main Gentoo system. There are other parameters you can pass to crossdev too.

root #emerge --ask crossdev -t aarch64-unknown-linux-gnu

This will create a cross toolchain that builds things into /usr/aarch64-unknown-linux-gnu/ This is the target root.

Eventually, you copy this directory to / on your Pi SDCard.

Once crossdev has finished, you will have a /usr/aarch64-unknown-linux-gnu/etc/portage which corresponds to your build host /etc/portage. The difference being that /usr/aarch64-unknown-linux-gnu/etc/portage applies when cross compiling for aarch64-unknown-linux-gnu.

Any number of cross compilers con be installed. Practice on something more well proven before you try 64 bit Raspberry Pi.

Target configuration

This consists of setting up /usr/aarch64-armv8a-unknown-gnu/etc/portage by hand.

make.conf

Copy your mirror list from the build host and any of your favourite options.

Optimists can set VIDEO_CARDS= and so on. Just like the Gentoo on your build host, these settings can be changed as needed.

Note that the default settings will share the hosts portage tree and distfiles but keep binary packaces and ongoing builds separate from the build hosts. This means its permissible to build for several targets at the same time.

cross make.profile

The default generic make.profile symbolic link needs to be changed. The cross make.profile (by hand)

root #rm /usr/aarch64-unknown-linux-gnu/etc/portage/make.profile
root #ln -s /usr/portage/profiles/default/linux/arm64/13.0/desktop /usr/aarch64-unknown-linux-gnu/etc/portage/make.profile

Or use another arm64 profile that's more to your liking. Don't use eselect profile.


Cross Compiling the Kernel

Mainline kernels do not have the Raspberry Pi3 64 bit patches. Fetch a kernel using git.

Make the normal home for the kernel.

root #mkdir /usr/aarch64-unknown-linux-gnu/usr/src
root #cd /usr/aarch64-unknown-linux-gnu/usr/src
root #git clone --depth 1 -b rpi3 https://github.com/zeldin/linux.git linux64-rpi3
root #ln -s linux64-rpi3 linux

With thanks to marcus_c at the Raspberry Pi Forum. https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=137963&start=25

Cross Compiling Das U-Boot

May as well put Das U-Boot beside the kernel,

With thanks to marcus_c at the Raspberry Pi Forum. https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=137963

Cross Building User Space Packages

Execute

root #emerge-wrapper --target aarch64-unknomn-linux-gnu --init

This sets up some symlinks so that

root #aarch64-unknown-linux-gnu-emerge <package/atom>

uses emerge, with the cross toolchain to install armv8a packages into /usr/aarch64-unknown-linux-gnu

emerge-wrapper needs to be run every time you open a new shell to cross compile in.

Important
emerge <package/atom> still builds on the host.

Random Warnings And Caveats

/usr/aarch64-unknown-linux-gnu/ is empty when you start to cross compile, unlike a stage3 install, where a working host chroot is provided. None of the code here will run on the build host.

In theory
root #aarch64-unknown-linux-gnu-emerge @system
will cross compile the system set. It will have a good go but it will fail for a number of reasons.
* To avoid circular dependencies the @system set is assumed to always be present.
* Some build systems don't support cross compiling
* arm64 is not well tested - don't even think of using ACCEPT_KEYWORDS=arm64
* expect to use the keyword ** on a number of packages

USE=-cxx aarch64-unknown-linux-gnueabi-emerge @system -uDNav almost works.

Missing deps - fix by hand

gcc - uses wrong linker /usr/lib/gcc/x86_64-pc-linux-gnu/5.3.0/../../../../x86_64-pc-linux-gnu/bin/ld

sys-apps/util-linux fails configure: error: udev selected but libudev not found even when libudev built

sys-devel/llvm needs dev-util/ninja. dev-util/ninja builds then sys-devel/llvm fails later when the build system tries to execute armv8a code on the build host.

groff

checking whether g++ accepts -g... yes checking that header files support C++... no configure: error: header files do not support C++

udev builds eudev fails.

armv8a is not well tested - be prepared keyword things you want to try

Optional Additions to @system USE="-cxx -fortran" aarch64-armv8a-linux-gnueabi-emerge dcron metalog ntpd dhcpcd wpa_supplicant

libepoxy

graphite2 -- Check for working CXX compiler: /usr/bin/g++ -- Check for working CXX compiler: /usr/bin/g++ -- broken CMake Error at /usr/share/cmake/Modules/CMakeTestCXXCompiler.cmake:54 (message):

 The C++ compiler "/usr/bin/g++" is not able to compile a simple test
 program.

libical -- Check for working CXX compiler: /usr/bin/g++ -- Check for working CXX compiler: /usr/bin/g++ -- broken CMake Error at /usr/share/cmake/Modules/CMakeTestCXXCompiler.cmake:54 (message):

 The C++ compiler "/usr/bin/g++" is not able to compile a simple test
 program.


fortune-mod /bin/sh: line 6: ../util/strfile: cannot execute binary file: Exec format error

libgpg

gnome-icon-theme checking icon-naming-utils >= 0.8.7... no configure: error: icon-naming-utils >= 0.8.7 is required to build and install gnome-icon-theme That needs perl, which is designed to not cross compile.

mupdf

dialog /usr/lib/gcc/x86_64-pc-linux-gnu/5.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: .libs/trace.o: Relocations in generic ELF (EM: 183) .libs/trace.o: error adding symbols: File in wrong format collect2: error: ld returned 1 exit status

screen ... configure: error: in `/usr/aarch64-armv8a-linux-gnueabi/tmp/portage/app-misc/screen-4.3.1-r1/work/screen-4.3.1': configure: error: cannot run test program while cross compiling See `config.log' for more details dbus-glib

Root Filesystem

Missing Dirs

The /usr/aarch64-unknown-linux-gnu/ directory has a few key directories missing that are needed to boot. A few are nice to have.

root #cd /usr/aarch64-unknown-linux-gnu
root #mkdir dev home opt proc sys boot root

Set the owner, group and modes to match the build host.

/etc/fstab

Only the SDCard slot and the serial console will work. Boot must be on the SDCard and in must be in vfat format.

Root can be anywhere but the SDcard is a good choice right now.

This /etc/fstab is consistent with the rest of this page but this being Gentoo you are free to change it to suit yourself.

Beware of using partitions beyond /dev/mmcblk0p7 as the kernel defaults to eight minor device numbers for mmc cards. Thats mmcblk0 and mmcblk0p1 ... mmcblk0p7.

# <fs>			<mountpoint>	<type>		<opts>		<dump/pass>

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.
/dev/mmcblk0p1		/boot		vfat		noauto,noatime	1 2
/dev/mmcblk0p2		/		ext4		noatime		0 1

#None of these ever
#/dev/cdrom		/mnt/cdrom	auto		noauto,ro	0 0
#/dev/fd0		/mnt/floppy	auto		noauto		0 0

Add a swap later.

mtab

Make /etc/mtab -> /proc/self/mounts in /usr/aarch64-unknown-linux-gnu/

/etc/inittab

A serial console must be configured. Das U-Boot can drive the HDMI output but the kernel cannot.

The last message that appears on the HDMI is "Starting Kernel ..."

Edit /etc/inittab for the first serial terminal.

# SERIAL CONSOLES
s0:12345:respawn:/sbin/agetty -L 115200 ttyAMA0 vt340

and comment out the 9600 baud entry below.

# Architecture specific features
#f0:12345:respawn:/sbin/agetty 9600 ttyAMA0 vt100

This avoids the serial terminal baud rate or the Pi being changed to 9600 baud, while the terminal emulator works at 115200 baud.

Clear /usr/aarch64-unknown-linux-gnu/tmp

This is the portage build location for aarch64-unknown-linux-gnu cross compiles. On the SDcard, it will become /tmp, which gets cleared every boot. There will be a lot of junk there, so you may need to wait a long time for the Pi to clear it if you forget.

Copy /usr/aarch64-unknown-linux-gnu/ to the root partition on your microSD card. This is in place of what was there.

Copy u-boot.bin, the dtb and kernel to the boot partition on your microSD Card. This is in addition to the files already there.

config.txt

There are few changes to config.txt but back it up anyway. Add the following ...

# boot in 64 bit mode !!!
arm_control=0x200 
kernel=u-boot.bin
dtoverlay=pi3-miniuart-bt-overlay
kernel_old=1
disable_commandline_tags=1
# get the serial console baud rate right
force_turbo=1

arm_control=0x200 sets the CPU into 64 bit mode.

kernel=u-boot.bin starts u-boot instead of a kernel.

dtoverlay=pi3-miniuart-bt-overlay gets the details for the serial port on the Pi.

disable_commandline_tags disables any kernel command line built into the kernel.

force_turbo=1 is needed to stop the serial port baud rate changing with CPU clock speed.

Configuring the system

Follow Configuring the system in the amd64 handbook. Adjust the paths given. Root is now
FILE {{{filename}}}
/usr/aarch64-armv8a-linux-gnueabi
.

Booting In 64 Bit Mode

Best test your serial console in 32 bit mode first. Its all you will have so you want to know it works

First Boot Only

To automatically boot with Das U-Boot, it reads its u-boot.env to know what to do. It looks like it may be a text file but its not quite.

At the U-Boot prompt, enter the following, a line at a time.

setenv linux_load 'fatload mmc 0:1 $loadaddr image'
setenv fdt_load 'fatload mmc 0:1 $fdt_addr_r $fdtfile'
setenv fdtfile 'bcm2837-rpi-3-b.dtb'
setenv bootcmd 'run fdt_load ; run linux_load ; booti $loadaddr - $fdt_addr_r'
setenv bootargs 'dwc_otg.lpm_enable=0 console=ttyAMA0,115200 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait'
saveenv

This creates your u-boot.env

Just like more common boot loaders, bootargs must be a single line. From time to time, its useful to add init=/bin/bash to bookargs.

Power cycle the Pi or type boot to test.

What could Possibly Go Wrong?

Booting stops at

* Wiping /tmp directory ...

All the build leftovers are still there. This is where the cross compiler puts its portage build directory. Make a coffee, it will eventually clear /tmp and move on.

The serial console stops at

* Starting local ...
[ ok ]

/etc/inittab is in a mess. You should see

* Starting local ...
[ ok ]


This is localhost.unknown_domain (Linux aarch64 4.5.0+) 00:00:10

localhost login:

Both ends of the serial link are running with different setting. Lucky users will get gibberish, so its obvious that there are serial link issues.

The Easy Way

Read the README Fetch /boot and untar it to partition 1 on the microSD Card Fetch / and untar it to partition 1 on a USB storage device.

Don't forget the p option to tar.

Edit the Pis

/etc/conf.d/keymaps 

because you really won't like my

keymap="dvorak-uk"

Rinse and repeat in

/etc/X11/xorg.conf 

While you are there, comment out

Option "XkbOptions"  "ctrl:swapcaps"

Move the microSD card and USB storage to the Pi3 and power up. The only user is root and the password is raspberry.

Ignore all the missing special /dev entries. They are listed in /etc/fstab along with swap but are not provided in the tarball.

Building Java

The bad news is that like gcc, you need java to build java. There are two paths, gcj or a binary java like dev-java/oracle-jre-bin.

The dev-java/oracle-jre-bin patch is described at the end of this section.

Using gcj to build icedtea

The good news is that contrary to conventional wisdom that says gcj can't be used after gcc-4.x, it seems that gcj-5.4 works.

The preferred path to java on arm64 is then

  • build gcc-5.4 with gcj
  • build icedtea-7.2.6.8 with gcj (java 1.7)
  • build icedtea-3.2.0 with icedtea7 (java 1.8)

Building gcc-5.4 with gcj

Do not try gcc-6.3.0 it will build gcj but the resulting gcj will not bootstrap icedtea.

The gcj USE flag is profile masked on arm64.

root #cd nano /etc/portage/profile/package.use.mask
# gcc java compiler is hard masked
# it goes away in gcc-7 
sys-devel/gcc -gcj

A version mask should work and may be preferred because on a Pi3, gcc now needs to be built with -j1 or it appears to hang during the build.

Turn on the gcj USE flag

root #cd nano /etc/portage/package.use/gcc
sys-devel/gcc gcj

Again a version mask should work, since only gcc-5.4 is required with gcj.

Now (re)build gcc-5.4. On a Pi3, don't forget the MAKEOPTS="-j1"

Building icedtea-7.2.6.8 with gcj

Lots of things are keyword masked, portage will tell you exactly what.

Important
These packages are now keyworded ~arm64. This step is not required
Make yourself a
root #cd nano /etc/portage/package.accept_keywords/iced_tea
with the following content.
<dev-java/java-config-9999 **
<virtual/jre-9999 **
<dev-java/icedtea-web-9999 **
<dev-util/systemtap-9999 **
<app-eselect/eselect-java-9999 **
<dev-java/icedtea-9999 **
<dev-java/javatoolkit-9999 **
<virtual/jdk-9999 **
<dev-java/ant-core-9999 **
<sys-apps/baselayout-java-9999 **
<dev-java/gcj-jdk-9999 **
<dev-java/ecj-gcj-9999 **
<media-libs/libart_lgpl-9999 **

Other than icedtea itself, everything else builds with gcc-6.3

Building icedtea-3.2.0 with icedtea7

Now icedtea-7 is installed icedtea-8 just works. It needs the same package.accept_keywords as above.

Using dev-java/oracle-jre-bin to Build Icedtea

This section describes the use of dev-java/oracle-jre-bin-1.8.x to build icedtea. There is real reason to do this. Its documented here for completeness.

Building icedtea

Install the fetch restricted dev-java/oracle-jre-bin. Its masked on arm64.

There are two icedtea ebuilds in the tree icedtea-3.2.0.ebuild and icedtea-7.2.6.8.ebuild. icedtea-7 would get you a java-1.7 VM. However, it won't build with the java-1.8 VM provided by dev-java/oracle-jre-bin. The icedtea-3.2.0.ebuild gets you icedtea-1.8 but it refuses to build against dev-java/oracle-jre-bin.

Edit the icedtea-3.2.0.ebuild

This is a one time thing (until you have icedtea), so you can do this in the portage tree if you wish. Find the DEPEND statement

DEPEND="${COMMON_DEP} ${ALSA_COMMON_DEP} ${CUPS_COMMON_DEP} ${X_COMMON_DEP} ${X_DEPEND}
       || (
               dev-java/icedtea-bin:8
               dev-java/icedtea-bin:7
               dev-java/icedtea:8
               dev-java/icedtea:7
       )

and add dev-java/oracle-jre-bin to the || list and regenerate the digest.

Build icedtea

Add the non-keyworded dependencies to /etc/portage/package.accept_keywords/iced_tea. Portage will tell you what they are.

It still needs a bit of coaxing to build

MAKEOPTS="j2" JAVA_PKG_FORCE_VM="oracle-jdk-bin" emerge icedtea -av
Pi3 64bit ~ # genlop -t icedtea                                             
* dev-java/icedtea
    Thu Dec  1 23:52:58 2016 >>> dev-java/icedtea-3.2.0
      merge time: 3 hours, 6 minutes and 51 seconds.

Lastly, select icedtea as your VM.

icedtea should now build with itself. That's not tested here yet.

USEing java

The java USE flag is forced off in the arm64 profiles.

Important
The arm64 profiles have been updated - This is no longer correct

Remove the hard mask on a package by package basis with an entry in /etc/portage/profile/package.use.mask, for example,

# java is hard masked on arm64
app-office/libreoffice -java

This removes the hard mask on app-office/libreoffice only. USE=java still needs to be set in the normal way.

app-office/libreoffice does build on arm64 with USE=java.