Gentoo for NanoPI Neo3
The NanoPI Neo3 is a tiny IoT offered by NanoPI having many more features than initially shown This document tries to guide you how to install Gentoo on a SD card, and boot a bare system on it. Probably you'll want to keep the crossdev live, as the cortex A53 chips are not really suited for recompiling the whole device. NanoPI R2S and Neo 3 are very very similar: they only differ by a few hw features, so this guide should be a good start for R2S too. This guide focuses on a minimal setup with all performance we could get, A53 chips are made for power-saving, not performance, and as such we need to use every feature that may be useful
U-boot: GPL-2+ https://github.com/01org/edison-u-boot
- working u-boot
- working kernel
- able to build using crossdev
- able to unlock special features not in standard dts
- TODO: fix USB3 functionality
Steps to install
1. prepare the crossdev setup:
emerge --ask crossdev
Please note that it may be useful to add multilib support for ARMv8, as it _may_ allow armv7 code to compile. Also, I prefer to use custom cflags here, as the Rockchip RK3328 chip has SIMD + crypto extensions not defined in standard ARMv8-A.
In order to tune options for gcc, glibc, ...:
echo "cross-aarch64-unknown-linux-gnu/glibc multilib custom-cfags" >> /etc/portage/package.use/crossdev
echo "cross-aarch64-unknown-linux-gnu/gcc multilib graphite lto" >> /etc/portage/package.use/crossdev
other details could be found in Embedded Handbook, Run it until at compiling the kernel.
Starting from below, all packages which are not in portage are fetched into /usr/aarch64-unknown-linux-gnu/tmp
2. Prepare U-BOOT
U-boot only contains an image for NanoPi R2S, which requires a few modifications to run properly. Also, fancy stuff like boot from pxe, etc .. makes no sense because the device needs an SDcard anyway. In order to compile everything, it could be useful to download the latest ARM firmware first:
and then u-boot (may be useful to add -d 5 to limit the history): see here on github
This is where it becomes tricky, in order to understand, The following boot steps are important:
- initial CPU start (onboard SRAM), search for MMC card block 0x40
- initial boot loader (U-BOOT) (also referenced as idbloader)
- Second program loader (U-BOOT) (setup initial devices, buses, and loads a primary device tree)
- Tertiary program loader (U-BOOT) (runs DDR setup)
- U-BOOT executable (U-BOOT) (somewhat comparable to EFI boot loader)
- Kernel loading
Previously, steps 2-5 all required separate files on separate partitions. This is no longer the case: u-boot generates an image "u-boot-rockchip.bin" which could be written to sector 0x40 of your SD card:
dd if=u-boot-rockchip.bin of=/dev/sdd bs=512 seek=64
Knowing this, the SDCard partition layout (using msdos here, but gpt should work as well) can be created. It is important to keep the order, size and start sector of the FAT32 boot partition. Other things like root fs, swap partition etc are not, and enjoy the flexibility of Gentoo OS.
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x31a5b84a
Device Boot Start End Sectors Size Id Type
/dev/sdd1 * 32768 262143 229376 112M b W95 FAT32
/dev/sdd2 262144 113246174 112984031 53.9G 83 Linux
/dev/sdd3 113246175 121634815 8388641 4G 82 Linux swap / Solaris
There are 2 ways U-BOOT can boot from the SDCard: 1. Using an exlinux config file 2. Using default UEFI image
It is possible to make U-BOOT load grub and configure it before booting the kernel, but the advantage of it is pretty minimal: if boot customization is required, use extlinux.
neo3 ~ # ls /boot
efi extlinux rockchip
neo3 ~ # ls /boot/efi/boot/
neo3 ~ # ls /boot/extlinux
neo3 ~ # ls /boot/rockchip
And for the extlinux.conf file:
neo3 ~ # cat /boot/extlinux/extlinux.conf
append earlycon rootwait root=/dev/mmcblk1p2 rootfstype=ext4 init=/sbin/init email@example.com/eth0,6666@(ip of log server)/(mac of log server)
A little explanation about the layout here:
- the boot partition contains a default EFI folder (for the kernel image), an extlinux folder (for extlinux config), and a rockchip folder for the device tree, which is necessary to present the devices to the OS.
- the EFI partition has the kernel stored as bootaarch4.efi, which makes it the default boot image in case extlinux wouldn't work.
- rockchip folder contains the dtb file (a compiled dts file) which was generated while compiling u-boot / the linux kernel. It is located at (u-boot)/arch/arm/dts/, or use the default linux dts file in (linux source)/arch/arm64/boot/dts/
- In the extlinux folder, write something like explained above
- Default image is gento
- the kernel image is at ....
- the fdt file is at ...
- the kernel arguments are .... it could be, depending on your DTS settings, that mmcblk1 or mmcblk0 is required: if the eMMC controller of the rockchip is enabled in DTS and thus in DTB, it will take first place, even though not specified. rootwait is required, as it is not guaranteed that the MMC controller is online when Linux tries to mount the root filesystem.
- In this example, the netconsole is enabled. This is for debugging: when something goes wrong inside the kernel, it possible to perform some diagnosis using the logs here.
if no configuration is required, simply append the kernel arguments while compiling the kernel, but the dts magic may be a little more complicated.
Setting up base system
There are a number of _essential_ things that need to be verified before gentoo can boot:
- It must be able to mount root fs (see extlinux.conf layout above)
- It must be able to run init. This is a caveat: ARM setups tend to default to busybox, but it also replaces the /sbin/init with its own variant. This variant does not load /etc/inittab the way openrc init does. As such, due to these confusion, it is recommended to check /sbin/init after every major upgrade: if it a symlink to busybox, a busybox version of inittab is required, otherwise, the default one is fine
- sysinit:/sbin/openrc sysinit
- wait:/sbin/openrc boot
- wait:/sbin/openrc default
- ctrlaltdel:/sbin/shutdown -r now
- shutdown:/sbin/swapoff -a
- shutdown:/bin/umount -a -r
- Default runlevel.
- System initialization, mount local filesystems, etc.
- si::sysinit:/sbin/openrc sysinit
- Further system initialization, brings up the boot runlevel.
- rc::bootwait:/sbin/openrc boot
- l0u:0:wait:/sbin/telinit u
- l0:0:wait:/sbin/openrc shutdown
- l1:1:wait:/sbin/openrc single
- l2:2:wait:/sbin/openrc nonetwork
- l3:3:wait:/sbin/openrc default
- l4:4:wait:/sbin/openrc default
- l5:5:wait:/sbin/openrc default
- l6u:6:wait:/sbin/telinit u
- l6:6:wait:/sbin/openrc reboot
- l6r:6:wait:/sbin/reboot -dkn
(to be continued...)