StarFive VisionFive 2
This page describes several methods for installing Gentoo on the StarFive VisionFive 2.
Introduction
This page aims to provide a comprehensive introduction to the various methods of installing Gentoo onto Embedded hardware using the VisionFive 2 as an example device.
The majority of the instructions here are not specific to the VisionFive 2 and may be applied to other devices with similar hardware. Please apply some common sense when adapting these instructions for other devices and do not blindly copy and paste commands without understanding what they do.
It should also be noted that the processes described hereafter are not always the most efficient in terms of commands used, and multiple commands may be used with, for example, environment variables that would typically be exported, or repeated additional options where a wrapper would be useful. This is intentional as it is intended to be a learning experience for the reader.
This article is intended to supplement the Embedded Handbook.
Hardware
For a detailed overview of the StarFive hardware, please visit StarFive_VisionFive_2/Hardware
Boot sequence
Do not trust the "ON" label on the boot selector DIP switch as that latter may be soldered in the reverse position as it is the case in the photo here-after. The true states vs switch positions are (with GPIO header seen upper-left):
- OFF (low or 0 level): switch positioned towards to the "ON" label (switch is on the right)
- ON (high or 1 level): switch positioned backwards to the "ON" label (switch is on the left)
The boot sequence of a typical RISC-V device is as follows:
- When the SoC is powered on, the CPU fetches instructions beginning at address 0x0, where is the BootROM (BROM AKA Primary Program Loader) is located.
- The BROM loads the Secondary Program Loader (SPL) from some form of Non Volatile Memory (NVM).
- The SPL loads a U-Boot Flattened Image Tree (FIT) image (presumably from the same device). This FIT image contains the U-Boot binary, the Device Tree Blob (DTB) and the OpenSBI binary. This image may be combined with the SPL.
- U-Boot loads the Linux Kernel.
In the case of the VisionFive 2/JH7110, the BROM is located on 32k of onboard (on the SoC) memory which may be seen on the SoC block diagram. The BROM is what it is: a ROM (Read Only Memory) so it is impossible to update it.
The VisionFive 2 BROM uses the state of the RGPIO pins on the board to determine which attached storage device (NVM) to load the the U-Boot Secondary Program Loader (U-Boot SPL) (u-boot-spl.bin.normal.out
) from. By default this is a partition with a GUID type code of 2E54B353-1271-4842-806F-E436D6AF6985
. The number of this partition is irrelevant, though it is typically partition 1.
In its default configuration, the U-Boot SPL then loads the U-Boot FIT image (u-boot.itb
) from partition 2 (CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_PARTITION=0x2
). When formatting this partition the recommended GUID type code is BC13C2FF-59E6-4262-A352-B275FD6F7172
.
The FIT image (u-boot.itb
) is a combination of OpenSBI's fw_dynamic.bin
, u-boot-nodtb.bin
and the device tree blob (jh7110-starfive-visionfive-2-v1.3b.dtb
or jh7110-starfive-visionfive-2-v1.2a.dtb
).
The VisionFive 2 has a two-switch RGPIO header on the board to select the device to load the firmware from. The configurations are as follows:
Option | RGPIO_0 | RGPIO_1 |
---|---|---|
Onboard QSPI Flash (factory setting) | 0 | 0 |
SDIO3.0 (TF / microSD card) | 1 | 0 |
eMMC | 0 | 1 |
UART | 1 | 1 |
First start
Serial connection
The only way to interact with the board is to use a serial link as its HDMI output is not functional (yet). The following is valid for board revision 1.3B.
Notice the RGPIO switches positions as the DIP switch has been mounted upside-down: both RGPIO switches are set in their right position (OFF) thus selecting the onboard SPI FLash as a boot device despite being pushed towards the ON label.
The StarFive VisionFive 2 supports a serial connection (3.3V) through its GPIO pins. Its UART is accessible via the pins 6 (GND), 8 (TX) and 10 (RX) so a crossed-over cable is required:
StarFive VisionFive 2 (1.3B) GPIO pin | Serial adapter pin |
---|---|
6 (Ground) | Ground |
8 (TX) | RX |
10 (RX) | TX |
Illustration for a 1.3B board revision with a connected USB adapter (from left to right its pins are GND - black wire, RXD - blue wire, TXD - violet wire, 3.3V + VCC - yellow jumper, 5V - not connected):
Install a communication program that can handle a serial port connection and transfer files over X-Modem or Y-Modem protocols like net-dialup/minicom (will be used for the rest of the explanations):
root #
emerge --ask net-dialup/minicom
Then launch Minicom by specifying the serial device to use (likely /dev/ttyUSB0
):
user $
minicom -D /dev/ttyUSB0 -b 115200
The serial port connection parameters should be set as being:
- Speed: 115200 bps
- Data: 8 bits
- Parity: None
- Stop bits: 1 bit
- No control flow (hardware and software)
To set those parameters directly in Minicom rather than on the command line, use the keys Ctrl+A then press Z then press O when the menu shows up.
Powering up
As the board's QSPI flash already contains a firmware (preloaded at factory), it is possible to use the board when unboxing it. QSPI flash is too small to contain a full-fledged OS nevertheless it is more than enough to contain an OpenSBI/U-Boot environment (comparable to OpenBOOT for SPARC machines users).
With the serial connection setup in Minicom and the serial cable connected to the StarFive VisionFive 2 UART pins, powering the board up should lead to the following display in Minicom:
U-Boot SPL 2021.10 (Sep 28 2024 - 01:14:56 +0800)
LPDDR4: 8G version: g8ad50857. Trying to boot from SPI OpenSBI v1.2 ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|___/_____| | | |_| Platform Name : StarFive VisionFive V2 Platform Features : medeleg Platform HART Count : 5 Platform IPI Device : aclint-mswi Platform Timer Device : aclint-mtimer @ 4000000Hz Platform Console Device : uart8250 Platform HSM Device : --- Platform PMU Device : --- Platform Reboot Device : pm-reset Platform Shutdown Device : pm-reset Platform Suspend Device : --- Firmware Base : 0x40000000 Firmware Size : 392 KB Firmware RW Offset : 0x40000 Runtime SBI Version : 1.0 Domain0 Name : root Domain0 Boot HART : 1 Domain0 HARTs : 0*,1*,2*,3*,4* Domain0 Region00 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: () Domain0 Region01 : 0x0000000040000000-0x000000004003ffff M: (R,X) S/U: () Domain0 Region02 : 0x0000000040040000-0x000000004007ffff M: (R,W) S/U: () Domain0 Region03 : 0x0000000000000000-0xffffffffffffffff M: (R,W,X) S/U: (R,W,X) Domain0 Next Address : 0x0000000040200000 Domain0 Next Arg1 : 0x0000000042200000 Domain0 Next Mode : S-mode Domain0 SysReset : yes Domain0 SysSuspend : yes Boot HART ID : 1 Boot HART Domain : root Boot HART Priv Version : v1.11 Boot HART Base ISA : rv64imafdcbx Boot HART ISA Extensions : none Boot HART PMP Count : 8 Boot HART PMP Granularity : 4096 Boot HART PMP Address Bits: 34 Boot HART MHPM Count : 2 Boot HART MIDELEG : 0x0000000000000222 Boot HART MEDELEG : 0x000000000000b109 U-Boot 2021.10 (Sep 28 2024 - 01:14:56 +0800), Build: jenkins-github_visionfive2_6.6-5 CPU: rv64imacu_zba_zbb Model: StarFive VisionFive V2 DRAM: 8 GiB MMC: sdio0@16010000: 0, sdio1@16020000: 1 Loading Environment from SPIFlash... SF: Detected gd25lq128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB OK StarFive EEPROM format v2 --------EEPROM INFO-------- Vendor : StarFive Technology Co., Ltd. Product full SN: VF7110B1-2318-D008E000-00000000 data version: 0x2 PCB revision: 0xb2 BOM revision: A Ethernet MAC0 address: 6c:cf:39:00:b0:16 Ethernet MAC1 address: 6c:cf:39:00:b0:17 --------EEPROM INFO-------- In: serial Out: serial Err: serial Model: StarFive VisionFive V2 Net: eth0: ethernet@16030000, eth1: ethernet@16040000 Hit any key to stop autoboot: 0 Card did not respond to voltage select! : -110 Card did not respond to voltage select! : -110 starfive_pcie pcie@2B000000: Port link up. starfive_pcie pcie@2B000000: Starfive PCIe bus probed. PCI: Failed autoconfig bar 10 starfive_pcie pcie@2C000000: Port link down. starfive_pcie pcie@2C000000: Starfive PCIe bus probed. PCI: Failed autoconfig bar 10 Device 0: unknown device Device 0: unknown device Tring booting distro ... Card did not respond to voltage select! : -110 Card did not respond to voltage select! : -110 Device 0: unknown device Device 0: unknown device StarFive #
So far so good, an U-Boot command prompt beginning with StarFive #
should show up at the end of the booting process. No worries for error messages like Card did not respond to voltage select! : -110 and Device 0: unknown device, they are normal at this point. Enter something like help
to get a list of what valid commands are under the U-Boot shell as a test:
StarFive #
help
? - alias for 'help' base - print or set address offset bdinfo - print Board Info structure blkcache - block cache diagnostics and control boot - boot default, i.e., run 'bootcmd' bootd - boot default, i.e., run 'bootcmd' bootefi - Boots an EFI payload from memory bootelf - Boot from an ELF image in memory booti - boot Linux kernel 'Image' format from memory bootm - boot application image from memory bootp - boot image via network using BOOTP/TFTP protocol bootvx - Boot vxWorks from an ELF image cmp - memory compare config - print .config coninfo - print console devices and information cp - memory copy cpu - display information about CPUs crc32 - checksum calculation dhcp - boot image via network using DHCP/TFTP protocol dm - Driver model low level access echo - echo args to console editenv - edit environment variable eeprom - EEPROM sub-system efidebug - Configure UEFI environment env - environment handling commands erase - erase FLASH memory eraseenv - erase environment variables from persistent storage exit - exit script ext2load - load binary file from a Ext2 filesystem (...) showvar - print local hushshell variables size - determine a file's size sleep - delay execution for some time source - run script from memory sysboot - command to get and boot from syslinux files test - minimal test like /bin/sh tftpboot - boot image via network using TFTP protocol tftpput - TFTP put command, for uploading files to a server true - do nothing, successfully unlz4 - lz4 uncompress a memory region unzip - unzip a memory region usb - USB sub-system usbboot - boot from USB device version - print monitor, compiler and linker version
Everything is in good working order at this point. The StarFive VisionFive 2 can be powered down by removing its USB-C power cable (safe operation, will not corrupt anything).
First aid kit
Nothing being perfect, let's take precautions. There is a recovery image available on GitHub that can be used if, for a reason or another, the onboard SPI Flash memory would become corrupt. If that would happen, the recovery image will enable you to start your StarFive VisionFive 2 and reflash it.
First things first, download the most recent recovery binary file on https://github.com/starfive-tech/Tools/tree/master/
recovery (that binary file is named jh7110-recovery-YYYYMMDD.bin
, e.g. jh7110-recovery-20230322.bin
).
With the StarFive VisionFive 2 board powered down, change the boot selector setting to make the board boot via its UART (i.e. serial link) rather than its SPI Flash memory (RGPIO_0 and RGPIO_1 must be both set to ON, i.e on their left position, GPIO header seen upper-left). Once the boot selector setting has been changed, power the board up again. At this point, Minicom should show a series of "C", meaning that the board is patiently waiting for its firmware image to be sent to it by Minicom via the X-Modem protocol:
U-Boot SPL 2021.10 (Sep 28 2024 - 01:14:56 +0800)
(C)StarFive CCCCCCCCCCCCCCCCCCCCC
Within Minicom, transfer the recovery binary file just downloaded from Github using the said X-Modem protocol (Ctrl+A then press S then select xmodem
). Once the recovery image has been transferred, the StarFive VisionFive 2 board will automatically boot on it and will present a menu from where an operation to perform can be chosen:
U-Boot SPL 2021.10 (Sep 28 2024 - 01:14:56 +0800)
(C)StarFive CCCCCCCCCCCCCCCCCCCCC JH7110 secondboot version: 230322-c514da9 CPU freq: 1250MHz idcode: 0x1860C8 ddr 0x00000000, 4M test ddr 0x00400000, 8M test DDR clk 2133M, size 8GB ********************************************************* ****************** JH7110 program tool ****************** ********************************************************* 0: update 2ndboot/SPL in flash 1: update 2ndboot/SPL in emmc 2: update fw_verif/uboot in flash 3: update fw_verif/uboot in emmc 4: update otp, caution!!!! 5: exit NOTE: current xmodem receive buff = 0x40000000, 'load 0x********' to change. select the function to test:
Do not use option 4 (OTP) unless being in good knowledge: an OTP is a One Time Programmable memory or WORM (Write-Once Read Many memory). Once an OTP memory has been programmed ("fused"), it cannot be programmed again unless using specialized equipment or procedures. OTP fuses dot not have a public documentation, see the following discussion thread: https://github.com/starfive-tech/Tools/issues/8
Options 0 and 2 can be used to reprogram the onboard SPI Flash memory to make it bootable again. Options 1 and 2 do exactly the same if you have plugged an eMMC memory module in (at the back of the board).
Updating the stock firmware preloaded at factory
Before attempting to install Gentoo onto the VisionFive 2 it is essential to update the firmware. Depending on the age of the firmware the current (modern) official images from StarFive may fail to load.
There are several methods for updating the firmware, including:
- Using the U-Boot console to retrieve and install firmware images via TFTP
- Invoking
flashcp
command (from sys-fs/mtd-utils) on a running system - Sending firmware binaries over a UART console interface
- Booting updated firmware off removable media
Path #3 will be taken here as it involves the least additional software and does not require a bootable board.
Updating U-Boot will likely reset the contents of U-Boot variables to their default value. If some of them were customized to have a fully working system it is highly advised to keep a trace of their current value before proceeding!
First, gather the following firmware files from the latest VisionFive2 software release:
Firmware file | Base address in SPI flash memory |
---|---|
u-boot-spl.bin.normal.out | 0x0 |
visionfive2_fw_payload.img | 0x100000 |
Second, calculate their CRC-32 with /usr/bin/crc32 (provided by dev-perl/Archive-Zip) on the host, results might defer fomr what is show here:
user $
crc32 u-boot-spl.bin.normal.out visionfive2_fw_payload.img
4f92e463 u-boot-spl.bin.normal.out 6949ec27 visionfive2_fw_payload.img
Third, ensure the StarFive VisionFive 2 is set up to boot off the onboard SPI Flash memory: RGPIO_0 and RGPIO_1 switches must, both, be set to their OFF position (i.e. on the right, having the GPIO header seen upper-left). Power the board on and the U-Boot command prompt should be reached after a few seconds. From that prompt there, run the following command to initialize a file transfer via the Y-Modem protocol (calculates a CRC for each chunk of transferred data to ensure no corruption has occurred):
StarFive #
loady
## Ready for binary (ymodem) download to 0x60000000 at 115200 bps... CCCCC
At this point, use Ctrl+A then S in Minicom to enter the file transfer menu, then select Ymodem, then a single firmware file of your choice (beginning with u-boot-spl.bin.normal.out or visionfive2_fw_payload.img is not important as long as both are processed) and wait for the transfer to complete.
StarFive #
loady
## Ready for binary (ymodem) download to 0x60000000 at 115200 bps... CC0(STX)/0(CAN) packets, 6 retries ## Total Size = 0x00025a30 = 154160 Bytes
At this point the firmware payload lies in board's RAM but remains to be written on the SPI flash memory. Before committing it to the SPI flash memory, check the firmware payload's CRC-32 as it is seen on the StarFive VisionFive 2 board (the result should match what has been computed on the host):
StarFive #
crc32 $loadaddr $filesize
$filesize is a shell variable automatically initialized when the transfer completes. A couple of shell variables like $loadaddr are preset at system startup. To see all defined variables use the shell command: printenv.
If the CRC-32 value calculated on the board matches the one calculated on the host, the firmware file contents can be comitted to the onboard SPI flash memory. Depending on which firmware file has been transferred, the command to use is:
- For u-boot-spl.bin.normal.out:
StarFive #
sf update $loadaddr 0x0 $filesize
- For visionfive2_fw_payload.img:
StarFive #
sf update $loadaddr 0x100000 $filesize
Repeat the operations above for the second firmware file. Once both firmware files have been transferred and committed to the onboard SPI flash memory, powercycle the board and the U-Boot command prompt should show up again after a couple of seconds. In the case some badluck would have happened making the board unable to boot on the onboard SPI flash memory, it is possible to recover from the situation using recovery boot image, see section First aid kit.
Prerequisites
The process to build a Gentoo stage 4 for the StarFive VisionFive 2 involve several "big" steps:
- Setting up QEMU to make it handle RISC-V binaries
- Generating a cross-build envoironment used to generate a Catalyst seed
- Using Catalyst to build a fully-functional RISC-V rootfs
- Chrooting into the RISC-V rootfs
- Customizing the RISC-V rootfs (here is where QEMU userland emulation appears in the landscape)
Configure QEMU-user and binfmt
To get the benefit of the QEMU/KVM speed-up, add the user (e.g. larry) under which target environment will be built to the kvm group: gpasswd -a larry kvm
QEMU should be configured to, at least, support:
- full-system emulation (i.e. QEMU_SOFTMMU_TARGETS): AMD64 (
x86_64
) and RISC-V (risc-v
) - userland emulation (i.e. QEMU_USER_TARGETS): RISC-V (
risc-v
)
The very first step is to check what lies inside /etc/portage/make.conf and add the required targets discussed herebefore if they were missing:
QEMU_SOFTMMU_TARGETS="riscv64 x86_64"
QEMU_USER_TARGETS="riscv64"
The next step is to ensure that app-emulation/qemu has, at least, the USE flag static-user enabled as anything outside the chroot will be inaccessible (QEMU binaries should be fully autonomous):
app-emulation/qemu static-user
Now, re-emerge app-emulation/qemu and package it to be able to be re-deployed inside the chrooted target environment:
root #
emerge --ask app-emulation/qemu
root #
quickpkg app-emulation/qemu
It is time to register the ELF/RISC-V signature to the miscellaneous binary formats known by the Linux kernel to make the kernel invoke the adequate QEMU userland emulator when needed:
root #
mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc
root #
echo ':riscv64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-riscv64:' > /etc/binfmt.d/qemu-riscv64-static.conf
Then restart binfmt support:
- OpenRC:
root #
rc-service binfmt restart
- Systemd:
root #
systemctl restart systemd-binfmt
At this point a pseudo-file called riscv64 should appear under /proc/sys/fs/binfmt_misc :
user $
ls -l /proc/sys/fs/binfmt_misc
total 0 --w------- 1 root root 0 Nov 12 10:13 register -rw-r--r-- 1 root root 0 Nov 12 10:13 riscv64 -rw-r--r-- 1 root root 0 Nov 6 10:57 status
The pseudo-file /proc/sys/fs/binfmt_misc/riscv64 contains: Template:CmdRoot
Generate a Cross Build Environment (crossdev)
Rather than using the (already outdated) version of GCC specified in the VisionFive 2 SDK crossdev may instead be used to to build an up-to-date cross-compiler from the Gentoo repository. This will then be used to bootstrap a Catalyst stage, build the kernel, and any required firmware binaries.
Install the sys-devel/crossdev package and generate a RISC-V cross toolchain (see Cross Build Environment for further information):
root #
emerge --ask sys-devel/crossdev
Create an ebuild repository for crossdev, preventing it from choosing a (seemingly) random repository to store its packages:
root #
mkdir -p /var/db/repos/crossdev/{profiles,metadata}
root #
echo 'crossdev' > /var/db/repos/crossdev/profiles/repo_name
root #
echo 'masters = gentoo' > /var/db/repos/crossdev/metadata/layout.conf
root #
chown -R portage:portage /var/db/repos/crossdev
If the Gentoo ebuild repository is synchronized using Git, or any other method with Manifest files that do not include checksums for ebuilds:
masters = gentoo
thin-manifests = true
Instruct Portage and crossdev to use this ebuild repository:
[crossdev]
location = /var/db/repos/crossdev
priority = 10
masters = gentoo
auto-sync = no
Then build the cross-build toolchain:
root #
crossdev --target riscv64-unknown-linux-gnu
Once crossdev has built the cross-toolchain, that latter will be installed to /usr/<target> (i.e. /usr/ in the present case). The cross-compiler may be used by prefixing the target to the command, e.g. riscv64-unknown-linux-gnu-gcc:
root #
riscv64-unknown-linux-gnu-gcc -v
Using built-in specs. COLLECT_GCC=riscv64-unknown-linux-gnu-gcc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/riscv64-unknown-linux-gnu/14/lto-wrapper Target: riscv64-unknown-linux-gnu Configured with: /var/tmp/portage/cross-riscv64-unknown-linux-gnu/gcc-14.2.1_p20241026/work/gcc-14-20241026/configure --host=x86_64-pc-linux-gnu --target=riscv64-unknown-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/riscv64-unknown-linux-gnu/gcc-bin/14 --includedir=/usr/lib/gcc/riscv64-unknown-linux-gnu/14/include --datadir=/usr/share/gcc-data/riscv64-unknown-linux-gnu/14 --mandir=/usr/share/gcc-data/riscv64-unknown-linux-gnu/14/man --infodir=/usr/share/gcc-data/riscv64-unknown-linux-gnu/14/info --with-gxx-include-dir=/usr/lib/gcc/riscv64-unknown-linux-gnu/14/include/g++-v14 --disable-silent-rules --disable-dependency-tracking --with-python-dir=/share/gcc-data/riscv64-unknown-linux-gnu/14/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --disable-libunwind-exceptions --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 14.2.1_p20241026 p3' --with-gcc-major-version-only --enable-libstdcxx-time --enable-lto --disable-libstdcxx-pch --enable-poison-system-directories --with-sysroot=/usr/riscv64-unknown-linux-gnu --disable-gcov --disable-bootstrap --enable-__cxa_atexit --enable-clocale=gnu --disable-multilib --disable-fixed-point --with-abi=lp64d --enable-libgomp --disable-libssp --disable-libada --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --with-zstd --without-isl --disable-libsanitizer --enable-default-pie --enable-host-pie --disable-host-bind-now --enable-default-ssp --disable-fixincludes Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 14.2.1 20241026 (Gentoo 14.2.1_p20241026 p3)
Building a system image
There are two philosophies when it comes to installing an operating system onto a SBC / embedded device such as the VisionFive 2.
The first involves writing a static system image, typically a squashfs, onto some form of media (typically eMMC, though NVMe is on the rise). The initramfs is able to load this image into RAM and use it as a rootfs; when an update is required the whole image is replaced as a single operation. There are advantages to this approach, particularly for embedded devices where users are not expected to update individual packages and recovery 'in-the-field' may be impractical, or to provide an A/B partition layout for updates. Systems configured in this way are also resilient when it comes to unexpected shutdowns as the only time that the rootfs storage volume is performing writes is when this image is being updated. This approach will be described as an 'embedded' installation going forward and called out where possible.
The second approach involves writing a rootfs onto some accessible storage media the device and using a package manager to install and update packages as required. For a Gentoo system this approach is more flexible and allows for a more traditional Linux experience, but requires more effort to set up and maintain. This approach will be described as a 'traditional' installation going forward and called out where possible.
The process of generating a system image for a Gentoo installation on the VisionFive 2 may be broadly described as follows:
- Gather the installation files and generate a cross-compiler
- Check out the VisionFive 2 SDK OR kernel sources (until the upstream kernel has full hardware support)
- Build a cross-compiler and use it to build a kernel, initramfs, and FIT image
- Generate a Gentoo rootfs
- Unpack and customize a Gentoo Stage 3 tarball to create a Gentoo rootfs
- Use Catalyst to generate an image from scratch
- Load the Flattened Image Tree (FIT) onto the device
- Write a Gentoo rootfs to the selected storage medium.
Generate a cross-compiler
A cross-compiler will typically also be required as the VisionFive 2 is a RISC-V device and most users will be running an x86_64 (amd64) host.
Check out the VisionFive 2 BSP
See https://rvspace.org/en/project/JH7110_Upstream_Plan to get the upstreaming process status.
The VisionFive 2 Board Support Package (BSP) (sometimes called an SDK by manufacturers) is a git repository containing a collection of scripts (and git submodules) that may be used to bootstrap a cross-compiler and build a Linux kernel, initramfs, rootfs, U-Boot, and FIT image for the VisionFive 2.
As most consumers of this article already use Gentoo, this step is not essential; Gentoo users have the option of using crossdev (from sys-devel/crossdev) to get an up-to-date riscv64 cross-compiler with which to build the kernel, initramfs, and FIT image. However, the SDK is still useful: it provides a convenient way to build U-Boot and a rootfs, contains a great deal of information about how the developers intend for images to be written to the device, and, for inexperienced embedded users, provides a way to generate a guaranteed bootable image and a less-complex approach which be preferable.
The VisionFive 2 BSP repository is available at: https://github.com/starfive-tech/VisionFive2
Use the following commands to check out the repository:
user $
git clone https://github.com/starfive-tech/VisionFive2.git
Cloning into 'VisionFive2'... remote: Enumerating objects: 4479, done. remote: Counting objects: 100% (603/603), done. remote: Compressing objects: 100% (311/311), done. remote: Total 4479 (delta 331), reused 552 (delta 292), pack-reused 3876 Receiving objects: 100% (4479/4479), 290.58 MiB | 6.31 MiB/s, done. Resolving deltas: 100% (2457/2457), done.
user $
cd VisionFive2
user $
git submodule update --init --recursive
Submodule 'buildroot' (https://github.com/starfive-tech/buildroot.git) registered for path 'buildroot' Submodule 'linux' (https://github.com/starfive-tech/linux.git) registered for path 'linux' Submodule 'opensbi' (https://github.com/starfive-tech/opensbi.git) registered for path 'opensbi' Submodule 'soft_3rdpart' (https://github.com/starfive-tech/soft_3rdpart.git) registered for path 'soft_3rdpart' Submodule 'u-boot' (https://github.com/starfive-tech/u-boot.git) registered for path 'u-boot' Cloning into '/data/development/visionfive/VisionFive2/buildroot'... Cloning into '/data/development/visionfive/VisionFive2/linux'... Cloning into '/data/development/visionfive/VisionFive2/opensbi'... Cloning into '/data/development/visionfive/VisionFive2/soft_3rdpart'... Cloning into '/data/development/visionfive/VisionFive2/u-boot'... Submodule path 'buildroot': checked out '762ee9bc4e1fbdaf09675acaed9516d6c136d5b1' Submodule path 'linux': checked out 'a87c6861c6d96621026ee53b94f081a1a00a4cc7' Submodule path 'opensbi': checked out 'c6a092cd80112529cb2e92e180767ff5341b22a3' Submodule path 'soft_3rdpart': checked out 'cd7b50cd9f9eca66c23ebd19f06a172ce0be591f' Submodule path 'u-boot': checked out '688befadf1d337dee3593e6cc0fe1c737cc150bd'
Use the BSP to build everything
For inexperienced embedded developers it may be desirable to use the VisionFive 2 SDK to build the kernel, initramfs, U-Boot, etc. This is not the recommended approach as the BSP uses outdated dependencies that include a number of bugs and issues that have been fixed in the upstream components.
Use the SDK's build scripts to build a cross-compiler, kernel, initramfs, U-Boot, etc.
user $
make -j$(nproc)
The output of this process (kernel, initramfs, device tree blobs [dtb], OR the FIT image containing them) may be used to boot a Gentoo rootfs; if this is all that is desired there is no need to build custom Gentoo versions.
Use crossdev
Rather than using the (already outdated) version of GCC specified in the VisionFive 2 SDK crossdev may instead be used to to build an up-to-date cross-compiler from the Gentoo repository. This will then be used to bootstrap a Catalyst stage, build the kernel, and any required firmware binaries.
Install the sys-devel/crossdev package and generate a RISC-V cross toolchain (see Cross Build Environment for further information):
root #
emerge --ask sys-devel/crossdev
Create an ebuild repository for crossdev, preventing it from choosing a (seemingly) random repository to store its packages:
root #
mkdir -p /var/db/repos/crossdev/{profiles,metadata}
root #
echo 'crossdev' > /var/db/repos/crossdev/profiles/repo_name
root #
echo 'masters = gentoo' > /var/db/repos/crossdev/metadata/layout.conf
root #
chown -R portage:portage /var/db/repos/crossdev
If the Gentoo ebuild repository is synchronized using Git, or any other method with Manifest files that do not include checksums for ebuilds:
masters = gentoo
thin-manifests = true
Instruct Portage and crossdev to use this ebuild repository:
[crossdev]
location = /var/db/repos/crossdev
priority = 10
masters = gentoo
auto-sync = no
root #
crossdev --target riscv64-unknown-linux-gnu
Once crossdev has built the cross-toolchain it will be installed to /usr/<target>. The cross-compiler may be used by prefixing the target to the command, e.g.
user $
riscv64-unknown-linux-gnu-gcc
Until the VisionFive 2 is fully supported by the upstream Linux Kernel it is necessary to use the VisionFive 2 kernel fork. Clone the repository:
user $
git clone https://github.com/starfive-tech/linux.git
user $
cd linux
user $
git checkout tags/VF2_v2.11.5
Build the kernel
In order for the VisionFive 2 to run Linux a Linux Kernel is required. This is built using the cross-compiler provided by crossdev.
It's recommended to use GCC 12 to build the VisionFive 2's 5.15.0 kernel
CONFIG_SECCOMP=y is not set in the defconfig and should be manually enabled if packages such as net-libs/webkit-gtk will be installed with USE=
seccomp
.There are some out-of-tree kernel modules (jpu/venc/vdec) that should be installed in addition to this kernel. These modules are available from the VisionFive 2 soft_3rdpart repo, or by emerging media-video/vf2vpudev from the bingch overlay
user $
cd linux
user $
HWBOARD_FLAG=HWBOARD_VISIONFIVE2 ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- make starfive_visionfive2_defconfig
user $
HWBOARD_FLAG=HWBOARD_VISIONFIVE2 ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- make menuconfig
user $
HWBOARD_FLAG=HWBOARD_VISIONFIVE2 ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- make -j$(nproc) vmlinux all modules
On such error: use of enum ‘gsi_iterator_update’ without previous declaration
Just use this patch: https://lore.kernel.org/lkml/DB6P189MB05681E9F4785DF2758B9875B9CA49@DB6P189MB0568.EURP189.PROD.OUTLOOK.COM/t/
Skipping ahead a bit, once a rootfs is available, install the kernel modules to the rootfs:
root #
HWBOARD_FLAG=HWBOARD_VISIONFIVE2 ARCH=riscv INSTALL_MOD_PATH=/path/to/rootfs make modules_install
Once a Gentoo initramfs has been generated, mkimage
from dev-embedded/u-boot-tools may be used to consolidate the kernel, initramfs, and dtb into a single file that U-Boot can load. This is not required; the kernel, initramfs, and dtb may also be loaded separately by U-Boot.
root #
echo "sys-apps/dtc yaml" > /etc/portage/package.use/device-tree-compiler
root #
emerge --ask sys-apps/dtc dev-embedded/u-boot-tools
Update the paths of the kernel, initramfs, and dtb in the
.its
file before running mkimage.user $
curl https://raw.githubusercontent.com/starfive-tech/VisionFive2/JH7110_VisionFive2_devel/conf/visionfive2-fit-image.its -O
user $
mkimage -f visionfive2-fit-image.its -A riscv -O linux -T flat_dt gentoo.fit
Root filesystem
There are several methods that may be used to create or obtain a Gentoo root filesystem for the JH7110 SoC. The simplest method is simply downloading and unpacking a stage3 tarball from https://www.gentoo.org/downloads/, if a suitable one is available:
root #
mkdir rootfs
root #
tar xpvf stage3-*.tar.xz --xattrs-include='*.*' --numeric-owner -C rootfs
This example will assume that this has not been selected and instead Catalyst will be used to generate an appropriate stage3 tarball from scratch. If using an upstream stage3 tarball is desired, skip ahead to customising the rootfs.
It is possible directly use the crossdev root under /usr to build a rootfs; it is broadly similar to using Catalyst however instead of generating a seed tarball the rootfs is built by chrooting directly into the crossdev root. There are some advantages to this approach, particularly that it is possible to save a significant amount of time.
While this may be faster for quick development or building for a single device, if intending to target multiple configurations it is usually better to use Catalyst as a generic stage 1 image may be created and used as the base for multiple stage 3 images. Using Catalyst also provides better isolation between the host and target systems.
Build a seed tarball
To create a stage3 tarball, Calalyst requires a seed tarball. Catalyst will chroot into the seed and emerge packages for the new stage to ensure that packages generated for stage tarballs are isolated from the host system.
This example will build a seed tarball from scratch; an appropriate stage3 tarball from upstream may be placed in /var/tmp/catalyst/builds/default and used instead.
Set the system profile
root #
PORTAGE_CONFIGROOT=/usr/riscv64-unknown-linux-gnu eselect profile list
Available profile symlink targets: [1] default/linux/riscv/20.0/rv64gc/lp64d (stable) [2] default/linux/riscv/20.0/rv64gc/lp64d/desktop (dev) [3] default/linux/riscv/20.0/rv64gc/lp64d/desktop/gnome (dev) [4] default/linux/riscv/20.0/rv64gc/lp64d/desktop/gnome/systemd (dev) [5] default/linux/riscv/20.0/rv64gc/lp64d/desktop/gnome/systemd/merged-usr (dev) [6] default/linux/riscv/20.0/rv64gc/lp64d/desktop/plasma (dev) [7] default/linux/riscv/20.0/rv64gc/lp64d/desktop/plasma/systemd (dev) [8] default/linux/riscv/20.0/rv64gc/lp64d/desktop/plasma/systemd/merged-usr (dev) [9] default/linux/riscv/20.0/rv64gc/lp64d/desktop/systemd (dev) [10] default/linux/riscv/20.0/rv64gc/lp64d/desktop/systemd/merged-usr (dev) [11] default/linux/riscv/20.0/rv64gc/lp64d/systemd (stable) [12] default/linux/riscv/20.0/rv64gc/lp64d/systemd/merged-usr (stable) [13] default/linux/riscv/20.0/rv64gc/lp64 (stable) [14] default/linux/riscv/20.0/rv64gc/lp64/desktop (dev) [15] default/linux/riscv/20.0/rv64gc/lp64/desktop/gnome (dev) [16] default/linux/riscv/20.0/rv64gc/lp64/desktop/gnome/systemd (dev) [17] default/linux/riscv/20.0/rv64gc/lp64/desktop/gnome/systemd/merged-usr (dev) [18] default/linux/riscv/20.0/rv64gc/lp64/desktop/plasma (dev) [19] default/linux/riscv/20.0/rv64gc/lp64/desktop/plasma/systemd (dev) [20] default/linux/riscv/20.0/rv64gc/lp64/desktop/plasma/systemd/merged-usr (dev) [21] default/linux/riscv/20.0/rv64gc/lp64/desktop/systemd (dev) [22] default/linux/riscv/20.0/rv64gc/lp64/desktop/systemd/merged-usr (dev) [23] default/linux/riscv/20.0/rv64gc/lp64/systemd (stable) [24] default/linux/riscv/20.0/rv64gc/lp64/systemd/merged-usr (stable) [25] default/linux/riscv/20.0/rv64gc/multilib (exp) [26] default/linux/riscv/20.0/rv64gc/multilib/systemd (exp) [27] default/linux/riscv/20.0/rv64gc/multilib/systemd/merged-usr (exp) [28] default/linux/riscv/20.0/rv64gc/lp64d/musl (dev) [29] default/linux/riscv/20.0/rv64gc/lp64/musl (dev)
root #
PORTAGE_CONFIGROOT=/usr/riscv64-unknown-linux-gnu eselect profile set 10
As of profile 23.0, the equivalent profile is 35:
default/linux/riscv/23.0/rv64/lp64d/systemd (stable)
as documented in the 23.0 Profile update table.If a profile marked experimental (exp) is desired, use the --force
flag to enable the profile.
To prevent errors from occurring while building the seed, the following USE flags should be set to prevent conflicts over the default su provider:
root #
mkdir /usr/riscv64-unknown-linux-gnu/etc/portage/package.use
root #
echo "sys-apps/util-linux -su" > /usr/riscv64-unknown-linux-gnu/etc/portage/package.use/system
or
root #
sed -i -e "s:-pam::" /usr/riscv64-unknown-linux-gnu/etc/portage/make.conf
Emerge the system:
root #
riscv64-unknown-linux-gnu-emerge -va1 @system --keep-going
At this point in the process, the Catalyst stage generation may be skipped and instead the system may be built by chrooting into the crossdev environment.
Create a seed tarball:
root #
cd /usr/riscv64-unknown-linux-gnu/
root #
tar -cvJf /tmp/riscv64-glibc-seed.tar.xz *
Catalyst
Install catalyst:
root #
emerge --ask dev-util/catalyst
As of July 2023 Catalyst is only supported as a live ebuild release, See the Catalyst article for more information.
Create a Catalyst work directory, move the seed tarball to Catalyst's workdir, and build a Portage snapshot:
root #
mkdir -p /var/tmp/catalyst/builds
root #
mv /tmp/riscv64-glibc-seed.tar.xz /var/tmp/catalyst/builds/
root #
emerge --sync
root #
mkdir -p /var/tmp/catalyst/repos; pushd /var/tmp/catalyst/repos/
root #
git clone --mirror /var/db/repos/gentoo
root #
popd
root #
catalyst --snapshot stable
18 May 2023 10:31:46 AEST: NOTICE : Loading configuration file: /etc/catalyst/catalyst.conf NOTICE:catalyst:Loading configuration file: /etc/catalyst/catalyst.conf 18 May 2023 10:31:46 AEST: NOTICE : conf_values[options] = ['autoresume', 'bindist', 'kerncache', 'pkgcache', 'seedcache'] NOTICE:catalyst:conf_values[options] = ['autoresume', 'bindist', 'kerncache', 'pkgcache', 'seedcache'] 18 May 2023 10:31:46 AEST: NOTICE : >>> /usr/bin/git -C /var/tmp/catalyst/repos/gentoo.git fetch --quiet --depth=1 NOTICE:catalyst:>>> /usr/bin/git -C /var/tmp/catalyst/repos/gentoo.git fetch --quiet --depth=1 18 May 2023 10:31:46 AEST: NOTICE : >>> /usr/bin/git -C /var/tmp/catalyst/repos/gentoo.git update-ref HEAD FETCH_HEAD NOTICE:catalyst:>>> /usr/bin/git -C /var/tmp/catalyst/repos/gentoo.git update-ref HEAD FETCH_HEAD 18 May 2023 10:31:46 AEST: NOTICE : >>> /usr/bin/git -C /var/tmp/catalyst/repos/gentoo.git gc --quiet NOTICE:catalyst:>>> /usr/bin/git -C /var/tmp/catalyst/repos/gentoo.git gc --quiet 18 May 2023 10:31:47 AEST: NOTICE : Creating gentoo tree snapshot afe106ae95ed7ba6536c870774c1b7e62d940ebd from /var/tmp/catalyst/repos/gentoo.git NOTICE:catalyst:Creating gentoo tree snapshot afe106ae95ed7ba6536c870774c1b7e62d940ebd from /var/tmp/catalyst/repos/gentoo.git 18 May 2023 10:31:47 AEST: NOTICE : >>> /usr/bin/git -C /var/tmp/catalyst/repos/gentoo.git archive --format=tar afe106ae95ed7ba6536c870774c1b7e62d940ebd | NOTICE:catalyst:>>> /usr/bin/git -C /var/tmp/catalyst/repos/gentoo.git archive --format=tar afe106ae95ed7ba6536c870774c1b7e62d940ebd | 18 May 2023 10:31:47 AEST: NOTICE : /usr/bin/tar2sqfs /var/tmp/catalyst/snapshots/gentoo-afe106ae95ed7ba6536c870774c1b7e62d940ebd.sqfs -q -f -j1 -c gzip NOTICE:catalyst: /usr/bin/tar2sqfs /var/tmp/catalyst/snapshots/gentoo-afe106ae95ed7ba6536c870774c1b7e62d940ebd.sqfs -q -f -j1 -c gzip 18 May 2023 10:31:55 AEST: NOTICE : Wrote snapshot to /var/tmp/catalyst/snapshots/gentoo- .sqfs NOTICE:catalyst:Wrote snapshot to /var/tmp/catalyst/snapshots/gentoo-afe106ae95ed7ba6536c870774c1b7e62d940ebd.sqfs
Create the Catalyst spec files that match the desired stage type:
Replace afe106ae95ed7ba6536c870774c1b7e62d940ebd
in snapshot_treeish with the commit id that was given when running catalyst -s stable
root #
cd /var/tmp/catalyst
subarch: rv64_lp64d
target: stage1
version_stamp: systemd-mergedusr-20230518
interpreter: /usr/bin/qemu-riscv64
rel_type: default
profile: default/linux/riscv/20.0/rv64gc/lp64/systemd/merged-usr
snapshot_treeish: afe106ae95ed7ba6536c870774c1b7e62d940ebd
source_subpath: riscv64-glibc-seed
compression_mode: pixz
decompressor_search_order: xz bzip2
update_seed: yes
update_seed_command: -uDN @world
subarch: rv64_lp64d
target: stage3
version_stamp: systemd-mergedusr-20230518
interpreter: /usr/bin/qemu-riscv64
rel_type: default
profile: default/linux/riscv/20.0/rv64gc/lp64/systemd/merged-usr
snapshot_treeish: afe106ae95ed7ba6536c870774c1b7e62d940ebd
source_subpath: default/stage1-rv64_lp64d_systemd-mergedusr-20230518.tar.xz
compression_mode: pixz
decompressor_search_order: xz bzip2
Finally, using Catalyst, build a Stage 1 image from the seed tarball, and a Stage 3 image from the Stage 1 image:
root #
catalyst -f stage1-riscv64-systemd-mergedusr.spec
root #
catalyst -f stage3-riscv64-systemd-mergedusr.spec
If @system fails to build try checking out the releng repo and setting the portage_confdir variable to its location.
root #
git clone -o upstream https://github.com/gentoo/releng.git
root #
echo "portage_confdir = /path/to/releng/releases/portage/stages-qemu" >> /var/tmp/catalyst/builds/default/stage1-riscv64-musl-openrc.spec
This ensures that the most up-to-date portage configuration is available to the build process.
If this fails, and a suitable stage 3 image is available, try using that as the seed. If that fails, or is unavailable, ask for support in#gentoo-releng
.If a stage successfully builds the output will be located at /var/tmp/catalyst/builds/default.
Customise the RootFS
Once a a stage 3 image has been obtained or constructed the next task is personalisation of the root filesystem. This involves mounting the root filesystem, followed by executing a chroot command to enter it. For the purpose of the following commands, it is assumed that the root filesystem is stored at /var/tmp/catalyst/builds/default/stage3-riscv64-systemd-mergedusr-20230518.tar.xz.
root #
mkdir rootfs
root #
tar xpvf /var/tmp/catalyst/builds/default/stage3-*.tar.xz --xattrs-include='*.*' --numeric-owner -C rootfs
Emerge QEMU into the target and chroot; customise the system image (at least set the root password and add a regular user). Proceed with a typical Stage 3 configuration outside of Kernel and Bootloader.
root #
mount --bind rootfs rootfs
root #
cd rootfs
root #
ROOT=$PWD/ emerge --usepkgonly --oneshot --nodeps qemu
root #
mount --bind /proc proc
root #
mount --bind /sys sys
root #
mount --bind /dev dev
root #
mount --bind /dev/pts dev/pts
root #
mkdir -p var/db/repos/gentoo
root #
mount --bind /var/db/repos/gentoo var/db/repos/gentoo
root #
mkdir -p usr/src/linux
root #
mount --bind ../linux usr/src/linux
root #
chroot . /bin/bash --login
To enable accelerated graphics, add the bingch ebuild repository to utilise its custom ebuilds for the Imagination BXE-4-32 GPU.
root #
eselect repository add bingch git https://gitlab.com/bingch/gentoo-overlay.git
root #
emerge --sync
root #
echo 'media-libs/mesa::bingch vulkan' > /etc/portage/package.use/powervr_graphics
root #
echo 'VIDEO_CARDS="imagination"' >> /etc/portage/make.conf
root #
echo 'media-libs/img-gpu-powervr-bin restricted' > /etc/portage/package.license/powervr-gpu-blobs
LLVM 15 is required to emerge bingch's customized mesa, (or USE=
"-llvm"
which results in a slower swrast which is not ideal for X11); until the Imagination BXE-4-32 is well supported upstream it is unlikely that ::gentoo will contain a version of LLVM that can build the older Mesa releases.root #
emerge --ask media-libs/mesa::bingch
Install the kernel and modules to the rootfs image:
root #
pushd /usr/src/linux
root #
make install && make modules_install
Generate an initramfs from within the chroot using your rootfs and a tool like Dracut:
root #
dracut --kver 5.15.0
Finally, exit the chroot and recursively unmount it:
root #
exit
root #
umount -R rootfs
Bootloader configuration
There are several methods of of configuring the U-Boot bootloader, each with their own advantages and disadvantages.
An external extlinux configuration file (also known as U-Boot Standard Boot) to determine bootup parameters. This enables U-Boot to load a dynamic configuation from the disk, enabling the user to amend the bootloader configuration without requiring changes to the U-Boot environment.
U-boot is capable of reading from ext2, ext4 and FAT filesystems; each of these filesystems may be used to store the kernel and extlinux/syslinux configuration file.
Regardless of the choice of filesystem, each of these files must be located in a directory called /boot on the root of the partition that it is located. This means that the kernel and extlinuxconfiguration file must be located at /boot/Image and /boot/extlinux/extlinux.conf respectively.
Within the unpacked rootfs, with the kernel at /boot/Image, create a file at /boot/extlinux/extlinux.conf that looks similar to this:
label default
linux /Image
append root=/dev/mmcblk0p2 rootwait console=ttyS0,115200 earlycon=sbi debug
The extlinux configuration file does not support FIT images.
The bootloader may also be configured by saving the U-Boot environment. This method of configuring the bootloader offers the most control, but requires that the user has a good understanding of the U-Boot environment variables and the boot process.
Imaging the device
There are several ways to get an image onto the device with the most straightforward method being simply writing an image to a TF (MicroSD) card. The VisionFive 2 has a built-in TF card reader, so this is a matter of imaging the card, inserting the card, selecting the appropriate boot option via the RGPIO switches, and powering on the device.
An alternate method where an image is loaded via TFTP will be described here.
The following files may required to boot the VF2 board, depending on the selected boot method:
├── visionfive2_fw_payload.img ├── image.fit ├── initramfs.cpio.gz ├── u-boot-spl.bin.normal.out ├── linux/arch/riscv/boot ├── dts │ └── starfive │ ├── jh7110-visionfive-v2-ac108.dtb │ ├── jh7110-visionfive-v2.dtb │ ├── jh7110-visionfive-v2-wm8960.dtb │ ├── vf2-overlay │ │ └── vf2-overlay-uart3-i2c.dtbo └── Image.gz
A note about firmware
While the process of imaging the SPI NAND was documented earlier in this guide, it is not the only location that firmware may be loaded from. Depending on the configuration of the RGPIO switches (and saved or compiled-in U-Boot configuration) the VF2 device may attempt to load firmware from partitions on a variety of block devices or over UART.
M.2 installation using a single rootfs partition
This example utilises the QSPI NAND to store the firmware and U-Boot environment alongside with a single rootfs partition on the M.2 NVMe SSD, used to store the rootfs with a FIT image being stored (and updated if necessary) under /boot. This configuration is broadly analogous to booting a legacy desktop with a monolithic partition and GRUB or LILO installed to the MBR with some additional, manual, bootloader configuration.
It is not a requirement to store the FIT image on the same block device as the rootfs, however it is often convenient to do so. A potential downside to this approach is that U-Boot only knows how to read ext2, ext4 and FAT file systems which restricts the choice of rootfs. It is entirely possible to make /boot another partition and store the FIT image elsewhere. If this option is selected ensure that partition numbers and device names are updated (or, ideally, use UUIDs for the rootfs in the kernel cmdline).
It is assumed that the user has some familiarity with running a TFTP server such as net-ftp/tftp-hpa.
To begin, validate the produced image.fit
. Set environment parameters, load the image from the TFTP sever into memory, and boot it:
StarFive #
setenv bootfile vmlinuz; setenv fdtcontroladdr 0xffffffffffffffff; setenv serverip 192.168.1.x; setenv ipaddr 192.168.1.x;
StarFive #
tftpboot ${loadaddr} ${serverip}:image.fit;
StarFive #
bootm start ${loadaddr};bootm loados ${loadaddr};run chipa_set_linux;run cpu_vol_set;booti 0x40200000 0x46100000:${filesize} 0x46000000
## Loading kernel from FIT Image at a0000000 ... Using 'config-1' configuration Trying 'vmlinux' kernel subimage Description: vmlinux Type: Kernel Image Compression: uncompressed Data Start: 0xa00000c8 Data Size: 24161280 Bytes = 23 MiB Architecture: RISC-V OS: Linux Load Address: 0x40200000 Entry Point: 0x40200000 Verifying Hash Integrity ... OK ## Loading fdt from FIT Image at a0000000 ... Using 'config-1' configuration Trying 'fdt' fdt subimage Description: unavailable Type: Flat Device Tree Compression: uncompressed Data Start: 0xa7f57d50 Data Size: 48366 Bytes = 47.2 KiB Architecture: RISC-V Load Address: 0x46000000 Hash algo: sha256 Hash value: dae1bdb73c5a4806cc8ff17df2552c3152c5d858e76f89212a3de4714e63e40b Verifying Hash Integrity ... sha256+ OK Loading fdt from 0xa7f57d50 to 0x46000000 Booting using the fdt blob at 0x46000000 ## Loading loadables from FIT Image at a0000000 ... Trying 'ramdisk' loadables subimage Description: buildroot initramfs Type: RAMDisk Image Compression: uncompressed Data Start: 0xa170ad7c Data Size: 109367045 Bytes = 104.3 MiB Architecture: RISC-V OS: Linux Load Address: 0x46100000 Entry Point: unavailable Hash algo: sha256 Hash value: 9b07bf94f17c3ae38607e09944ebd036c962e91d731d2b81231088a7e56ad46e Verifying Hash Integrity ... sha256+ OK Loading loadables from 0xa170ad7c to 0x46100000 Loading Kernel Image ## Flattened Device Tree blob at 46000000 Booting using the fdt blob at 0x46000000 Using Device Tree in place at 0000000046000000, end 000000004600eced Starting kernel ...
If the device successfully boots using the generated FIT image it may be used to install the rootfs onto attached storage.
Use mkstage4 to generate a rootfs tarball:
root #
pushd rootfs
root #
mkstage4 -C gz -t $(pwd) ../work/visionfive2-rootfs
root #
popd
Using the initramfs UART console, partition the NVMe device, set an IP, and copy the rootfs tarball to the device:
#
gdisk /dev/nvme0n1
#
mkdir /mnt/gentoo
#
ip addr add 192.168.1.x/24 dev eth1
#
ip route add default via 192.168.1.1 dev eth1
#
scp larry@buildhost:visionfive2-rootfs.tar.gz /mnt/gentoo/
Unpack the tarball and configure the stage3 as usual. Copy image.fit to /boot. The FIT image produced by the VF2 SDK may be used to boot the Gentoo installation however, if desired, a new FIT image may be generated using the Gentoo kernel and device tree.
#
tar -xpf visionfive2-rootfs.tar.gz -C /mnt/gentoo
It is recommended that the dhcpcd
(or an alternative DHCP client) and sshd
services be enabled at this time.
From a U-Boot console, scan and identify the NVMe device:
StarFive #
nvme scan
StarFive #
nvme info
Device 0: Vendor: 0x1179 Rev: 10604107 Prod: Y1BFC3Z2F8R3 Type: Hard Disk Capacity: 976762.3 MB = 953.8 GB (2000409264 x 512)
As U-Boot is able to see the device, configure the bootloader to add an NVMe boot target.
It doesn't appear to be possible to load a FIT image using a sysboot/extlinux/standard boot configuration. It would be ideal for this to be configured dynamically rather than requiring a U-Boot configuration change if (e.g.) the kernel cmdline needs to be updated. Investigate this further and add to the wiki!
First, load the image manually from disk:
StarFive #
setenv bootfile vmlinuz; setenv fileaddr a0000000; setenv fdtcontroladdr 0xffffffffffffffff; nvme scan; ext4load nvme 0:1 $fileaddr boot/image.fit
StarFive #
bootm start ${fileaddr};bootm loados ${fileaddr};run chipa_set_linux;run cpu_vol_set;booti 0x40200000 0x46100000:${filesize} 0x46000000
To debug a failing boot, try a combination of the following additional kernel cmdline options:
- PID1 fails to start:
systemd.log_level=debug systemd.log_target=console
- Essential service fails to start after control handed to systemd:
systemd.journald.forward_to_console=1
The 'debug' option may be removed from the default 'bootargs' environment setting if desired, once the system is booting successfully.
Once safe commands for booting the system are identified they need to be saved to the SPI flash as macros that can be easily run; several variables will be saved to make the boot process easier to parse and debug. It is important to note that any lines containing other macros or environment variables must be enclosed in '
to ensure that they are expanded correctly at runtime. The following commands may be used to save the boot commands to the SPI flash:
StarFive #
setenv nvme_bootargs 'setenv bootargs ${bootargs} root=/dev/nvme0n1p1 rw'
StarFive #
setenv nvme_fitload 'setenv bootfile vmlinuz; setenv fdtcontroladdr 0xffffffffffffffff; nvme scan; ext4load nvme 0:1 $loadaddr boot/gentoo.fit'
StarFive #
setenv nvme_bootfit 'bootm start ${loadaddr};bootm loados ${loadaddr};run chipa_set_linux;run cpu_vol_set;booti 0x40200000 0x46100000:${filesize} 0x46000000'
StarFive #
setenv bootcmd_nvme0 'run nvme_bootargs nvme_fitload nvme_bootfit'
StarFive #
setenv boot_targets 'mmc0 dhcp nvme0'
StarFive #
saveenv
StarFive #
reset
Saving Environment to SPI Flash... Erasing SPI flash...Writing to SPI flash...done
Upon the next boot the device will load the kernel and initramfs from the NVMe device without interaction.
TF Card/eMMC
In this example the BROM is instructed to load firmware from an attached TF/eMMC device. It is very similar to the the way that aarch64 devices are imaged due to the similar boot mechanisms. The TF card will be partitioned using GPT.
An image will be generated manually on-disk which may then be installed onto the TF card using tools such as dd.
Create an image and mount it on an available loop device:
user $
fallocate -l 8G visionfive2.img
user $
doas losetup --find --show visionfive2.img
The sectors used below are pulled from the VF2 BSP image generation script. The numbers assume a 512 Byte sector size with partitions aligned to 2048-sector boundaries. It is possible to use a different sector size, refer to the earlier guidance around the SoC's boot sequence.
root #
sgdisk --clear --set-alignment=2 \
--new=1:4096:8191 --change-name=1:"spl" --typecode=1:2E54B353-1271-4842-806F-E436D6AF6985 \
--new=2:8192:16383 --change-name=2:"uboot" --typecode=2:5B193300-FC78-40CD-8002-E86C45580B47 \
--new=3:16385:614399 --change-name=3:"image" --typecode=3:EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 \
--new=4:614400:0 --change-name=4:"root" --typecode=4:0FC63DAF-8483-4772-8E79-3D69D8477DE4 \
/dev/loop0
Creating new GPT entries in memory. Warning: The kernel is still using the old partition table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) The operation has completed successfully.
If the TF card is larger than the size of the disk image (which is recommended; it saves a lot of time when imaging) resize2fs or a similar utility depending on file system may be used to resize the rootfs partition later.
Write the SPL and U-Boot
It is possible to combine the SPL and U-Boot into a single image FIT image. This will require a different partition layout but should otherwise be similar to this example. This is not covered here, but should be. Please help out by updating the wiki!
The SPL and U-Boot may be written to the image using dd.
root #
dd if=u-boot/spl/u-boot-spl.bin of=/dev/loop0p1
root #
dd if=u-boot/u-boot.itb of=/dev/loop0p2
Format and mount the boot and root partitions
The boot partition may be formatted with mkfs.fat, and the root partition with mkfs.ext4. The following commands may be used to format and mount the partitions:
root #
mkfs.fat -F 32 -n boot /dev/loop0p3
root #
mkfs.ext4 -L root /dev/loop0p4
For a more conventional /boot layout, make /boot a symlink to /mnt/sdcard/boot and mount sdcard at /mnt/sdcard.
root #
mkdir -p /mnt/visionfive2
root #
mount /dev/loop0p4 /mnt/visionfive2
root #
cp -a rootfs/* /mnt/visionfive2
root #
mkdir -p /mnt/visionfive2/mnt/sdcard
root #
mount /dev/loop0p3 /mnt/visionfive2/mnt/sdcard
root #
mv /mnt/visionfive2/boot /mnt/visionfive2/mnt/sdcard/
root #
ln -s /mnt/sdcard/boot /mnt/visionfive2/boot
Edit fstab to cater for any file systems that need to be mounted on the device, then unmount the image:
root #
umount -R /mnt/visionfive2/mnt/sdcard
root #
sync
root #
losetup -d /dev/loop0
Write the image to a TF card
There are many methods available to do this. sys-boot/etcher-bin is a common method for those that want an easy-to-use GUI, however this example will use dd:
root #
dd if=visionfive2.img | pv | dd of=/dev/mmcblk0 bs=4M
Move the secondary GPT to the actual end of the disk, delete the last partition then recreate it, and finally resize the file system:
root #
sgdisk -e /dev/mmcblk0
root #
sgdisk -d 4 /dev/mmcblk0
root #
sgdisk --new=4:614400:0 --change-name=4:"root" --typecode=4:0FC63DAF-8483-4772-8E79-3D69D8477DE4 /dev/mmcblk0
root #
partprobe /dev/mmcblk0
root #
resize2fs /dev/mmcblk0p4
Boot the device
Once the TF card has been inserted into the device and the RGPIO switch is set to TF, the device should boot from the TF card.
Useful notes
Some useful notes that may be of interest to the reader can be found below.
Musl
This example uses a glib libc. It is possible to use musl libc as the systems C library. The TL;DR is:
- Use the tuple riscv64-unknown-linux-musl instead of riscv64-unknown-linux-gnu wherever crossdev is in use.
- Obtain (or build) any lp64d musl stage3 tarball and use that.
- Select an appropriate musl profile.
Faster installation
Anywhere that QEMU-user is invoked to build a cross-arch package, using Portage within a chroot may be replaced with an external installation utilizing crossdev to cross-compile binaries and portage to install them into the image as follows:
root #
riscv64-unknown-linux-gnu-emerge --ask sys-kernel/dracut
root #
cd rootfs
root #
ROOT=$PWD/ riscv64-unknown-linux-gnu-emerge --ask --usepkgonly --oneshot sys-kernel/dracut
It will be faster to cross-compile packages and install them into the image than to use QEMU-user to build them within the chroot, though this is not the preferred approach.
make.conf
Some useful additions for cross-compiling packages and identifying breakage in failed package builds:
# Colour in portage output, useful for debugging
# Needed for ninja (e.g. z3)
CLICOLOR_FORCE=1
# https://gitlab.kitware.com/cmake/cmake/-/merge_requests/6747
# https://github.com/ninja-build/ninja/issues/174
CMAKE_COMPILER_COLOR_DIAGNOSTICS=ON
CMAKE_COLOR_DIAGNOSTICS=ON
# Common flags for cross-compiling and colour; params pulled from -march=native
COMMON_FLAGS="-mabi=lp64d -march=rv64imafdc_zicsr_zba_zbb -mcpu=sifive-u74 -mtune=sifive-7-series -O2 -pipe -fdiagnostics-color=always -frecord-gcc-switches --param l1-cache-size=32 --param l2-cache-size=2048"
# Enable QA messages for from iwdevtools
PORTAGE_ELOG_CLASSES="${PORTAGE_ELOG_CLASSES} qa"
See also
- RISC-V hardware list — a list of RISC-V hardware owned by the Gentoo community within the #gentoo-riscv (webchat) channel.
- Embedded Handbook — a collection of community maintained documents providing a consolidation of embedded and SoC knowledge for Gentoo.