User:Idella4/Xen arm howto

XEN in ARM
Support for the arm arch in xen began with the 4.3.0 release mid 2013. It was provisional support for only 2 of the arm boards. The arm boards are single control boards (SCB) and represent the first expansion from the x86 and x86_64 arches. At the time of writing, the development of xen support of arm is still under heavy active development and will remain so in the medium to long term. Formal support will be in place with the release of xen-4.4.

This HowTo cover the requirements to boot into dom0 in one selected sample arm SCB, the cubieboard2, a new generation 'light weight' model released also around mid 2013. This guide provides a working model of an effective setup that boots a dom0. By their nature, specs of the various arm boards demand settings. Key constituents that require custom configurations to yield xen capbility on arm are:


 * 1) Kernel support
 * 2) Device tree file
 * 3) Bootloader package u-boot
 * 4) Serial console

Key differences
The traditional boot into dom0 in x86(_64) is via the standard bzImage or vmlinuz kernel image + the xen.gz file takes effect via the cmds linux & multiboot in a menu entry of grub.cfg of grub2. With arm, the kernel image saved to /boot is a zImage coupled with a device tree file generated in the build of the kernel. Booting takes place via u-boot which is likewise also under heavy development. u-boot supplies optional ways of booting; by setting a 'boot script' file and directly from a shell.

More on u-boot later.

System setup
In gentoo style, the partioning of the selected drive will have a booting partition for parition one. The prepatory code might resemble;


 * parted /dev/${card} --script -- mkpart primary fat32 2048s 264191s
 * parted /dev/${card} --script -- mkpart primary ext4 264192s 100%
 * mkfs.vfat /dev/${card}p1;mkfs.ext4 /dev/${card}p2

While vfat is often favoured for /boot, testing found that it had issue in reading an essential file for booting, therfore it's recommended to select an ext fs for /boot. The drive itself in the above code is a match for an SD card, the file being /dev/mmcblk0, + the pn suffix. The cubieboard2 boots most reliably with a micro SD card fitted into the slot on or in the board,

This is fine for an inital system, however the better option is to utilise a sata drive for whichg the kernel will make /dev/sdn. While the cubieboard2 provides for connecting a sata drive of large storage capacity, the setup requirements are far more demanding. The kernel must provide SATA support AND the drive needs powering. The cable that comes with the cb2 is limited to a usb rate of 5V and is best 'put aside'. Some SATA drives demand a 12V power source which demands powering the drive by separate means. [See the link.]

D'load this script which effects an install of a neophyte gentoo on a micr SD card. This code snippet


 * mount /dev/${card}p2 /mnt/gemtoo/
 * mkdir /mnt/kernel/boot
 * mount /dev/${card}p1 /mnt/gentoo/boot

taken from it effectively mounts the boot partition gentoo style. The script above acquires a stage/sunxi-3.4 branch kernel, however this is unusable with xen.

The Kernel
To achieve bootup of an arm arch board, linux-sunxi developed an androidish kernel-3.4, however it is 'xenless'. The support of arm in mainline kernels is already a long standing work in progress. While the initial xen support was added at 2.6.38, futher watershed development took place around 3.7. linux-sunxi have 2 branches dedicated to mainline development; next and devel.

At the time of writing, patching of the mmc card support has only just been added to the devel branch, making the setup of booting a dom0 viable. Use git to clone the devel branch of https://github.com/linux-sunxi/linux-sunxi

The settings needed for MMC/SD support and sata support are:


 * CONFIG_MMC_BLOCK
 * CONFIG_MMC_SUNXI
 * CONFIG_SATA_AHCI_PLATFORM

A wiki at xenproject.org also provides a guide on configuring and building the kernel. See http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Allwinner.

Having acquired a viable kernel, the kernel build yields a zImage and a full set of all arm board device tree files. The file for the cubieboartd2 is sun7i-a20-cubieboard2.dtb. At this point it is recommended to initially build a xenless kernel and boot the zImage and device tree before attempting to boot the xen hypervisor binary. The make and copy of sun7i-a20-cubieboard2.dtb are detailed in the 'gentoo system install script here

Build, emerge xen
At the time of writing, the xen ebuilds are still in prepatory processes of being keyworded to ~arm. The major cause for delay stems from a fundamental change yielding the xen.gz file unusable in arm. Booting a dom0 requires a xen binary executable, a zImage file in the style built by a kernel. This change was set in the xen-4.4.0-rc branch at xenproject.org, making it the minimal required version of xen. The ebuilds to build xen and xen-tools version 4.4.0_rc6 are currently stored in my dev space here and here.

Boot to dom0 currently requires a serial console from a host source subsequent to the necessaty code still absent in the mainline kernel or @ kernel.org. [See sub-heading "Serial console"] The build requires setting early_printk (as a configure option) in order to to yield full view of output to the serial console.

Booting the xen binary requires an environment set to non secure hyper mode. The mode is set in the build of the bootloader u-boot.

Bootloader u-boot
This also is outlined at the xenproject wiki here To elaborate briefly, the mainline support of u-boot @ linux-sunxi builds in secure mode. At this point in time, the branch by jwrdegoede is the required tree and is entitled the `sunxi-next`. Attempts to set the "NS' mode in this tree fail. The `sunxi-next` branch has the settings required for 'NS' mode set as default.

Anatomy of u-boot methods

Here are 3 main methods or modes


 * u-boot shell
 * uEnv.txt
 * script.bin & boot.scr

On boot, there is the option to enter any key to halt autoload. This 'places' the user in a shell where, like grub or bash, cmds are entered directly. --help options are made available. The bootloader seeks a script file to read. uEnv.txt is a text file of u-boot script cmds. The alternate script file is named boot.scr and requires the presence of script.bin in the boot partition / folder. The boot.scr is generated from a text file, by convention normally named boot.cmd, by the executable mkimage. The naming of the file is quite arbitrary and in this case is aptly named boot.xen.

From the system-install.txt, this line demonstrates mkinage making the boot.scr:


 * mkimage -A arm -T script -C none -d /mnt/gentoo/boot/boot.xen /mnt/gentoo/boot/boot.scr

Install u-boot, boot

The u-boot loader is installed to the start of the drive or card like grub. The install cmd (by dd) is found in the 'gentoo system install script @ my dev space here To boot in secure hyper mode via u-boot requires a re-install of a build by a mainline branch @ linux-sunxi, also found in the link. The boot sripts both implement use of the boot.scr. The boot.xen is here.

In this example, the dev file is /dev/mmcblk0. All required files are copied to the 1st. partition of the SD card, root on the 2nd. partition. The effective boot firstly loads the xen binary file, then the device tree file and the kernel zImage. Note from the boot.xen, the console at login is the xen style hvco. To observe bootup however, the final element is currently an essential, the serial console.

Serial console
While the kernel-3.4 supports a tty console, keyboard and output to vga monitors, the mainline kernel falls far short of these functionalities. To observe bootup and to login, a serial console is mandatory. The host computer displaying the serial console, in my case, was a tower with an x86_64 cpu. The packaged cubieboard2 (cb2) includes a usb adpator cable to service a serial console for the cb2. (Images and specs. of the board's 4 pins are to be found at cubieboard related websites) Only the TX and RX pins are essential for connnecting. The host computer receives the usb plug. To effect the console, emerge minicom, a terminal emulator that supplies a VT102.

The host will need configuring of both its kernel and the installed minicom to achieve a working serial console. For the kernel, set


 *   USB Prolific 2303 Single Port Serial Driver

This will supply the driver pl2303. Add the driver to /etc/conf.d/modules for auto loading at boot. This will create a /dev/ttyUSB0.

Minicom requires settings to match the physical specs of the cb2. To configure minicom, under Serial port setup, set


 * Serial Device     : /dev/ttyUSB0
 * Bps/Par/Bits      : 115200 8N1
 * Hardware Flow Control : No
 * Software Flow Control : No

and save. Next, edit the section # SERIAL CONSOLES of /etc/inittab to


 * 0:12345:respawn:/sbin/agetty -L 115200 ttyS0 vt100
 * s1:12345:respawn:/sbin/agetty -L 115200 hvc0 vt100

The ttyS0 yields a serial console for a straight kernel image, and the hvc0 for the xen binary.