MIPS/ERLite-3

The EdgeRouter Lite (ERLite-3) is an MIPS64 router (MIPS64r2, Cavium Octeon) with 512 MB of RAM, which uses a (removable) USB pendrive for storage. This document describes how to run Gentoo on this device.



GPL Sources
Ubiquiti provides a GPL archive containing the sources of all open-source software that runs on this router. The GPL archive can be found here

The Processor
system type            : UBNT_E100 (CN5020p1.1-500-SCP) processor              : 0 cpu model              : Cavium Octeon+ V0.1 BogoMIPS               : 1000.00 wait instruction       : yes microsecond timers     : yes tlb_entries            : 64 extra interrupt vector : yes hardware watchpoint    : yes, count: 2, address/irw mask: [0x0ffc, 0x0ffb] isa                    : mips1 mips2 mips3 mips4 mips5 mips64r2 ASEs implemented       : shadow register sets   : 1 kscratch registers     : 0 core                   : 0 VCED exceptions        : not available VCEI exceptions        : not available

processor              : 1 cpu model              : Cavium Octeon+ V0.1 BogoMIPS               : 1000.00 wait instruction       : yes microsecond timers     : yes tlb_entries            : 64 extra interrupt vector : yes hardware watchpoint    : yes, count: 2, address/irw mask: [0x0ffc, 0x0ffb] isa                    : mips1 mips2 mips3 mips4 mips5 mips64r2 ASEs implemented       : shadow register sets   : 1 kscratch registers     : 0 core                   : 1 VCED exceptions        : not available VCEI exceptions        : not available

The USB Flash Drive
The details of the USB flash drive that comes with this board are the following:

Partitioning
The USB flash drive on the board has two partitions. A small FAT32 one for the Linux Kernel image and a big ext3 one containing the root filesystem in a squashfs file. The squashfs file is in the GPL archive as well.

The Linux Kernel
The Linux kernel that comes with the board is a modified 2.6.32.13 one. If you ever want to rebuild it yourself from the GPL archive, you need a mips64-octeon-linux-gnu toolchain that comes with the Cavium SDK. Typical distro toolchains (mips64-unknonwn-*) will fail to compile these kernel sources with errors like these:

The RootFS
This router comes with Debian 6.0.6

Getting the MIPS64 stage3 tarball
The ERLite-3 uses a 64-bit MIPS64r2 Big-Endian Cavium Octeon processor. So the stage3 we want for this board is a mip64-* one. For this guide, we will pick a glibc based one.

Configure /etc/fstab
If you are going to use NFSroot to boot your ERLite-3 board, you need to edit the entry in your fstab like this

Reset root password
In order to be able to login after the first boot, you need to reset the root password. For this edit the file and remove the '*' from the second column.

Building the toolchain
The are many different ways to build a mips64 toolchain. We will use the Gentoo script to build one.

Create a cross toolchain for MIPS64 (big-endian system):

Make sure you read this and this  to understand how to use crossdev in your day-to-day development.

The MIPS64 Linux Kernel
Since this is a brand new board, not many things are supported upstream. For example, the Cavium Octeon Ethernet driver (currently in Staging) will flood the stdout with warnings like the following ones:

Moreover, this is no driver for the USB controller, meaning that if you want to build another kernel yourself, you will not be able to use the USB flash drive.

Getting the Kernel
We will use the Linux Kernel (v3.9) from the Linus' git repo for now, but there might be other repos more suitable for this board out there. If you find one, please edit this guide as appropriate.

Patching the Kernel
To workaround the previously mentioned problem with the Ethernet driver (and possibly other drivers as well), you can use this patch which is based on the previously mentioned GPL archive provided by Ubiquiti. And to support USB in the kernel, you may use this patch.

Install and Configure the TFTP Server
The TFTP server will be used to load the new kernel image on the board. For this, you need the package.

Edit the file as appropriate:

And start the service (add it to the default runlevel if you wish)

Install and Configure the NFS Server
The NFS Server will export the (previously extracted and prepared) stage3 (/mnt/erlite-3) to the ERLite-3 board.

In order to use your PC as an NFS server, you need the following kernel options to be enabled:

You also need to build and configure the as follows:

Then edit the as follows:

Prepare U-Boot for NFS boot
U-boot is pre-configured as follows:

For NFS boot, we need to set the following variables: ipaddr : IP for the ERLite-3 board (e.g. 192.168.1.2) serverip : IP for the PC acting as tftp server (e.g. 192.168.1.3) bootcmd : We need to override the existing 'bootcmd' command with one suitable for tftp boot

For this, use the following commands in the u-boot command line (Press Ctrl-C to interrupt the boot process)

setenv ipaddr "192.168.1.2" setenv serverip "192.168.1.3" setenv bootcmd 'tftpboot $loadaddr vmlinux; bootoctlinux $loadaddr coremask=0x3 ip=192.168.1.2::192.168.1.254:255.255.255.0:edge:eth0:off root=/dev/nfs nfsroot=192.168.1.3:/mnt/erlite-3/,tcp,vers=3

And now save the new configuration

saveenv

Now, it is time to reset the router and boot into your shiny new Gentoo MIPS64 rootfs.

USB Boot
Successful boot from USB drive is confirmed on vanilla kernel 3.11_rc4

Kernel command line parameters can be adjusted through U-boot.