QEMU with Open vSwitch network

This article is designed to show the steps needed to create a basic and minimal Open vSwitch network to be used by a QEMU virtual machines(s) managed with libvirt. This type of network provides a much more powerful switching solution than the legacy NAT and bridge forwarding solutions.

Context
There all quite a few different networking designs that can be created; but for the sake of keeping this simple we are going to work under the assumption that we are trying to configure a host OS on a computer that has 2 physical ethernet ports. The first port (eth0) is to be exclusively used by the host OS. The second port (eth1) is to be reserved for use by the vSwitch, which has the guest OS(s) attached to it. We also assume that the two physical ports are connected to a simple hardware ethernet switch, without anything advanced like VLAN tagging for instance.

Background on OpenFlow and Open vSwitch
Open vSwitch has 3 service components to it: database, server and controller. The database daemon keeps track of interfaces that are created or modified, so that after a reboot they can be automatically re-created and configured. The server daemon actually sets up and manages the network, as a well as interfaces with the switching functionality within the kernel. The controller daemon may be the part that you are not very familiar with OpenFlow. In non-openflow switchers, there is the hardware that does all the work, and there is a controller. It is the "brains" of the switch that contains all the logic used to determine how packets are routed. With OpenFlow, the controller logic and packet routing functionality are decoupled from each other. This allows for the centralization of the controller logic into one controller that provides the logic for any number of switches. This provides the advantages of quicker deployment, cheaper hardware, and tight integration of both physical and virtual switchers. This also blurs the distinction between the roles of switcher and router. The controller daemon provides this functionality and can control multiple Open vSwitchers, as well as hardware switchers support OpenFlow. Or it can be turned off with the controller functionality provided by a controller elsewhere in the network.

For the scope of this article, we will just assume that the controller daemon is only being used by the single virtual switch we are creating.

Kernel
You need to activate the following kernel options:

libvirt
You will want to make sure is built with support for the various virtual networking solutions:

Open vSwtich
Install :

We will be using the default settings, however feel free to view them.
 * - Database daemon
 * - vSwitch daemon
 * - Controller daemon

Startup and enable the daemons:
Database daemon:

If the database daemon complains about the non-existing conf.db please make sure you did run the emerge --config command.

Controller daemon:

vSwitch daemon (this also starts/enables the database daemon since it is a dependency):

Setup eth1
We don't want eth1 to be used by the host, and we also don't want it to be assigned an IP address.

Change the net config file:

If eth1 was configured differently before changing it, you might want to restart your system to apply the changes. (Restarting net.eth1 by itself is a mess)

Setup vSwitch
First, create the bridge. We'll call it "vbr0".

Next, we will add eth1 to this bridge.

The final change we need to make is to assign the bridge a controller. Without it, the bridge doesn't know what to do with the packets.

One setting that is optional, but very highly recommended, is to turn on the spanning tree protocol.

Setup libvirt
Recent versions of libvirt support this type of bridge, we just have to configure the virtual machine to use it.

Connect to the local QEMU manager:

Once logged into the virsh shell, get a listing of the virtual machines:

Select the virtual machine you need to configure:

This will open up the xml config in your default text editor. Find the section that defines the virtual OS's network interfaces. It will usually look like something similar to this:

There will probably be a few other tags inside the tag, however, we are only showing the ones you will need to change. They will need to change to the following:

If you virtual machines is setup up with more than one network interface, you will need to edit each additional network interface tag accordingly.

Save the file and exit. You will return to the virsh shell, which should confirm that the virtual machine config has been updated.

Start the virtual machine:

OpenRC
All the pieces are in place, and will be automatically restored on reboot. However there is nothing in place to automatically bring up the bridge interface and eth1. We will need to add an init service to do that for us.

Create the new file :

Remember to set the correct mode bits:

Next start and enable the service:

At this point the virtual machine should be just like another computer on the network connected to the physical switch.

Closing
There is a lot of functionality available with Open vSwitch that this article does not touch. You can setup VLAN tagging, QoS rules, create re-routing rules, block IP address, block ports, and much more. Feel free to reference online documentation and man pages if you are interested in adding complex functionality to the controller.