OVirt

Background

oVirt project is a complete open sourced virtualization management platform working with kvm. More can be found here: [1]
Additional information (including presentations) from a Workshop held early this November is available here: [2]
The Project is made of:

• The Engine core, which is the backend server that does the management.
• The various VDSM agents installed on each host you 'll use as a hypervisor for VMs
• A client side UI (GWT based) and/or RESTful API to control the engine core.

Description

This article will try to explain how to install oVirt on Gentoo system, more specifically we'll set up the following:

• For the engine core
  • The Java runtime environment
  • A JBoss server instance
  • A PostgreSQL server
  • The engine core and UI

• For the VDSM agents
  • (Work in progress)

Install Java runtime

Ovirt manual is suggesting installing openJDK. My server has sun-jdk already set so I will point you to Gentoo Java guide for setting java environment. Make sure you are setting jdk version 1.6!!!

Install Java application server – JBoss

oVirt runs on JBoss application server, here is a quick guide to set up jboss for our needs. The plan is to have JBoss under /opt, make a jboss user and set the installation dir as the user's home dir (I did this because I wanted to have everything on a single place for portability and because it makes things easier with permissions etc). Then we 'll put ovirt inside this directory and set it up.

• First go to /opt as root

• Download latest JBoss version 7

• Untar the archive

• Create the jboss user
  Note

  For compatibility with Fedora we use uid/gid 36, on gentoo this one is used for the "games" user/group so make sure you change uid/gids inside /etc/passwd and /etc/group and reset permissions on /usr/games, without it we'll have some trouble setting up NFS storage domains later. Also the -Z option is there in case you are on the SELinux profile, if not ignore it

• Now set permissions for jboss's home dir

• Test that JBoss is working by starting the server as jboss user (-b 0.0.0.0 option makes server bind to all IP addresses)

• Now open a browser to your server's hostname (let's assume it's "engine-dev") and check out http://engine-dev:8080 you should see JBoss's welcome screen.

Troubleshooting JBoss

If you're being attacked by exceptions, follow this list:

• Verify jboss folder owner and permissions.
• For external connections, make sure your firewall allows incoming traffic on 8080 port.
• If your machine has and selinux policy installed, make sure it will not block JBoss. Here is a very dirty and insecure jboss.te just to temporarily pass these denials (you'll also need selinux-java)

module jboss 1.0;

require {
        type user_tmp_t;
        type user_home_dir_t;
        type bin_t;
        type usr_t;
        type port_t;
        type node_t;
        type sysfs_t;
        type newrole_t;
        type java_t;
        class tcp_socket { name_bind listen accept node_bind };
        class fd use;
        class capability dac_override;
        class file{ read write getattr open create unlink };
        class dir { getattr read write open search add_name create remove_name };
}

#============= java_t ==============
allow java_t newrole_t:fd use;
allow java_t bin_t:file { getattr read open };
allow java_t self:capability dac_override;
allow java_t sysfs_t:dir { read search open };
allow java_t user_tmp_t:dir { read getattr open search write add_name create remove_name };
allow java_t user_home_dir_t:dir search;
allow java_t user_tmp_t:file { create read write open unlink };
allow java_t usr_t:dir { write add_name create remove_name };
allow java_t usr_t:file { create open write };
allow java_t port_t:tcp_socket name_bind;
allow java_t self:tcp_socket { listen accept };
allow java_t node_t:tcp_socket node_bind;

• Used TCP ports: 8080/8443/8083/1090/4457
  • These ports may be used by other applications. Either close other application or change JBoss configuration (/opt/jboss-as-7.1.0.CR1b/standalone/configuration/standalone.xml)
  • Since JBoss binds to the hostname, your hostname should be resolvable, or you may add it to /etc/hosts for local resolution.

127.0.0.1 localhost engine-dev

• Verify you gave JBoss's enough RAM to work with

Installing PostgreSQL

Ovirt Engine works with postgresql 8.4.x or 9.1.x

If you didn't already, specify default locale for your server (as root):

• Check /etc/locale.gen

Output (may be different)

en_US ISO-8859-1
en_US.UTF-8 UTF-8
sr_RS UTF-8
sr_RS@latin UTF-8 

• Create /etc/env.d/02locale

• Update environment for current shell (and every new shell you open) or simply reboot

Now lets install PostgreSQL...

• Emerge Postgresql server. Make sure uuid useflag is enabled!

• Make sure the jdbc installed is 9.1_XXX (you may need to unmask it).
• Once emerged, verify the config files in /etc/postgresql-9.1 and /etc/conf.d directory. Use Gentoo Postgresql Guide as a reference for configuration.
• Make sure pg_hba.conf file has access rules set to trust
• Add jboss user to postgres group

• Run the relevant emerge --config … command.
• Start the database server

Installing oVirt Engine Core

Now that our needed components are in place it's time to download and build the engine core. Sticking to the plan lets get ovirt engine's source code and place it inside ~jboss/ We 'll do the whole process as user jboss

Install maven

• Install dev-java/maven-bin. It is a must to build version 2 of maven. Version 3 is not supported!!! As of time of writing, stable version in portage is 2.2.1 so it is safe to just emerge it:

• If unsure, emerge strict version:

• Set up the needed repositories for Maven. To create default repository directory, run maven as non-privileged user. DO NOT use root user.

• This command will create .m2 directory in jboss user's HOME directory.
• Copy paste the content of the file below into user's ~/.m2/settings.xml file. See the [Maven personal settings] on the oVirt wiki.

 
<?xml version="1.0"?>
<settings xmlns="http://maven.apache.org/POM/4.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
          http://maven.apache.org/xsd/settings-1.0.0.xsd">

<!--**************************** PROFILES ****************************-->

        <activeProfiles>
                <activeProfile>oVirtEnvSettings</activeProfile>
        </activeProfiles>

        <profiles>
                <profile>
                        <id>oVirtEnvSettings</id>
                        <properties>
                                <jbossHome>/opt/jboss-as-7.1.0.CR1b</jbossHome>
                                <JAVA_1_6_HOME>/opt/sun-jdk-1.6.0.26</JAVA_1_6_HOME>
                                <forkTests>always</forkTests>
                        </properties>
                </profile>
        </profiles>
</settings>

• Modify jbossHome and JAVA_1_6_HOME tags as needed. If these parameters are incorrect, compiling and installing using maven will fail!

Build oVirt-engine

Prepare build environment

• Increase some maximum values by default. Add the following to ~jboss/.mavenrc:

export MAVEN_OPTS="-Xmx2048M -XX:MaxPermSize=2048M"

• Values for mx and MaxPermSize may be different!
• Build GWT to work with a specific browser (this is needed in case you don't have that much RAM to compile support for all 6). Modify
  • ~jboss/ovirt-engine/frontend/webadmin/modules/webadmin/src/main/resources/org/ovirt/engine/ui/webadmin/WebAdmin.gwt.xml
  • ~jboss/ovirt-engine/frontend/webadmin/modules/userportal/src/main/java/org/ovirt/engine/ui/userportal/UserPortal.gwt.xml

by adding the lines:

<!-- Reduce the number of permutations to compile, just for tests: -->
<set-property name="user.agent" value="gecko1_8"/>

Building ovirt-engine and REST api

• As unprivileged user

Compile web-admin and user-portal

• As unprivileged user

Application deployment

Time to put oVirt Engine Core and its components to JBoss

• The first deployment of the application to JBoss AS container should use the setup profile:

• There is a issue with the dep and setup_postgres profiles getting in the way of each other. The setup_postgres profile will prevent the deployment of the quartz jar to the JBoss server. So after this step completes, run:

• From this point on, since postgres is already set up, every time you deploy you can simply run:

New ROOT container deployment

We need to replace the JBoss's root container (what you see under http://engine-dev:8080/) with oVirt's ROOT.war since it'll hold the Engine Core's certificate and public ssh key, used by oVirt nodes.

• Copy ROOT.war to the deployments directory

• Tweak standalone.xml to enable the new server's root

[...]
<virtual-server name="default-host" enable-welcome-root="false">
[...]

• And tell JBoss to deploy it next time it runs

Creating the database schema

Finally it's time to create the database schema for Engine Core

Create the oVirt Engine Core's Certificate Authority and keys

Now we need to create the Engine Core's CA certificate that 'll be used to sign the certificate of each Hypervisor and its public ssh key, needed for engine <-> Hypervisor communication. The plan is to put everything under ~/pki (on Fedora /etc/pki is used) and put the needed values on the database.

• First lets make the needed dirs

• Now compile pubkey2ssh program needed to create the public ssh key from your CA certificate

• Create and set up the CA

• Create and set up the SSH keys

• Put CA certificate and public SSH key under /

#lock_file=$7
if [ -z "$lock_file" ]; then
        lock_file=/tmp/.openssl.exclusivelock
fi

• Set CA directories on the database

Time to add our new CA to the database, this script will do the trick...

#!/bin/bash

CA_DIR=/opt/jboss-as-7.1.0.CR1b/pki/vps-ca/ca

psql engine postgres -c "update vdc_options set option_value = '$CA_DIR/certs/engine.cer' where option_name = 'CertificateFileName';"
psql engine postgres -c "update vdc_options set option_value = '$CA_DIR/.keystore' where option_name = 'TruststoreUrl';"
psql engine postgres -c "update vdc_options set option_value = '$CA_DIR' where option_name = 'CABaseDirectory';"
psql engine postgres -c "update vdc_options set option_value = 'ca.pem' where option_name  = 'CACertificatePath';"
psql engine postgres -c "update vdc_options set option_value = '$CA_DIR/.keystore' where option_name = 'keystoreUrl';"
psql engine postgres -c "update vdc_options set option_value = '$CA_DIR/private/ca.pem' where option_name = 'CAEngineKey';"

Testing

• Start JBoss application server

• Test JBoss by accessing from browser

http://engine-dev:8080/

• Use username admin@internal and password letmein! for testing access.
• Accessing the web-admin:

 http://engine-dev:8080/webadmin

• Accessing the user-portal:

 http://engine-dev:8080/UserPortal

• Accessing the RESTful API:

• From the browser

http://engine-dev:8080/api

Install vdsm

• Install dev-python/pyflakes as it is needed by vdsm

• Obtain VDSM source RPM

• Convert rpm package to tgz

• Unpack the archive

• Enter the directory and do the configure-make-make install magic (Recommended to use –prefix when compiling from source so you can have all files under one directory per package)

How to contribute

• oVirt project is working with Gerrit code review for code contribution.
• In order to register and login to oVirt's Gerrit, you'll need an OpenID account.
  • You can use a Google OpenID, or register to some other provider and use it,
• All other details can be found here: http://www.ovirt.org/wiki/Working_with_oVirt_Gerrit

Errors & Solutions

• Theese are the errors and solutions as I was going through the process of installing oVirt. Feel free to add your own :)

Error: When compiling projects, getting errors about unmappable characters.

Solution: My install is UTF8, but default locale was not set. So default locale by Maven/JBoss (not sure which one) was US_ASCII. Set default locale and errors are history.

Error: Compile process dies with PermGenSpace error.

Solution: This error means that no more memory is available to java compiler. This is fixed by setting MAVEN_OPTS with higher values for mx and MaxPermSize parameters. Another trick that helped was choosing only one browser to compile GWT for by modifying above mentioned files.

Error: Compile process dies with error code 137.

Solution: This one I'm not quite sure about. The only info I could google out was a suggestion to try change the java compiler (oVirt wiki uses openJDK, me sun-jdk). The solution for me was to add -e -DskipTests=true options to maven. Again, I'm not sure how this is relevant as I don't know what error 137 means or how it is caused.

• Errors while deploying the root container

Error: Message: JBAS014724: Missing required attribute(s): cache-ref

Solution:

165c158
<                 <stateful default-access-timeout="5000"/>
---
>                 <stateful default-access-timeout="5000" cache-ref="simple"/>

Error: Message: JBAS014788: Unexpected attribute 'show-model' encountered

Solution:

222c215
<         <subsystem xmlns="urn:jboss:domain:jmx:1.1" show-model="true"/>
---
>         <subsystem xmlns="urn:jboss:domain:jmx:1.1"/>

Advanced features

oVirt Node integration

• By default development setup works with hosts based on base distro's installations.
• In order to be able to work with oVirt Node (which is a sub-set of the base OS), you'll need to setup a Public Key environment.
• More details on Engine and oVirt Node integration can be found here: http://www.ovirt.org/wiki/Engine_Node_Integration.
  • Note that by default Gentoo does not have /etc/pki folder, and you'll need to create it (or write an eBuild which will do that).