User:Rabisg/GSOC 14

[GSoC'14] netifrc on Systemd

Abstract

The goal of this project is to abstract away the tight dependence of netifrc on OpenRC and write a compatibility layer for netifrc to work with other init systems like Systemd

Further details of the proposal can be found at this page

Progress Report

April 21 - May 18 : Community bonding period

1. Read up shell programming in general
2. Read up systemd - its management and philosophies Freedesktop: Systemd
3. Read through the source code of netifrc.
4. Checked out tools like checkbashism (suggested by @robbat2) and shellcheck and fixed the bugs in the module for iw which I had started writing earlier

May 19 - May 25 : Initial systemd design

• Created a systemd wrapper to be called by the unit file
  • It sets up the initial environment for non-systemd settings
  • Sources the conf.d/net file manually
  • Sources the net.lo file and relays calls to start, stop and restart to the net.lo script

• Refactor common functionality in sh/functions.sh
  • Functionality like finding the interface name from the runscript which is required on any target but which differs depending on the target (Gentoo v/s non-Gentoo, openrc v/s systemd) are to be refactored in this file. For example it presently contains get_interface() which finds out the interface name depending on the init system
  • It also checks whether ebegin, etc. is present in the scope. If not, then it tries to source /lib/gentoo/functions.sh. Else as a fallback it contains the definitions of those functions (Adapted from sys-apps/gentoo-functions)

May 26 - June 1 : OpenRC Compatibility Layer

• Discussed the design further with @robbat2 and decided to remove e* functions from functions.sh and instead add gentoo-functions as a dependency
• Changed the build process to include the new files and tested it locally. Subsequently fixed the errors that I could I find.
• Updated the ebuild (maintained at https://github.com/rabisg/gentoo-overlay/tree/master/net-misc/netifrc) and added a systemd service file (To be tested)

• Started with the compatibility layer: Discussed the semantics of service_set_value and service_get_value at #openrc and also had a peek at openrc's codebase. Finally decided that the best way to implement it using files in /run/netifrc similar to what is done by openrc (which stores it at $SVCDIR/options/$SVCNAME/$KEY )

June 2nd - June 8

I could not work the usual duration because I had to move to college from home and had some things I had to take care of at college.

• However I set up the various VMs which will be required to test the project
  • Gentoo Stable
  • Gentoo Unstable
  • Gentoo with Systemd
  • Gentoo Unstable with Systemd
  • Archlinux

Setting up systemd helped me better understand systemd itself, especially unit files and their handling. During this I also had to to look into netctl (the archlinux project to manage network connections)which has many similarities to my current work

• Also started looking into mark_service calls in openrc

June 9th - June 15 : Testing on a Systemd system

• Read through the codebase of openrc and tried to understand the semantics of marking services. I have also discussed this with people at #openrc but am still not 100% confident about understanding the semantics of the same. I have also written a small wiki entry trying to explain my understanding of the concerned openrc functionality here

Please feel free to point out mistakes and help me out in understanding it further

• Successfully tested netifrc running on a Gentoo systemd stable machine (only tested setting up "lo" for now). The changes I had to make in the code and the service file are recorded in the main repo and my local overlay respectively

June 16 - June 22

• This week I tested the codebase on gentoo+systemd machines. Upon testing it on a actual systemd machine, I figured out the missing functions and fixed a few bugs. I could successfully test eth0 and lo interfaces. I also tested it on a gentoo+openrc system to make sure that I did not introduce any breaking changes
• Added the prototype for is_net_fs function and defined functions primarily used in the depend block (program, after, need, etc.)

VirtualBox image: Gentoo - https://drive.google.com/folderview?id=0B8yZctFMulwlT2hwanNpTVdJWkE&usp=sharing

Build Instructions:

1. After getting the VM running, I generally mount /usr/portage and /usr/local/portage from my local machine /usr/local/portage is my local overlay: https://github.com/rabisg/gentoo-overlay. But this image has netifrc-9999 is installed anyways
2. You can directly run

CODE

sudo systemctl restart net@eth0
sudo systemctl status net@eth0

June 23 - June 29

• This week I had been further testing the code written so far:
• Upon discussion with WilliamH at #openrc I discovered and patched a few bugs
• Wrote the implementation of marking services and querying service interface status
• Patched the compatibility functions to enable netifrc to work with multiple instantiated units
• I also started testing the codebase on Archlinux but didn't make report-able progress till now.

June 30 - July 6

• This week i had been working on writing the test suite for netifrc on systems.
• We are planning to write the test suite such that it matches the output on a traditional openrc system vs a systemd system.
• After looking into various testing frameworks I decided to write my own implementation using a yaml parser and a simple testing framework.
• It would work in 2 modes. Master and slave, where it stores key values and matches them respectively.

July 7 - July 13

This week I added the final touches to the testing module. Now we have support for

1. The testing module puts up the interface with the test configuration file(s) and then runs various commands to check the status of the network.
2. The list of commands to run and the specifications of how to read the output is specified in the specs file(s) https://github.com/gentoo/netifrc-systemd/blob/master/test/specs/eth_static.yaml
3. The test runner is divided into two parts. When it runs in the master mode it runs the test specified in the spec and stores the output in a file which serves as the basis for future tests (The plan is that this file is also bundled with the test suite). In slave mode it matches runs the tests again but this time matches the output with the expected output

You can find the code here: https://github.com/gentoo/netifrc-systemd/tree/master/test

July 14 - July 20

This week the following progress has been made:

1. Add support for constraints on the values itself in the test mode Now even while running the tests in master mode, the test suite can make sure that the values being extracted follow a certain pattern. For example, the IP address can specify the IP regex in order to make sure that the value being extracted is valid IP address.
2. Support for variables in the conf.d/ files
3. Test specs for DHCP and static IP address.

July 21 - July 27

• This week I worked on the testing part of Vlan, Bonding and Bridging operations.
• For each of them, I wrote the configuration file, test specs and verified the output generated.

July 28 - August 3

• This week I have been working on writing test cases for tunneling.
• Moreover I have been investing more time in learning about bridging, bonding, Vlans, etc. I have playing around with VMs trying to set up the network, and reading literature on the same. I plan to invest some more time in it before moving on to other things so that I can write the test cases in a better fashion

August 4 - August 10

1. This week has been spent on further testing and fixing bugs.
2. I had planned on completing the testing by this weekend and spending the next week in writing documentation, improving code if possible, etc. but I hit a roadblock with the testing part and thus was not able to complete it as expected.