Opennebula: dhcpd contextualization magic

One of the most frequent questions on the Opennebula lists relates to network contextualization of Virtual Machines (VMs). Specifically, contrary to Eucalyptus or Nimbus, Opennebula does not directly manage a DHCP server. Instead Opennebula:

  • suggests using a simple rule for extracting the IPv4 address from the MAC address within the VM
  • manages just MAC addresses

This moves the burden of IPv4 configuration to the VM operating system, which has to dynamically calculate the IPv4 address details based on each interface MAC address. While Opennebula provides a relevant sample VM template and script to do this, it comes up a little bit short. Specifically, the script is linux specific, it will probably not work with other Unix O/S like Solaris or FreeBSD, let alone Windows. In addition, extra work is required in order to configure additional but required network parameters, like a default router or a DNS server.

This got me thinking that it should be possible to configure an ISC dhcpd server to do the “MAC to IPv4 address” work. This would make the VM configuration simpler (just configure DHCP for all network interfaces), the solution cross-platform and leverage any existing knowledge (existing subnets, routers, DNS servers, etc).

40,000 foot view

Here is how the solution works from a 40,000 ft. view:

  1. A new, unknown Opennebula (ONE) VM performs a DHCP REQ
  2. The VM gets assigned a temporary short-lived (i.e. 10 second) IPv4 address in the subnet it resides
  3. The DHCP server determines this is a ONE VM based on the MAC prefix
  4. The DHCP server determines this is an unknown ONE VM
  5. The DHCP server creates on-the-fly (using omshell) a new static host entry for the unknown VM based on the MAC to IPv4 rule
  6. The lease of the VM gets expired and a renewal is requested
  7. A long term lease based on the reservation created in step 5 is assigned

Subnet configuration

Assuming that the subnet of interest is 192.168.254.0/24, its configuration in the DHCP server follows:

subnet 192.168.254.0 netmask 255.255.255.0 {
  option routers 192.168.254.1;
  option broadcast-address 192.168.254.255;
  pool {
    range 192.168.254.251 192.168.254.254;
    max-lease-time 10;
  }
}

Note the small pool of 4 IP addresses reserved within the subnet. This reflects the “short-lived leases” mentioned above. One should make sure that the relevant Opennebula network configuration leaves these out. For example:

NAME = "mynet"
TYPE = FIXED

BRIDGE = vbr1

LEASES = [IP=192.168.254.2]
LEASES = [IP=192.168.254.3]
...
LEASES = [IP=192.168.254.249]
LEASES = [IP=192.168.254.250]

One will notice that I’ve also left out the router IP address.

Enable OMAPI

One needs to enable OMAPI in order to update the DHCPD configuration during runtime. The following line does the trick:

# grep omapi /etc/dhcp/dhcpd.conf
omapi-port 7911;

In a production environment you should configure a key so that unauthorized users don’t wreak havoc in your server. There are a myriad of posts explaining how to do this, just google for them (random example)

Create an OMAPI script frontend

While OMAPI is useful, it is interactive. Towards this end we need a script that takes a MAC address, an IPv4 address and a hostname and generates a static lease.

# cat /var/tmp/omshell.sh
#!/bin/bash -i

. /root/.bashrc

if [ "$4" == "delayed" ]; then
  #fork to background
  bash $0 $1 $2 $3 "run" &
  exit 0
fi
if [ "$4" != "run" ]; then
  echo error
  exit 1
fi

# /bin/cat <<FOO
/usr/bin/omshell <<FOO
port 7911
connect
new host
set name = "$3"
set hardware-address = $1
set hardware-type = 1
set ip-address = $2
set known = 1
create
FOO

The above script also includes an extra “feature”: it allows specifying whether it will run normally or fork a copy of itself in the background to do the work.

Tying it all together: dhcp-eval

dhcp-eval(5) is the duct tape that ties it all together. It allows us to:

  1. Define a class that applies only to DHCP clients with the magic MAC prefix
  2. Calculate the appropriate IPv4 address
  3. Create a DHCP reservation for the respective MAC:IPv4 pair (if one doesn’t exist already)
# more /etc/dhcp/dhcpd.conf
...
class "one-clients" {
  match if binary-to-ascii (16, 8, "-", substring (hardware, 1, 2)) = "0-3";
  if not (substring(host-decl-name,0,4) = "one-") {
    log (info, "hi one client; creating host entry");
    set onemacaddr = concat(binary-to-ascii (16, 8, "-", substring (hardware, 1, 1)), ":",
                            binary-to-ascii (16, 8, "-", substring (hardware, 2, 1)), ":",
                            binary-to-ascii (16, 8, "-", substring (hardware, 3, 1)), ":",
                            binary-to-ascii (16, 8, "-", substring (hardware, 4, 1)), ":",
                            binary-to-ascii (16, 8, "-", substring (hardware, 5, 1)), ":",
                            binary-to-ascii (16, 8, "-", substring (hardware, 6, 1)));
    set oneipaddr = concat(binary-to-ascii (10, 8, "-", substring (hardware, 3, 1)), ".",
                           binary-to-ascii (10, 8, "-", substring (hardware, 4, 1)), ".",
                           binary-to-ascii (10, 8, "-", substring (hardware, 5, 1)), ".",
                           binary-to-ascii (10, 8, "-", substring (hardware, 6, 1)));
    set onename = concat("one-",
                         binary-to-ascii (10, 8, "-", substring (hardware, 3, 1)),
                         binary-to-ascii (10, 8, "-", substring (hardware, 4, 1)),
                         binary-to-ascii (10, 8, "-", substring (hardware, 5, 1)),
                         binary-to-ascii (10, 8, "-", substring (hardware, 6, 1)));
    execute ("/var/tmp/omshell.sh", onemacaddr, oneipaddr, onename, "delayed");
  }
}
...

Why the short-lived lease?

Some readers will wonder why go through the short-lived 10-second lease. Why not go through the following steps:

  1. DHCP req comes in
  2. Client is recognized as a ONE VM
  3. omshell is used to create static host entry
  4. DHCP response is provided based on above entry

Which was what I initially shot for as well. Turns out that the ISC dhcpd doesn’t really allow that, for instance you can’t use the execute keyword unless a valid lease is reserved for the client. And once a valid lease has been reserved there is some kind of “lock” in place which prevents you from tweaking the lease database, unless the appropriate DHCP response has been sent.

Does it work?

Serving my conscription duty means (among other things) that I don’t have handy access to my OpenNebula setup.Hence I resorted to a simple Virtualbox testbed (one DHCP server, one DHCP client) leveraging a private network and dhclient(8) to verify the above. The test MAC address used was 00:03:C0:A8:FE:CF, corresponding to the 192.168.254.207 IP address.

dhcp-client# ifconfig eth0 | grep HWaddr | awk '{print $NF}'
00:03:C0:A8:FE:CF
dhcp-client# dhclient -v -d eth0
Internet Systems Consortium DHCP Client 4.2.0
Copyright 2004-2010 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on LPF/eth0/00:03:c0:a8:fe:cf
Sending on   LPF/eth0/00:03:c0:a8:fe:cf
Sending on   Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8
DHCPOFFER from 192.168.254.1
DHCPREQUEST on eth0 to 255.255.255.255 port 67
DHCPNAK from 192.168.254.1
DHCPREQUEST on eth0 to 255.255.255.255 port 67
DHCPNAK from 192.168.254.1
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 7
DHCPOFFER from 192.168.254.1
DHCPREQUEST on eth0 to 255.255.255.255 port 67
DHCPACK from 192.168.254.1
bound to 192.168.254.207 -- renewal in 228 seconds.

The respective log entries from the server side follow. One can clearly see that renewal requests for the “short-lived” lease are promptly rejected once the permanent entry has been added via omshell.


Feb 20 14:01:28 f14-server dhcpd: DHCPDISCOVER from 00:03:c0:a8:fe:cf via eth0
Feb 20 14:01:29 f14-server dhcpd: DHCPOFFER on 192.168.254.251 to 00:03:c0:a8:fe:cf via eth0
Feb 20 14:01:29 f14-server dhcpd: DHCPREQUEST for 192.168.254.251 (192.168.254.1) from 00:03:c0:a8:fe:cf via eth0: lease 192.168.254.251 unavailable.
Feb 20 14:01:29 f14-server dhcpd: DHCPNAK on 192.168.254.251 to 00:03:c0:a8:fe:cf via eth0
Feb 20 14:01:32 f14-server dhcpd: DHCPREQUEST for 192.168.254.251 (192.168.254.1) from 00:03:c0:a8:fe:cf via eth0: lease 192.168.254.251 unavailable.
Feb 20 14:01:32 f14-server dhcpd: DHCPNAK on 192.168.254.251 to 00:03:c0:a8:fe:cf via eth0
Feb 20 14:01:38 f14-server dhcpd: DHCPREQUEST for 192.168.254.251 (192.168.254.1) from 00:03:c0:a8:fe:cf via eth0: lease 192.168.254.251 unavailable.
Feb 20 14:01:38 f14-server dhcpd: DHCPNAK on 192.168.254.251 to 00:03:c0:a8:fe:cf via eth0
Feb 20 14:01:49 f14-server dhcpd: DHCPDISCOVER from 00:03:c0:a8:fe:cf via eth0
Feb 20 14:01:49 f14-server dhcpd: DHCPOFFER on 192.168.254.207 to 00:03:c0:a8:fe:cf via eth0
Feb 20 14:01:49 f14-server dhcpd: DHCPREQUEST for 192.168.254.207 (192.168.254.1) from 00:03:c0:a8:fe:cf via eth0
Feb 20 14:01:49 f14-server dhcpd: DHCPACK on 192.168.254.207 to 00:03:c0:a8:fe:cf via eth0

Potential issues

That said, the “short-lived” lease could affect the start-up of an actual system (if nothing else it will probably briefly result in the network being down for 3-5 seconds between getting a DHCPNACK for the short-lived lease and getting a DHCPACK for the final IPv4 address).

Other issues could include the short-lived leases being susceptible to a “DoS” attack, malicious clients/VMs getting hold of them and preventing new ones joining the cloud. Some ebtables magic (example) should do the trick here.

Advertisements

Tags: , , , ,

4 Responses to “Opennebula: dhcpd contextualization magic”

  1. Tweets that mention Opennebula: dhcpd contextualization magic « ~mperedim/weblog -- Topsy.com Says:

    […] This post was mentioned on Twitter by mperedim and Jaime Melis, Javi Fontan. Javi Fontan said: RT @mperedim: New post: "Opennebula: dhcpd contextualization magic" – http://wp.me/pprn9-62 […]

  2. zenet Says:

    I didn’t know OpenNebula works with Virtualbox…

    • mperedim Says:

      It doesn’t, I just used Virtualbox (and two VMs) to verify that the above solution works.

      I updated the “does it work” section to hopefully better reflect this.

  3. Installing and Configuring the DHCPD Server Says:

    […] Opennebula: dhcpd contextualization magic […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: