Solaris + xenserver + ovswitch

This has troubling me for quite some time, hopefully someone else can save a few hours by bumping in this post.

For some reason my Solaris 10 Virtual Machines on Xenserver failed when the Distributed Virtual Switch Controller was also running. I didn’t really troubleshoot the issue until recently since I could live without cross-server private networks. This no longer being the case I decided to look into it again.

Scroll forward a couple of hours and after losing quite some time on trying various tricks on the VM (disabling NIC checksum offload, lower MTUs etc) to no avail I concluded that it must be a hypervisor issue. Digging into the openvswitch tools revealed something interesting.

[root@xenserver ~]# ovs-vsctl list-ports xapi25

Specifically, for my Linux VMs only a vifX.Y interface was being added to the bridge, while for my Solaris ones both a tapX.Y and a vifX.Y. Clickety-click.

[root@xenserver ~]# ovs-vsctl del-port xapi25 tap47.0

Et voila! Network connectivity to the Solaris VM works like a charm. Now to make this change permanent:

[root@xenserver ~]# diff /etc/xensource/scripts/vif.orig /etc/xensource/scripts/vif
if [[ $dev != tap* ]]; then
> $vsctl --timeout=30 -- --if-exists del-port $dev -- add-port $bridge $dev $vif_details
> else
> echo Skipping command $vsctl --timeout=30 -- --if-exists del-port $dev -- add-port $bridge $dev $vif_details
> fi

I am not really certain of the ugly side-effects that this may have. But it does the trick for me.

Update 2013/03/10: A better workaround is to have the above behavior apply only to Solaris VMs. For example, assuming that these are based on the “Solaris 10 (experimental)” template, the following snippet skips the offending command only for the Solaris VMs:

if [[ $dev != tap* ]]; then
    $vsctl --timeout=30 -- --if-exists del-port $dev -- add-port $bridge $dev $vif_details
    xe vm-list dom-id=$DOMID params=name-description | grep 'Solaris 10' 2>&1 >/dev/null || \
        $vsctl --timeout=30 -- --if-exists del-port $dev -- add-port $bridge $dev $vif_details

Tags: , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: