The ability to manage virtual machines is something which is receiving a lot of focus right now. Xen, KVM, QEMU and others provide the infrastructure required to run a virtual machine, and each can provide guests with a virtual network interface. This proposal addresses the problem of how guests are networked together.
We aim:
To make virtual networking "just work".
Guests should be able to communicate with each other, their host and the Internet without any fuss or configuration. This should be the case even with laptops and offline machines.
To allow a greater flexibily with how guests are networked.
It should be possible to isolate groups of guests in different networks, allow guests on different physical machines to communicate, firewall guests' networks from physical networks or allow guests to appear just like physical machines on physical networks.
To make networking virtual machines analogous with networking physical machines.
To support inter-networking between virtualisation technologies.
It's important to consider the manner in which we expose the functionality of virtual networking. What concepts will be exposing through the UI? Are those concepts well defined and consistent? Are those concepts more complex than neccessary? Or are the too simple to be able to support the functionality we want?
Real world, or "physical", concepts[1]:
With virtual networking, we will be exposing the following "virtual" concepts:
(Note, unprivileged users may create any of the above)
Finally, where the physical world meets the virtual world:
Shared Physical Interface - if a physical interface is configured to be "shared", then any number of virtual interfaces may be connected to it allowing virtual machines to be connected to the same physical network which the physical interface is connected to.
Only privileged users may configure a physical interface to be shared and/or connect guests to it.
There are a few problems with all of the above:
[1] - Yes, these definitions aren't entirely accurate, but they describe the kind of understanding a moderately technical user might have of the concepts.
Below are some example networks users may configure and an explanation of how that network would be implemented in practice.
A privileged user creates two (Xen) guests, each with a Virtual Network Interface. Without any special networking configuration, these two guests are connected to a default Virtual Network which contains a combined Virtual Bridge/Router/Firewall.
+-----------+ D +-----------+
| Guest | N D H | Guest |
| A | A N C | B |
| +---+ | T S P | +---+ |
| |NIC| | ^ ^ ^ | |NIC| |
+---+-+-+---+ +---+---+ +---+-+-+---+
^ | ^
| +--------+ +---+---+ +--------+ |
+-->+ vif1.0 +----+ vnbr0 +----+ vif2.0 +<--+
+--------+ +-------+ +--------+
Notes:
A privileged user does exactly the same thing as (1), but with QEMU guests.
D
N D H
A N C
T S P
^ ^ ^
+---+---+
|
+---+---+
+-----------+ | vnbr0 | +-----------+
| Guest | +---+---+ | Guest |
| A | | | B |
| +---+ | +---+---+ | +---+ |
| |NIC| | | vtap0 | | |NIC| |
+---+-+-+---+ +---+---+ +---+-+-+---+
^ +-------+ | +-------+ ^
| | | +---+---+ | | |
+------>+ VLAN0 +-+ VDE +-+ VLAN0 +<------+
| | +-------+ | |
+-------+ +-------+
Notes:
An unprivileged user does exactly the same thing as (2).
+-----------+ +-----------+
| Guest | +----+----+ | Guest |
| A | |userspace| | B |
| +---+ | | network | | +---+ |
| |NIC| | | stack | | |NIC| |
+---+-+-+---+ +----+----+ +---+-+-+---+
^ +-------+ | +-------+ ^
| | | +---+---+ | | |
+------>+ VLAN0 +-+ VDE +-+ VLAN0 +<------+
| | +-------+ | |
+-------+ +-------+
Notes:
Same as (2), except the user also creates two Xen guests.
+-----------+ D +-----------+
| Guest | N D H | Guest |
| A | A N C | B |
| +---+ | T S P | +---+ |
| |NIC| | ^ ^ ^ | |NIC| |
+---+-+-+---+ +---+---+ +---+-+-+---+
^ | ^
| +--------+ +---+---+ +--------+ |
+-->+ vif1.0 +----+ vnbr0 +----+ vif2.0 +<--+
+--------+ +---+---+ +--------+
|
+---+---+
| vtap0 |
+---+---+
|
+-------+ +--+--+ +-------+
+---->+ VLAN0 +----+ VDE +---+ VLAN0 +<-----+
| +-------+ +-----+ +-------+ |
V V
+---+-+-+---+ +---+-+-+---+
| |NIC| | | |NIC| |
| +---+ | | +---+ |
| Guest | | Guest |
| C | | D |
+-----------+ +-----------+
Notes:
Same as (3) except Guests A and C are connected to a Shared Physical Interface.
+-----------+ | D +-----------+
| Guest | ^ | N D H | Guest |
| A | | | A N C | B |
| +---+ | +---+---+ | T S P | +---+ |
| |NIC| | | eth0 | | ^ ^ ^ | |NIC| |
+---+-+-+---+ +---+---+ | +---+---+ +---+-+-+---+
^ | | | ^
| +--------+ +---+---+ | +---+---+ +--------+ |
+>+ vif1.0 +-+ ebr0 + | + vnbr0 +-+ vif2.0 +<-+
+--------+ +---+---+ | +---+---+ +--------+
| | |
+---+---+ | +---+---+
| vtap1 | | | vtap0 |
+---+---+ | +---+---+
| | |
+-------+ +--+--+ | +--+--+ +-------+
+->+ VLAN0 +--+ VDE + | + VDE +--+ VLAN0 +<-+
| +-------+ +-----+ | +-----+ +-------+ |
V | V
+---+-+-+---+ | +---+-+-+---+
| |NIC| | | | |NIC| |
| +---+ | | | +---+ |
| Guest | | | Guest |
| C | | | D |
+-----------+ | +-----------+
Notes:
Same as 2) except the QEMU guests are on a Virtual Network on another physical machine which is, in turn, connected to the Virtual Network on the first physical machine
+-----------+ D +-----------+
| Guest | N D H | Guest |
| A | A N C | B |
| +---+ | T S P | +---+ |
| |NIC| | ^ ^ ^ | |NIC| |
+---+-+-+---+ +---+---+ +---+-+-+---+
^ | ^
| +--------+ +---+---+ +--------+ |
+-->+ vif1.0 +----+ vnbr0 +----+ vif2.0 +<--+
+--------+ +---+---+ +--------+
|
+---+---+
| vtap0 |
+---+---+
|
+--+--+
| VDE |
+--+--+
|
First Physical Machine V
-------------------------------------------------------------
Second Physical Machine ^
|
+-------+ +--+--+ +-------+
+---->+ VLAN0 +----+ VDE +---+ VLAN0 +<-----+
| +-------+ +-----+ +-------+ |
V V
+---+-+-+---+ +---+-+-+---+
| |NIC| | | |NIC| |
| +---+ | | +---+ |
| Guest | | Guest |
| C | | D |
+-----------+ +-----------+
Notes:
One interesting thing to note from all of those examples is that although QEMU's networking options are very interesting, it doesn't actually make sense for a network to be implemented inside a guest. The network needs to be external to any guests, and so we use VDE to offer similar networking options to the ones QEMU provides. All QEMU needs to be able to do is to connect to VDE.
This isn't meant a UI specification, but just some notes on how this stuff might be exposed in virt-manager.
Parity with the current state of networking with Xen will be achieved by:
Implementing "shared physical interface" support in Fedora's initscripts and network configuration tool. It boils down to configuring the interface (e.g. eth0) something like:
ifcfg-peth0:
DEVICE=peth0
ONBOOT=yes
Bridge=eth0
HWADDR=00:30:48:30:73:19
ifcfg-eth0
DEVICE=eth0
Type=Bridge
ONBOOT=yes
BOOTPROTO=dhcp
Virtual Networks will be implemented in libvirt. First, there will be an XML description of Virtual Networks e.g.:
<network id="0">
<name>Foo</name>
<uuid>596a5d2171f48fb2e068e2386a5c413e</uuid>
<listen address="172.31.0.5" port="1234" />
<connections>
<connection address="172.31.0.6" port="4321" />
</conections>
<dhcp enabled="true">
<ip address="10.0.0.1"
netmask="255.255.255.0"
start="10.0.0.128"
end="10.0.0.254" />
</dhcp>
<forwarding enabled="true">
<incoming default="deny">
<allow port="123" domain="foobar" destport="321" />
</incoming>
<outgoing default="allow">
<deny port="25" />
</outgoing>
</forwarding>
<network>
In a manner similar to libvirt's QEMU support, there will be a daemon to manage Virtual Networks. The daemon will have access to a store of network definitions. The daemon will be responsible for managing the bridge devices, vde_switch/dhcp/dnses processes and the iptables rules needed for SNAT/DNAT etc.
virsh command line interface would look like:
$> virsh network-create foo.xml $> virsh network-dumpxml > foo.xml $> virsh network-define foo.xml $> virsh network-list $> virsh network-start Foo $> virsh network-stop Foo $> virsh network-restart Foo
The libvirt API for virtual networks would be modelled on the API for virtual machines:
/*
* Virtual Networks API
*/
/**
* virNetwork:
*
* a virNetwork is a private structure representing a virtual network.
*/
typedef struct _virNetwork virNetwork;
/**
* virNetworkPtr:
*
* a virNetworkPtr is pointer to a virNetwork private structure, this is the
* type used to reference a virtual network in the API.
*/
typedef virNetwork *virNetworkPtr;
/**
* virNetworkCreateFlags:
*
* Flags OR'ed together to provide specific behaviour when creating a
* Network.
*/
typedef enum {
VIR_NETWORK_NONE = 0
} virNetworkCreateFlags;
/*
* List active networks
*/
int virConnectNumOfNetworks (virConnectPtr conn);
int virConnectListNetworks (virConnectPtr conn,
int *ids,
int maxids);
/*
* List inactive networks
*/
int virConnectNumOfDefinedNetworks (virConnectPtr conn);
int virConnectListDefinedNetworks (virConnectPtr conn,
const char **names,
int maxnames);
/*
* Lookup network by name, id or uuid
*/
virNetworkPtr virNetworkLookupByName (virConnectPtr conn,
const char *name);
virNetworkPtr virNetworkLookupByID (virConnectPtr conn,
int id);
virNetworkPtr virNetworkLookupByUUID (virConnectPtr conn,
const unsigned char *uuid);
virNetworkPtr virNetworkLookupByUUIDString (virConnectPtr conn,
const char *uuid);
/*
* Create active transient network
*/
virNetworkPtr virNetworkCreateXML (virConnectPtr conn,
const char *xmlDesc,
unsigned int flags);
/*
* Define inactive persistent network
*/
virNetworkPtr virNetworkDefineXML (virConnectPtr conn,
const char *xmlDesc);
/*
* Delete persistent network
*/
int virNetworkUndefine (virNetworkPtr network);
/*
* Activate persistent network
*/
int virNetworkCreate (virNetworkPtr network);
/*
* Network destroy/free
*/
int virNetworkDestroy (virNetworkPtr network);
int virNetworkFree (virNetworkPtr network);
/*
* Network informations
*/
const char* virNetworkGetName (virNetworkPtr network);
unsigned int virNetworkGetID (virNetworkPtr network);
int virNetworkGetUUID (virNetworkPtr network,
unsigned char *uuid);
int virNetworkGetUUIDString (virNetworkPtr network,
char *buf);
char * virNetworkGetXMLDesc (virNetworkPtr network,
int flags);
Discussion points on the XML format and API:
Mark McLoughlin. Jan 15, 2007.