xenapi - cleanup network configuration
The Xenserver/XCP part of devstack was configuring the hypervisor's
connectivity to the outer world, by adding VLANs, and physical
interfaces to the virtual networks. It added a lot of complexity, and
made it hard to get started with XenServer. This patch removes that
extra complexity, so it is left as an exercise for the user.
Related to blueprint blueprint xenapi-devstack-cleanup
Change-Id: If3367335c3da8621d0afe1f6cae77511fbdbb3e2
diff --git a/tools/xen/README.md b/tools/xen/README.md
index 258d7a3..8f0c10d 100644
--- a/tools/xen/README.md
+++ b/tools/xen/README.md
@@ -8,6 +8,34 @@
machine on the Xenserver host (called OS domU). The VM uses the XAPI toolstack
to communicate with the host.
+The provided localrc helps to build a basic environment.
+The requirements are:
+ - An internet-enabled network with a DHCP server on it
+ - XenServer box plugged in to the same network
+This network will be used as the OpenStack management network. The VM Network
+and the Public Network will not be connected to any physical interfaces, only
+new virtual networks will be created by the `install_os_domU.sh` script.
+
+Steps to follow:
+ - Install XenServer
+ - Download Devstack to XenServer
+ - Customise `localrc`
+ - Start `install_os_domU.sh` script
+
+The `install_os_domU.sh` script will:
+ - Setup XenAPI plugins
+ - Create the named networks, if they don't exist
+ - Install an Ubuntu Virtual Machine, with 4 network interfaces:
+ - eth0 - internal xapi interface
+ - eth1 - VM interface, connected to `VM_BRIDGE_OR_NET_NAME` defaults to
+ `"OpenStack VM Network"`.
+ - eth2 - Management interface, connected to `MGT_BRIDGE_OR_NET_NAME`,
+ defaults to `xenbr0`, XenServer's bridge associated with the Hypervisors
+ `eth0`.
+ - eth3 - Public interface, connected to `PUB_BRIDGE_OR_NET_NAME` defaults to
+ `"OpenStack Public Network"`.
+ - Start devstack inside the created OpenStack VM
+
## Step 1: Install Xenserver
Install XenServer 5.6+ on a clean box. You can get XenServer by signing
up for an account on citrix.com, and then visiting:
@@ -15,13 +43,12 @@
For details on installation, see: http://wiki.openstack.org/XenServer/Install
-Here are some sample Xenserver network settings for when you are just
-getting started (Settings like this have been used with a laptop + cheap wifi router):
-
-* XenServer Host IP: 192.168.1.10
-* XenServer Netmask: 255.255.255.0
-* XenServer Gateway: 192.168.1.1
-* XenServer DNS: 192.168.1.1
+The XenServer IP configuration depends on your local network setup. If you are
+using dhcp, make a reservation for XenServer, so its IP address won't change
+over time. Make a note of the XenServer's IP address, as it has to be specified
+in `localrc`. The other option is to manually specify the IP setup for the
+XenServer box. Please make sure, that a gateway and a nameserver is configured,
+as `install_os_domU.sh` will connect to github.com to get source-code snapshots.
## Step 2: Download devstack
On your XenServer host, run the following commands as root:
@@ -32,45 +59,63 @@
## Step 3: Configure your localrc inside the devstack directory
Devstack uses a localrc for user-specific configuration. Note that
-the XENAPI_PASSWORD must be your dom0 root password.
+the `XENAPI_PASSWORD` must be your dom0 root password.
Of course, use real passwords if this machine is exposed.
cat > ./localrc <<EOF
+ # Passwords
+ # NOTE: these need to be specified, otherwise devstack will try
+ # to prompt for these passwords, blocking the install process.
+
MYSQL_PASSWORD=my_super_secret
SERVICE_TOKEN=my_super_secret
ADMIN_PASSWORD=my_super_secret
SERVICE_PASSWORD=my_super_secret
RABBIT_PASSWORD=my_super_secret
SWIFT_HASH="66a3d6b56c1f479c8b4e70ab5c2000f5"
- # This is the password for the OpenStack VM (for both stack and root users)
+ # This will be the password for the OpenStack VM (both stack and root users)
GUEST_PASSWORD=my_super_secret
# XenAPI parameters
- # IMPORTANT: The following must be set to your dom0 root password!
+ # NOTE: The following must be set to your XenServer root password!
+
XENAPI_PASSWORD=my_xenserver_root_password
+
XENAPI_CONNECTION_URL="http://address_of_your_xenserver"
VNCSERVER_PROXYCLIENT_ADDRESS=address_of_your_xenserver
- # Do not download the usual images yet!
+ # Do not download the usual images
IMAGE_URLS=""
# Explicitly set virt driver here
VIRT_DRIVER=xenserver
- # Explicitly set multi-host
+ # Explicitly enable multi-host
MULTI_HOST=1
# Give extra time for boot
ACTIVE_TIMEOUT=45
- # Host Interface, i.e. the interface on the nova vm you want to expose the
- # services on. Usually eth2 (management network) or eth3 (public network) and
- # not eth0 (private network with XenServer host) or eth1 (VM traffic network)
- # The default is eth3.
- # HOST_IP_IFACE=eth3
+ # Host Interface, i.e. the interface on the OpenStack vm you want to expose
+ # the services on. The default is eth3, which means the public network, but
+ # as the public network is going to be virtual, we are setting the services
+ # to listen on the management network, which defaults to 'xenbr0', the
+ # XenServer's network.
+ HOST_IP_IFACE=eth2
+
+ # Use DHCP server to configure the Management IP of OpenStack VM
+ MGT_IP="dhcp"
# Settings for netinstalling Ubuntu
- # UBUNTU_INST_RELEASE=precise
+ UBUNTU_INST_RELEASE=precise
- # First time Ubuntu network install params
- # UBUNTU_INST_IFACE="eth3"
- # UBUNTU_INST_IP="dhcp"
+ # First time Ubuntu network install params, use the DHCP server on the
+ # management network
+ UBUNTU_INST_IFACE="eth2"
+ UBUNTU_INST_IP="dhcp"
+
+ # NOTE: the value of FLAT_NETWORK_BRIDGE will automatically be determined
+ # by install_os_domU.sh script.
+
+ # Public IP address is aligned with the devstack defaults (see FLOATING_RANGE)
+ PUB_IP=172.24.4.10
+ PUB_NETMASK=255.255.255.0
EOF
## Step 4: Run `./install_os_domU.sh` from the `tools/xen` directory
diff --git a/tools/xen/functions b/tools/xen/functions
index 26ddb8b..c6e484d 100644
--- a/tools/xen/functions
+++ b/tools/xen/functions
@@ -93,3 +93,84 @@
fi
done
}
+
+function _create_new_network() {
+ local name_label
+ name_label=$1
+
+ xe network-create name-label="$name_label"
+}
+
+function _multiple_networks_with_name() {
+ local name_label
+ name_label=$1
+
+ # A comma indicates multiple matches
+ xe network-list name-label="$name_label" --minimal | grep -q ","
+}
+
+function _network_exists() {
+ local name_label
+ name_label=$1
+
+ ! [ -z $(xe network-list name-label="$name_label" --minimal) ]
+}
+
+function _bridge_exists() {
+ local bridge
+ bridge=$1
+
+ ! [ -z $(xe network-list bridge="$bridge" --minimal) ]
+}
+
+
+function setup_network() {
+ local bridge_or_net_name
+ bridge_or_net_name=$1
+
+ if ! _bridge_exists "$bridge_or_net_name"; then
+ if _network_exists "$bridge_or_net_name"; then
+ if _multiple_networks_with_name "$bridge_or_net_name"; then
+ cat >&2 << EOF
+ERROR: Multiple networks found matching name-label to "$bridge_or_net_name"
+please review your XenServer network configuration / localrc file.
+EOF
+ exit 1
+ fi
+ else
+ _create_new_network "$bridge_or_net_name"
+ fi
+ fi
+}
+
+function bridge_for() {
+ local bridge_or_net_name
+ bridge_or_net_name=$1
+
+ if _bridge_exists "$bridge_or_net_name"; then
+ echo "$bridge_or_net_name"
+ else
+ xe network-list name-label="$bridge_or_net_name" params=bridge --minimal
+ fi
+}
+
+function xenapi_ip_on() {
+ local bridge_or_net_name
+ bridge_or_net_name=$1
+
+ ifconfig $(bridge_for "$bridge_or_net_name") | grep "inet addr" | cut -d ":" -f2 | sed "s/ .*//"
+}
+
+function xenapi_is_listening_on() {
+ local bridge_or_net_name
+ bridge_or_net_name=$1
+
+ ! [ -z $(xenapi_ip_on "$bridge_or_net_name") ]
+}
+
+function parameter_is_specified() {
+ local parameter_name
+ parameter_name=$1
+
+ compgen -v | grep "$parameter_name"
+}
diff --git a/tools/xen/install_os_domU.sh b/tools/xen/install_os_domU.sh
index 0d5e31e..161d7e7 100755
--- a/tools/xen/install_os_domU.sh
+++ b/tools/xen/install_os_domU.sh
@@ -67,97 +67,28 @@
#
# Configure Networking
#
+setup_network "$VM_BRIDGE_OR_NET_NAME"
+setup_network "$MGT_BRIDGE_OR_NET_NAME"
+setup_network "$PUB_BRIDGE_OR_NET_NAME"
-# Helper to create networks
-# Uses echo trickery to return network uuid
-function create_network() {
- br=$1
- dev=$2
- vlan=$3
- netname=$4
- if [ -z $br ]
- then
- pif=$(xe_min pif-list device=$dev VLAN=$vlan)
- if [ -z $pif ]
- then
- net=$(xe network-create name-label=$netname)
- else
- net=$(xe_min network-list PIF-uuids=$pif)
- fi
- echo $net
- return 0
- fi
- if [ ! $(xe_min network-list params=bridge | grep -w --only-matching $br) ]
- then
- echo "Specified bridge $br does not exist"
- echo "If you wish to use defaults, please keep the bridge name empty"
- exit 1
- else
- net=$(xe_min network-list bridge=$br)
- echo $net
- fi
-}
-
-function errorcheck() {
- rc=$?
- if [ $rc -ne 0 ]
- then
- exit $rc
- fi
-}
-
-# Create host, vm, mgmt, pub networks on XenServer
-VM_NET=$(create_network "$VM_BR" "$VM_DEV" "$VM_VLAN" "vmbr")
-errorcheck
-MGT_NET=$(create_network "$MGT_BR" "$MGT_DEV" "$MGT_VLAN" "mgtbr")
-errorcheck
-PUB_NET=$(create_network "$PUB_BR" "$PUB_DEV" "$PUB_VLAN" "pubbr")
-errorcheck
-
-# Helper to create vlans
-function create_vlan() {
- dev=$1
- vlan=$2
- net=$3
- # VLAN -1 refers to no VLAN (physical network)
- if [ $vlan -eq -1 ]
- then
- return
- fi
- if [ -z $(xe_min vlan-list tag=$vlan) ]
- then
- pif=$(xe_min pif-list network-uuid=$net)
- # We created a brand new network this time
- if [ -z $pif ]
- then
- pif=$(xe_min pif-list device=$dev VLAN=-1)
- xe vlan-create pif-uuid=$pif vlan=$vlan network-uuid=$net
- else
- echo "VLAN does not exist but PIF attached to this network"
- echo "How did we reach here?"
- exit 1
- fi
- fi
-}
-
-# Create vlans for vm and management
-create_vlan $PUB_DEV $PUB_VLAN $PUB_NET
-create_vlan $VM_DEV $VM_VLAN $VM_NET
-create_vlan $MGT_DEV $MGT_VLAN $MGT_NET
-
-# Get final bridge names
-if [ -z $VM_BR ]; then
- VM_BR=$(xe_min network-list uuid=$VM_NET params=bridge)
-fi
-if [ -z $MGT_BR ]; then
- MGT_BR=$(xe_min network-list uuid=$MGT_NET params=bridge)
-fi
-if [ -z $PUB_BR ]; then
- PUB_BR=$(xe_min network-list uuid=$PUB_NET params=bridge)
+if parameter_is_specified "FLAT_NETWORK_BRIDGE"; then
+ cat >&2 << EOF
+ERROR: FLAT_NETWORK_BRIDGE is specified in localrc file
+This is considered as an error, as its value will be derived from the
+VM_BRIDGE_OR_NET_NAME variable's value.
+EOF
+ exit 1
fi
-# dom0 ip, XenAPI is assumed to be listening
-HOST_IP=${HOST_IP:-`ifconfig xenbr0 | grep "inet addr" | cut -d ":" -f2 | sed "s/ .*//"`}
+if ! xenapi_is_listening_on "$MGT_BRIDGE_OR_NET_NAME"; then
+ cat >&2 << EOF
+ERROR: XenAPI does not have an assigned IP address on the management network.
+please review your XenServer network configuration / localrc file.
+EOF
+ exit 1
+fi
+
+HOST_IP=$(xenapi_ip_on "$MGT_BRIDGE_OR_NET_NAME")
# Set up ip forwarding, but skip on xcp-xapi
if [ -a /etc/sysconfig/network ]; then
@@ -263,7 +194,15 @@
# create a new VM with the given template
# creating the correct VIFs and metadata
- $THIS_DIR/scripts/install-os-vpx.sh -t "$UBUNTU_INST_TEMPLATE_NAME" -v $VM_BR -m $MGT_BR -p $PUB_BR -l $GUEST_NAME -r $OSDOMU_MEM_MB -k "flat_network_bridge=${VM_BR}"
+ FLAT_NETWORK_BRIDGE=$(bridge_for "$VM_BRIDGE_OR_NET_NAME")
+ $THIS_DIR/scripts/install-os-vpx.sh \
+ -t "$UBUNTU_INST_TEMPLATE_NAME" \
+ -v "$VM_BRIDGE_OR_NET_NAME" \
+ -m "$MGT_BRIDGE_OR_NET_NAME" \
+ -p "$PUB_BRIDGE_OR_NET_NAME" \
+ -l "$GUEST_NAME" \
+ -r "$OSDOMU_MEM_MB" \
+ -k "flat_network_bridge=${FLAT_NETWORK_BRIDGE}"
# wait for install to finish
wait_for_VM_to_halt
diff --git a/tools/xen/scripts/install-os-vpx.sh b/tools/xen/scripts/install-os-vpx.sh
index 241296b..6105a1e 100755
--- a/tools/xen/scripts/install-os-vpx.sh
+++ b/tools/xen/scripts/install-os-vpx.sh
@@ -235,7 +235,7 @@
create_vm_vif()
{
local v="$1"
- echo "Installing management interface on $BRIDGE_V."
+ echo "Installing VM interface on $BRIDGE_V."
local out_network_uuid=$(find_network "$BRIDGE_V")
create_vif "$v" "$out_network_uuid" "1" >/dev/null
}
diff --git a/tools/xen/xenrc b/tools/xen/xenrc
index e50f954..7aaafd2 100644
--- a/tools/xen/xenrc
+++ b/tools/xen/xenrc
@@ -16,6 +16,16 @@
OSDOMU_MEM_MB=1024
OSDOMU_VDI_GB=8
+# Network mapping. Specify bridge names or network names. Network names may
+# differ across localised versions of XenServer. If a given bridge/network
+# was not found, a new network will be created with the specified name.
+
+# The management network is specified by the bridge name. xenbr0 is usually
+# the name of the bridge of the network associated with the hypervisor's eth0.
+MGT_BRIDGE_OR_NET_NAME="xenbr0"
+VM_BRIDGE_OR_NET_NAME="OpenStack VM Network"
+PUB_BRIDGE_OR_NET_NAME="OpenStack Public Network"
+
# VM Password
GUEST_PASSWORD=${GUEST_PASSWORD:-secrete}
@@ -28,28 +38,17 @@
# Our nova host's network info
#
-# A host-only ip that let's the interface come up, otherwise unused
-VM_IP=${VM_IP:-10.255.255.255}
+# Management network
MGT_IP=${MGT_IP:-172.16.100.55}
-PUB_IP=${PUB_IP:-192.168.1.55}
+MGT_NETMASK=${MGT_NETMASK:-255.255.255.0}
+
+# VM Network
+VM_IP=${VM_IP:-10.255.255.255}
+VM_NETMASK=${VM_NETMASK:-255.255.255.0}
# Public network
+PUB_IP=${PUB_IP:-192.168.1.55}
PUB_NETMASK=${PUB_NETMASK:-255.255.255.0}
-PUB_BR=${PUB_BR:-"xenbr0"}
-PUB_VLAN=${PUB_VLAN:--1}
-PUB_DEV=${PUB_DEV:-eth0}
-
-# VM network params
-VM_NETMASK=${VM_NETMASK:-255.255.255.0}
-VM_BR=${VM_BR:-""}
-VM_VLAN=${VM_VLAN:-100}
-VM_DEV=${VM_DEV:-eth0}
-
-# MGMT network params
-MGT_NETMASK=${MGT_NETMASK:-255.255.255.0}
-MGT_BR=${MGT_BR:-""}
-MGT_VLAN=${MGT_VLAN:-101}
-MGT_DEV=${MGT_DEV:-eth0}
# Decide if you should enable eth0,
# the guest installer network