Merge "XenAPI: Use round-robin mirror for Ubuntu"
diff --git a/doc/source/guides/devstack-with-nested-kvm.rst b/doc/source/guides/devstack-with-nested-kvm.rst
new file mode 100644
index 0000000..2538c8d
--- /dev/null
+++ b/doc/source/guides/devstack-with-nested-kvm.rst
@@ -0,0 +1,139 @@
+=======================================================
+Configure DevStack with KVM-based Nested Virtualization
+=======================================================
+
+When using virtualization technologies like KVM, one can take advantage
+of "Nested VMX" (i.e. the ability to run KVM on KVM) so that the VMs in
+cloud (Nova guests) can run relatively faster than with plain QEMU
+emulation.
+
+Kernels shipped with Linux distributions doesn't have this enabled by
+default. This guide outlines the configuration details to enable nested
+virtualization in KVM-based environments. And how to setup DevStack
+(that'll run in a VM) to take advantage of this.
+
+
+Nested Virtualization Configuration
+===================================
+
+Configure Nested KVM for Intel-based Machines
+---------------------------------------------
+
+Procedure to enable nested KVM virtualization on AMD-based machines.
+
+Check if the nested KVM Kernel parameter is enabled:
+
+::
+
+ cat /sys/module/kvm_intel/parameters/nested
+ N
+
+Temporarily remove the KVM intel Kernel module, enable nested
+virtualization to be persistent across reboots and add the Kernel
+module back:
+
+::
+
+ sudo rmmod kvm-intel
+ sudo sh -c "echo 'options kvm-intel nested=y' >> /etc/modprobe.d/dist.conf"
+ sudo modprobe kvm-intel
+
+Ensure the Nested KVM Kernel module parameter for Intel is enabled on
+the host:
+
+::
+
+ cat /sys/module/kvm_intel/parameters/nested
+ Y
+
+ modinfo kvm_intel | grep nested
+ parm: nested:bool
+
+Start your VM, now it should have KVM capabilities -- you can verify
+that by ensuring `/dev/kvm` character device is present.
+
+
+Configure Nested KVM for AMD-based Machines
+--------------------------------------------
+
+Procedure to enable nested KVM virtualization on AMD-based machines.
+
+Check if the nested KVM Kernel parameter is enabled:
+
+::
+
+ cat /sys/module/kvm_amd/parameters/nested
+ 0
+
+
+Temporarily remove the KVM AMD Kernel module, enable nested
+virtualization to be persistent across reboots and add the Kernel module
+back:
+
+::
+
+ sudo rmmod kvm-amd
+ sudo sh -c "echo 'options amd nested=1' >> /etc/modprobe.d/dist.conf"
+ sudo modprobe kvm-amd
+
+Ensure the Nested KVM Kernel module parameter for AMD is enabled on the
+host:
+
+::
+
+ cat /sys/module/kvm_amd/parameters/nested
+ 1
+
+ modinfo kvm_amd | grep -i nested
+ parm: nested:int
+
+To make the above value persistent across reboots, add an entry in
+/etc/modprobe.ddist.conf so it looks as below::
+
+ cat /etc/modprobe.d/dist.conf
+ options kvm-amd nested=y
+
+
+Expose Virtualization Extensions to DevStack VM
+-----------------------------------------------
+
+Edit the VM's libvirt XML configuration via `virsh` utility:
+
+::
+
+ sudo virsh edit devstack-vm
+
+Add the below snippet to expose the host CPU features to the VM:
+
+::
+
+ <cpu mode='host-passthrough'>
+ </cpu>
+
+
+Ensure DevStack VM is Using KVM
+-------------------------------
+
+Before invoking ``stack.sh`` in the VM, ensure that KVM is enabled. This
+can be verified by checking for the presence of the file `/dev/kvm` in
+your VM. If it is present, DevStack will default to using the config
+attribute `virt_type = kvm` in `/etc/nova.conf`; otherwise, it'll fall
+back to `virt_type=qemu`, i.e. plain QEMU emulation.
+
+Optionally, to explicitly set the type of virtualization, to KVM, by the
+libvirt driver in Nova, the below config attribute can be used in
+DevStack's ``local.conf``:
+
+::
+
+ LIBVIRT_TYPE=kvm
+
+
+Once DevStack is configured succesfully, verify if the Nova instances
+are using KVM by noticing the QEMU CLI invoked by Nova is using the
+parameter `accel=kvm`, e.g.:
+
+::
+
+ ps -ef | grep -i qemu
+ root 29773 1 0 11:24 ? 00:00:00 /usr/bin/qemu-system-x86_64 -machine accel=kvm [. . .]
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 0763fb8..0790d1e 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -66,6 +66,7 @@
guides/single-machine
guides/multinode-lab
guides/neutron
+ guides/devstack-with-nested-kvm
All-In-One Single VM
--------------------
@@ -94,6 +95,13 @@
This guide is meant for building lab environments with a dedicated
control node and multiple compute nodes.
+DevStack with KVM-based Nested Virtualization
+---------------------------------------------
+
+Procedure to setup :doc:`DevStack with KVM-based Nested Virtualization
+<guides/devstack-with-nested-kvm>`. With this setup, Nova instances
+will be more performant than with plain QEMU emulation.
+
DevStack Documentation
======================
diff --git a/doc/source/plugins.rst b/doc/source/plugins.rst
index d1f7377..8bb92ed 100644
--- a/doc/source/plugins.rst
+++ b/doc/source/plugins.rst
@@ -16,7 +16,7 @@
The script in ``extras.d`` is expected to be mostly a dispatcher to
functions in a ``lib/*`` script. The scripts are named with a
zero-padded two digits sequence number prefix to control the order that
-the scripts are called, and with a suffix of ``.sh``. DevSack reserves
+the scripts are called, and with a suffix of ``.sh``. DevStack reserves
for itself the sequence numbers 00 through 09 and 90 through 99.
Below is a template that shows handlers for the possible command-line
diff --git a/extras.d/70-gantt.sh b/extras.d/70-gantt.sh
deleted file mode 100644
index ac1efba..0000000
--- a/extras.d/70-gantt.sh
+++ /dev/null
@@ -1,31 +0,0 @@
-# gantt.sh - Devstack extras script to install Gantt
-
-if is_service_enabled n-sch; then
- disable_service gantt
-fi
-
-if is_service_enabled gantt; then
- if [[ "$1" == "source" ]]; then
- # Initial source
- source $TOP_DIR/lib/gantt
- elif [[ "$1" == "stack" && "$2" == "install" ]]; then
- echo_summary "Installing Gantt"
- install_gantt
- cleanup_gantt
- elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
- echo_summary "Configuring Gantt"
- configure_gantt
-
- elif [[ "$1" == "stack" && "$2" == "extra" ]]; then
- # Initialize gantt
- init_gantt
-
- # Start gantt
- echo_summary "Starting Gantt"
- start_gantt
- fi
-
- if [[ "$1" == "unstack" ]]; then
- stop_gantt
- fi
-fi
diff --git a/extras.d/70-tuskar.sh b/extras.d/70-tuskar.sh
deleted file mode 100644
index 551916f..0000000
--- a/extras.d/70-tuskar.sh
+++ /dev/null
@@ -1,204 +0,0 @@
-# Install and start the **Tuskar** service
-#
-# To enable, add the following to your localrc
-#
-# enable_service tuskar
-# enable_service tuskar-api
-
-
-if is_service_enabled tuskar; then
- if [[ "$1" == "source" ]]; then
- # Initial source, do nothing as functions sourced
- # are below rather than in lib/tuskar
- echo_summary "source extras tuskar"
- elif [[ "$1" == "stack" && "$2" == "install" ]]; then
- echo_summary "Installing Tuskar"
- install_tuskarclient
- install_tuskar
- elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
- echo_summary "Configuring Tuskar"
- configure_tuskar
- configure_tuskarclient
-
- if is_service_enabled key; then
- create_tuskar_accounts
- fi
-
- elif [[ "$1" == "stack" && "$2" == "extra" ]]; then
- echo_summary "Initializing Tuskar"
- init_tuskar
- start_tuskar
- fi
-
- if [[ "$1" == "unstack" ]]; then
- stop_tuskar
- fi
-fi
-
-# library code (equivalent to lib/tuskar)
-# ---------
-# - install_tuskarclient
-# - install_tuskar
-# - configure_tuskarclient
-# - configure_tuskar
-# - init_tuskar
-# - start_tuskar
-# - stop_tuskar
-# - cleanup_tuskar
-
-# Save trace setting
-XTRACE=$(set +o | grep xtrace)
-set +o xtrace
-
-
-# Defaults
-# --------
-
-# tuskar repos
-TUSKAR_REPO=${TUSKAR_REPO:-${GIT_BASE}/openstack/tuskar.git}
-TUSKAR_BRANCH=${TUSKAR_BRANCH:-master}
-
-TUSKARCLIENT_REPO=${TUSKARCLIENT_REPO:-${GIT_BASE}/openstack/python-tuskarclient.git}
-TUSKARCLIENT_BRANCH=${TUSKARCLIENT_BRANCH:-master}
-
-# set up default directories
-TUSKAR_DIR=$DEST/tuskar
-TUSKARCLIENT_DIR=$DEST/python-tuskarclient
-TUSKAR_AUTH_CACHE_DIR=${TUSKAR_AUTH_CACHE_DIR:-/var/cache/tuskar}
-TUSKAR_STANDALONE=$(trueorfalse False TUSKAR_STANDALONE)
-TUSKAR_CONF_DIR=/etc/tuskar
-TUSKAR_CONF=$TUSKAR_CONF_DIR/tuskar.conf
-TUSKAR_API_HOST=${TUSKAR_API_HOST:-$HOST_IP}
-TUSKAR_API_PORT=${TUSKAR_API_PORT:-8585}
-
-# Tell Tempest this project is present
-TEMPEST_SERVICES+=,tuskar
-
-# Functions
-# ---------
-
-# Test if any Tuskar services are enabled
-# is_tuskar_enabled
-function is_tuskar_enabled {
- [[ ,${ENABLED_SERVICES} =~ ,"tuskar-" ]] && return 0
- return 1
-}
-
-# cleanup_tuskar() - Remove residual data files, anything left over from previous
-# runs that a clean run would need to clean up
-function cleanup_tuskar {
- sudo rm -rf $TUSKAR_AUTH_CACHE_DIR
-}
-
-# configure_tuskar() - Set config files, create data dirs, etc
-function configure_tuskar {
- setup_develop $TUSKAR_DIR
-
- if [[ ! -d $TUSKAR_CONF_DIR ]]; then
- sudo mkdir -p $TUSKAR_CONF_DIR
- fi
- sudo chown $STACK_USER $TUSKAR_CONF_DIR
- # remove old config files
- rm -f $TUSKAR_CONF_DIR/tuskar-*.conf
-
- TUSKAR_POLICY_FILE=$TUSKAR_CONF_DIR/policy.json
-
- cp $TUSKAR_DIR/etc/tuskar/policy.json $TUSKAR_POLICY_FILE
- cp $TUSKAR_DIR/etc/tuskar/tuskar.conf.sample $TUSKAR_CONF
-
- # common options
- iniset $TUSKAR_CONF database connection `database_connection_url tuskar`
-
- # logging
- iniset $TUSKAR_CONF DEFAULT debug $ENABLE_DEBUG_LOG_LEVEL
- iniset $TUSKAR_CONF DEFAULT use_syslog $SYSLOG
- if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ]; then
- # Add color to logging output
- setup_colorized_logging $TUSKAR_CONF DEFAULT tenant user
- fi
-
- configure_auth_token_middleware $TUSKAR_CONF tuskar $TUSKAR_AUTH_CACHE_DIR
-
- if is_ssl_enabled_service "key"; then
- iniset $TUSKAR_CONF clients_keystone ca_file $SSL_BUNDLE_FILE
- fi
-
- iniset $TUSKAR_CONF tuskar_api bind_port $TUSKAR_API_PORT
-
-}
-
-# init_tuskar() - Initialize database
-function init_tuskar {
-
- # (re)create tuskar database
- recreate_database tuskar
-
- tuskar-dbsync --config-file $TUSKAR_CONF
- create_tuskar_cache_dir
-}
-
-# create_tuskar_cache_dir() - Part of the init_tuskar() process
-function create_tuskar_cache_dir {
- # Create cache dirs
- sudo mkdir -p $TUSKAR_AUTH_CACHE_DIR
- sudo chown $STACK_USER $TUSKAR_AUTH_CACHE_DIR
-}
-
-# install_tuskarclient() - Collect source and prepare
-function install_tuskarclient {
- git_clone $TUSKARCLIENT_REPO $TUSKARCLIENT_DIR $TUSKARCLIENT_BRANCH
- setup_develop $TUSKARCLIENT_DIR
-}
-
-# configure_tuskarclient() - Set config files, create data dirs, etc
-function configure_tuskarclient {
- setup_develop $TUSKARCLIENT_DIR
-}
-
-# install_tuskar() - Collect source and prepare
-function install_tuskar {
- git_clone $TUSKAR_REPO $TUSKAR_DIR $TUSKAR_BRANCH
-}
-
-# start_tuskar() - Start running processes, including screen
-function start_tuskar {
- run_process tuskar-api "tuskar-api --config-file=$TUSKAR_CONF"
-}
-
-# stop_tuskar() - Stop running processes
-function stop_tuskar {
- # Kill the screen windows
- local serv
- for serv in tuskar-api; do
- stop_process $serv
- done
-}
-
-# create_tuskar_accounts() - Set up common required tuskar accounts
-function create_tuskar_accounts {
- # migrated from files/keystone_data.sh
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- local admin_role=$(openstack role list | awk "/ admin / { print \$2 }")
-
- local tuskar_user=$(get_or_create_user "tuskar" "$SERVICE_PASSWORD")
- get_or_add_user_role $admin_role $tuskar_user $service_tenant
-
- if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
-
- local tuskar_service=$(get_or_create_service "tuskar" \
- "management" "Tuskar Management Service")
- get_or_create_endpoint $tuskar_service \
- "$REGION_NAME" \
- "$SERVICE_PROTOCOL://$TUSKAR_API_HOST:$TUSKAR_API_PORT" \
- "$SERVICE_PROTOCOL://$TUSKAR_API_HOST:$TUSKAR_API_PORT" \
- "$SERVICE_PROTOCOL://$TUSKAR_API_HOST:$TUSKAR_API_PORT"
- fi
-}
-
-# Restore xtrace
-$XTRACE
-
-# Tell emacs to use shell-script-mode
-## Local variables:
-## mode: shell-script
-## End:
diff --git a/files/debs/neutron b/files/debs/neutron
index 5a59b22..3f4b6d2 100644
--- a/files/debs/neutron
+++ b/files/debs/neutron
@@ -6,6 +6,7 @@
libmysqlclient-dev # testonly
mysql-server #NOPRIME
sudo
+postgresql-server-dev-all # testonly
python-iso8601
python-paste
python-routes
diff --git a/files/rpms-suse/neutron b/files/rpms-suse/neutron
index 50ee145..66d6e4c 100644
--- a/files/rpms-suse/neutron
+++ b/files/rpms-suse/neutron
@@ -5,6 +5,7 @@
iptables
iputils
mariadb # NOPRIME
+postgresql-devel # testonly
python-eventlet
python-greenlet
python-iso8601
diff --git a/files/rpms/neutron b/files/rpms/neutron
index 59152d6..d11dab7 100644
--- a/files/rpms/neutron
+++ b/files/rpms/neutron
@@ -9,6 +9,7 @@
mysql-devel # testonly
mysql-server # NOPRIME
openvswitch # NOPRIME
+postgresql-devel # testonly
python-eventlet
python-greenlet
python-iso8601
diff --git a/lib/ceilometer b/lib/ceilometer
index f1617fb..f03bab2 100644
--- a/lib/ceilometer
+++ b/lib/ceilometer
@@ -105,13 +105,10 @@
# SERVICE_TENANT_NAME ceilometer ResellerAdmin (if Swift is enabled)
function create_ceilometer_accounts {
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- local admin_role=$(openstack role list | awk "/ admin / { print \$2 }")
-
# Ceilometer
if [[ "$ENABLED_SERVICES" =~ "ceilometer-api" ]]; then
- local ceilometer_user=$(get_or_create_user "ceilometer" "$SERVICE_PASSWORD")
- get_or_add_user_role $admin_role $ceilometer_user $service_tenant
+
+ create_service_user "ceilometer" "admin"
if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
local ceilometer_service=$(get_or_create_service "ceilometer" \
diff --git a/lib/cinder b/lib/cinder
index 937689a..12ba51e 100644
--- a/lib/cinder
+++ b/lib/cinder
@@ -330,14 +330,10 @@
# Migrated from keystone_data.sh
function create_cinder_accounts {
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- local admin_role=$(openstack role list | awk "/ admin / { print \$2 }")
-
# Cinder
if [[ "$ENABLED_SERVICES" =~ "c-api" ]]; then
- local cinder_user=$(get_or_create_user "cinder" "$SERVICE_PASSWORD")
- get_or_add_user_role $admin_role $cinder_user $service_tenant
+ create_service_user "cinder" "admin"
if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
diff --git a/lib/gantt b/lib/gantt
deleted file mode 100644
index 5bd28c2..0000000
--- a/lib/gantt
+++ /dev/null
@@ -1,98 +0,0 @@
-#!/bin/bash
-#
-# lib/gantt
-# Install and start **Gantt** scheduler service
-
-# Dependencies:
-#
-# - functions
-# - DEST, DATA_DIR, STACK_USER must be defined
-
-# stack.sh
-# ---------
-# - install_gantt
-# - configure_gantt
-# - init_gantt
-# - start_gantt
-# - stop_gantt
-# - cleanup_gantt
-
-# Save trace setting
-XTRACE=$(set +o | grep xtrace)
-set +o xtrace
-
-# Defaults
-# --------
-
-# set up default directories
-GANTT_DIR=$DEST/gantt
-GANTT_STATE_PATH=${GANTT_STATE_PATH:=$DATA_DIR/gantt}
-GANTT_REPO=${GANTT_REPO:-${GIT_BASE}/openstack/gantt.git}
-GANTT_BRANCH=${GANTT_BRANCH:-master}
-
-GANTTCLIENT_DIR=$DEST/python-ganttclient
-GANTTCLIENT_REPO=${GANTT_REPO:-${GIT_BASE}/openstack/python-ganttclient.git}
-GANTTCLIENT_BRANCH=${GANTT_BRANCH:-master}
-
-# eventually we will have a separate gantt config
-# file but for compatibility reasone stick with
-# nova.conf for now
-GANTT_CONF_DIR=${GANTT_CONF_DIR:-/etc/nova}
-GANTT_CONF=$GANTT_CONF_DIR/nova.conf
-
-# Support entry points installation of console scripts
-GANTT_BIN_DIR=$(get_python_exec_prefix)
-
-
-# Functions
-# ---------
-
-# cleanup_gantt() - Remove residual data files, anything left over from previous
-# runs that a clean run would need to clean up
-function cleanup_gantt {
- echo "Cleanup Gantt"
-}
-
-# configure_gantt() - Set config files, create data dirs, etc
-function configure_gantt {
- echo "Configure Gantt"
-}
-
-# init_gantt() - Initialize database and volume group
-function init_gantt {
- echo "Initialize Gantt"
-}
-
-# install_gantt() - Collect source and prepare
-function install_gantt {
- git_clone $GANTT_REPO $GANTT_DIR $GANTT_BRANCH
- setup_develop $GANTT_DIR
-}
-
-# install_ganttclient() - Collect source and prepare
-function install_ganttclient {
- echo "Install Gantt Client"
-# git_clone $GANTTCLIENT_REPO $GANTTCLIENT_DIR $GANTTCLIENT_BRANCH
-# setup_develop $GANTTCLIENT_DIR
-}
-
-# start_gantt() - Start running processes, including screen
-function start_gantt {
- if is_service_enabled gantt; then
- run_process gantt "$GANTT_BIN_DIR/gantt-scheduler --config-file $GANTT_CONF"
- fi
-}
-
-# stop_gantt() - Stop running processes
-function stop_gantt {
- echo "Stop Gantt"
- stop_process gantt
-}
-
-# Restore xtrace
-$XTRACE
-
-# Tell emacs to use shell-script-mode
-## Local variables:
-## mode: shell-script
-## End:
diff --git a/lib/glance b/lib/glance
index bee57a3..0340c21 100644
--- a/lib/glance
+++ b/lib/glance
@@ -232,8 +232,7 @@
function create_glance_accounts {
if is_service_enabled g-api; then
- local glance_user=$(get_or_create_user "glance" "$SERVICE_PASSWORD")
- get_or_add_user_role service $glance_user $SERVICE_TENANT_NAME
+ create_service_user "glance"
# required for swift access
if is_service_enabled s-proxy; then
diff --git a/lib/heat b/lib/heat
index 58439d6..c102163 100644
--- a/lib/heat
+++ b/lib/heat
@@ -134,10 +134,6 @@
iniset $HEAT_CONF keystone_authtoken cafile $SSL_BUNDLE_FILE
iniset $HEAT_CONF keystone_authtoken signing_dir $HEAT_AUTH_CACHE_DIR
- if is_ssl_enabled_service "key"; then
- iniset $HEAT_CONF clients_keystone ca_file $SSL_BUNDLE_FILE
- fi
-
# ec2authtoken
iniset $HEAT_CONF ec2authtoken auth_uri $KEYSTONE_SERVICE_URI/v2.0
@@ -246,12 +242,7 @@
# create_heat_accounts() - Set up common required heat accounts
function create_heat_accounts {
- # migrated from files/keystone_data.sh
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- local admin_role=$(openstack role list | awk "/ admin / { print \$2 }")
-
- local heat_user=$(get_or_create_user "heat" "$SERVICE_PASSWORD")
- get_or_add_user_role $admin_role $heat_user $service_tenant
+ create_service_user "heat" "admin"
if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
diff --git a/lib/ironic b/lib/ironic
index fced294..921bcf1 100644
--- a/lib/ironic
+++ b/lib/ironic
@@ -358,15 +358,11 @@
# service ironic admin # if enabled
function create_ironic_accounts {
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- local admin_role=$(openstack role list | awk "/ admin / { print \$2 }")
-
# Ironic
if [[ "$ENABLED_SERVICES" =~ "ir-api" ]]; then
# Get ironic user if exists
- local ironic_user=$(get_or_create_user "ironic" "$SERVICE_PASSWORD")
- get_or_add_user_role $admin_role $ironic_user $service_tenant
+ create_service_user "ironic" "admin"
if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
diff --git a/lib/keystone b/lib/keystone
index d5ccc2f..79806b8 100644
--- a/lib/keystone
+++ b/lib/keystone
@@ -309,8 +309,9 @@
setup_colorized_logging $KEYSTONE_CONF DEFAULT
fi
+ iniset $KEYSTONE_CONF DEFAULT debug $ENABLE_DEBUG_LOG_LEVEL
+
if [ "$KEYSTONE_USE_MOD_WSGI" == "True" ]; then
- iniset $KEYSTONE_CONF DEFAULT debug $ENABLE_DEBUG_LOG_LEVEL
# Eliminate the %(asctime)s.%(msecs)03d from the log format strings
iniset $KEYSTONE_CONF DEFAULT logging_context_format_string "%(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s"
iniset $KEYSTONE_CONF DEFAULT logging_default_format_string "%(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s"
@@ -414,6 +415,20 @@
fi
}
+# Create a user that is capable of verifying keystone tokens for use with auth_token middleware.
+#
+# create_service_user <name> [role]
+#
+# The role defaults to the service role. It is allowed to be provided as optional as historically
+# a lot of projects have configured themselves with the admin or other role here if they are
+# using this user for other purposes beyond simply auth_token middleware.
+function create_service_user {
+ local role=${2:-service}
+
+ local user=$(get_or_create_user "$1" "$SERVICE_PASSWORD")
+ get_or_add_user_role "$role" "$user" "$SERVICE_TENANT_NAME"
+}
+
# Configure the service to use the auth token middleware.
#
# configure_auth_token_middleware conf_file admin_user signing_dir [section]
@@ -532,12 +547,8 @@
tail_log key /var/log/$APACHE_NAME/keystone.log
tail_log key-access /var/log/$APACHE_NAME/keystone_access.log
else
- local EXTRA_PARAMS=""
- if [ "$ENABLE_DEBUG_LOG_LEVEL" == "True" ]; then
- EXTRA_PARAMS="--debug"
- fi
# Start Keystone in a screen window
- run_process key "$KEYSTONE_DIR/bin/keystone-all --config-file $KEYSTONE_CONF $EXTRA_PARAMS"
+ run_process key "$KEYSTONE_DIR/bin/keystone-all --config-file $KEYSTONE_CONF"
fi
echo "Waiting for keystone to start..."
diff --git a/lib/neutron b/lib/neutron
index 2c7ec94..15a5f00 100755
--- a/lib/neutron
+++ b/lib/neutron
@@ -10,24 +10,25 @@
# ``stack.sh`` calls the entry points in this order:
#
-# - install_neutron
-# - install_neutronclient
# - install_neutron_agent_packages
+# - install_neutronclient
+# - install_neutron
# - install_neutron_third_party
# - configure_neutron
# - init_neutron
# - configure_neutron_third_party
# - init_neutron_third_party
# - start_neutron_third_party
-# - create_neutron_cache_dir
# - create_nova_conf_neutron
# - start_neutron_service_and_check
+# - check_neutron_third_party_integration
# - start_neutron_agents
# - create_neutron_initial_network
# - setup_neutron_debug
#
# ``unstack.sh`` calls the entry points in this order:
#
+# - teardown_neutron_debug
# - stop_neutron
# - stop_neutron_third_party
# - cleanup_neutron
@@ -507,14 +508,9 @@
# Migrated from keystone_data.sh
function create_neutron_accounts {
-
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- local service_role=$(openstack role list | awk "/ service / { print \$2 }")
-
if [[ "$ENABLED_SERVICES" =~ "q-svc" ]]; then
- local neutron_user=$(get_or_create_user "neutron" "$SERVICE_PASSWORD")
- get_or_add_user_role $service_role $neutron_user $service_tenant
+ create_service_user "neutron"
if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
diff --git a/lib/nova b/lib/nova
index 0f4729f..c760066 100644
--- a/lib/nova
+++ b/lib/nova
@@ -353,14 +353,10 @@
# SERVICE_TENANT_NAME nova ResellerAdmin (if Swift is enabled)
function create_nova_accounts {
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- local admin_role=$(openstack role list | awk "/ admin / { print \$2 }")
-
# Nova
if [[ "$ENABLED_SERVICES" =~ "n-api" ]]; then
- local nova_user=$(get_or_create_user "nova" "$SERVICE_PASSWORD")
- get_or_add_user_role $admin_role $nova_user $service_tenant
+ create_service_user "nova" "admin"
if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
diff --git a/lib/sahara b/lib/sahara
index 44c06d3..cb6ecc3 100644
--- a/lib/sahara
+++ b/lib/sahara
@@ -61,11 +61,7 @@
# service sahara admin
function create_sahara_accounts {
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- local admin_role=$(openstack role list | awk "/ admin / { print \$2 }")
-
- local sahara_user=$(get_or_create_user "sahara" "$SERVICE_PASSWORD")
- get_or_add_user_role $admin_role $sahara_user $service_tenant
+ create_service_user "sahara" "admin"
if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
diff --git a/lib/swift b/lib/swift
index 683bc17..d9f750c 100644
--- a/lib/swift
+++ b/lib/swift
@@ -601,12 +601,9 @@
KEYSTONE_CATALOG_BACKEND=${KEYSTONE_CATALOG_BACKEND:-sql}
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- local admin_role=$(openstack role list | awk "/ admin / { print \$2 }")
local another_role=$(openstack role list | awk "/ anotherrole / { print \$2 }")
- local swift_user=$(get_or_create_user "swift" "$SERVICE_PASSWORD")
- get_or_add_user_role $admin_role $swift_user $service_tenant
+ create_service_user "swift" "admin"
if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
@@ -623,7 +620,7 @@
die_if_not_set $LINENO swift_tenant_test1 "Failure creating swift_tenant_test1"
SWIFT_USER_TEST1=$(get_or_create_user swiftusertest1 $swiftusertest1_password "test@example.com")
die_if_not_set $LINENO SWIFT_USER_TEST1 "Failure creating SWIFT_USER_TEST1"
- get_or_add_user_role $admin_role $SWIFT_USER_TEST1 $swift_tenant_test1
+ get_or_add_user_role admin $SWIFT_USER_TEST1 $swift_tenant_test1
local swift_user_test3=$(get_or_create_user swiftusertest3 $swiftusertest3_password "test3@example.com")
die_if_not_set $LINENO swift_user_test3 "Failure creating swift_user_test3"
@@ -634,7 +631,7 @@
local swift_user_test2=$(get_or_create_user swiftusertest2 $swiftusertest2_password "test2@example.com")
die_if_not_set $LINENO swift_user_test2 "Failure creating swift_user_test2"
- get_or_add_user_role $admin_role $swift_user_test2 $swift_tenant_test2
+ get_or_add_user_role admin $swift_user_test2 $swift_tenant_test2
local swift_domain=$(get_or_create_domain swift_test 'Used for swift functional testing')
die_if_not_set $LINENO swift_domain "Failure creating swift_test domain"
@@ -644,7 +641,7 @@
local swift_user_test4=$(get_or_create_user swiftusertest4 $swiftusertest4_password "test4@example.com" $swift_domain)
die_if_not_set $LINENO swift_user_test4 "Failure creating swift_user_test4"
- get_or_add_user_role $admin_role $swift_user_test4 $swift_tenant_test4
+ get_or_add_user_role admin $swift_user_test4 $swift_tenant_test4
}
# init_swift() - Initialize rings
diff --git a/lib/tempest b/lib/tempest
index 86f30b4..777d03e 100644
--- a/lib/tempest
+++ b/lib/tempest
@@ -95,7 +95,8 @@
# configure_tempest() - Set config files, create data dirs, etc
function configure_tempest {
- setup_develop $TEMPEST_DIR
+ # install testr since its used to process tempest logs
+ pip_install `grep -h testrepository $REQUIREMENTS_DIR/global-requirements.txt | cut -d\# -f1`
local image_lines
local images
local num_images
@@ -319,7 +320,8 @@
# Run verify_tempest_config -ur to retrieve enabled extensions on API endpoints
# NOTE(mtreinish): This must be done after auth settings are added to the tempest config
local tmp_cfg_file=$(mktemp)
- $TEMPEST_DIR/tempest/cmd/verify_tempest_config.py -uro $tmp_cfg_file
+ cd $TEMPEST_DIR
+ tox -evenv -- verify-tempest-config -uro $tmp_cfg_file
local compute_api_extensions=${COMPUTE_API_EXTENSIONS:-"all"}
if [[ ! -z "$DISABLE_COMPUTE_API_EXTENSIONS" ]]; then
diff --git a/lib/trove b/lib/trove
index 5e6b1b3..d32c776 100644
--- a/lib/trove
+++ b/lib/trove
@@ -79,13 +79,9 @@
# service trove admin # if enabled
function create_trove_accounts {
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- local service_role=$(openstack role list | awk "/ admin / { print \$2 }")
-
if [[ "$ENABLED_SERVICES" =~ "trove" ]]; then
- local trove_user=$(get_or_create_user "trove" "$SERVICE_PASSWORD")
- get_or_add_user_role $service_role $trove_user $service_tenant
+ create_service_user "trove" "admin"
if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
diff --git a/lib/zaqar b/lib/zaqar
index 618ac30..8b560bb 100644
--- a/lib/zaqar
+++ b/lib/zaqar
@@ -215,11 +215,7 @@
}
function create_zaqar_accounts {
- local service_tenant=$(openstack project list | awk "/ $SERVICE_TENANT_NAME / { print \$2 }")
- ADMIN_ROLE=$(openstack role list | awk "/ admin / { print \$2 }")
-
- local zaqar_user=$(get_or_create_user "zaqar" "$SERVICE_PASSWORD")
- get_or_add_user_role $ADMIN_ROLE $zaqar_user $service_tenant
+ create_service_user "zaqar" "admin"
if [[ "$KEYSTONE_CATALOG_BACKEND" = 'sql' ]]; then
diff --git a/pkg/elasticsearch.sh b/pkg/elasticsearch.sh
new file mode 100755
index 0000000..15e1b2b
--- /dev/null
+++ b/pkg/elasticsearch.sh
@@ -0,0 +1,126 @@
+#!/bin/bash -xe
+
+# basic reference point for things like filecache
+#
+# TODO(sdague): once we have a few of these I imagine the download
+# step can probably be factored out to something nicer
+TOP_DIR=$(cd $(dirname "$0")/.. && pwd)
+FILES=$TOP_DIR/files
+source $TOP_DIR/functions
+
+# Package source and version, all pkg files are expected to have
+# something like this, as well as a way to override them.
+ELASTICSEARCH_VERSION=${ELASTICSEARCH_VERSION:-1.4.2}
+ELASTICSEARCH_BASEURL=${ELASTICSEARCH_BASEURL:-https://download.elasticsearch.org/elasticsearch/elasticsearch}
+
+# Elastic search actual implementation
+function wget_elasticsearch {
+ local file=${1}
+
+ if [ ! -f ${FILES}/${file} ]; then
+ wget $ELASTICSEARCH_BASEURL/${file} -O ${FILES}/${file}
+ fi
+
+ if [ ! -f ${FILES}/${file}.sha1.txt ]; then
+ wget $ELASTICSEARCH_BASEURL/${file}.sha1.txt -O ${FILES}/${file}.sha1.txt
+ fi
+
+ pushd ${FILES}; sha1sum ${file} > ${file}.sha1.gen; popd
+
+ if ! diff ${FILES}/${file}.sha1.gen ${FILES}/${file}.sha1.txt; then
+ echo "Invalid elasticsearch download. Could not install."
+ return 1
+ fi
+ return 0
+}
+
+function download_elasticsearch {
+ if is_ubuntu; then
+ wget_elasticsearch elasticsearch-${ELASTICSEARCH_VERSION}.deb
+ elif is_fedora; then
+ wget_elasticsearch elasticsearch-${ELASTICSEARCH_VERSION}.noarch.rpm
+ fi
+}
+
+function configure_elasticsearch {
+ # currently a no op
+ ::
+}
+
+function start_elasticsearch {
+ if is_ubuntu; then
+ sudo /etc/init.d/elasticsearch start
+ elif is_fedora; then
+ sudo /bin/systemctl start elasticsearch.service
+ else
+ echo "Unsupported architecture...can not start elasticsearch."
+ fi
+}
+
+function stop_elasticsearch {
+ if is_ubuntu; then
+ sudo /etc/init.d/elasticsearch stop
+ elif is_fedora; then
+ sudo /bin/systemctl stop elasticsearch.service
+ else
+ echo "Unsupported architecture...can not stop elasticsearch."
+ fi
+}
+
+function install_elasticsearch {
+ if is_package_installed elasticsearch; then
+ echo "Note: elasticsearch was already installed."
+ return
+ fi
+ if is_ubuntu; then
+ is_package_installed openjdk-7-jre-headless || install_package openjdk-7-jre-headless
+
+ sudo dpkg -i ${FILES}/elasticsearch-${ELASTICSEARCH_VERSION}.deb
+ sudo update-rc.d elasticsearch defaults 95 10
+ elif is_fedora; then
+ is_package_installed java-1.7.0-openjdk-headless || install_package java-1.7.0-openjdk-headless
+ yum_install ${FILES}/elasticsearch-${ELASTICSEARCH_VERSION}.noarch.rpm
+ sudo /bin/systemctl daemon-reload
+ sudo /bin/systemctl enable elasticsearch.service
+ else
+ echo "Unsupported install of elasticsearch on this architecture."
+ fi
+}
+
+function uninstall_elasticsearch {
+ if is_package_installed elasticsearch; then
+ if is_ubuntu; then
+ sudo apt-get purge elasticsearch
+ elif is_fedora; then
+ sudo yum remove elasticsearch
+ else
+ echo "Unsupported install of elasticsearch on this architecture."
+ fi
+ fi
+}
+
+# The PHASE dispatcher. All pkg files are expected to basically cargo
+# cult the case statement.
+PHASE=$1
+echo "Phase is $PHASE"
+
+case $PHASE in
+ download)
+ download_elasticsearch
+ ;;
+ install)
+ install_elasticsearch
+ ;;
+ configure)
+ configure_elasticsearch
+ ;;
+ start)
+ start_elasticsearch
+ ;;
+ stop)
+ stop_elasticsearch
+ ;;
+ uninstall)
+ uninstall_elasticsearch
+ ;;
+esac