Merge "Don't set xtrace directly in local call"
diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst
index 7ca82c7..aae4f33 100644
--- a/doc/source/configuration.rst
+++ b/doc/source/configuration.rst
@@ -408,6 +408,28 @@
IMAGE_URLS="http://foo.bar.com/image.qcow,"
IMAGE_URLS+="http://foo.bar.com/image2.qcow"
+
+Instance Type
+-------------
+
+``DEFAULT_INSTANCE_TYPE`` can be used to configure the default instance
+type. When this parameter is not specified, Devstack creates additional
+micro & nano flavors for really small instances to run Tempest tests.
+
+For guests with larger memory requirements, ``DEFAULT_INSTANCE_TYPE``
+should be specified in the configuration file so Tempest selects the
+default flavors instead.
+
+KVM on Power with QEMU 2.4 requires 512 MB to load the firmware -
+`QEMU 2.4 - PowerPC <http://wiki.qemu.org/ChangeLog/2.4>`__ so users
+running instances on ppc64/ppc64le can choose one of the default
+created flavors as follows:
+
+ ::
+
+ DEFAULT_INSTANCE_TYPE=m1.tiny
+
+
IP Version
----------
diff --git a/doc/source/guides/neutron.rst b/doc/source/guides/neutron.rst
index 2973eb6..4248445 100644
--- a/doc/source/guides/neutron.rst
+++ b/doc/source/guides/neutron.rst
@@ -125,39 +125,6 @@
-Disabling Next Generation Firewall Tools
-========================================
-
-DevStack does not properly operate with modern firewall tools. Specifically
-it will appear as if the guest VM can access the external network via ICMP,
-but UDP and TCP packets will not be delivered to the guest VM. The root cause
-of the issue is that both ufw (Uncomplicated Firewall) and firewalld (Fedora's
-firewall manager) apply firewall rules to all interfaces in the system, rather
-then per-device. One solution to this problem is to revert to iptables
-functionality.
-
-To get a functional firewall configuration for Fedora do the following:
-
-::
-
- sudo service iptables save
- sudo systemctl disable firewalld
- sudo systemctl enable iptables
- sudo systemctl stop firewalld
- sudo systemctl start iptables
-
-
-To get a functional firewall configuration for distributions containing ufw,
-disable ufw. Note ufw is generally not enabled by default in Ubuntu. To
-disable ufw if it was enabled, do the following:
-
-::
-
- sudo service iptables save
- sudo ufw disable
-
-
-
Neutron Networking with Open vSwitch
====================================
@@ -301,3 +268,41 @@
created, named `br-ex` which is managed by Open vSwitch, and the
second interface on the compute node, `eth1` is attached to the
bridge, to forward traffic sent by guest VMs.
+
+Miscellaneous Tips
+==================
+
+
+Disabling Next Generation Firewall Tools
+----------------------------------------
+
+DevStack does not properly operate with modern firewall tools. Specifically
+it will appear as if the guest VM can access the external network via ICMP,
+but UDP and TCP packets will not be delivered to the guest VM. The root cause
+of the issue is that both ufw (Uncomplicated Firewall) and firewalld (Fedora's
+firewall manager) apply firewall rules to all interfaces in the system, rather
+then per-device. One solution to this problem is to revert to iptables
+functionality.
+
+To get a functional firewall configuration for Fedora do the following:
+
+::
+
+ sudo service iptables save
+ sudo systemctl disable firewalld
+ sudo systemctl enable iptables
+ sudo systemctl stop firewalld
+ sudo systemctl start iptables
+
+
+To get a functional firewall configuration for distributions containing ufw,
+disable ufw. Note ufw is generally not enabled by default in Ubuntu. To
+disable ufw if it was enabled, do the following:
+
+::
+
+ sudo service iptables save
+ sudo ufw disable
+
+
+
diff --git a/doc/source/plugin-registry.rst b/doc/source/plugin-registry.rst
index 85fd7cc..eb09988 100644
--- a/doc/source/plugin-registry.rst
+++ b/doc/source/plugin-registry.rst
@@ -22,6 +22,8 @@
+------------------+---------------------------------------------+--------------------+
|aodh |git://git.openstack.org/openstack/aodh | alarming |
+------------------+---------------------------------------------+--------------------+
+|barbican |git://git.openstack.org/openstack/barbican | key management |
++------------------+---------------------------------------------+--------------------+
|ceilometer |git://git.openstack.org/openstack/ceilometer | metering |
+------------------+---------------------------------------------+--------------------+
|gnocchi |git://git.openstack.org/openstack/gnocchi | metric |
diff --git a/functions b/functions
index ff95c89..3e17f2f 100644
--- a/functions
+++ b/functions
@@ -341,7 +341,7 @@
# No backends registered means this is likely called from ``localrc``
# This is now deprecated usage
DATABASE_TYPE=$1
- DEPRECATED_TEXT="$DEPRECATED_TEXT\nThe database backend needs to be properly set in ENABLED_SERVICES; use_database is deprecated localrc\n"
+ deprecated "The database backend needs to be properly set in ENABLED_SERVICES; use_database is deprecated localrc"
else
# This should no longer get called...here for posterity
use_exclusive_service DATABASE_BACKENDS DATABASE_TYPE $1
diff --git a/functions-common b/functions-common
index 53b64d6..f9e0b5a 100644
--- a/functions-common
+++ b/functions-common
@@ -88,9 +88,9 @@
--file $CLOUDS_YAML \
--os-cloud devstack \
--os-region-name $REGION_NAME \
- --os-identity-api-version $IDENTITY_API_VERSION \
+ --os-identity-api-version 3 \
$CA_CERT_ARG \
- --os-auth-url $KEYSTONE_AUTH_URI/v$IDENTITY_API_VERSION \
+ --os-auth-url $KEYSTONE_AUTH_URI \
--os-username demo \
--os-password $ADMIN_PASSWORD \
--os-project-name demo
@@ -98,9 +98,9 @@
--file $CLOUDS_YAML \
--os-cloud devstack-admin \
--os-region-name $REGION_NAME \
- --os-identity-api-version $IDENTITY_API_VERSION \
+ --os-identity-api-version 3 \
$CA_CERT_ARG \
- --os-auth-url $KEYSTONE_AUTH_URI/v$IDENTITY_API_VERSION \
+ --os-auth-url $KEYSTONE_AUTH_URI \
--os-username admin \
--os-password $ADMIN_PASSWORD \
--os-project-name admin
@@ -181,6 +181,12 @@
$xtrace
}
+function deprecated {
+ local text=$1
+ DEPRECATED_TEXT+="\n$text"
+ echo "WARNING: $text"
+}
+
# Prints line number and "message" in error format
# err $LINENO "message"
function err {
@@ -740,16 +746,13 @@
# Usage: get_or_create_domain <name> <description>
function get_or_create_domain {
local domain_id
- local os_url="$KEYSTONE_SERVICE_URI_V3"
# Gets domain id
domain_id=$(
# Gets domain id
- openstack --os-token=$OS_TOKEN --os-url=$os_url \
- --os-identity-api-version=3 domain show $1 \
+ openstack domain show $1 \
-f value -c id 2>/dev/null ||
# Creates new domain
- openstack --os-token=$OS_TOKEN --os-url=$os_url \
- --os-identity-api-version=3 domain create $1 \
+ openstack domain create $1 \
--description "$2" \
-f value -c id
)
@@ -760,13 +763,11 @@
# Usage: get_or_create_group <groupname> <domain> [<description>]
function get_or_create_group {
local desc="${3:-}"
- local os_url="$KEYSTONE_SERVICE_URI_V3"
local group_id
# Gets group id
group_id=$(
# Creates new group with --or-show
- openstack --os-token=$OS_TOKEN --os-url=$os_url \
- --os-identity-api-version=3 group create $1 \
+ openstack group create $1 \
--domain $2 --description "$desc" --or-show \
-f value -c id
)
@@ -788,8 +789,6 @@
openstack user create \
$1 \
--password "$2" \
- --os-url=$KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
--domain=$3 \
$email \
--or-show \
@@ -804,9 +803,7 @@
local project_id
project_id=$(
# Creates new project with --or-show
- openstack --os-url=$KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
- project create $1 \
+ openstack project create $1 \
--domain=$2 \
--or-show -f value -c id
)
@@ -820,8 +817,6 @@
role_id=$(
# Creates role with --or-show
openstack role create $1 \
- --os-url=$KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
--or-show -f value -c id
)
echo $role_id
@@ -834,8 +829,6 @@
# Gets user role id
user_role_id=$(openstack role list \
--user $2 \
- --os-url=$KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
--column "ID" \
--project $3 \
--column "Name" \
@@ -844,13 +837,9 @@
# Adds role to user and get it
openstack role add $1 \
--user $2 \
- --project $3 \
- --os-url=$KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3
+ --project $3
user_role_id=$(openstack role list \
--user $2 \
- --os-url=$KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
--column "ID" \
--project $3 \
--column "Name" \
@@ -865,21 +854,15 @@
local group_role_id
# Gets group role id
group_role_id=$(openstack role list \
- --os-url=$KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
--group $2 \
--project $3 \
-c "ID" -f value)
if [[ -z "$group_role_id" ]]; then
# Adds role to group and get it
openstack role add $1 \
- --os-url=$KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
--group $2 \
--project $3
group_role_id=$(openstack role list \
- --os-url=$KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
--group $2 \
--project $3 \
-c "ID" -f value)
@@ -897,8 +880,6 @@
openstack service show $2 -f value -c id 2>/dev/null ||
# Creates new service if not exists
openstack service create \
- --os-url $KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
$2 \
--name $1 \
--description="$3" \
@@ -917,8 +898,6 @@
# gets support for this, the check for the region name can be removed.
# Related bug in keystone: https://bugs.launchpad.net/keystone/+bug/1482772
endpoint_id=$(openstack endpoint list \
- --os-url $KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
--service $1 \
--interface $2 \
--region $4 \
@@ -926,8 +905,6 @@
if [[ -z "$endpoint_id" ]]; then
# Creates new endpoint
endpoint_id=$(openstack endpoint create \
- --os-url $KEYSTONE_SERVICE_URI_V3 \
- --os-identity-api-version=3 \
$1 $2 $3 --region $4 -f value -c id)
fi
@@ -1737,6 +1714,16 @@
if [[ -d $TOP_DIR/extras.d ]]; then
for i in $TOP_DIR/extras.d/*.sh; do
[[ -r $i ]] && source $i $mode $phase
+ # NOTE(sdague): generate a big warning about using
+ # extras.d in an unsupported way which will let us track
+ # unsupported usage in the gate.
+ local exceptions="50-ironic.sh 60-ceph.sh 80-tempest.sh"
+ local extra=$(basename $i)
+ if [[ ! ( $exceptions =~ "$extra" ) ]]; then
+ deprecated "extras.d support is being removed in Mitaka-1"
+ deprecated "jobs for project $extra will break after that point"
+ deprecated "please move project to a supported devstack plugin model"
+ fi
done
fi
# the source phase corresponds to settings loading in plugins
diff --git a/lib/cinder b/lib/cinder
index 1014411..2cda8b7 100644
--- a/lib/cinder
+++ b/lib/cinder
@@ -93,7 +93,7 @@
if [[ $CINDER_SECURE_DELETE == "False" ]]; then
CINDER_VOLUME_CLEAR_DEFAULT="none"
fi
- DEPRECATED_TEXT="$DEPRECATED_TEXT\nConfigure secure Cinder volume deletion using CINDER_VOLUME_CLEAR instead of CINDER_SECURE_DELETE.\n"
+ deprecated "Configure secure Cinder volume deletion using CINDER_VOLUME_CLEAR instead of CINDER_SECURE_DELETE."
fi
CINDER_VOLUME_CLEAR=${CINDER_VOLUME_CLEAR:-${CINDER_VOLUME_CLEAR_DEFAULT:-zero}}
CINDER_VOLUME_CLEAR=$(echo ${CINDER_VOLUME_CLEAR} | tr '[:upper:]' '[:lower:]')
diff --git a/lib/dstat b/lib/dstat
index f11bfa5..fe4790b 100644
--- a/lib/dstat
+++ b/lib/dstat
@@ -19,8 +19,7 @@
# start_dstat() - Start running processes, including screen
function start_dstat {
# A better kind of sysstat, with the top process per time slice
- DSTAT_OPTS="-tcmndrylpg --top-cpu-adv --top-io-adv"
- run_process dstat "dstat $DSTAT_OPTS"
+ run_process dstat "$TOP_DIR/tools/dstat.sh $LOGDIR"
# To enable peakmem_tracker add:
# enable_service peakmem_tracker
diff --git a/lib/neutron_plugins/oneconvergence b/lib/neutron_plugins/oneconvergence
deleted file mode 100644
index 0c570e5..0000000
--- a/lib/neutron_plugins/oneconvergence
+++ /dev/null
@@ -1,77 +0,0 @@
-#!/bin/bash
-#
-# Neutron One Convergence plugin
-# ------------------------------
-
-# Save trace setting
-OC_XTRACE=$(set +o | grep xtrace)
-set +o xtrace
-
-source $TOP_DIR/lib/neutron_plugins/ovs_base
-
-Q_L3_ENABLED=true
-Q_L3_ROUTER_PER_TENANT=true
-Q_USE_NAMESPACE=true
-
-function neutron_plugin_install_agent_packages {
- _neutron_ovs_base_install_agent_packages
-}
-# Configure common parameters
-function neutron_plugin_configure_common {
-
- Q_PLUGIN_CONF_PATH=etc/neutron/plugins/oneconvergence
- Q_PLUGIN_CONF_FILENAME=nvsdplugin.ini
- Q_PLUGIN_CLASS="neutron.plugins.oneconvergence.plugin.OneConvergencePluginV2"
-}
-
-# Configure plugin specific information
-function neutron_plugin_configure_service {
- iniset /$Q_PLUGIN_CONF_FILE nvsd nvsd_ip $NVSD_IP
- iniset /$Q_PLUGIN_CONF_FILE nvsd nvsd_port $NVSD_PORT
- iniset /$Q_PLUGIN_CONF_FILE nvsd nvsd_user $NVSD_USER
- iniset /$Q_PLUGIN_CONF_FILE nvsd nvsd_passwd $NVSD_PASSWD
-}
-
-function neutron_plugin_configure_debug_command {
- _neutron_ovs_base_configure_debug_command
-}
-
-function neutron_plugin_setup_interface_driver {
- local conf_file=$1
- iniset $conf_file DEFAULT interface_driver neutron.agent.linux.interface.OVSInterfaceDriver
-}
-
-function has_neutron_plugin_security_group {
- # 1 means False here
- return 0
-}
-
-function setup_integration_bridge {
- _neutron_ovs_base_setup_bridge $OVS_BRIDGE
-}
-
-function neutron_plugin_configure_dhcp_agent {
- setup_integration_bridge
- iniset $Q_DHCP_CONF_FILE DEFAULT dhcp_agent_manager neutron.agent.dhcp_agent.DhcpAgentWithStateReport
-}
-
-function neutron_plugin_configure_l3_agent {
- _neutron_ovs_base_configure_l3_agent
- iniset $Q_L3_CONF_FILE DEFAULT l3_agent_manager neutron.agent.l3_agent.L3NATAgentWithStateReport
-}
-
-function neutron_plugin_configure_plugin_agent {
-
- AGENT_BINARY="$NEUTRON_BIN_DIR/neutron-nvsd-agent"
-
- _neutron_ovs_base_configure_firewall_driver
-}
-
-function neutron_plugin_create_nova_conf {
- if ( is_service_enabled n-cpu && ! ( is_service_enabled q-dhcp )) ; then
- setup_integration_bridge
- fi
-}
-
-# Restore xtrace
-$OC_XTRACE
diff --git a/lib/swift b/lib/swift
index 645bfd7..2c4ddfe 100644
--- a/lib/swift
+++ b/lib/swift
@@ -130,9 +130,9 @@
# Port bases used in port number calclution for the service "nodes"
# The specified port number will be used, the additinal ports calculated by
# base_port + node_num * 10
-OBJECT_PORT_BASE=${OBJECT_PORT_BASE:-6013}
-CONTAINER_PORT_BASE=${CONTAINER_PORT_BASE:-6011}
-ACCOUNT_PORT_BASE=${ACCOUNT_PORT_BASE:-6012}
+OBJECT_PORT_BASE=${OBJECT_PORT_BASE:-6613}
+CONTAINER_PORT_BASE=${CONTAINER_PORT_BASE:-6611}
+ACCOUNT_PORT_BASE=${ACCOUNT_PORT_BASE:-6612}
# Enable tempurl feature
SWIFT_ENABLE_TEMPURLS=${SWIFT_ENABLE_TEMPURLS:-False}
@@ -799,10 +799,10 @@
function swift_configure_tempurls {
OS_USERNAME=swift \
- OS_TENANT_NAME=$SERVICE_TENANT_NAME \
+ OS_PROJECT_NAME=$SERVICE_TENANT_NAME \
OS_PASSWORD=$SERVICE_PASSWORD \
- OS_AUTH_URL=$KEYSTONE_AUTH_URI/v$IDENTITY_API_VERSION \
- swift post -m "Temp-URL-Key: $SWIFT_TEMPURL_KEY"
+ OS_AUTH_URL=$SERVICE_ENDPOINT \
+ swift post --auth-version 3 -m "Temp-URL-Key: $SWIFT_TEMPURL_KEY"
}
# Restore xtrace
diff --git a/lib/tempest b/lib/tempest
index f4d0a6d..6eeab4e 100644
--- a/lib/tempest
+++ b/lib/tempest
@@ -205,21 +205,12 @@
if [[ -z "$DEFAULT_INSTANCE_TYPE" ]]; then
available_flavors=$(nova flavor-list)
if [[ ! ( $available_flavors =~ 'm1.nano' ) ]]; then
- if is_arch "ppc64"; then
- # Qemu needs at least 128MB of memory to boot on ppc64
- nova flavor-create m1.nano 42 128 0 1
- else
- nova flavor-create m1.nano 42 64 0 1
- fi
+ nova flavor-create m1.nano 42 64 0 1
fi
flavor_ref=42
boto_instance_type=m1.nano
if [[ ! ( $available_flavors =~ 'm1.micro' ) ]]; then
- if is_arch "ppc64"; then
- nova flavor-create m1.micro 84 256 0 1
- else
- nova flavor-create m1.micro 84 128 0 1
- fi
+ nova flavor-create m1.micro 84 128 0 1
fi
flavor_ref_alt=84
else
diff --git a/stack.sh b/stack.sh
index c2eeaee..01668c2 100755
--- a/stack.sh
+++ b/stack.sh
@@ -975,13 +975,15 @@
start_keystone
fi
+ export OS_IDENTITY_API_VERSION=3
+
# Set up a temporary admin URI for Keystone
- SERVICE_ENDPOINT=$KEYSTONE_AUTH_URI/v2.0
+ SERVICE_ENDPOINT=$KEYSTONE_AUTH_URI/v3
if is_service_enabled tls-proxy; then
export OS_CACERT=$INT_CA_DIR/ca-chain.pem
# Until the client support is fixed, just use the internal endpoint
- SERVICE_ENDPOINT=http://$KEYSTONE_AUTH_HOST:$KEYSTONE_AUTH_PORT_INT/v2.0
+ SERVICE_ENDPOINT=http://$KEYSTONE_AUTH_HOST:$KEYSTONE_AUTH_PORT_INT/v3
fi
# Setup OpenStackClient token-endpoint auth
@@ -1005,14 +1007,13 @@
# Begone token auth
unset OS_TOKEN OS_URL
- # force set to use v2 identity authentication even with v3 commands
- export OS_AUTH_TYPE=v2password
-
# Set up password auth credentials now that Keystone is bootstrapped
- export OS_AUTH_URL=$SERVICE_ENDPOINT
- export OS_TENANT_NAME=admin
+ export OS_AUTH_URL=$KEYSTONE_AUTH_URI
export OS_USERNAME=admin
+ export OS_USER_DOMAIN_ID=default
export OS_PASSWORD=$ADMIN_PASSWORD
+ export OS_PROJECT_NAME=admin
+ export OS_PROJECT_DOMAIN_ID=default
export OS_REGION_NAME=$REGION_NAME
fi
diff --git a/tools/build_docs.sh b/tools/build_docs.sh
index fa84343..7dc492e 100755
--- a/tools/build_docs.sh
+++ b/tools/build_docs.sh
@@ -81,7 +81,7 @@
mkdir -p $FQ_HTML_BUILD/`dirname $f`;
$SHOCCO $f > $FQ_HTML_BUILD/$f.html
done
-for f in $(find functions functions-common inc lib pkg samples -type f -name \*); do
+for f in $(find functions functions-common inc lib pkg samples -type f -name \* ! -name *.md ! -name *.conf); do
echo $f
FILES+="$f "
mkdir -p $FQ_HTML_BUILD/`dirname $f`;
diff --git a/tools/dstat.sh b/tools/dstat.sh
new file mode 100755
index 0000000..6ba4515
--- /dev/null
+++ b/tools/dstat.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# **tools/dstat.sh** - Execute instances of DStat to log system load info
+#
+# Multiple instances of DStat are executed in order to take advantage of
+# incompatible features, particularly CSV output and the "top-cpu-adv" and
+# "top-io-adv" flags.
+#
+# Assumes:
+# - dstat command is installed
+
+# Retreive log directory as argument from calling script.
+LOGDIR=$1
+
+# Command line arguments for primary DStat process.
+DSTAT_OPTS="-tcmndrylpg --top-cpu-adv --top-io-adv"
+
+# Command-line arguments for secondary background DStat process.
+DSTAT_CSV_OPTS="-tcmndrylpg --output $LOGDIR/dstat-csv.log"
+
+# Execute and background the secondary dstat process and discard its output.
+dstat $DSTAT_CSV_OPTS >& /dev/null &
+
+# Execute and background the primary dstat process, but keep its output in this
+# TTY.
+dstat $DSTAT_OPTS &
+
+# Catch any exit signals, making sure to also terminate any child processes.
+trap "kill -- -$$" EXIT
+
+# Keep this script running as long as child dstat processes are alive.
+wait
diff --git a/tools/install_pip.sh b/tools/install_pip.sh
index 7b42c8c..4126180 100755
--- a/tools/install_pip.sh
+++ b/tools/install_pip.sh
@@ -53,8 +53,15 @@
# since and only download if a new version is out -- but only if
# it seems we downloaded the file originally.
if [[ ! -r $LOCAL_PIP || -r $LOCAL_PIP.downloaded ]]; then
+ # only test freshness if LOCAL_PIP is actually there,
+ # otherwise we generate a scary warning.
+ local timecond=""
+ if [[ -r $LOCAL_PIP ]]; then
+ timecond="-z $LOCAL_PIP"
+ fi
+
curl --retry 6 --retry-delay 5 \
- -z $LOCAL_PIP -o $LOCAL_PIP $PIP_GET_PIP_URL || \
+ $timecond -o $LOCAL_PIP $PIP_GET_PIP_URL || \
die $LINENO "Download of get-pip.py failed"
touch $LOCAL_PIP.downloaded
fi