Merge "Tune mysql a bit better for concurrent operations"
diff --git a/doc/source/guides/neutron.rst b/doc/source/guides/neutron.rst
index bdfd3a4..40a5632 100644
--- a/doc/source/guides/neutron.rst
+++ b/doc/source/guides/neutron.rst
@@ -261,15 +261,18 @@
## Neutron Networking options used to create Neutron Subnets
- FIXED_RANGE="10.1.1.0/24"
+ FIXED_RANGE="203.0.113.0/24"
PROVIDER_SUBNET_NAME="provider_net"
PROVIDER_NETWORK_TYPE="vlan"
SEGMENTATION_ID=2010
In this configuration we are defining FIXED_RANGE to be a
-subnet that exists in the private RFC1918 address space - however
-in a real setup FIXED_RANGE would be a public IP address range, so
-that you could access your instances from the public internet.
+publicly routed IPv4 subnet. In this specific instance we are using
+the special TEST-NET-3 subnet defined in `RFC 5737 <http://tools.ietf.org/html/rfc5737>`_,
+which is used for documentation. In your DevStack setup, FIXED_RANGE
+would be a public IP address range that you or your organization has
+allocated to you, so that you could access your instances from the
+public internet.
The following is a snippet of the DevStack configuration on the
compute node.
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 6d0edb0..f15c306 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -10,6 +10,7 @@
overview
configuration
plugins
+ plugin-registry
faq
changes
hacking
diff --git a/doc/source/plugin-registry.rst b/doc/source/plugin-registry.rst
new file mode 100644
index 0000000..2dd70d8
--- /dev/null
+++ b/doc/source/plugin-registry.rst
@@ -0,0 +1,73 @@
+..
+ Note to reviewers: the intent of this file is to be easy for
+ community members to update. As such fast approving (single core +2)
+ is fine as long as you've identified that the plugin listed actually exists.
+
+==========================
+ DevStack Plugin Registry
+==========================
+
+Since we've created the external plugin mechanism, it's gotten used by
+a lot of projects. The following is a list of plugins that currently
+exist. Any project that wishes to list their plugin here is welcomed
+to.
+
+Official OpenStack Projects
+===========================
+
+The following are plugins that exist for official OpenStack projects.
+
++--------------------+-------------------------------------------+--------------------+
+|Plugin Name |URL |Comments |
++--------------------+-------------------------------------------+--------------------+
+|magnum |git://git.openstack.org/openstack/magnum | |
++--------------------+-------------------------------------------+--------------------+
+|trove |git://git.openstack.org/openstack/trove | |
++--------------------+-------------------------------------------+--------------------+
+|zaqar |git://git.openstack.org/openstack/zarar | |
++--------------------+-------------------------------------------+--------------------+
+
+
+
+Drivers
+=======
+
++--------------------+-------------------------------------------------+------------------+
+|Plugin Name |URL |Comments |
++--------------------+-------------------------------------------------+------------------+
+|dragonflow |git://git.openstack.org/openstack/dragonflow |[d1]_ |
++--------------------+-------------------------------------------------+------------------+
+|odl |git://git.openstack.org/openstack/networking-odl |[d2]_ |
++--------------------+-------------------------------------------------+------------------+
+
+.. [d1] demonstrates example of installing 3rd party SDN controller
+.. [d2] demonstrates a pretty advanced set of modes that that allow
+ one to run OpenDayLight either from a pre-existing install, or
+ also from source
+
+Alternate Configs
+=================
+
++-------------+------------------------------------------------------------+------------+
+| Plugin Name | URL | Comments |
+| | | |
++-------------+------------------------------------------------------------+------------+
+|glusterfs |git://git.openstack.org/stackforge/devstack-plugin-glusterfs| |
++-------------+------------------------------------------------------------+------------+
+| | | |
++-------------+------------------------------------------------------------+------------+
+
+Additional Services
+===================
+
++-------------+------------------------------------------+------------+
+| Plugin Name | URL | Comments |
+| | | |
++-------------+------------------------------------------+------------+
+|ec2-api |git://git.openstack.org/stackforge/ec2api |[as1]_ |
++-------------+------------------------------------------+------------+
+| | | |
++-------------+------------------------------------------+------------+
+
+.. [as1] first functional devstack plugin, hence why used in most of
+ the examples.
diff --git a/doc/source/plugins.rst b/doc/source/plugins.rst
index c4ed228..b166936 100644
--- a/doc/source/plugins.rst
+++ b/doc/source/plugins.rst
@@ -2,103 +2,21 @@
Plugins
=======
-DevStack has a couple of plugin mechanisms to allow easily adding
-support for additional projects and features.
+The OpenStack ecosystem is wide and deep, and only growing more so
+every day. The value of DevStack is that it's simple enough to
+understand what it's doing clearly. And yet we'd like to support as
+much of the OpenStack Ecosystem as possible. We do that with plugins.
-Extras.d Hooks
-==============
+DevStack plugins are bits of bash code that live outside the DevStack
+tree. They are called through a strong contract, so these plugins can
+be sure that they will continue to work in the future as DevStack
+evolves.
-These hooks are an extension of the service calls in
-``stack.sh`` at specific points in its run, plus ``unstack.sh`` and
-``clean.sh``. A number of the higher-layer projects are implemented in
-DevStack using this mechanism.
+Plugin Interface
+================
-The script in ``extras.d`` is expected to be mostly a dispatcher to
-functions in a ``lib/*`` script. The scripts are named with a
-zero-padded two digits sequence number prefix to control the order that
-the scripts are called, and with a suffix of ``.sh``. DevStack reserves
-for itself the sequence numbers 00 through 09 and 90 through 99.
-
-Below is a template that shows handlers for the possible command-line
-arguments:
-
-::
-
- # template.sh - DevStack extras.d dispatch script template
-
- # check for service enabled
- if is_service_enabled template; then
-
- if [[ "$1" == "source" ]]; then
- # Initial source of lib script
- source $TOP_DIR/lib/template
- fi
-
- if [[ "$1" == "stack" && "$2" == "pre-install" ]]; then
- # Set up system services
- echo_summary "Configuring system services Template"
- install_package cowsay
-
- elif [[ "$1" == "stack" && "$2" == "install" ]]; then
- # Perform installation of service source
- echo_summary "Installing Template"
- install_template
-
- elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
- # Configure after the other layer 1 and 2 services have been configured
- echo_summary "Configuring Template"
- configure_template
-
- elif [[ "$1" == "stack" && "$2" == "extra" ]]; then
- # Initialize and start the template service
- echo_summary "Initializing Template"
- ##init_template
- fi
-
- if [[ "$1" == "unstack" ]]; then
- # Shut down template services
- # no-op
- :
- fi
-
- if [[ "$1" == "clean" ]]; then
- # Remove state and transient data
- # Remember clean.sh first calls unstack.sh
- # no-op
- :
- fi
- fi
-
-The arguments are:
-
-- **source** - Called by each script that utilizes ``extras.d`` hooks;
- this replaces directly sourcing the ``lib/*`` script.
-- **stack** - Called by ``stack.sh`` three times for different phases
- of its run:
-
- - **pre-install** - Called after system (OS) setup is complete and
- before project source is installed.
- - **install** - Called after the layer 1 and 2 projects source and
- their dependencies have been installed.
- - **post-config** - Called after the layer 1 and 2 services have
- been configured. All configuration files for enabled services
- should exist at this point.
- - **extra** - Called near the end after layer 1 and 2 services have
- been started. This is the existing hook and has not otherwise
- changed.
-
-- **unstack** - Called by ``unstack.sh`` before other services are shut
- down.
-- **clean** - Called by ``clean.sh`` before other services are cleaned,
- but after ``unstack.sh`` has been called.
-
-
-Externally Hosted Plugins
-=========================
-
-Based on the extras.d hooks, DevStack supports a standard mechansim
-for including plugins from external repositories. The plugin interface
-assumes the following:
+DevStack supports a standard mechansim for including plugins from
+external repositories. The plugin interface assumes the following:
An external git repository that includes a ``devstack/`` top level
directory. Inside this directory there can be 2 files.
@@ -118,11 +36,10 @@
default value only if the variable is unset or empty; e.g. in bash
syntax ``FOO=${FOO:-default}``.
-- ``plugin.sh`` - the actual plugin. It will be executed by devstack
- during it's run. The run order will be done in the registration
- order for these plugins, and will occur immediately after all in
- tree extras.d dispatch at the phase in question. The plugin.sh
- looks like the extras.d dispatcher above.
+- ``plugin.sh`` - the actual plugin. It is executed by devstack at
+ well defined points during a ``stack.sh`` run. The plugin.sh
+ internal structure is discussed bellow.
+
Plugins are registered by adding the following to the localrc section
of ``local.conf``.
@@ -141,49 +58,121 @@
enable_plugin ec2api git://git.openstack.org/stackforge/ec2api
-Plugins for gate jobs
----------------------
+plugin.sh contract
+==================
-All OpenStack plugins that wish to be used as gate jobs need to exist
-in OpenStack's gerrit. Both ``openstack`` namespace and ``stackforge``
-namespace are fine. This allows testing of the plugin as well as
-provides network isolation against upstream git repository failures
-(which we see often enough to be an issue).
+``plugin.sh`` is a bash script that will be called at specific points
+during ``stack.sh``, ``unstack.sh``, and ``clean.sh``. It will be
+called in the following way::
-Ideally plugins will be implemented as ``devstack`` directory inside
-the project they are testing. For example, the stackforge/ec2-api
-project has it's pluggin support in it's tree.
+ source $PATH/TO/plugin.sh <mode> [phase]
-In the cases where there is no "project tree" per say (like
-integrating a backend storage configuration such as ceph or glusterfs)
-it's also allowed to build a dedicated
-``stackforge/devstack-plugin-FOO`` project to house the plugin.
+``mode`` can be thought of as the major mode being called, currently
+one of: ``stack``, ``unstack``, ``clean``. ``phase`` is used by modes
+which have multiple points during their run where it's necessary to
+be able to execute code. All existing ``mode`` and ``phase`` points
+are considered **strong contracts** and won't be removed without a
+reasonable deprecation period. Additional new ``mode`` or ``phase``
+points may be added at any time if we discover we need them to support
+additional kinds of plugins in devstack.
-Note jobs must not require cloning of repositories during tests.
-Tests must list their repository in the ``PROJECTS`` variable for
-`devstack-gate
-<https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/devstack-vm-gate-wrap.sh>`_
-for the repository to be available to the test. Further information
-is provided in the project creator's guide.
+The current full list of ``mode`` and ``phase`` are:
-Hypervisor
-==========
+- **stack** - Called by ``stack.sh`` four times for different phases
+ of its run:
-Hypervisor plugins are fairly new and condense most hypervisor
-configuration into one place.
+ - **pre-install** - Called after system (OS) setup is complete and
+ before project source is installed.
+ - **install** - Called after the layer 1 and 2 projects source and
+ their dependencies have been installed.
+ - **post-config** - Called after the layer 1 and 2 services have
+ been configured. All configuration files for enabled services
+ should exist at this point.
+ - **extra** - Called near the end after layer 1 and 2 services have
+ been started.
-The initial plugin implemented was for Docker support and is a useful
-template for the required support. Plugins are placed in
-``lib/nova_plugins`` and named ``hypervisor-<name>`` where ``<name>`` is
-the value of ``VIRT_DRIVER``. Plugins must define the following
-functions:
+- **unstack** - Called by ``unstack.sh`` before other services are shut
+ down.
+- **clean** - Called by ``clean.sh`` before other services are cleaned,
+ but after ``unstack.sh`` has been called.
-- ``install_nova_hypervisor`` - install any external requirements
-- ``configure_nova_hypervisor`` - make configuration changes, including
- those to other services
-- ``start_nova_hypervisor`` - start any external services
-- ``stop_nova_hypervisor`` - stop any external services
-- ``cleanup_nova_hypervisor`` - remove transient data and cache
+Example plugin
+====================
+
+An example plugin would look something as follows.
+
+``devstack/settings``::
+
+ # settings file for template
+ enable_service template
+
+
+``devstack/plugin.sh``::
+
+ # plugin.sh - DevStack plugin.sh dispatch script template
+
+ function install_template {
+ ...
+ }
+
+ function init_template {
+ ...
+ }
+
+ function configure_template {
+ ...
+ }
+
+ # check for service enabled
+ if is_service_enabled template; then
+
+ if [[ "$1" == "stack" && "$2" == "pre-install" ]]; then
+ # Set up system services
+ echo_summary "Configuring system services Template"
+ install_package cowsay
+
+ elif [[ "$1" == "stack" && "$2" == "install" ]]; then
+ # Perform installation of service source
+ echo_summary "Installing Template"
+ install_template
+
+ elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
+ # Configure after the other layer 1 and 2 services have been configured
+ echo_summary "Configuring Template"
+ configure_template
+
+ elif [[ "$1" == "stack" && "$2" == "extra" ]]; then
+ # Initialize and start the template service
+ echo_summary "Initializing Template"
+ init_template
+ fi
+
+ if [[ "$1" == "unstack" ]]; then
+ # Shut down template services
+ # no-op
+ :
+ fi
+
+ if [[ "$1" == "clean" ]]; then
+ # Remove state and transient data
+ # Remember clean.sh first calls unstack.sh
+ # no-op
+ :
+ fi
+ fi
+
+Plugin Execution Order
+======================
+
+Plugins are run after in tree services at each of the stages
+above. For example, if you need something to happen before Keystone
+starts, you should do that at the ``post-config`` phase.
+
+Multiple plugins can be specified in your ``local.conf``. When that
+happens the plugins will be executed **in order** at each phase. This
+allows plugins to conceptually depend on each other through
+documenting to the user the order they must be declared. A formal
+dependency mechanism is beyond the scope of the current work.
System Packages
===============
@@ -205,3 +194,47 @@
- ``./devstack/files/rpms-suse/$plugin_name`` - Packages to install when
running on SUSE Linux or openSUSE.
+
+
+Using Plugins in the OpenStack Gate
+===================================
+
+For everyday use, DevStack plugins can exist in any git tree that's
+accessible on the internet. However, when using DevStack plugins in
+the OpenStack gate, they must live in projects in OpenStack's
+gerrit. Both ``openstack`` namespace and ``stackforge`` namespace are
+fine. This allows testing of the plugin as well as provides network
+isolation against upstream git repository failures (which we see often
+enough to be an issue).
+
+Ideally a plugin will be included within the ``devstack`` directory of
+the project they are being tested. For example, the stackforge/ec2-api
+project has its pluggin support in its own tree.
+
+However, some times a DevStack plugin might be used solely to
+configure a backend service that will be used by the rest of
+OpenStack, so there is no "project tree" per say. Good examples
+include: integration of back end storage (e.g. ceph or glusterfs),
+integration of SDN controllers (e.g. ovn, OpenDayLight), or
+integration of alternate RPC systems (e.g. zmq, qpid). In these cases
+the best practice is to build a dedicated
+``stackforge/devstack-plugin-FOO`` project.
+
+To enable a plugin to be used in a gate job, the following lines will
+be needed in your project.yaml definition::
+
+ # Because we are testing a non standard project, add the
+ # our project repository. This makes zuul do the right
+ # reference magic for testing changes.
+ export PROJECTS="stackforge/ec2-api $PROJECTS"
+
+ # note the actual url here is somewhat irrelevant because it
+ # caches in nodepool, however make it a valid url for
+ # documentation purposes.
+ export DEVSTACK_LOCAL_CONFIG="enable_plugin ec2-api git://git.openstack.org/stackforge/ec2-api"
+
+See Also
+========
+
+For additional inspiration on devstack plugins you can check out the
+`Plugin Registry <plugin-registry.html>`_.
diff --git a/exercises/sahara.sh b/exercises/sahara.sh
index 2589e28..8cad945 100755
--- a/exercises/sahara.sh
+++ b/exercises/sahara.sh
@@ -35,7 +35,13 @@
is_service_enabled sahara || exit 55
-$CURL_GET http://$SERVICE_HOST:8386/ 2>/dev/null | grep -q 'Auth' || die $LINENO "Sahara API isn't functioning!"
+if is_ssl_enabled_service "sahara" || is_service_enabled tls-proxy; then
+ SAHARA_SERVICE_PROTOCOL="https"
+fi
+
+SAHARA_SERVICE_PROTOCOL=${SAHARA_SERVICE_PROTOCOL:-$SERVICE_PROTOCOL}
+
+$CURL_GET $SAHARA_SERVICE_PROTOCOL://$SERVICE_HOST:8386/ 2>/dev/null | grep -q 'Auth' || die $LINENO "Sahara API isn't functioning!"
set +o xtrace
echo "*********************************************************************"
diff --git a/files/rpms-suse/devlibs b/files/rpms-suse/devlibs
index bdb630a..54d13a3 100644
--- a/files/rpms-suse/devlibs
+++ b/files/rpms-suse/devlibs
@@ -1,6 +1,5 @@
libffi-devel # pyOpenSSL
libopenssl-devel # pyOpenSSL
-libxml2-devel # lxml
libxslt-devel # lxml
postgresql-devel # psycopg2
libmysqlclient-devel # MySQL-python
diff --git a/files/rpms-suse/glance b/files/rpms-suse/glance
index 0e58425..bf512de 100644
--- a/files/rpms-suse/glance
+++ b/files/rpms-suse/glance
@@ -1,2 +1 @@
-libxml2-devel
python-devel
diff --git a/files/rpms-suse/trove b/files/rpms-suse/trove
deleted file mode 100644
index 96f8f29..0000000
--- a/files/rpms-suse/trove
+++ /dev/null
@@ -1 +0,0 @@
-libxslt1-dev
diff --git a/files/rpms/general b/files/rpms/general
index 67b084e..c3f3de8 100644
--- a/files/rpms/general
+++ b/files/rpms/general
@@ -28,3 +28,4 @@
java-1.7.0-openjdk-headless # NOPRIME rhel7
java-1.8.0-openjdk-headless # NOPRIME f21,f22
pyOpenSSL # version in pip uses too much memory
+iptables-services # NOPRIME f21,f22
diff --git a/files/rpms/nova b/files/rpms/nova
index ebd6674..d32c332 100644
--- a/files/rpms/nova
+++ b/files/rpms/nova
@@ -10,6 +10,7 @@
iputils
kpartx
kvm # NOPRIME
+qemu-kvm # NOPRIME
libvirt-bin # NOPRIME
libvirt-devel # NOPRIME
libvirt-python # NOPRIME
diff --git a/functions-common b/functions-common
index 3a2f5f7..061a935 100644
--- a/functions-common
+++ b/functions-common
@@ -43,6 +43,25 @@
TRACK_DEPENDS=${TRACK_DEPENDS:-False}
+# Save these variables to .stackenv
+STACK_ENV_VARS="BASE_SQL_CONN DATA_DIR DEST ENABLED_SERVICES HOST_IP \
+ KEYSTONE_AUTH_PROTOCOL KEYSTONE_AUTH_URI KEYSTONE_SERVICE_URI \
+ LOGFILE OS_CACERT SERVICE_HOST SERVICE_PROTOCOL STACK_USER TLS_IP"
+
+
+# Saves significant environment variables to .stackenv for later use
+# Refers to a lot of globals, only TOP_DIR and STACK_ENV_VARS are required to
+# function, the rest are simply saved and do not cause problems if they are undefined.
+# save_stackenv [tag]
+function save_stackenv {
+ local tag=${1:-""}
+ # Save some values we generated for later use
+ time_stamp=$(date "+$TIMESTAMP_FORMAT")
+ echo "# $time_stamp $tag" >$TOP_DIR/.stackenv
+ for i in $STACK_ENV_VARS; do
+ echo $i=${!i} >>$TOP_DIR/.stackenv
+ done
+}
# Normalize config values to True or False
# Accepts as False: 0 no No NO false False FALSE
@@ -68,6 +87,7 @@
[[ -v "$1" ]]
}
+
# Control Functions
# =================
@@ -1947,6 +1967,19 @@
fi
}
+# Test with a finite retry loop.
+#
+function test_with_retry {
+ local testcmd=$1
+ local failmsg=$2
+ local until=${3:-10}
+ local sleep=${4:-0.5}
+
+ if ! timeout $until sh -c "while ! $testcmd; do sleep $sleep; done"; then
+ die $LINENO "$failmsg"
+ fi
+}
+
# Restore xtrace
$XTRACE
diff --git a/inc/python b/inc/python
index 3d329b5..07a811e 100644
--- a/inc/python
+++ b/inc/python
@@ -216,18 +216,14 @@
local update_requirements=$(cd $project_dir && git diff --exit-code >/dev/null || echo "changed")
if [[ $update_requirements != "changed" ]]; then
- if [[ "$REQUIREMENTS_MODE" == "soft" ]]; then
- if is_in_projects_txt $project_dir; then
- (cd $REQUIREMENTS_DIR; \
- python update.py $project_dir)
- else
- # soft update projects not found in requirements project.txt
- (cd $REQUIREMENTS_DIR; \
- python update.py -s $project_dir)
- fi
- else
+ if is_in_projects_txt $project_dir; then
(cd $REQUIREMENTS_DIR; \
- python update.py $project_dir)
+ ./.venv/bin/python update.py $project_dir)
+ else
+ # soft update projects not found in requirements project.txt
+ echo "$project_dir not a constrained repository, soft enforcing requirements"
+ (cd $REQUIREMENTS_DIR; \
+ ./.venv/bin/python update.py -s $project_dir)
fi
fi
diff --git a/lib/ceilometer b/lib/ceilometer
index a577ee9..d7888d9 100644
--- a/lib/ceilometer
+++ b/lib/ceilometer
@@ -366,10 +366,7 @@
git_clone_by_name "ceilometermiddleware"
setup_dev_lib "ceilometermiddleware"
else
- # BUG: this should be a pip_install_gr except it was never
- # included in global-requirements. Needs to be fixed by
- # https://bugs.launchpad.net/ceilometer/+bug/1441655
- pip_install ceilometermiddleware
+ pip_install_gr ceilometermiddleware
fi
}
diff --git a/lib/cinder b/lib/cinder
index ade3b82..8117447 100644
--- a/lib/cinder
+++ b/lib/cinder
@@ -66,6 +66,10 @@
CINDER_SERVICE_PORT_INT=${CINDER_SERVICE_PORT_INT:-18776}
CINDER_SERVICE_PROTOCOL=${CINDER_SERVICE_PROTOCOL:-$SERVICE_PROTOCOL}
+# What type of LVM device should Cinder use for LVM backend
+# Defaults to default, which is thick, the other valid choice
+# is thin, which as the name implies utilizes lvm thin provisioning.
+CINDER_LVM_TYPE=${CINDER_LVM_TYPE:-default}
# Default backends
# The backend format is type:name where type is one of the supported backend
diff --git a/lib/cinder_backends/lvm b/lib/cinder_backends/lvm
index 35ad209..411b82c 100644
--- a/lib/cinder_backends/lvm
+++ b/lib/cinder_backends/lvm
@@ -51,6 +51,7 @@
iniset $CINDER_CONF $be_name volume_driver "cinder.volume.drivers.lvm.LVMVolumeDriver"
iniset $CINDER_CONF $be_name volume_group $VOLUME_GROUP_NAME-$be_name
iniset $CINDER_CONF $be_name iscsi_helper "$CINDER_ISCSI_HELPER"
+ iniset $CINDER_CONF $be_name lvm_type "$CINDER_LVM_TYPE"
if [[ "$CINDER_SECURE_DELETE" == "False" ]]; then
iniset $CINDER_CONF $be_name volume_clear none
diff --git a/lib/glance b/lib/glance
index 016ade3..47bad0e 100644
--- a/lib/glance
+++ b/lib/glance
@@ -56,6 +56,7 @@
GLANCE_CACHE_CONF=$GLANCE_CONF_DIR/glance-cache.conf
GLANCE_POLICY_JSON=$GLANCE_CONF_DIR/policy.json
GLANCE_SCHEMA_JSON=$GLANCE_CONF_DIR/schema-image.json
+GLANCE_SWIFT_STORE_CONF=$GLANCE_CONF_DIR/glance-swift-store.conf
if is_ssl_enabled_service "glance" || is_service_enabled tls-proxy; then
GLANCE_SERVICE_PROTOCOL="https"
@@ -145,11 +146,20 @@
# Store the images in swift if enabled.
if is_service_enabled s-proxy; then
iniset $GLANCE_API_CONF glance_store default_store swift
- iniset $GLANCE_API_CONF glance_store swift_store_auth_address $KEYSTONE_SERVICE_URI/v2.0/
- iniset $GLANCE_API_CONF glance_store swift_store_user $SERVICE_TENANT_NAME:glance-swift
- iniset $GLANCE_API_CONF glance_store swift_store_key $SERVICE_PASSWORD
iniset $GLANCE_API_CONF glance_store swift_store_create_container_on_put True
+
+ iniset $GLANCE_API_CONF glance_store swift_store_config_file $GLANCE_SWIFT_STORE_CONF
+ iniset $GLANCE_API_CONF glance_store default_swift_reference ref1
iniset $GLANCE_API_CONF glance_store stores "file, http, swift"
+
+ iniset $GLANCE_SWIFT_STORE_CONF ref1 user $SERVICE_TENANT_NAME:glance-swift
+ iniset $GLANCE_SWIFT_STORE_CONF ref1 key $SERVICE_PASSWORD
+ iniset $GLANCE_SWIFT_STORE_CONF ref1 auth_address $KEYSTONE_SERVICE_URI/v2.0/
+
+ # commenting is not strictly necessary but it's confusing to have bad values in conf
+ inicomment $GLANCE_API_CONF glance_store swift_store_user
+ inicomment $GLANCE_API_CONF glance_store swift_store_key
+ inicomment $GLANCE_API_CONF glance_store swift_store_auth_address
fi
if is_service_enabled tls-proxy; then
diff --git a/lib/infra b/lib/infra
index 5fb185f..3d68e45 100644
--- a/lib/infra
+++ b/lib/infra
@@ -29,6 +29,18 @@
# install_infra() - Collect source and prepare
function install_infra {
+ local PIP_VIRTUAL_ENV="$REQUIREMENTS_DIR/.venv"
+ # bring down global requirements
+ git_clone $REQUIREMENTS_REPO $REQUIREMENTS_DIR $REQUIREMENTS_BRANCH
+ [ ! -d $PIP_VIRTUAL_ENV ] && virtualenv $PIP_VIRTUAL_ENV
+ # We don't care about testing git pbr in the requirements venv.
+ PIP_VIRTUAL_ENV=$PIP_VIRTUAL_ENV pip_install -U pbr
+ PIP_VIRTUAL_ENV=$PIP_VIRTUAL_ENV pip_install $REQUIREMENTS_DIR
+
+ # Unset the PIP_VIRTUAL_ENV so that PBR does not end up trapped
+ # down the VENV well
+ unset PIP_VIRTUAL_ENV
+
# Install pbr
if use_library_from_git "pbr"; then
git_clone_by_name "pbr"
@@ -38,10 +50,6 @@
# in via system packages.
pip_install "-U" "pbr"
fi
-
- # bring down global requirements
- git_clone $REQUIREMENTS_REPO $REQUIREMENTS_DIR $REQUIREMENTS_BRANCH
- pip_install $REQUIREMENTS_DIR
}
# Restore xtrace
diff --git a/lib/neutron-legacy b/lib/neutron-legacy
index 3ac76a2..ee72b60 100644
--- a/lib/neutron-legacy
+++ b/lib/neutron-legacy
@@ -696,9 +696,10 @@
if is_ssl_enabled_service "neutron"; then
ssl_ca="--ca-certificate=${SSL_BUNDLE_FILE}"
fi
- if ! timeout $SERVICE_TIMEOUT sh -c "while ! wget ${ssl_ca} --no-proxy -q -O- $service_protocol://$Q_HOST:$service_port; do sleep 1; done"; then
- die $LINENO "Neutron did not start"
- fi
+
+ local testcmd="wget ${ssl_ca} --no-proxy -q -O- $service_protocol://$Q_HOST:$service_port"
+ test_with_retry "$testcmd" "Neutron did not start" $SERVICE_TIMEOUT
+
# Start proxy if enabled
if is_service_enabled tls-proxy; then
start_tls_proxy '*' $Q_PORT $Q_HOST $Q_PORT_INT &
@@ -1266,9 +1267,19 @@
# This logic is specific to using the l3-agent for layer 3
if is_service_enabled q-l3; then
# Configure and enable public bridge
+ local ext_gw_interface="none"
if is_neutron_ovs_base_plugin && [[ "$Q_USE_NAMESPACE" = "True" ]]; then
- local ext_gw_interface=$(_neutron_get_ext_gw_interface)
+ ext_gw_interface=$(_neutron_get_ext_gw_interface)
+ elif [[ "$Q_AGENT" = "linuxbridge" ]]; then
+ # Search for the brq device the neutron router and network for $FIXED_RANGE
+ # will be using.
+ # e.x. brq3592e767-da for NET_ID 3592e767-da66-4bcb-9bec-cdb03cd96102
+ ext_gw_interface=brq${EXT_NET_ID:0:11}
+ fi
+ if [[ "$ext_gw_interface" != "none" ]]; then
local cidr_len=${FLOATING_RANGE#*/}
+ local testcmd="ip -o link | grep -q $ext_gw_interface"
+ test_with_retry "$testcmd" "$ext_gw_interface creation failed"
if [[ $(ip addr show dev $ext_gw_interface | grep -c $ext_gw_ip) == 0 && ( $Q_USE_PROVIDERNET_FOR_PUBLIC == "False" || $Q_USE_PUBLIC_VETH == "True" ) ]]; then
sudo ip addr add $ext_gw_ip/$cidr_len dev $ext_gw_interface
sudo ip link set $ext_gw_interface up
@@ -1380,9 +1391,8 @@
local timeout_sec=$5
local probe_cmd = ""
probe_cmd=`_get_probe_cmd_prefix $from_net`
- if ! timeout $timeout_sec sh -c "while ! $probe_cmd ssh -o StrictHostKeyChecking=no -i $key_file ${user}@$ip echo success; do sleep 1; done"; then
- die $LINENO "server didn't become ssh-able!"
- fi
+ local testcmd="$probe_cmd ssh -o StrictHostKeyChecking=no -i $key_file ${user}@$ip echo success"
+ test_with_retry "$testcmd" "server $ip didn't become ssh-able" $timeout_sec
}
# Neutron 3rd party programs
diff --git a/lib/nova_plugins/functions-libvirt b/lib/nova_plugins/functions-libvirt
index 96d8a44..5525cfd 100755
--- a/lib/nova_plugins/functions-libvirt
+++ b/lib/nova_plugins/functions-libvirt
@@ -28,16 +28,21 @@
else
install_package qemu-kvm
install_package libguestfs0
- install_package python-guestfs
fi
install_package libvirt-bin libvirt-dev
pip_install_gr libvirt-python
#pip_install_gr <there-si-no-guestfs-in-pypi>
elif is_fedora || is_suse; then
install_package kvm
+ # there is a dependency issue with kvm (which is really just a
+ # wrapper to qemu-system-x86) that leaves some bios files out,
+ # so install qemu-kvm (which shouldn't strictly be needed, as
+ # everything has been merged into qemu-system-x86) to bring in
+ # the right packages. see
+ # https://bugzilla.redhat.com/show_bug.cgi?id=1235890
+ install_package qemu-kvm
install_package libvirt libvirt-devel
pip_install_gr libvirt-python
- install_package python-libguestfs
fi
}
diff --git a/lib/nova_plugins/hypervisor-libvirt b/lib/nova_plugins/hypervisor-libvirt
index a6a87f9..f70b21a 100644
--- a/lib/nova_plugins/hypervisor-libvirt
+++ b/lib/nova_plugins/hypervisor-libvirt
@@ -26,7 +26,7 @@
# --------
# File injection is disabled by default in Nova. This will turn it back on.
-ENABLE_FILE_INJECTION=${ENABLE_FILE_INJECTION:-False}
+ENABLE_FILE_INJECTION=$(trueorfalse False ENABLE_FILE_INJECTION)
# Entry Points
@@ -60,7 +60,6 @@
iniset $NOVA_CONF DEFAULT vnc_enabled "false"
fi
- ENABLE_FILE_INJECTION=$(trueorfalse False ENABLE_FILE_INJECTION)
if [[ "$ENABLE_FILE_INJECTION" = "True" ]] ; then
# When libguestfs is available for file injection, enable using
# libguestfs to inspect the image and figure out the proper
@@ -97,6 +96,14 @@
yum_install libcgroup-tools
fi
fi
+
+ if [[ "$ENABLE_FILE_INJECTION" = "True" ]] ; then
+ if is_ubuntu; then
+ install_package python-guestfs
+ elif is_fedora || is_suse; then
+ install_package python-libguestfs
+ fi
+ fi
}
# start_nova_hypervisor - Start any required external services
diff --git a/lib/tempest b/lib/tempest
index c4ae05f..9fba0aa 100644
--- a/lib/tempest
+++ b/lib/tempest
@@ -329,6 +329,7 @@
if [[ ! -z "$TEMPEST_HTTP_IMAGE" ]]; then
iniset $TEMPEST_CONFIG image http_image $TEMPEST_HTTP_IMAGE
fi
+ iniset $TEMPEST_CONFIG image-feature-enabled deactivate_image true
# Image Features
iniset $TEMPEST_CONFIG image-feature-enabled deactivate_image True
@@ -451,6 +452,9 @@
iniset $TEMPEST_CONFIG object-storage-feature-enabled discoverable_apis $object_storage_api_extensions
# Volume
+ # TODO(dkranz): Remove the bootable flag when Juno is end of life.
+ iniset $TEMPEST_CONFIG volume-feature-enabled bootable True
+
local volume_api_extensions=${VOLUME_API_EXTENSIONS:-"all"}
if [[ ! -z "$DISABLE_VOLUME_API_EXTENSIONS" ]]; then
# Enabled extensions are either the ones explicitly specified or those available on the API endpoint
diff --git a/stack.sh b/stack.sh
index 0c01165..7a5ed04 100755
--- a/stack.sh
+++ b/stack.sh
@@ -263,6 +263,7 @@
EOF
# Enable a bootstrap repo. It is removed after finishing
# the epel-release installation.
+ is_package_installed yum-utils || install_package yum-utils
sudo yum-config-manager --enable epel-bootstrap
yum_install epel-release || \
die $LINENO "Error installing EPEL repo, cannot continue"
@@ -270,7 +271,6 @@
sudo rm -f /etc/yum.repos.d/epel-bootstrap.repo
# ... and also optional to be enabled
- is_package_installed yum-utils || install_package yum-utils
sudo yum-config-manager --enable rhel-7-server-optional-rpms
RHEL_RDO_REPO_RPM=${RHEL7_RDO_REPO_RPM:-"https://repos.fedorapeople.org/repos/openstack/openstack-juno/rdo-release-juno-1.noarch.rpm"}
@@ -669,6 +669,9 @@
fi
fi
+# Save configuration values
+save_stackenv $LINENO
+
# Install Packages
# ================
@@ -950,6 +953,9 @@
# Initialize the directory for service status check
init_service_check
+# Save configuration values
+save_stackenv $LINENO
+
# Start Services
# ==============
@@ -1287,35 +1293,44 @@
# Save some values we generated for later use
-CURRENT_RUN_TIME=$(date "+$TIMESTAMP_FORMAT")
-echo "# $CURRENT_RUN_TIME" >$TOP_DIR/.stackenv
-for i in BASE_SQL_CONN ENABLED_SERVICES HOST_IP LOGFILE \
- SERVICE_HOST SERVICE_PROTOCOL STACK_USER TLS_IP KEYSTONE_AUTH_PROTOCOL OS_CACERT; do
- echo $i=${!i} >>$TOP_DIR/.stackenv
-done
+save_stackenv
-# Write out a clouds.yaml file
-# putting the location into a variable to allow for easier refactoring later
-# to make it overridable. There is current no usecase where doing so makes
-# sense, so I'm not actually doing it now.
+# Update/create user clouds.yaml file.
+# clouds.yaml will have
+# - A `devstack` entry for the `demo` user for the `demo` project.
+# - A `devstack-admin` entry for the `admin` user for the `admin` project.
+
+# The location is a variable to allow for easier refactoring later to make it
+# overridable. There is currently no usecase where doing so makes sense, so
+# it's not currently configurable.
CLOUDS_YAML=~/.config/openstack/clouds.yaml
-if [ ! -e $CLOUDS_YAML ]; then
- mkdir -p $(dirname $CLOUDS_YAML)
- cat >"$CLOUDS_YAML" <<EOF
-clouds:
- devstack:
- auth:
- auth_url: $KEYSTONE_AUTH_URI/v$IDENTITY_API_VERSION
- username: demo
- project_name: demo
- password: $ADMIN_PASSWORD
- region_name: $REGION_NAME
- identity_api_version: $IDENTITY_API_VERSION
-EOF
- if [ -f "$SSL_BUNDLE_FILE" ]; then
- echo " cacert: $SSL_BUNDLE_FILE" >>"$CLOUDS_YAML"
- fi
+
+mkdir -p $(dirname $CLOUDS_YAML)
+
+CA_CERT_ARG=''
+if [ -f "$SSL_BUNDLE_FILE" ]; then
+ CA_CERT_ARG="--os-cacert $SSL_BUNDLE_FILE"
fi
+$TOP_DIR/tools/update_clouds_yaml.py \
+ --file $CLOUDS_YAML \
+ --os-cloud devstack \
+ --os-region-name $REGION_NAME \
+ --os-identity-api-version $IDENTITY_API_VERSION \
+ $CA_CERT_ARG \
+ --os-auth-url $KEYSTONE_AUTH_URI/v$IDENTITY_API_VERSION \
+ --os-username demo \
+ --os-password $ADMIN_PASSWORD \
+ --os-project-name demo
+$TOP_DIR/tools/update_clouds_yaml.py \
+ --file $CLOUDS_YAML \
+ --os-cloud devstack-admin \
+ --os-region-name $REGION_NAME \
+ --os-identity-api-version $IDENTITY_API_VERSION \
+ $CA_CERT_ARG \
+ --os-auth-url $KEYSTONE_AUTH_URI/v$IDENTITY_API_VERSION \
+ --os-username admin \
+ --os-password $ADMIN_PASSWORD \
+ --os-project-name admin
# Wrapup configuration
diff --git a/stackrc b/stackrc
index 1ac1338..9cd9c05 100644
--- a/stackrc
+++ b/stackrc
@@ -149,17 +149,6 @@
# Zero disables timeouts
GIT_TIMEOUT=${GIT_TIMEOUT:-0}
-# Requirements enforcing mode
-#
-# - strict (default) : ensure all project requirements files match
-# what's in global requirements.
-#
-# - soft : enforce requirements on everything in
-# requirements/projects.txt, but do soft updates on all other
-# repositories (i.e. sync versions for requirements that are in g-r,
-# but pass through any extras)
-REQUIREMENTS_MODE=${REQUIREMENTS_MODE:-strict}
-
# Repositories
# ------------
diff --git a/tools/fixup_stuff.sh b/tools/fixup_stuff.sh
index 31258d1..4fff57f 100755
--- a/tools/fixup_stuff.sh
+++ b/tools/fixup_stuff.sh
@@ -126,6 +126,9 @@
# [4] http://docs.openstack.org/developer/devstack/guides/neutron.html
if is_package_installed firewalld; then
sudo systemctl disable firewalld
+ # The iptables service files are no longer included by default,
+ # at least on a baremetal Fedora 21 Server install.
+ install_package iptables-services
sudo systemctl enable iptables
sudo systemctl stop firewalld
sudo systemctl start iptables
diff --git a/tools/update_clouds_yaml.py b/tools/update_clouds_yaml.py
new file mode 100755
index 0000000..0862135
--- /dev/null
+++ b/tools/update_clouds_yaml.py
@@ -0,0 +1,95 @@
+#!/usr/bin/env python
+
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+# Update the clouds.yaml file.
+
+
+import argparse
+import os.path
+
+import yaml
+
+
+class UpdateCloudsYaml(object):
+ def __init__(self, args):
+ if args.file:
+ self._clouds_path = args.file
+ self._create_directory = False
+ else:
+ self._clouds_path = os.path.expanduser(
+ '~/.config/openstack/clouds.yaml')
+ self._create_directory = True
+ self._clouds = {}
+
+ self._cloud = args.os_cloud
+ self._cloud_data = {
+ 'region_name': args.os_region_name,
+ 'identity_api_version': args.os_identity_api_version,
+ 'auth': {
+ 'auth_url': args.os_auth_url,
+ 'username': args.os_username,
+ 'password': args.os_password,
+ 'project_name': args.os_project_name,
+ },
+ }
+ if args.os_cacert:
+ self._cloud_data['cacert'] = args.os_cacert
+
+ def run(self):
+ self._read_clouds()
+ self._update_clouds()
+ self._write_clouds()
+
+ def _read_clouds(self):
+ try:
+ with open(self._clouds_path) as clouds_file:
+ self._clouds = yaml.load(clouds_file)
+ except IOError:
+ # The user doesn't have a clouds.yaml file.
+ print("The user clouds.yaml file didn't exist.")
+ self._clouds = {}
+
+ def _update_clouds(self):
+ self._clouds.setdefault('clouds', {})[self._cloud] = self._cloud_data
+
+ def _write_clouds(self):
+
+ if self._create_directory:
+ clouds_dir = os.path.dirname(self._clouds_path)
+ os.makedirs(clouds_dir)
+
+ with open(self._clouds_path, 'w') as clouds_file:
+ yaml.dump(self._clouds, clouds_file, default_flow_style=False)
+
+
+def main():
+ parser = argparse.ArgumentParser('Update clouds.yaml file.')
+ parser.add_argument('--file')
+ parser.add_argument('--os-cloud', required=True)
+ parser.add_argument('--os-region-name', default='RegionOne')
+ parser.add_argument('--os-identity-api-version', default='3')
+ parser.add_argument('--os-cacert')
+ parser.add_argument('--os-auth-url', required=True)
+ parser.add_argument('--os-username', required=True)
+ parser.add_argument('--os-password', required=True)
+ parser.add_argument('--os-project-name', required=True)
+
+ args = parser.parse_args()
+
+ update_clouds_yaml = UpdateCloudsYaml(args)
+ update_clouds_yaml.run()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/unstack.sh b/unstack.sh
index f0da971..10e5958 100755
--- a/unstack.sh
+++ b/unstack.sh
@@ -187,5 +187,10 @@
fi
# BUG: maybe it doesn't exist? We should isolate this further down.
-clean_lvm_volume_group $DEFAULT_VOLUME_GROUP_NAME || /bin/true
-clean_lvm_filter
+# NOTE: Cinder automatically installs the lvm2 package, independently of the
+# enabled backends. So if Cinder is enabled, we are sure lvm (lvremove,
+# /etc/lvm/lvm.conf, etc.) is here.
+if is_service_enabled cinder; then
+ clean_lvm_volume_group $DEFAULT_VOLUME_GROUP_NAME || /bin/true
+ clean_lvm_filter
+fi