Merge "Tox: don't check stack-screenrc with bashate"
diff --git a/doc/source/guides/lxc.rst b/doc/source/guides/lxc.rst
new file mode 100644
index 0000000..a719d60
--- /dev/null
+++ b/doc/source/guides/lxc.rst
@@ -0,0 +1,164 @@
+================================
+All-In-One Single LXC Container
+================================
+
+This guide walks you through the process of deploying OpenStack using devstack
+in an LXC container instead of a VM.
+
+The primary benefits to running devstack inside a container instead of a VM is
+faster performance and lower memory overhead while still providing a suitable
+level of isolation. This can be particularly useful when you want to simulate
+running OpenStack on multiple nodes.
+
+.. Warning:: Containers do not provide the same level of isolation as a virtual
+ machine.
+
+.. Note:: Not all OpenStack features support running inside of a container. See
+ `Limitations`_ section below for details. :doc:`OpenStack in a VM <single-vm>`
+ is recommended for beginners.
+
+Prerequisites
+==============
+
+This guide is written for Ubuntu 14.04 but should be adaptable for any modern
+Linux distribution.
+
+Install the LXC package::
+
+ sudo apt-get install lxc
+
+You can verify support for containerization features in your currently running
+kernel using the ``lxc-checkconfig`` command.
+
+Container Setup
+===============
+
+Configuration
+---------------
+
+For a successful run of ``stack.sh`` and to permit use of KVM to run the VMs you
+launch inside your container, we need to use the following additional
+configuration options. Place the following in a file called
+``devstack-lxc.conf``::
+
+ # Permit access to /dev/loop*
+ lxc.cgroup.devices.allow = b 7:* rwm
+
+ # Setup access to /dev/net/tun and /dev/kvm
+ lxc.mount.entry = /dev/net/tun dev/net/tun none bind,create=file 0 0
+ lxc.mount.entry = /dev/kvm dev/kvm none bind,create=file 0 0
+
+ # Networking
+ lxc.network.type = veth
+ lxc.network.flags = up
+ lxc.network.link = lxcbr0
+
+
+Create Container
+-------------------
+
+The configuration and rootfs for LXC containers are created using the
+``lxc-create`` command.
+
+We will name our container ``devstack`` and use the ``ubuntu`` template which
+will use ``debootstrap`` to build a Ubuntu rootfs. It will default to the same
+release and architecture as the host system. We also install the additional
+packages ``bsdmainutils`` and ``git`` as we'll need them to run devstack::
+
+ sudo lxc-create -n devstack -t ubuntu -f devstack-lxc.conf -- --packages=bsdmainutils,git
+
+The first time it builds the rootfs will take a few minutes to download, unpack,
+and configure all the necessary packages for a minimal installation of Ubuntu.
+LXC will cache this and subsequent containers will only take seconds to create.
+
+.. Note:: To speed up the initial rootfs creation, you can specify a mirror to
+ download the Ubuntu packages from by appending ``--mirror=`` and then the URL
+ of a Ubuntu mirror. To see other other template options, you can run
+ ``lxc-create -t ubuntu -h``.
+
+Start Container
+----------------
+
+To start the container, run::
+
+ sudo lxc-start -n devstack
+
+A moment later you should be presented with the login prompt for your container.
+You can login using the username ``ubuntu`` and password ``ubuntu``.
+
+You can also ssh into your container. On your host, run
+``sudo lxc-info -n devstack`` to get the IP address (e.g.
+``ssh ubuntu@$(sudo lxc-info -n p2 | awk '/IP/ { print $2 }')``).
+
+Run Devstack
+-------------
+
+You should now be logged into your container and almost ready to run devstack.
+The commands in this section should all be run inside your container.
+
+.. Tip:: You can greatly reduce the runtime of your initial devstack setup by
+ ensuring you have your apt sources.list configured to use a fast mirror.
+ Check and update ``/etc/apt/sources.list`` if necessary and then run
+ ``apt-get update``.
+
+#. Download DevStack
+
+ ::
+
+ git clone https://git.openstack.org/openstack-dev/devstack
+
+#. Configure
+
+ Refer to :ref:`minimal-configuration` if you wish to configure the behaviour
+ of devstack.
+
+#. Start the install
+
+ ::
+
+ cd devstack
+ ./stack.sh
+
+Cleanup
+-------
+
+To stop the container::
+
+ lxc-stop -n devstack
+
+To delete the container::
+
+ lxc-destroy -n devstack
+
+Limitations
+============
+
+Not all OpenStack features may function correctly or at all when ran from within
+a container.
+
+Cinder
+-------
+
+Unable to create LVM backed volume
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+ In our configuration, we have not whitelisted access to device-mapper or LVM
+ devices. Doing so will permit your container to have access and control of LVM
+ on the host system. To enable, add the following to your
+ ``devstack-lxc.conf`` before running ``lxc-create``::
+
+ lxc.cgroup.devices.allow = c 10:236 rwm
+ lxc.cgroup.devices.allow = b 252:* rwm
+
+ Additionally you'll need to set ``udev_rules = 0`` in the ``activation``
+ section of ``/etc/lvm/lvm.conf`` unless you mount devtmpfs in your container.
+
+Unable to attach volume to instance
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+ It is not possible to attach cinder volumes to nova instances due to parts of
+ the Linux iSCSI implementation not being network namespace aware. This can be
+ worked around by using network pass-through instead of a separate network
+ namespace but such a setup significantly reduces the isolation of the
+ container (e.g. a ``halt`` command issued in the container will cause the host
+ system to shutdown).
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 4a1d93d..3e324ad 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -76,6 +76,7 @@
guides/single-vm
guides/single-machine
+ guides/lxc
guides/multinode-lab
guides/neutron
guides/devstack-with-nested-kvm
@@ -96,6 +97,13 @@
server-class machine or a laptop at home.
:doc:`[Read] <guides/single-machine>`
+All-In-One LXC Container
+-------------------------
+
+Run :doc:`OpenStack in a LXC container <guides/lxc>`. Beneficial for intermediate
+and advanced users. The VMs launched in this cloud will be fully accelerated but
+not all OpenStack features are supported. :doc:`[Read] <guides/lxc>`
+
Multi-Node Lab
--------------
diff --git a/files/rpms/general b/files/rpms/general
index e0ef54c..a0906e2 100644
--- a/files/rpms/general
+++ b/files/rpms/general
@@ -26,6 +26,7 @@
psmisc
pyOpenSSL # version in pip uses too much memory
python-devel
+redhat-rpm-config # missing dep for gcc hardening flags, see rhbz#1217376
screen
tar
tcpdump
diff --git a/functions b/functions
index 29d0518..8e9820c 100644
--- a/functions
+++ b/functions
@@ -301,8 +301,8 @@
*) echo "Do not know what to do with $image_fname"; false;;
esac
- if is_arch "ppc64"; then
- img_property="--property hw_cdrom_bus=scsi"
+ if is_arch "ppc64le" || is_arch "ppc64" || is_arch "ppc"; then
+ img_property="--property hw_disk_bus=scsi --property hw_scsi_model=virtio-scsi --property hw_cdrom_bus=scsi --property os_command_line=console=hvc0"
fi
if is_arch "aarch64"; then
diff --git a/functions-common b/functions-common
index 80bdbea..8e52853 100644
--- a/functions-common
+++ b/functions-common
@@ -1429,14 +1429,17 @@
local service=$1
local command="$2"
local group=$3
+ local subservice=$4
+
+ local name=${subservice:-$service}
time_start "run_process"
if is_service_enabled $service; then
if [[ "$USE_SCREEN" = "True" ]]; then
- screen_process "$service" "$command" "$group"
+ screen_process "$name" "$command" "$group"
else
# Spawn directly without screen
- _run_process "$service" "$command" "$group" &
+ _run_process "$name" "$command" "$group" &
fi
fi
time_stop "run_process"
diff --git a/lib/heat b/lib/heat
index 1bb753d..4131878 100644
--- a/lib/heat
+++ b/lib/heat
@@ -196,6 +196,9 @@
iniset $HEAT_CONF DEFAULT enable_stack_abandon true
fi
+ iniset $HEAT_CONF cache enabled "True"
+ iniset $HEAT_CONF cache backend "dogpile.cache.memory"
+
sudo install -d -o $STACK_USER $HEAT_ENV_DIR $HEAT_TEMPLATES_DIR
# copy the default environment
diff --git a/lib/keystone b/lib/keystone
index 3c67693..7d5fd41 100644
--- a/lib/keystone
+++ b/lib/keystone
@@ -56,8 +56,23 @@
KEYSTONE_CATALOG_BACKEND="sql"
# Toggle for deploying Keystone under HTTPD + mod_wsgi
+# Deprecated in Mitaka, use KEYSTONE_DEPLOY instead.
KEYSTONE_USE_MOD_WSGI=${KEYSTONE_USE_MOD_WSGI:-${ENABLE_HTTPD_MOD_WSGI_SERVICES}}
+# KEYSTONE_DEPLOY defines how keystone is deployed, allowed values:
+# - mod_wsgi : Run keystone under Apache HTTPd mod_wsgi
+# - eventlet : Run keystone-all
+# - uwsgi : Run keystone under uwsgi
+if [ -z "$KEYSTONE_DEPLOY" ]; then
+ if [ -z "$KEYSTONE_USE_MOD_WSGI" ]; then
+ KEYSTONE_DEPLOY=mod_wsgi
+ elif [ "$KEYSTONE_USE_MOD_WSGI" == True ]; then
+ KEYSTONE_DEPLOY=mod_wsgi
+ else
+ KEYSTONE_DEPLOY=eventlet
+ fi
+fi
+
# Select the token persistence backend driver
KEYSTONE_TOKEN_BACKEND=${KEYSTONE_TOKEN_BACKEND:-sql}
@@ -230,16 +245,15 @@
# Register SSL certificates if provided
if is_ssl_enabled_service key; then
ensure_certificates KEYSTONE
-
- iniset $KEYSTONE_CONF eventlet_server_ssl enable True
- iniset $KEYSTONE_CONF eventlet_server_ssl certfile $KEYSTONE_SSL_CERT
- iniset $KEYSTONE_CONF eventlet_server_ssl keyfile $KEYSTONE_SSL_KEY
fi
+ local service_port=$KEYSTONE_SERVICE_PORT
+ local auth_port=$KEYSTONE_AUTH_PORT
+
if is_service_enabled tls-proxy; then
# Set the service ports for a proxy to take the originals
- iniset $KEYSTONE_CONF eventlet_server public_port $KEYSTONE_SERVICE_PORT_INT
- iniset $KEYSTONE_CONF eventlet_server admin_port $KEYSTONE_AUTH_PORT_INT
+ service_port=$KEYSTONE_SERVICE_PORT_INT
+ auth_port=$KEYSTONE_AUTH_PORT_INT
iniset $KEYSTONE_CONF DEFAULT public_endpoint $KEYSTONE_SERVICE_URI
iniset $KEYSTONE_CONF DEFAULT admin_endpoint $KEYSTONE_AUTH_URI
@@ -259,19 +273,70 @@
fi
# Format logging
- if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ] && [ "$KEYSTONE_USE_MOD_WSGI" == "False" ] ; then
+ if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ] && [ "$KEYSTONE_DEPLOY" != "mod_wsgi" ] ; then
setup_colorized_logging $KEYSTONE_CONF DEFAULT
fi
iniset $KEYSTONE_CONF DEFAULT debug $ENABLE_DEBUG_LOG_LEVEL
- if [ "$KEYSTONE_USE_MOD_WSGI" == "True" ]; then
+ if [ "$KEYSTONE_DEPLOY" == "mod_wsgi" ]; then
iniset $KEYSTONE_CONF DEFAULT logging_context_format_string "%(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s"
iniset $KEYSTONE_CONF DEFAULT logging_default_format_string "%(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s"
iniset $KEYSTONE_CONF DEFAULT logging_debug_format_suffix "%(asctime)s.%(msecs)03d %(funcName)s %(pathname)s:%(lineno)d"
iniset $KEYSTONE_CONF DEFAULT logging_exception_prefix "%(asctime)s.%(msecs)03d %(process)d TRACE %(name)s %(instance)s"
_config_keystone_apache_wsgi
- else
+ elif [ "$KEYSTONE_DEPLOY" == "uwsgi" ]; then
+ # iniset creates these files when it's called if they don't exist.
+ KEYSTONE_PUBLIC_UWSGI_FILE=$KEYSTONE_CONF_DIR/keystone-uwsgi-public.ini
+ KEYSTONE_ADMIN_UWSGI_FILE=$KEYSTONE_CONF_DIR/keystone-uwsgi-admin.ini
+
+ rm -f "$KEYSTONE_PUBLIC_UWSGI_FILE"
+ rm -f "$KEYSTONE_ADMIN_UWSGI_FILE"
+
+ if is_ssl_enabled_service key; then
+ iniset "$KEYSTONE_PUBLIC_UWSGI_FILE" uwsgi https $KEYSTONE_SERVICE_HOST:$service_port,$KEYSTONE_SSL_CERT,$KEYSTONE_SSL_KEY
+ iniset "$KEYSTONE_ADMIN_UWSGI_FILE" uwsgi https $KEYSTONE_ADMIN_BIND_HOST:$auth_port,$KEYSTONE_SSL_CERT,$KEYSTONE_SSL_KEY
+ else
+ iniset "$KEYSTONE_PUBLIC_UWSGI_FILE" uwsgi http $KEYSTONE_SERVICE_HOST:$service_port
+ iniset "$KEYSTONE_ADMIN_UWSGI_FILE" uwsgi http $KEYSTONE_ADMIN_BIND_HOST:$auth_port
+ fi
+
+ iniset "$KEYSTONE_PUBLIC_UWSGI_FILE" uwsgi wsgi-file "$KEYSTONE_BIN_DIR/keystone-wsgi-public"
+ # This is running standalone
+ iniset "$KEYSTONE_PUBLIC_UWSGI_FILE" uwsgi master true
+ iniset "$KEYSTONE_PUBLIC_UWSGI_FILE" uwsgi threads $(nproc)
+ iniset "$KEYSTONE_PUBLIC_UWSGI_FILE" uwsgi enable-threads true
+ iniset "$KEYSTONE_PUBLIC_UWSGI_FILE" uwsgi plugins python
+ # uwsgi recommends this to prevent thundering herd on accept.
+ iniset "$KEYSTONE_PUBLIC_UWSGI_FILE" uwsgi thunder-lock true
+ # Override the default size for headers from the 4k default.
+ iniset "$KEYSTONE_PUBLIC_UWSGI_FILE" uwsgi buffer-size 65535
+ # Make sure the client doesn't try to re-use the connection.
+ iniset "$KEYSTONE_PUBLIC_UWSGI_FILE" uwsgi add-header "Connection: close"
+
+ iniset "$KEYSTONE_ADMIN_UWSGI_FILE" uwsgi wsgi-file "$KEYSTONE_BIN_DIR/keystone-wsgi-admin"
+ # This is running standalone
+ iniset "$KEYSTONE_ADMIN_UWSGI_FILE" uwsgi master true
+ iniset "$KEYSTONE_ADMIN_UWSGI_FILE" uwsgi threads $API_WORKERS
+ iniset "$KEYSTONE_ADMIN_UWSGI_FILE" uwsgi enable-threads true
+ iniset "$KEYSTONE_ADMIN_UWSGI_FILE" uwsgi plugins python
+ # uwsgi recommends this to prevent thundering herd on accept.
+ iniset "$KEYSTONE_ADMIN_UWSGI_FILE" uwsgi thunder-lock true
+ # Override the default size for headers from the 4k default.
+ iniset "$KEYSTONE_ADMIN_UWSGI_FILE" uwsgi buffer-size 65535
+ # Make sure the client doesn't try to re-use the connection.
+ iniset "$KEYSTONE_ADMIN_UWSGI_FILE" uwsgi add-header "Connection: close"
+
+ else # eventlet
+ if is_ssl_enabled_service key; then
+ iniset $KEYSTONE_CONF eventlet_server_ssl enable True
+ iniset $KEYSTONE_CONF eventlet_server_ssl certfile $KEYSTONE_SSL_CERT
+ iniset $KEYSTONE_CONF eventlet_server_ssl keyfile $KEYSTONE_SSL_KEY
+ fi
+
+ iniset $KEYSTONE_CONF eventlet_server public_port $service_port
+ iniset $KEYSTONE_CONF eventlet_server admin_port $auth_port
+
iniset $KEYSTONE_CONF eventlet_server admin_bind_host "$KEYSTONE_ADMIN_BIND_HOST"
iniset $KEYSTONE_CONF eventlet_server admin_workers "$API_WORKERS"
# Public workers will use the server default, typically number of CPU.
@@ -493,7 +558,7 @@
setup_develop $KEYSTONE_DIR ldap
fi
- if [ "$KEYSTONE_USE_MOD_WSGI" == "True" ]; then
+ if [ "$KEYSTONE_DEPLOY" == "mod_wsgi" ]; then
install_apache_wsgi
if is_ssl_enabled_service "key"; then
enable_mod_ssl
@@ -511,12 +576,15 @@
auth_protocol="http"
fi
- if [ "$KEYSTONE_USE_MOD_WSGI" == "True" ]; then
+ if [ "$KEYSTONE_DEPLOY" == "mod_wsgi" ]; then
enable_apache_site keystone
restart_apache_server
tail_log key /var/log/$APACHE_NAME/keystone.log
tail_log key-access /var/log/$APACHE_NAME/keystone_access.log
- else
+ elif [ "$KEYSTONE_DEPLOY" == "uwsgi" ]; then
+ run_process key "uwsgi $KEYSTONE_PUBLIC_UWSGI_FILE" "" "key-p"
+ run_process key "uwsgi $KEYSTONE_ADMIN_UWSGI_FILE" "" "key-a"
+ else # eventlet
# Start Keystone in a screen window
run_process key "$KEYSTONE_BIN_DIR/keystone-all --config-file $KEYSTONE_CONF"
fi
@@ -541,7 +609,7 @@
# stop_keystone() - Stop running processes
function stop_keystone {
- if [ "$KEYSTONE_USE_MOD_WSGI" == "True" ]; then
+ if [ "$KEYSTONE_DEPLOY" == "mod_wsgi" ]; then
disable_apache_site keystone
restart_apache_server
fi
diff --git a/lib/tempest b/lib/tempest
index e90ff93..f75d755 100644
--- a/lib/tempest
+++ b/lib/tempest
@@ -321,7 +321,10 @@
local tmp_cfg_file
tmp_cfg_file=$(mktemp)
cd $TEMPEST_DIR
- tox -revenv -- tempest verify-config -u -r -o $tmp_cfg_file
+ tox -revenv --notest
+ # NOTE(mtreinish): Respect constraints on tempest verify-config venv
+ tox -evenv -- pip install -c $REQUIREMENTS_DIR/upper-constraints.txt -r requirements.txt
+ tox -evenv -- tempest verify-config -uro $tmp_cfg_file
local compute_api_extensions=${COMPUTE_API_EXTENSIONS:-"all"}
if [[ ! -z "$DISABLE_COMPUTE_API_EXTENSIONS" ]]; then
@@ -450,10 +453,6 @@
iniset $TEMPEST_CONFIG validation network_for_ssh $PRIVATE_NETWORK_NAME
# Volume
- # TODO(dkranz): Remove the bootable flag when Juno is end of life.
- iniset $TEMPEST_CONFIG volume-feature-enabled bootable True
- # TODO(jordanP): Remove the extend_with_snapshot flag when Juno is end of life.
- iniset $TEMPEST_CONFIG volume-feature-enabled extend_with_snapshot True
# TODO(obutenko): Remove the incremental_backup_force flag when Kilo and Juno is end of life.
iniset $TEMPEST_CONFIG volume-feature-enabled incremental_backup_force True
# TODO(ynesenenko): Remove the volume_services flag when Liberty and Kilo will correct work with host info.
@@ -588,6 +587,10 @@
pip_install tox
pushd $TEMPEST_DIR
tox --notest -efull
+ # NOTE(mtreinish) Respect constraints in the tempest full venv, things that
+ # are using a tox job other than full will not be respecting constraints but
+ # running pip install -U on tempest requirements
+ $TEMPEST_DIR/.tox/full/bin/pip install -c $REQUIREMENTS_DIR/upper-constraints.txt -r requirements.txt
PROJECT_VENV["tempest"]=${TEMPEST_DIR}/.tox/full
install_tempest_lib
popd
diff --git a/tools/worlddump.py b/tools/worlddump.py
index 198bb7e..d129374 100755
--- a/tools/worlddump.py
+++ b/tools/worlddump.py
@@ -101,14 +101,24 @@
_dump_cmd("sudo iptables --line-numbers -L -nv -t %s" % table)
+def _netns_list():
+ process = subprocess.Popen(['ip', 'netns'], stdout=subprocess.PIPE)
+ stdout, _ = process.communicate()
+ return stdout.split()
+
+
def network_dump():
_header("Network Dump")
_dump_cmd("brctl show")
_dump_cmd("arp -n")
- _dump_cmd("ip addr")
- _dump_cmd("ip link")
- _dump_cmd("ip route")
+ ip_cmds = ["addr", "link", "route"]
+ for cmd in ip_cmds + ['netns']:
+ _dump_cmd("ip %s" % cmd)
+ for netns_ in _netns_list():
+ for cmd in ip_cmds:
+ args = {'netns': netns_, 'cmd': cmd}
+ _dump_cmd('sudo ip netns exec %(netns)s ip %(cmd)s' % args)
def ovs_dump():
diff --git a/unstack.sh b/unstack.sh
index d69e3f5..d7670e3 100755
--- a/unstack.sh
+++ b/unstack.sh
@@ -9,12 +9,12 @@
# Stop all processes by setting ``UNSTACK_ALL`` or specifying ``-a``
# on the command line
-UNSTACK_ALL=""
+UNSTACK_ALL=${UNSTACK_ALL:-""}
while getopts ":a" opt; do
case $opt in
a)
- UNSTACK_ALL=""
+ UNSTACK_ALL="-1"
;;
esac
done