Merge "add a table of contents"
diff --git a/.gitignore b/.gitignore
index d1781bc..7967e14 100644
--- a/.gitignore
+++ b/.gitignore
@@ -23,6 +23,7 @@
files/pip-*
files/get-pip.py*
files/ir-deploy*
+files/ironic-inspector*
local.conf
local.sh
localrc
diff --git a/HACKING.rst b/HACKING.rst
index b76cb6c..fc67f09 100644
--- a/HACKING.rst
+++ b/HACKING.rst
@@ -322,7 +322,7 @@
Review Criteria
-===============
+---------------
There are some broad criteria that will be followed when reviewing
your change
@@ -364,3 +364,26 @@
* **Reviewers** -- please see ``MAINTAINERS.rst`` for a list of people
that should be added to reviews of various sub-systems.
+
+
+Making Changes, Testing, and CI
+-------------------------------
+
+Changes to Devstack are tested by automated continuous integration jobs
+that run on a variety of Linux Distros using a handful of common
+configurations. What this means is that every change to Devstack is
+self testing. One major benefit of this is that developers do not
+typically need to add new non voting test jobs to add features to
+Devstack. Instead the features can be added, then if testing passes
+with the feature enabled the change is ready to merge (pending code
+review).
+
+A concrete example of this was the switch from screen based service
+management to systemd based service management. No new jobs were
+created for this. Instead the features were added to devstack, tested
+locally and in CI using a change that enabled the feature, then once
+the enabling change was passing and the new behavior communicated and
+documented it was merged.
+
+Using this process has been proven to be effective and leads to
+quicker implementation of desired features.
diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst
index 318e044..66b8702 100644
--- a/doc/source/configuration.rst
+++ b/doc/source/configuration.rst
@@ -609,29 +609,6 @@
Only Swift proxy server is launched in the systemd system all other
services are started in background and managed by ``swift-init`` tool.
-Heat
-~~~~
-
-Heat is disabled by default (see ``stackrc`` file). To enable it
-explicitly you'll need the following settings in your ``localrc``
-section
-
-::
-
- enable_service heat h-api h-api-cfn h-api-cw h-eng
-
-Heat can also run in standalone mode, and be configured to orchestrate
-on an external OpenStack cloud. To launch only Heat in standalone mode
-you'll need the following settings in your ``localrc`` section
-
-::
-
- disable_all_services
- enable_service rabbit mysql heat h-api h-api-cfn h-api-cw h-eng
- HEAT_STANDALONE=True
- KEYSTONE_SERVICE_HOST=...
- KEYSTONE_AUTH_HOST=...
-
Tempest
~~~~~~~
diff --git a/doc/source/faq.rst b/doc/source/faq.rst
index cb2f328..a186336 100644
--- a/doc/source/faq.rst
+++ b/doc/source/faq.rst
@@ -159,16 +159,6 @@
often good enough for a single-node installation. And in an extreme
case, use ``clean.sh`` to eradicate it and try again.
-Configure ``local.conf`` thusly:
-
- ::
-
- [[local|localrc]]
- HEAT_STANDALONE=True
- ENABLED_SERVICES=rabbit,mysql,heat,h-api,h-api-cfn,h-api-cw,h-eng
- KEYSTONE_SERVICE_HOST=<keystone-host>
- KEYSTONE_AUTH_HOST=<keystone-host>
-
Why are my configuration changes ignored?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/source/overview.rst b/doc/source/overview.rst
index e648193..c07a8e6 100644
--- a/doc/source/overview.rst
+++ b/doc/source/overview.rst
@@ -64,8 +64,7 @@
The default services configured by DevStack are Identity (keystone),
Object Storage (swift), Image Service (glance), Block Storage
-(cinder), Compute (nova), Networking (neutron), Dashboard (horizon),
-Orchestration (heat)
+(cinder), Compute (nova), Networking (neutron), Dashboard (horizon)
Additional services not included directly in DevStack can be tied in to
``stack.sh`` using the :doc:`plugin mechanism <plugins>` to call
diff --git a/doc/source/systemd.rst b/doc/source/systemd.rst
index efe79e4..60a7719 100644
--- a/doc/source/systemd.rst
+++ b/doc/source/systemd.rst
@@ -109,6 +109,17 @@
journalctl -f -o short-precise --unit devstack@n-cpu.service
+By default, journalctl strips out "unprintable" characters, including
+ASCII color codes. To keep the color codes (which can be interpreted by
+an appropriate terminal/pager - e.g. ``less``, the default)::
+
+ journalctl -a --unit devstack@n-cpu.service
+
+When outputting to the terminal using the default pager, long lines
+appear to be truncated, but horizontal scrolling is supported via the
+left/right arrow keys.
+
+See ``man 1 journalctl`` for more.
Known Issues
============
@@ -137,13 +148,6 @@
Future Work
===========
-log colorizing
---------------
-
-We lose log colorization through this process. We might want to build
-a custom colorizer that we could run journalctl output through
-optionally for people.
-
user units
----------
diff --git a/files/debs/n-api b/files/debs/n-api
deleted file mode 100644
index 0928cd5..0000000
--- a/files/debs/n-api
+++ /dev/null
@@ -1 +0,0 @@
-fping
diff --git a/files/rpms-suse/n-api b/files/rpms-suse/n-api
index af5ac2f..0f08daa 100644
--- a/files/rpms-suse/n-api
+++ b/files/rpms-suse/n-api
@@ -1,2 +1 @@
-fping
python-dateutil
diff --git a/files/rpms/n-api b/files/rpms/n-api
deleted file mode 100644
index 0928cd5..0000000
--- a/files/rpms/n-api
+++ /dev/null
@@ -1 +0,0 @@
-fping
diff --git a/functions b/functions
index 8968a18..52a82fa 100644
--- a/functions
+++ b/functions
@@ -310,6 +310,11 @@
disk_format=qcow2
container_format=bare
;;
+ *.raw)
+ image_name=$(basename "$image" ".raw")
+ disk_format=raw
+ container_format=bare
+ ;;
*.iso)
image_name=$(basename "$image" ".iso")
disk_format=iso
@@ -611,21 +616,18 @@
# request-id. However, there may be an eventlet interaction here,
# so going off for now.
USE_JOURNAL=$(trueorfalse USE_JOURNAL False)
+ local pidstr=""
if [[ "$USE_JOURNAL" == "True" ]]; then
iniset $conf_file $conf_section use_journal "True"
# if we are using the journal directly, our process id is already correct
- iniset $conf_file $conf_section logging_debug_format_suffix \
- "{{%(funcName)s %(pathname)s:%(lineno)d}}"
else
- iniset $conf_file $conf_section logging_debug_format_suffix \
- "{{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}}"
+ pidstr="(pid=%(process)d) "
fi
+ iniset $conf_file $conf_section logging_debug_format_suffix "[00;33m{{${pidstr}%(funcName)s %(pathname)s:%(lineno)d}}[00m"
- iniset $conf_file $conf_section logging_context_format_string \
- "%(levelname)s %(name)s [%(request_id)s %(project_name)s %(user_name)s] %(instance)s%(message)s"
- iniset $conf_file $conf_section logging_default_format_string \
- "%(levelname)s %(name)s [-] %(instance)s%(color)s%(message)s"
- iniset $conf_file $conf_section logging_exception_prefix "ERROR %(name)s %(instance)s"
+ iniset $conf_file $conf_section logging_context_format_string "%(color)s%(levelname)s %(name)s [[01;36m%(request_id)s [00;36m%(project_name)s %(user_name)s%(color)s] [01;35m%(instance)s%(color)s%(message)s[00m"
+ iniset $conf_file $conf_section logging_default_format_string "%(color)s%(levelname)s %(name)s [[00;36m-%(color)s] [01;35m%(instance)s%(color)s%(message)s[00m"
+ iniset $conf_file $conf_section logging_exception_prefix "ERROR %(name)s [01;35m%(instance)s[00m"
}
function setup_standard_logging_identity {
@@ -715,6 +717,22 @@
}
+# Set a systemd system override
+#
+# This sets a system-side override in system.conf. A per-service
+# override would be /etc/systemd/system/${service}.service/override.conf
+function set_systemd_override {
+ local key="$1"
+ local value="$2"
+
+ local sysconf="/etc/systemd/system.conf"
+ iniset -sudo "${sysconf}" "Manager" "$key" "$value"
+ echo "Set systemd system override for ${key}=${value}"
+
+ sudo systemctl daemon-reload
+}
+
+
# Restore xtrace
$_XTRACE_FUNCTIONS
diff --git a/functions-common b/functions-common
index 965f051..13559da 100644
--- a/functions-common
+++ b/functions-common
@@ -48,8 +48,8 @@
# Save these variables to .stackenv
STACK_ENV_VARS="BASE_SQL_CONN DATA_DIR DEST ENABLED_SERVICES HOST_IP \
- KEYSTONE_AUTH_PROTOCOL KEYSTONE_AUTH_URI KEYSTONE_SERVICE_URI \
- LOGFILE OS_CACERT SERVICE_HOST SERVICE_PROTOCOL STACK_USER TLS_IP \
+ KEYSTONE_AUTH_URI KEYSTONE_SERVICE_URI \
+ LOGFILE OS_CACERT SERVICE_HOST STACK_USER TLS_IP \
HOST_IPV6 SERVICE_IP_VERSION"
diff --git a/inc/python b/inc/python
index 2443c4d..4c443d6 100644
--- a/inc/python
+++ b/inc/python
@@ -351,6 +351,20 @@
return $result
}
+function pip_uninstall {
+ local name=$1
+ if [[ -n ${PIP_VIRTUAL_ENV:=} && -d ${PIP_VIRTUAL_ENV} ]]; then
+ local cmd_pip=$PIP_VIRTUAL_ENV/bin/pip
+ local sudo_pip="env"
+ else
+ local cmd_pip
+ cmd_pip=$(get_pip_command $PYTHON2_VERSION)
+ local sudo_pip="sudo -H"
+ fi
+ # don't error if we can't uninstall, it might not be there
+ $sudo_pip $cmd_pip uninstall -y $name || /bin/true
+}
+
# get version of a package from global requirements file
# get_from_global_requirements <package>
function get_from_global_requirements {
diff --git a/lib/cinder b/lib/cinder
index 9fc25c7..e3a687b 100644
--- a/lib/cinder
+++ b/lib/cinder
@@ -244,6 +244,10 @@
configure_rootwrap cinder
+ if [[ -f "$CINDER_DIR/etc/cinder/resource_filters.json" ]]; then
+ cp -p "$CINDER_DIR/etc/cinder/resource_filters.json" "$CINDER_CONF_DIR/resource_filters.json"
+ fi
+
cp $CINDER_DIR/etc/cinder/api-paste.ini $CINDER_API_PASTE_INI
inicomment $CINDER_API_PASTE_INI filter:authtoken auth_host
diff --git a/lib/nova_plugins/functions-libvirt b/lib/nova_plugins/functions-libvirt
index 47605af..326c8bf 100644
--- a/lib/nova_plugins/functions-libvirt
+++ b/lib/nova_plugins/functions-libvirt
@@ -67,6 +67,8 @@
else
install_package libvirt-clients libvirt-daemon-system libvirt-dev
fi
+ # uninstall in case the libvirt version changed
+ pip_uninstall libvirt-python
pip_install_gr libvirt-python
#pip_install_gr <there-si-no-guestfs-in-pypi>
elif is_fedora || is_suse; then
@@ -84,6 +86,7 @@
fi
install_package libvirt libvirt-devel
+ pip_uninstall libvirt-python
pip_install_gr libvirt-python
fi
diff --git a/lib/tempest b/lib/tempest
index fd8e3e7..47785ec 100644
--- a/lib/tempest
+++ b/lib/tempest
@@ -394,24 +394,6 @@
iniset $TEMPEST_CONFIG network-feature-enabled ipv6_subnet_attributes "$IPV6_SUBNET_ATTRIBUTES_ENABLED"
iniset $TEMPEST_CONFIG network-feature-enabled port_security $NEUTRON_PORT_SECURITY
- # Orchestration Tests
- if is_service_enabled heat; then
- if [[ ! -z "$HEAT_CFN_IMAGE_URL" ]]; then
- iniset $TEMPEST_CONFIG orchestration image_ref $(basename "${HEAT_CFN_IMAGE_URL%.*}")
- fi
- # Nova might not be enabled, especially when we want to test tempest scenario/API that only create Neutron resources
- if is_service_enabled nova; then
- # build a specialized heat flavor
- available_flavors=$(nova flavor-list)
- if [[ ! ( $available_flavors =~ 'm1.heat' ) ]]; then
- openstack flavor create --id 451 --ram 512 --disk 0 --vcpus 1 m1.heat
- fi
- iniset $TEMPEST_CONFIG orchestration instance_type "m1.heat"
- fi
- iniset $TEMPEST_CONFIG orchestration build_timeout 900
- iniset $TEMPEST_CONFIG orchestration stack_owner_role Member
- fi
-
# Scenario
if [ "$VIRT_DRIVER" = "xenserver" ]; then
SCENARIO_IMAGE_DIR=${SCENARIO_IMAGE_DIR:-$FILES}
diff --git a/openrc b/openrc
index 38f5c47..23c173c 100644
--- a/openrc
+++ b/openrc
@@ -72,8 +72,6 @@
GLANCE_HOST=${GLANCE_HOST:-$HOST_IP}
fi
-SERVICE_PROTOCOL=${SERVICE_PROTOCOL:-http}
-
# Identity API version
export OS_IDENTITY_API_VERSION=${IDENTITY_API_VERSION:-3}
@@ -85,7 +83,12 @@
# the user/project has access to - including nova, glance, keystone, swift, ...
# We currently recommend using the version 3 *identity api*.
#
-export OS_AUTH_URL=$KEYSTONE_AUTH_URI
+
+# If you don't have a working .stackenv, this is the backup possition
+KEYSTONE_BACKUP=$SERVICE_PROTOCOL://$SERVICE_HOST:5000
+KEYSTONE_AUTH_URI=${KEYSTONE_AUTH_URI:-$KEYSTONE_BACKUP}
+
+export OS_AUTH_URL=${OS_AUTH_URL:-$KEYSTONE_AUTH_URI}
# Currently, in order to use openstackclient with Identity API v3,
# we need to set the domain which the user and project belong to.
diff --git a/stack.sh b/stack.sh
index 9fdb47e..e83eaea 100755
--- a/stack.sh
+++ b/stack.sh
@@ -2,7 +2,7 @@
# ``stack.sh`` is an opinionated OpenStack developer installation. It
# installs and configures various combinations of **Cinder**, **Glance**,
-# **Heat**, **Horizon**, **Keystone**, **Nova**, **Neutron**, and **Swift**
+# **Horizon**, **Keystone**, **Nova**, **Neutron**, and **Swift**
# This script's options can be changed by setting appropriate environment
# variables. You can configure things like which git repositories to use,
@@ -765,6 +765,9 @@
# Phase: pre-install
run_phase stack pre-install
+# NOTE(danms): Set global limits before installing anything
+set_systemd_override DefaultLimitNOFILE ${ULIMIT_NOFILE}
+
install_rpc_backend
restart_rpc_backend
@@ -1332,10 +1335,6 @@
USERRC_PARAMS="$USERRC_PARAMS --os-cacert $SSL_BUNDLE_FILE"
fi
- if [[ "$HEAT_STANDALONE" = "True" ]]; then
- USERRC_PARAMS="$USERRC_PARAMS --heat-url http://$HEAT_API_HOST:$HEAT_API_PORT/v1"
- fi
-
$TOP_DIR/tools/create_userrc.sh $USERRC_PARAMS
fi
diff --git a/stackrc b/stackrc
index 1300f45..41ff268 100644
--- a/stackrc
+++ b/stackrc
@@ -100,11 +100,9 @@
if [[ "$USER_UNITS" == "True" ]]; then
SYSTEMD_DIR="$HOME/.local/share/systemd/user"
SYSTEMCTL="systemctl --user"
- JOURNALCTL_F="journalctl -f -o short-precise --user-unit"
else
SYSTEMD_DIR="/etc/systemd/system"
SYSTEMCTL="sudo systemctl"
- JOURNALCTL_F="journalctl -f -o short-precise --unit"
fi
@@ -518,14 +516,6 @@
GITREPO["glance_store"]=${GLANCE_STORE_REPO:-${GIT_BASE}/openstack/glance_store.git}
GITBRANCH["glance_store"]=${GLANCE_STORE_BRANCH:-master}
-# heat-cfntools server agent
-HEAT_CFNTOOLS_REPO=${HEAT_CFNTOOLS_REPO:-${GIT_BASE}/openstack/heat-cfntools.git}
-HEAT_CFNTOOLS_BRANCH=${HEAT_CFNTOOLS_BRANCH:-master}
-
-# heat example templates and elements
-HEAT_TEMPLATES_REPO=${HEAT_TEMPLATES_REPO:-${GIT_BASE}/openstack/heat-templates.git}
-HEAT_TEMPLATES_BRANCH=${HEAT_TEMPLATES_BRANCH:-master}
-
# django openstack_auth library
GITREPO["django_openstack_auth"]=${HORIZONAUTH_REPO:-${GIT_BASE}/openstack/django_openstack_auth.git}
GITBRANCH["django_openstack_auth"]=${HORIZONAUTH_BRANCH:-master}
@@ -933,6 +923,9 @@
# ``LOGDIR`` is always set at this point so it is not useful as a 'enable' for service logs
# ``SCREEN_LOGDIR`` may be set, it is useful to enable the compat symlinks
+# System-wide ulimit file descriptors override
+ULIMIT_NOFILE=${ULIMIT_NOFILE:-2048}
+
# Local variables:
# mode: shell-script
# End:
diff --git a/tools/dstat.sh b/tools/dstat.sh
index ae7306e..01c6d9b 100755
--- a/tools/dstat.sh
+++ b/tools/dstat.sh
@@ -13,10 +13,10 @@
LOGDIR=$1
# Command line arguments for primary DStat process.
-DSTAT_OPTS="-tcmndrylpg --top-cpu-adv --top-io-adv --top-mem --swap"
+DSTAT_OPTS="-tcmndrylpg --top-cpu-adv --top-io-adv --top-mem --swap --tcp"
# Command-line arguments for secondary background DStat process.
-DSTAT_CSV_OPTS="-tcmndrylpg --output $LOGDIR/dstat-csv.log"
+DSTAT_CSV_OPTS="-tcmndrylpg --tcp --output $LOGDIR/dstat-csv.log"
# Execute and background the secondary dstat process and discard its output.
dstat $DSTAT_CSV_OPTS >& /dev/null &
diff --git a/tools/fixup_stuff.sh b/tools/fixup_stuff.sh
index f3ba702..d07d267 100755
--- a/tools/fixup_stuff.sh
+++ b/tools/fixup_stuff.sh
@@ -82,15 +82,17 @@
source /etc/ci/mirror_info.sh
sudo apt-add-repository -y "deb $NODEPOOL_UCA_MIRROR xenial-updates/ocata main"
-
- # Disable use of libvirt wheel here as presence of mirror implies
- # presence of cached wheel build against older libvirt binary.
- # TODO(clarkb) figure out how to use wheel again.
- sudo bash -c 'echo "no-binary = libvirt-python" >> /etc/pip.conf'
else
# Otherwise use upstream UCA
sudo add-apt-repository -y cloud-archive:ocata
fi
+
+ # Disable use of libvirt wheel since a cached wheel build might be
+ # against older libvirt binary. Particularly a problem if using
+ # the openstack wheel mirrors, but can hit locally too.
+ # TODO(clarkb) figure out how to use upstream wheel again.
+ iniset -sudo /etc/pip.conf "global" "no-binary" "libvirt-python"
+
# Force update our APT repos, since we added UCA above.
REPOS_UPDATED=False
apt_get_update