Merge "Stop setting osapi_volume_base_URL"
diff --git a/.gitignore b/.gitignore
index d1781bc..d2c127d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -23,6 +23,8 @@
files/pip-*
files/get-pip.py*
files/ir-deploy*
+files/ironic-inspector*
+files/etcd*
local.conf
local.sh
localrc
diff --git a/HACKING.rst b/HACKING.rst
index b76cb6c..fc67f09 100644
--- a/HACKING.rst
+++ b/HACKING.rst
@@ -322,7 +322,7 @@
Review Criteria
-===============
+---------------
There are some broad criteria that will be followed when reviewing
your change
@@ -364,3 +364,26 @@
* **Reviewers** -- please see ``MAINTAINERS.rst`` for a list of people
that should be added to reviews of various sub-systems.
+
+
+Making Changes, Testing, and CI
+-------------------------------
+
+Changes to Devstack are tested by automated continuous integration jobs
+that run on a variety of Linux Distros using a handful of common
+configurations. What this means is that every change to Devstack is
+self testing. One major benefit of this is that developers do not
+typically need to add new non voting test jobs to add features to
+Devstack. Instead the features can be added, then if testing passes
+with the feature enabled the change is ready to merge (pending code
+review).
+
+A concrete example of this was the switch from screen based service
+management to systemd based service management. No new jobs were
+created for this. Instead the features were added to devstack, tested
+locally and in CI using a change that enabled the feature, then once
+the enabling change was passing and the new behavior communicated and
+documented it was merged.
+
+Using this process has been proven to be effective and leads to
+quicker implementation of desired features.
diff --git a/README.md b/README.rst
similarity index 88%
rename from README.md
rename to README.rst
index ff5598b..b4240bd 100644
--- a/README.md
+++ b/README.rst
@@ -1,6 +1,7 @@
DevStack is a set of scripts and utilities to quickly deploy an OpenStack cloud.
-# Goals
+Goals
+=====
* To quickly build dev OpenStack environments in a clean Ubuntu or Fedora
environment
@@ -20,12 +21,13 @@
networking configuration. We strongly recommend that you run `stack.sh`
in a clean and disposable vm when you are first getting started.
-# Versions
+Versions
+========
The DevStack master branch generally points to trunk versions of OpenStack
components. For older, stable versions, look for branches named
stable/[release] in the DevStack repo. For example, you can do the
-following to create a Newton OpenStack cloud:
+following to create a Newton OpenStack cloud::
git checkout stable/newton
./stack.sh
@@ -38,7 +40,8 @@
GLANCE_REPO=git://git.openstack.org/openstack/glance.git
GLANCE_BRANCH=milestone-proposed
-# Start A Dev Cloud
+Start A Dev Cloud
+=================
Installing in a dedicated disposable VM is safer than installing on your
dev machine! Plus you can pick one of the supported Linux distros for
@@ -54,14 +57,15 @@
* Keystone: http://myhost:5000/v2.0/
We also provide an environment file that you can use to interact with your
-cloud via CLI:
+cloud via CLI::
# source openrc file to load your environment with OpenStack CLI creds
. openrc
# list instances
- nova list
+ openstack server list
-# DevStack Execution Environment
+DevStack Execution Environment
+==============================
DevStack runs rampant over the system it runs on, installing things and
uninstalling other things. Running this on a system you care about is a recipe
@@ -81,10 +85,12 @@
it runs under. Many people simply use their usual login (the default
'ubuntu' login on a UEC image for example).
-# Customizing
+Customizing
+===========
DevStack can be extensively configured via the configuration file
`local.conf`. It is likely that you will need to provide and modify
this file if you want anything other than the most basic setup. Start
-by reading the [configuration guide](doc/source/configuration.rst) for
-details of the configuration file and the many available options.
+by reading the `configuration guide
+<https://docs.openstack.org/developer/devstack/configuration.html>`_
+for details of the configuration file and the many available options.
diff --git a/clean.sh b/clean.sh
index 90b21eb..9ffe3be 100755
--- a/clean.sh
+++ b/clean.sh
@@ -64,13 +64,8 @@
done
fi
-# See if there is anything running...
-# need to adapt when run_service is merged
-SESSION=$(screen -ls | awk '/[0-9].stack/ { print $1 }')
-if [[ -n "$SESSION" ]]; then
- # Let unstack.sh do its thing first
- $TOP_DIR/unstack.sh --all
-fi
+# Let unstack.sh do its thing first
+$TOP_DIR/unstack.sh --all
# Run extras
# ==========
@@ -130,6 +125,13 @@
sudo rm -rf $SCREEN_LOGDIR
fi
+# Clean out the sytemd user unit files if systemd was used.
+if [[ "$USE_SYSTEMD" = "True" ]]; then
+ sudo find $SYSTEMD_DIR -type f -name '*devstack@*service' -delete
+ # Make systemd aware of the deletion.
+ $SYSTEMCTL daemon-reload
+fi
+
# Clean up venvs
DIRS_TO_CLEAN="$WHEELHOUSE ${PROJECT_VENV[@]} .config/openstack"
rm -rf $DIRS_TO_CLEAN
diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst
index 318e044..064bf51 100644
--- a/doc/source/configuration.rst
+++ b/doc/source/configuration.rst
@@ -195,6 +195,9 @@
Setting it here also makes it available for ``openrc`` to set ``OS_AUTH_URL``.
``HOST_IPV6`` is not set by default.
+For architecture specific configurations which differ from the x86 default
+here, see `arch-configuration`_.
+
Historical Notes
================
@@ -609,29 +612,6 @@
Only Swift proxy server is launched in the systemd system all other
services are started in background and managed by ``swift-init`` tool.
-Heat
-~~~~
-
-Heat is disabled by default (see ``stackrc`` file). To enable it
-explicitly you'll need the following settings in your ``localrc``
-section
-
-::
-
- enable_service heat h-api h-api-cfn h-api-cw h-eng
-
-Heat can also run in standalone mode, and be configured to orchestrate
-on an external OpenStack cloud. To launch only Heat in standalone mode
-you'll need the following settings in your ``localrc`` section
-
-::
-
- disable_all_services
- enable_service rabbit mysql heat h-api h-api-cfn h-api-cw h-eng
- HEAT_STANDALONE=True
- KEYSTONE_SERVICE_HOST=...
- KEYSTONE_AUTH_HOST=...
-
Tempest
~~~~~~~
@@ -772,3 +752,60 @@
::
TERMINATE_TIMEOUT=30
+
+
+.. _arch-configuration:
+
+Architectures
+-------------
+
+The upstream CI runs exclusively on nodes with x86 architectures, but
+OpenStack supports even more architectures. Some of them need to configure
+Devstack in a certain way.
+
+KVM on s390x (IBM z Systems)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+KVM on s390x (IBM z Systems) is supported since the *Kilo* release. For
+an all-in-one setup, these minimal settings in the ``local.conf`` file
+are needed::
+
+ [[local|localrc]]
+ ADMIN_PASSWORD=secret
+ DATABASE_PASSWORD=$ADMIN_PASSWORD
+ RABBIT_PASSWORD=$ADMIN_PASSWORD
+ SERVICE_PASSWORD=$ADMIN_PASSWORD
+
+ DOWNLOAD_DEFAULT_IMAGES=False
+ IMAGE_URLS="https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-s390x-disk1.img"
+
+ enable_service n-sproxy
+ disable_service n-novnc
+ disable_service etcd3 # https://bugs.launchpad.net/devstack/+bug/1693192
+
+ [[post-config|$NOVA_CONF]]
+
+ [serial_console]
+ base_url=ws://$HOST_IP:6083/ # optional
+
+Reasoning:
+
+* The default image of Devstack is x86 only, so we deactivate the download
+ with ``DOWNLOAD_DEFAULT_IMAGES``. The referenced guest image
+ in the code above (``IMAGE_URLS``) serves as an example. The list of
+ possible s390x guest images is not limited to that.
+
+* This platform doesn't support a graphical console like VNC or SPICE.
+ The technical reason is the missing framebuffer on the platform. This
+ means we rely on the substitute feature *serial console* which needs the
+ proxy service ``n-sproxy``. We also disable VNC's proxy ``n-novnc`` for
+ that reason . The configuration in the ``post-config`` section is only
+ needed if you want to use the *serial console* outside of the all-in-one
+ setup.
+
+* The service ``etcd3`` needs to be disabled as long as bug report
+ https://bugs.launchpad.net/devstack/+bug/1693192 is not resolved.
+
+.. note:: To run *Tempest* against this *Devstack* all-in-one, you'll need
+ to use a guest image which is smaller than 1GB when uncompressed.
+ The example image from above is bigger than that!
diff --git a/doc/source/development.rst b/doc/source/development.rst
index a3f2747..957de9b 100644
--- a/doc/source/development.rst
+++ b/doc/source/development.rst
@@ -14,7 +14,7 @@
.. code-block:: bash
- sudo systemctl status --unit="devstack@*"
+ sudo systemctl status "devstack@*"
To learn more about the basics of systemd, see :doc:`/systemd`
@@ -27,14 +27,14 @@
.. code-block:: bash
- sudo systemctl restart --unit=devstack@n-cpu.service
+ sudo systemctl restart devstack@n-cpu.service
If your change impacts more than one daemon you can restart by
wildcard as well.
.. code-block:: bash
- sudo systemctl restart --unit="devstack@n-*"
+ sudo systemctl restart "devstack@n-*"
.. warning::
@@ -112,6 +112,6 @@
.. code-block:: bash
- sudo systemctl restart --unit="devstack@n-*"
+ sudo systemctl restart "devstack@n-*"
which will restart all nova services.
diff --git a/doc/source/faq.rst b/doc/source/faq.rst
index cb2f328..a186336 100644
--- a/doc/source/faq.rst
+++ b/doc/source/faq.rst
@@ -159,16 +159,6 @@
often good enough for a single-node installation. And in an extreme
case, use ``clean.sh`` to eradicate it and try again.
-Configure ``local.conf`` thusly:
-
- ::
-
- [[local|localrc]]
- HEAT_STANDALONE=True
- ENABLED_SERVICES=rabbit,mysql,heat,h-api,h-api-cfn,h-api-cw,h-eng
- KEYSTONE_SERVICE_HOST=<keystone-host>
- KEYSTONE_AUTH_HOST=<keystone-host>
-
Why are my configuration changes ignored?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/source/guides/devstack-with-lbaas-v2.rst b/doc/source/guides/devstack-with-lbaas-v2.rst
index 21bea99..4ed64bf 100644
--- a/doc/source/guides/devstack-with-lbaas-v2.rst
+++ b/doc/source/guides/devstack-with-lbaas-v2.rst
@@ -45,7 +45,7 @@
# Horizon
ENABLED_SERVICES+=,horizon
# Nova
- ENABLED_SERVICES+=,n-api,n-crt,n-cpu,n-cond,n-sch
+ ENABLED_SERVICES+=,n-api,n-cpu,n-cond,n-sch
# Glance
ENABLED_SERVICES+=,g-api,g-reg
# Neutron
diff --git a/doc/source/index.rst b/doc/source/index.rst
index cbd6971..47087c5 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -142,3 +142,12 @@
Get :doc:`the big picture <overview>` of what we are trying to do
with devstack, and help us by :doc:`contributing to the project
<hacking>`.
+
+Contents
+--------
+
+.. toctree::
+ :glob:
+ :maxdepth: 2
+
+ *
diff --git a/doc/source/overview.rst b/doc/source/overview.rst
index d245035..c07a8e6 100644
--- a/doc/source/overview.rst
+++ b/doc/source/overview.rst
@@ -20,11 +20,11 @@
*The OpenStack Technical Committee (TC) has defined the current CI
strategy to include the latest Ubuntu release and the latest RHEL
-release (for Python 2.6 testing).*
+release.*
- Ubuntu: current LTS release plus current development release
- Fedora: current release plus previous release
-- RHEL: current major release
+- RHEL/Centos: current major release
- Other OS platforms may continue to be included but the maintenance of
those platforms shall not be assumed simply due to their presence.
Having a listed point-of-contact for each additional OS will greatly
@@ -38,7 +38,6 @@
*As packaged by the host OS*
- MySQL
-- PostgreSQL
Queues
------
@@ -46,7 +45,6 @@
*As packaged by the host OS*
- Rabbit
-- Qpid
Web Server
----------
@@ -58,9 +56,6 @@
OpenStack Network
-----------------
-*Defaults to nova network, optionally use neutron*
-
-- Nova Network: FlatDHCP
- Neutron: A basic configuration approximating the original FlatDHCP
mode using linuxbridge or OpenVSwitch.
@@ -68,9 +63,8 @@
--------
The default services configured by DevStack are Identity (keystone),
-Object Storage (swift), Image Service (glance), Block Storage (cinder),
-Compute (nova), Networking (nova), Dashboard (horizon), Orchestration
-(heat)
+Object Storage (swift), Image Service (glance), Block Storage
+(cinder), Compute (nova), Networking (neutron), Dashboard (horizon)
Additional services not included directly in DevStack can be tied in to
``stack.sh`` using the :doc:`plugin mechanism <plugins>` to call
@@ -80,8 +74,7 @@
-------------------
- single node
-- multi-node is not tested regularly by the core team, and even then
- only minimal configurations are reviewed
+- multi-node configurations as are tested by the gate
Exercises
---------
diff --git a/doc/source/plugin-registry.rst b/doc/source/plugin-registry.rst
index 96a2733..92e5ecd 100644
--- a/doc/source/plugin-registry.rst
+++ b/doc/source/plugin-registry.rst
@@ -48,10 +48,12 @@
devstack-plugin-glusterfs `git://git.openstack.org/openstack/devstack-plugin-glusterfs <https://git.openstack.org/cgit/openstack/devstack-plugin-glusterfs>`__
devstack-plugin-hdfs `git://git.openstack.org/openstack/devstack-plugin-hdfs <https://git.openstack.org/cgit/openstack/devstack-plugin-hdfs>`__
devstack-plugin-kafka `git://git.openstack.org/openstack/devstack-plugin-kafka <https://git.openstack.org/cgit/openstack/devstack-plugin-kafka>`__
+devstack-plugin-libvirt-qemu `git://git.openstack.org/openstack/devstack-plugin-libvirt-qemu <https://git.openstack.org/cgit/openstack/devstack-plugin-libvirt-qemu>`__
devstack-plugin-mariadb `git://git.openstack.org/openstack/devstack-plugin-mariadb <https://git.openstack.org/cgit/openstack/devstack-plugin-mariadb>`__
devstack-plugin-nfs `git://git.openstack.org/openstack/devstack-plugin-nfs <https://git.openstack.org/cgit/openstack/devstack-plugin-nfs>`__
devstack-plugin-pika `git://git.openstack.org/openstack/devstack-plugin-pika <https://git.openstack.org/cgit/openstack/devstack-plugin-pika>`__
devstack-plugin-sheepdog `git://git.openstack.org/openstack/devstack-plugin-sheepdog <https://git.openstack.org/cgit/openstack/devstack-plugin-sheepdog>`__
+devstack-plugin-vmax `git://git.openstack.org/openstack/devstack-plugin-vmax <https://git.openstack.org/cgit/openstack/devstack-plugin-vmax>`__
devstack-plugin-zmq `git://git.openstack.org/openstack/devstack-plugin-zmq <https://git.openstack.org/cgit/openstack/devstack-plugin-zmq>`__
dragonflow `git://git.openstack.org/openstack/dragonflow <https://git.openstack.org/cgit/openstack/dragonflow>`__
drbd-devstack `git://git.openstack.org/openstack/drbd-devstack <https://git.openstack.org/cgit/openstack/drbd-devstack>`__
@@ -62,7 +64,6 @@
fuxi `git://git.openstack.org/openstack/fuxi <https://git.openstack.org/cgit/openstack/fuxi>`__
gce-api `git://git.openstack.org/openstack/gce-api <https://git.openstack.org/cgit/openstack/gce-api>`__
glare `git://git.openstack.org/openstack/glare <https://git.openstack.org/cgit/openstack/glare>`__
-gnocchi `git://git.openstack.org/openstack/gnocchi <https://git.openstack.org/cgit/openstack/gnocchi>`__
group-based-policy `git://git.openstack.org/openstack/group-based-policy <https://git.openstack.org/cgit/openstack/group-based-policy>`__
heat `git://git.openstack.org/openstack/heat <https://git.openstack.org/cgit/openstack/heat>`__
horizon-mellanox `git://git.openstack.org/openstack/horizon-mellanox <https://git.openstack.org/cgit/openstack/horizon-mellanox>`__
@@ -123,9 +124,11 @@
neutron `git://git.openstack.org/openstack/neutron <https://git.openstack.org/cgit/openstack/neutron>`__
neutron-dynamic-routing `git://git.openstack.org/openstack/neutron-dynamic-routing <https://git.openstack.org/cgit/openstack/neutron-dynamic-routing>`__
neutron-fwaas `git://git.openstack.org/openstack/neutron-fwaas <https://git.openstack.org/cgit/openstack/neutron-fwaas>`__
+neutron-fwaas-dashboard `git://git.openstack.org/openstack/neutron-fwaas-dashboard <https://git.openstack.org/cgit/openstack/neutron-fwaas-dashboard>`__
neutron-lbaas `git://git.openstack.org/openstack/neutron-lbaas <https://git.openstack.org/cgit/openstack/neutron-lbaas>`__
neutron-lbaas-dashboard `git://git.openstack.org/openstack/neutron-lbaas-dashboard <https://git.openstack.org/cgit/openstack/neutron-lbaas-dashboard>`__
neutron-vpnaas `git://git.openstack.org/openstack/neutron-vpnaas <https://git.openstack.org/cgit/openstack/neutron-vpnaas>`__
+neutron-vpnaas-dashboard `git://git.openstack.org/openstack/neutron-vpnaas-dashboard <https://git.openstack.org/cgit/openstack/neutron-vpnaas-dashboard>`__
nova-dpm `git://git.openstack.org/openstack/nova-dpm <https://git.openstack.org/cgit/openstack/nova-dpm>`__
nova-lxd `git://git.openstack.org/openstack/nova-lxd <https://git.openstack.org/cgit/openstack/nova-lxd>`__
nova-mksproxy `git://git.openstack.org/openstack/nova-mksproxy <https://git.openstack.org/cgit/openstack/nova-mksproxy>`__
@@ -136,6 +139,7 @@
os-xenapi `git://git.openstack.org/openstack/os-xenapi <https://git.openstack.org/cgit/openstack/os-xenapi>`__
osprofiler `git://git.openstack.org/openstack/osprofiler <https://git.openstack.org/cgit/openstack/osprofiler>`__
panko `git://git.openstack.org/openstack/panko <https://git.openstack.org/cgit/openstack/panko>`__
+patrole `git://git.openstack.org/openstack/patrole <https://git.openstack.org/cgit/openstack/patrole>`__
picasso `git://git.openstack.org/openstack/picasso <https://git.openstack.org/cgit/openstack/picasso>`__
rally `git://git.openstack.org/openstack/rally <https://git.openstack.org/cgit/openstack/rally>`__
sahara `git://git.openstack.org/openstack/sahara <https://git.openstack.org/cgit/openstack/sahara>`__
@@ -145,12 +149,15 @@
searchlight-ui `git://git.openstack.org/openstack/searchlight-ui <https://git.openstack.org/cgit/openstack/searchlight-ui>`__
senlin `git://git.openstack.org/openstack/senlin <https://git.openstack.org/cgit/openstack/senlin>`__
solum `git://git.openstack.org/openstack/solum <https://git.openstack.org/cgit/openstack/solum>`__
+stackube `git://git.openstack.org/openstack/stackube <https://git.openstack.org/cgit/openstack/stackube>`__
tacker `git://git.openstack.org/openstack/tacker <https://git.openstack.org/cgit/openstack/tacker>`__
tap-as-a-service `git://git.openstack.org/openstack/tap-as-a-service <https://git.openstack.org/cgit/openstack/tap-as-a-service>`__
+tap-as-a-service-dashboard `git://git.openstack.org/openstack/tap-as-a-service-dashboard <https://git.openstack.org/cgit/openstack/tap-as-a-service-dashboard>`__
tricircle `git://git.openstack.org/openstack/tricircle <https://git.openstack.org/cgit/openstack/tricircle>`__
trio2o `git://git.openstack.org/openstack/trio2o <https://git.openstack.org/cgit/openstack/trio2o>`__
trove `git://git.openstack.org/openstack/trove <https://git.openstack.org/cgit/openstack/trove>`__
trove-dashboard `git://git.openstack.org/openstack/trove-dashboard <https://git.openstack.org/cgit/openstack/trove-dashboard>`__
+valet `git://git.openstack.org/openstack/valet <https://git.openstack.org/cgit/openstack/valet>`__
vitrage `git://git.openstack.org/openstack/vitrage <https://git.openstack.org/cgit/openstack/vitrage>`__
vitrage-dashboard `git://git.openstack.org/openstack/vitrage-dashboard <https://git.openstack.org/cgit/openstack/vitrage-dashboard>`__
vmware-nsx `git://git.openstack.org/openstack/vmware-nsx <https://git.openstack.org/cgit/openstack/vmware-nsx>`__
diff --git a/doc/source/site-map.rst b/doc/source/site-map.rst
deleted file mode 100644
index 022cc73..0000000
--- a/doc/source/site-map.rst
+++ /dev/null
@@ -1,24 +0,0 @@
-:orphan:
-
-.. the TOC on the front page actually makes the document a lot more
- confusing. This lets us bury a toc which we can link in when
- appropriate.
-
-==========
- Site Map
-==========
-
-.. toctree::
- :glob:
- :maxdepth: 3
-
- overview
- configuration
- networking
- plugins
- plugin-registry
- faq
- development
- hacking
- guides
- systemd
diff --git a/doc/source/systemd.rst b/doc/source/systemd.rst
index efe79e4..60a7719 100644
--- a/doc/source/systemd.rst
+++ b/doc/source/systemd.rst
@@ -109,6 +109,17 @@
journalctl -f -o short-precise --unit devstack@n-cpu.service
+By default, journalctl strips out "unprintable" characters, including
+ASCII color codes. To keep the color codes (which can be interpreted by
+an appropriate terminal/pager - e.g. ``less``, the default)::
+
+ journalctl -a --unit devstack@n-cpu.service
+
+When outputting to the terminal using the default pager, long lines
+appear to be truncated, but horizontal scrolling is supported via the
+left/right arrow keys.
+
+See ``man 1 journalctl`` for more.
Known Issues
============
@@ -137,13 +148,6 @@
Future Work
===========
-log colorizing
---------------
-
-We lose log colorization through this process. We might want to build
-a custom colorizer that we could run journalctl output through
-optionally for people.
-
user units
----------
diff --git a/files/debs/n-api b/files/debs/n-api
deleted file mode 100644
index 0928cd5..0000000
--- a/files/debs/n-api
+++ /dev/null
@@ -1 +0,0 @@
-fping
diff --git a/files/debs/zookeeper b/files/debs/zookeeper
deleted file mode 100644
index f41b559..0000000
--- a/files/debs/zookeeper
+++ /dev/null
@@ -1 +0,0 @@
-zookeeperd
diff --git a/files/rpms-suse/general b/files/rpms-suse/general
index 1044c25..370f240 100644
--- a/files/rpms-suse/general
+++ b/files/rpms-suse/general
@@ -1,3 +1,5 @@
+apache2
+apache2-devel
bc
bridge-utils
ca-certificates-mozilla
@@ -23,9 +25,11 @@
python-devel # pyOpenSSL
python-xml
screen
+systemd-devel # for systemd-python
tar
tcpdump
unzip
util-linux
wget
+which
zlib-devel
diff --git a/files/rpms-suse/n-api b/files/rpms-suse/n-api
index af5ac2f..0f08daa 100644
--- a/files/rpms-suse/n-api
+++ b/files/rpms-suse/n-api
@@ -1,2 +1 @@
-fping
python-dateutil
diff --git a/files/rpms/n-api b/files/rpms/n-api
deleted file mode 100644
index 0928cd5..0000000
--- a/files/rpms/n-api
+++ /dev/null
@@ -1 +0,0 @@
-fping
diff --git a/files/rpms/zookeeper b/files/rpms/zookeeper
deleted file mode 100644
index 1bfac53..0000000
--- a/files/rpms/zookeeper
+++ /dev/null
@@ -1 +0,0 @@
-zookeeper
diff --git a/files/zookeeper/environment b/files/zookeeper/environment
deleted file mode 100644
index afa2d2f..0000000
--- a/files/zookeeper/environment
+++ /dev/null
@@ -1,36 +0,0 @@
-#
-# (C) Copyright 2015 Hewlett Packard Enterprise Development Company LP
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-# implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Modified from http://packages.ubuntu.com/saucy/zookeeperd
-NAME=zookeeper
-ZOOCFGDIR=/etc/zookeeper/conf
-
-# seems, that log4j requires the log4j.properties file to be in the classpath
-CLASSPATH="$ZOOCFGDIR:/usr/share/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/zookeeper.jar"
-
-ZOOCFG="$ZOOCFGDIR/zoo.cfg"
-ZOO_LOG_DIR=/var/log/zookeeper
-USER=$NAME
-GROUP=$NAME
-PIDDIR=/var/run/$NAME
-PIDFILE=$PIDDIR/$NAME.pid
-SCRIPTNAME=/etc/init.d/$NAME
-JAVA=/usr/bin/java
-ZOOMAIN="org.apache.zookeeper.server.quorum.QuorumPeerMain"
-ZOO_LOG4J_PROP="INFO,ROLLINGFILE"
-JMXLOCALONLY=false
-JAVA_OPTS=""
diff --git a/files/zookeeper/log4j.properties b/files/zookeeper/log4j.properties
deleted file mode 100644
index 6c45a4a..0000000
--- a/files/zookeeper/log4j.properties
+++ /dev/null
@@ -1,69 +0,0 @@
-#
-# (C) Copyright 2015 Hewlett Packard Enterprise Development Company LP
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-# implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# From http://packages.ubuntu.com/saucy/zookeeperd
-
-# ZooKeeper Logging Configuration
-#
-
-# Format is "<default threshold> (, <appender>)+
-
-log4j.rootLogger=${zookeeper.root.logger}
-
-# Example: console appender only
-# log4j.rootLogger=INFO, CONSOLE
-
-# Example with rolling log file
-#log4j.rootLogger=DEBUG, CONSOLE, ROLLINGFILE
-
-# Example with rolling log file and tracing
-#log4j.rootLogger=TRACE, CONSOLE, ROLLINGFILE, TRACEFILE
-
-#
-# Log INFO level and above messages to the console
-#
-log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
-log4j.appender.CONSOLE.Threshold=INFO
-log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
-log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n
-
-#
-# Add ROLLINGFILE to rootLogger to get log file output
-# Log DEBUG level and above messages to a log file
-log4j.appender.ROLLINGFILE=org.apache.log4j.RollingFileAppender
-log4j.appender.ROLLINGFILE.Threshold=WARN
-log4j.appender.ROLLINGFILE.File=${zookeeper.log.dir}/zookeeper.log
-
-# Max log file size of 10MB
-log4j.appender.ROLLINGFILE.MaxFileSize=10MB
-# uncomment the next line to limit number of backup files
-#log4j.appender.ROLLINGFILE.MaxBackupIndex=10
-
-log4j.appender.ROLLINGFILE.layout=org.apache.log4j.PatternLayout
-log4j.appender.ROLLINGFILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n
-
-
-#
-# Add TRACEFILE to rootLogger to get log file output
-# Log DEBUG level and above messages to a log file
-log4j.appender.TRACEFILE=org.apache.log4j.FileAppender
-log4j.appender.TRACEFILE.Threshold=TRACE
-log4j.appender.TRACEFILE.File=${zookeeper.log.dir}/zookeeper_trace.log
-
-log4j.appender.TRACEFILE.layout=org.apache.log4j.PatternLayout
-### Notice we are including log4j's NDC here (%x)
-log4j.appender.TRACEFILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L][%x] - %m%n
diff --git a/files/zookeeper/myid b/files/zookeeper/myid
deleted file mode 100644
index c227083..0000000
--- a/files/zookeeper/myid
+++ /dev/null
@@ -1 +0,0 @@
-0
\ No newline at end of file
diff --git a/files/zookeeper/zoo.cfg b/files/zookeeper/zoo.cfg
deleted file mode 100644
index b8f5582..0000000
--- a/files/zookeeper/zoo.cfg
+++ /dev/null
@@ -1,74 +0,0 @@
-#
-# (C) Copyright 2015 Hewlett Packard Enterprise Development Company LP
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-# implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html
-
-# The number of milliseconds of each tick
-tickTime=2000
-# The number of ticks that the initial
-# synchronization phase can take
-initLimit=10
-# The number of ticks that can pass between
-# sending a request and getting an acknowledgement
-syncLimit=5
-# the directory where the snapshot is stored.
-dataDir=/var/lib/zookeeper
-# Place the dataLogDir to a separate physical disc for better performance
-# dataLogDir=/disk2/zookeeper
-
-# the port at which the clients will connect
-clientPort=2181
-
-# Maximum number of clients that can connect from one client
-maxClientCnxns=60
-
-# specify all zookeeper servers
-# The fist port is used by followers to connect to the leader
-# The second one is used for leader election
-
-server.0=127.0.0.1:2888:3888
-
-# To avoid seeks ZooKeeper allocates space in the transaction log file in
-# blocks of preAllocSize kilobytes. The default block size is 64M. One reason
-# for changing the size of the blocks is to reduce the block size if snapshots
-# are taken more often. (Also, see snapCount).
-#preAllocSize=65536
-
-# Clients can submit requests faster than ZooKeeper can process them,
-# especially if there are a lot of clients. To prevent ZooKeeper from running
-# out of memory due to queued requests, ZooKeeper will throttle clients so that
-# there is no more than globalOutstandingLimit outstanding requests in the
-# system. The default limit is 1,000.ZooKeeper logs transactions to a
-# transaction log. After snapCount transactions are written to a log file a
-# snapshot is started and a new transaction log file is started. The default
-# snapCount is 10,000.
-#snapCount=1000
-
-# If this option is defined, requests will be will logged to a trace file named
-# traceFile.year.month.day.
-#traceFile=
-
-# Leader accepts client connections. Default value is "yes". The leader machine
-# coordinates updates. For higher update throughput at thes slight expense of
-# read throughput the leader can be configured to not accept clients and focus
-# on coordination.
-#leaderServes=yes
-
-# Autopurge every hour to avoid using lots of disk in bursts
-# Order of the next 2 properties matters.
-# autopurge.snapRetainCount must be before autopurge.purgeInterval.
-autopurge.snapRetainCount=3
-autopurge.purgeInterval=1
\ No newline at end of file
diff --git a/functions b/functions
index 8968a18..e056c3f 100644
--- a/functions
+++ b/functions
@@ -310,6 +310,11 @@
disk_format=qcow2
container_format=bare
;;
+ *.raw)
+ image_name=$(basename "$image" ".raw")
+ disk_format=raw
+ container_format=bare
+ ;;
*.iso)
image_name=$(basename "$image" ".iso")
disk_format=iso
@@ -318,7 +323,7 @@
*.vhd|*.vhdx|*.vhd.gz|*.vhdx.gz)
local extension="${image_fname#*.}"
image_name=$(basename "$image" ".$extension")
- disk_format=vhd
+ disk_format=$(echo $image_fname | grep -oP '(?<=\.)vhdx?(?=\.|$)')
container_format=bare
if [ "${image_fname##*.}" == "gz" ]; then
unpack=zcat
@@ -610,22 +615,19 @@
# native systemd path, which provides for things like search on
# request-id. However, there may be an eventlet interaction here,
# so going off for now.
- USE_JOURNAL=$(trueorfalse USE_JOURNAL False)
+ USE_JOURNAL=$(trueorfalse False USE_JOURNAL)
+ local pidstr=""
if [[ "$USE_JOURNAL" == "True" ]]; then
iniset $conf_file $conf_section use_journal "True"
# if we are using the journal directly, our process id is already correct
- iniset $conf_file $conf_section logging_debug_format_suffix \
- "{{%(funcName)s %(pathname)s:%(lineno)d}}"
else
- iniset $conf_file $conf_section logging_debug_format_suffix \
- "{{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}}"
+ pidstr="(pid=%(process)d) "
fi
+ iniset $conf_file $conf_section logging_debug_format_suffix "[00;33m{{${pidstr}%(funcName)s %(pathname)s:%(lineno)d}}[00m"
- iniset $conf_file $conf_section logging_context_format_string \
- "%(levelname)s %(name)s [%(request_id)s %(project_name)s %(user_name)s] %(instance)s%(message)s"
- iniset $conf_file $conf_section logging_default_format_string \
- "%(levelname)s %(name)s [-] %(instance)s%(color)s%(message)s"
- iniset $conf_file $conf_section logging_exception_prefix "ERROR %(name)s %(instance)s"
+ iniset $conf_file $conf_section logging_context_format_string "%(color)s%(levelname)s %(name)s [[01;36m%(global_request_id)s %(request_id)s [00;36m%(project_name)s %(user_name)s%(color)s] [01;35m%(instance)s%(color)s%(message)s[00m"
+ iniset $conf_file $conf_section logging_default_format_string "%(color)s%(levelname)s %(name)s [[00;36m-%(color)s] [01;35m%(instance)s%(color)s%(message)s[00m"
+ iniset $conf_file $conf_section logging_exception_prefix "ERROR %(name)s [01;35m%(instance)s[00m"
}
function setup_standard_logging_identity {
@@ -715,6 +717,40 @@
}
+# Set a systemd system override
+#
+# This sets a system-side override in system.conf. A per-service
+# override would be /etc/systemd/system/${service}.service/override.conf
+function set_systemd_override {
+ local key="$1"
+ local value="$2"
+
+ local sysconf="/etc/systemd/system.conf"
+ iniset -sudo "${sysconf}" "Manager" "$key" "$value"
+ echo "Set systemd system override for ${key}=${value}"
+
+ sudo systemctl daemon-reload
+}
+
+# Get a random port from the local port range
+#
+# This function returns an available port in the local port range. The search
+# order is not truly random, but should be considered a random value by the
+# user because it depends on the state of your local system.
+function get_random_port {
+ read lower_port upper_port < /proc/sys/net/ipv4/ip_local_port_range
+ while true; do
+ for (( port = upper_port ; port >= lower_port ; port-- )); do
+ sudo lsof -i ":$port" &> /dev/null
+ if [[ $? > 0 ]] ; then
+ break 2
+ fi
+ done
+ done
+ echo $port
+}
+
+
# Restore xtrace
$_XTRACE_FUNCTIONS
diff --git a/functions-common b/functions-common
index 90f5400..660df79 100644
--- a/functions-common
+++ b/functions-common
@@ -48,8 +48,8 @@
# Save these variables to .stackenv
STACK_ENV_VARS="BASE_SQL_CONN DATA_DIR DEST ENABLED_SERVICES HOST_IP \
- KEYSTONE_AUTH_PROTOCOL KEYSTONE_AUTH_URI KEYSTONE_SERVICE_URI \
- LOGFILE OS_CACERT SERVICE_HOST SERVICE_PROTOCOL STACK_USER TLS_IP \
+ KEYSTONE_AUTH_URI KEYSTONE_SERVICE_URI \
+ LOGFILE OS_CACERT SERVICE_HOST STACK_USER TLS_IP \
HOST_IPV6 SERVICE_IP_VERSION"
@@ -93,7 +93,7 @@
--os-region-name $REGION_NAME \
--os-identity-api-version 3 \
$CA_CERT_ARG \
- --os-auth-url $KEYSTONE_AUTH_URI \
+ --os-auth-url $KEYSTONE_SERVICE_URI \
--os-username demo \
--os-password $ADMIN_PASSWORD \
--os-project-name demo
@@ -105,7 +105,7 @@
--os-region-name $REGION_NAME \
--os-identity-api-version 3 \
$CA_CERT_ARG \
- --os-auth-url $KEYSTONE_AUTH_URI \
+ --os-auth-url $KEYSTONE_SERVICE_URI \
--os-username alt_demo \
--os-password $ADMIN_PASSWORD \
--os-project-name alt_demo
@@ -117,7 +117,7 @@
--os-region-name $REGION_NAME \
--os-identity-api-version 3 \
$CA_CERT_ARG \
- --os-auth-url $KEYSTONE_AUTH_URI \
+ --os-auth-url $KEYSTONE_SERVICE_URI \
--os-username admin \
--os-password $ADMIN_PASSWORD \
--os-project-name admin
@@ -864,10 +864,11 @@
# Gets user role id
user_role_id=$(openstack role assignment list \
+ --role $1 \
--user $2 \
--project $3 \
$domain_args \
- | grep " $1 " | get_field 1)
+ | grep '^|\s[a-f0-9]\+' | get_field 1)
if [[ -z "$user_role_id" ]]; then
# Adds role to user and get it
openstack role add $1 \
@@ -875,10 +876,11 @@
--project $3 \
$domain_args
user_role_id=$(openstack role assignment list \
+ --role $1 \
--user $2 \
--project $3 \
$domain_args \
- | grep " $1 " | get_field 1)
+ | grep '^|\s[a-f0-9]\+' | get_field 1)
fi
echo $user_role_id
}
@@ -889,18 +891,20 @@
local user_role_id
# Gets user role id
user_role_id=$(openstack role assignment list \
+ --role $1 \
--user $2 \
--domain $3 \
- | grep " $1 " | get_field 1)
+ | grep '^|\s[a-f0-9]\+' | get_field 1)
if [[ -z "$user_role_id" ]]; then
# Adds role to user and get it
openstack role add $1 \
--user $2 \
--domain $3
user_role_id=$(openstack role assignment list \
+ --role $1 \
--user $2 \
--domain $3 \
- | grep " $1 " | get_field 1)
+ | grep '^|\s[a-f0-9]\+' | get_field 1)
fi
echo $user_role_id
}
@@ -911,6 +915,7 @@
local group_role_id
# Gets group role id
group_role_id=$(openstack role assignment list \
+ --role $1 \
--group $2 \
--project $3 \
-f value)
@@ -920,6 +925,7 @@
--group $2 \
--project $3
group_role_id=$(openstack role assignment list \
+ --role $1 \
--group $2 \
--project $3 \
-f value)
@@ -1463,11 +1469,12 @@
mkdir -p $SYSTEMD_DIR
iniset -sudo $unitfile "Unit" "Description" "Devstack $service"
+ iniset -sudo $unitfile "Service" "SyslogIdentifier" "$service"
iniset -sudo $unitfile "Service" "User" "$user"
iniset -sudo $unitfile "Service" "ExecStart" "$command"
iniset -sudo $unitfile "Service" "Type" "notify"
iniset -sudo $unitfile "Service" "KillSignal" "SIGQUIT"
- iniset -sudo $unitfile "Service" "Restart" "Always"
+ iniset -sudo $unitfile "Service" "Restart" "always"
iniset -sudo $unitfile "Service" "NotifyAccess" "all"
iniset -sudo $unitfile "Service" "RestartForceExitStatus" "100"
@@ -1480,10 +1487,41 @@
$SYSTEMCTL daemon-reload
}
+function _common_systemd_pitfalls {
+ local cmd=$1
+ # do some sanity checks on $cmd to see things we don't expect to work
+
+ if [[ "$cmd" =~ "sudo" ]]; then
+ local msg=<<EOF
+You are trying to use run_process with sudo, this is not going to work under systemd.
+
+If you need to run a service as a user other than $STACK_USER call it with:
+
+ run_process \$name \$cmd \$group \$user
+EOF
+ die $LINENO $msg
+ fi
+
+ if [[ ! "$cmd" =~ ^/ ]]; then
+ local msg=<<EOF
+The cmd="$cmd" does not start with an absolute path. It will fail to
+start under systemd.
+
+Please update your run_process stanza to have an absolute path.
+EOF
+ die $LINENO $msg
+ fi
+
+}
+
+# Helper function to build a basic unit file and run it under systemd.
function _run_under_systemd {
local service=$1
local command="$2"
local cmd=$command
+ # sanity check the command
+ _common_systemd_pitfalls "$cmd"
+
local systemd_service="devstack@$service.service"
local group=$3
local user=${4:-$STACK_USER}
@@ -1527,7 +1565,7 @@
# If an optional group is provided sg will be used to run the
# command as that group.
# Uses globals ``USE_SCREEN``
-# run_process service "command-line" [group]
+# run_process service "command-line" [group] [user]
function run_process {
local service=$1
local command="$2"
@@ -2459,7 +2497,7 @@
if [[ -n "$start_time" ]]; then
die $LINENO "Trying to start the clock on $name, but it's already been started"
fi
- _TIME_START[$name]=$(date +%s)
+ _TIME_START[$name]=$(date +%s%3N)
}
# time_stop $name
@@ -2480,7 +2518,7 @@
if [[ -z "$start_time" ]]; then
die $LINENO "Trying to stop the clock on $name, but it was never started"
fi
- end_time=$(date +%s)
+ end_time=$(date +%s%3N)
elapsed_time=$(($end_time - $start_time))
total=${_TIME_TOTAL[$name]:-0}
# reset the clock so we can start it in the future
@@ -2488,6 +2526,49 @@
_TIME_TOTAL[$name]=$(($total + $elapsed_time))
}
+function oscwrap {
+ local out
+ local rc
+ local start
+ local end
+ # Cannot use timer_start and timer_stop as we run in subshells
+ # and those rely on modifying vars in the same process (which cannot
+ # happen from a subshell.
+ start=$(date +%s%3N)
+ out=$(command openstack "$@")
+ rc=$?
+ end=$(date +%s%3N)
+ echo $((end - start)) >> $OSCWRAP_TIMER_FILE
+
+ echo "$out"
+ return $rc
+}
+
+function install_oscwrap {
+ # File to accumulate our timing data
+ OSCWRAP_TIMER_FILE=$(mktemp)
+ # Bash by default doesn't expand aliases, allow it for the aliases
+ # we want to whitelist.
+ shopt -s expand_aliases
+ # Remove all aliases that might be expanded to preserve old unexpanded
+ # behavior
+ unalias -a
+ # Add only the alias we want for openstack
+ alias openstack=oscwrap
+}
+
+function cleanup_oscwrap {
+ local total=0
+ if python3_enabled ; then
+ local python=python3
+ else
+ local python=python
+ fi
+ total=$(cat $OSCWRAP_TIMER_FILE | $python -c "import sys; print(sum(int(l) for l in sys.stdin))")
+ _TIME_TOTAL["osc"]=$total
+ rm $OSCWRAP_TIMER_FILE
+}
+
# time_totals
# Print out total time summary
function time_totals {
@@ -2506,6 +2587,8 @@
fi
done
+ cleanup_oscwrap
+
xtrace=$(set +o | grep xtrace)
set +o xtrace
@@ -2517,6 +2600,8 @@
echo
for t in ${!_TIME_TOTAL[*]}; do
local v=${_TIME_TOTAL[$t]}
+ # because we're recording in milliseconds
+ v=$(($v / 1000))
printf "%-${len}s %3d\n" "$t" "$v"
done
echo "========================="
diff --git a/inc/python b/inc/python
index 2443c4d..718cbb2 100644
--- a/inc/python
+++ b/inc/python
@@ -320,6 +320,14 @@
fi
$xtrace
+
+ # Also install test requirements
+ local install_test_reqs=""
+ local test_req="${!#}/test-requirements.txt"
+ if [[ -e "$test_req" ]]; then
+ install_test_reqs="-r $test_req"
+ fi
+
# adding SETUPTOOLS_SYS_PATH_TECHNIQUE is a workaround to keep
# the same behaviour of setuptools before version 25.0.0.
# related issue: https://github.com/pypa/pip/issues/3874
@@ -329,28 +337,28 @@
no_proxy="${no_proxy:-}" \
PIP_FIND_LINKS=$PIP_FIND_LINKS \
SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite \
- $cmd_pip $upgrade \
+ $cmd_pip $upgrade $install_test_reqs \
$@
result=$?
- # Also install test requirements
- local test_req="${!#}/test-requirements.txt"
- if [[ $result == 0 ]] && [[ -e "$test_req" ]]; then
- echo "Installing test-requirements for $test_req"
- $sudo_pip \
- http_proxy=${http_proxy:-} \
- https_proxy=${https_proxy:-} \
- no_proxy=${no_proxy:-} \
- PIP_FIND_LINKS=$PIP_FIND_LINKS \
- $cmd_pip $upgrade \
- -r $test_req
- result=$?
- fi
-
time_stop "pip_install"
return $result
}
+function pip_uninstall {
+ local name=$1
+ if [[ -n ${PIP_VIRTUAL_ENV:=} && -d ${PIP_VIRTUAL_ENV} ]]; then
+ local cmd_pip=$PIP_VIRTUAL_ENV/bin/pip
+ local sudo_pip="env"
+ else
+ local cmd_pip
+ cmd_pip=$(get_pip_command $PYTHON2_VERSION)
+ local sudo_pip="sudo -H"
+ fi
+ # don't error if we can't uninstall, it might not be there
+ $sudo_pip $cmd_pip uninstall -y $name || /bin/true
+}
+
# get version of a package from global requirements file
# get_from_global_requirements <package>
function get_from_global_requirements {
diff --git a/lib/apache b/lib/apache
index 34ac660..25c65fe 100644
--- a/lib/apache
+++ b/lib/apache
@@ -53,8 +53,15 @@
function enable_apache_mod {
local mod=$1
# Apache installation, because we mark it NOPRIME
- if is_ubuntu || is_suse ; then
- if ! a2query -m $mod ; then
+ if is_ubuntu; then
+ # Skip mod_version as it is not a valid mod to enable
+ # on debuntu, instead it is built in.
+ if [[ "$mod" != "version" ]] && ! a2query -m $mod ; then
+ sudo a2enmod $mod
+ restart_apache_server
+ fi
+ elif is_suse; then
+ if ! a2enmod -q $mod ; then
sudo a2enmod $mod
restart_apache_server
fi
@@ -96,7 +103,7 @@
# delete the temp directory
sudo rm -rf $dir
- if is_ubuntu; then
+ if is_ubuntu || is_suse ; then
# we've got to enable proxy and proxy_uwsgi for this to work
sudo a2enmod proxy
sudo a2enmod proxy_uwsgi
@@ -171,6 +178,8 @@
# enable_apache_site() - Enable a particular apache site
function enable_apache_site {
local site=$@
+ # Many of our sites use mod version. Just enable it.
+ enable_apache_mod version
if is_ubuntu; then
sudo a2ensite ${site}
elif is_fedora || is_suse; then
@@ -229,7 +238,13 @@
# create a home for the sockets; note don't use /tmp -- apache has
# a private view of it on some platforms.
local socket_dir='/var/run/uwsgi'
- sudo install -d -o $STACK_USER -m 755 $socket_dir
+
+ # /var/run will be empty on ubuntu after reboot, so we can use systemd-temptiles
+ # to automatically create $socket_dir.
+ sudo mkdir -p /etc/tmpfiles.d/
+ echo "d $socket_dir 0755 $STACK_USER root" | sudo tee /etc/tmpfiles.d/uwsgi.conf
+ sudo systemd-tmpfiles --create /etc/tmpfiles.d/uwsgi.conf
+
local socket="$socket_dir/${name}.socket"
# always cleanup given that we are using iniset here
@@ -260,12 +275,64 @@
else
local apache_conf=""
apache_conf=$(apache_site_config_for $name)
- echo "ProxyPass \"${url}\" \"unix:${socket}|uwsgi://uwsgi-uds-${name}/\" retry=0 " | sudo tee $apache_conf
+ echo "SetEnv proxy-sendcl 1" | sudo tee $apache_conf
+ echo "ProxyPass \"${url}\" \"unix:${socket}|uwsgi://uwsgi-uds-${name}/\" retry=0 " | sudo tee -a $apache_conf
enable_apache_site $name
restart_apache_server
fi
}
+# For services using chunked encoding, the only services known to use this
+# currently are Glance and Swift, we need to use an http proxy instead of
+# mod_proxy_uwsgi because the chunked encoding gets dropped. See:
+# https://github.com/unbit/uwsgi/issues/1540 You can workaround this on python2
+# but that involves having apache buffer the request before sending it to
+# uswgi.
+function write_local_uwsgi_http_config {
+ local file=$1
+ local wsgi=$2
+ local url=$3
+ name=$(basename $wsgi)
+
+ # create a home for the sockets; note don't use /tmp -- apache has
+ # a private view of it on some platforms.
+
+ # always cleanup given that we are using iniset here
+ rm -rf $file
+ iniset "$file" uwsgi wsgi-file "$wsgi"
+ port=$(get_random_port)
+ iniset "$file" uwsgi http "127.0.0.1:$port"
+ iniset "$file" uwsgi processes $API_WORKERS
+ # This is running standalone
+ iniset "$file" uwsgi master true
+ # Set die-on-term & exit-on-reload so that uwsgi shuts down
+ iniset "$file" uwsgi die-on-term true
+ iniset "$file" uwsgi exit-on-reload true
+ iniset "$file" uwsgi enable-threads true
+ iniset "$file" uwsgi plugins python
+ # uwsgi recommends this to prevent thundering herd on accept.
+ iniset "$file" uwsgi thunder-lock true
+ # Override the default size for headers from the 4k default.
+ iniset "$file" uwsgi buffer-size 65535
+ # Make sure the client doesn't try to re-use the connection.
+ iniset "$file" uwsgi add-header "Connection: close"
+ # This ensures that file descriptors aren't shared between processes.
+ iniset "$file" uwsgi lazy-apps true
+ iniset "$file" uwsgi chmod-socket 666
+ iniset "$file" uwsgi http-raw-body true
+ iniset "$file" uwsgi http-chunked-input true
+ iniset "$file" uwsgi http-auto-chunked true
+
+ enable_apache_mod proxy
+ enable_apache_mod proxy_http
+ local apache_conf=""
+ apache_conf=$(apache_site_config_for $name)
+ echo "KeepAlive Off" | sudo tee $apache_conf
+ echo "ProxyPass \"${url}\" \"http://127.0.0.1:$port\" retry=0 " | sudo tee -a $apache_conf
+ enable_apache_site $name
+ restart_apache_server
+}
+
function remove_uwsgi_config {
local file=$1
local wsgi=$2
diff --git a/lib/cinder b/lib/cinder
index b3de191..998784b 100644
--- a/lib/cinder
+++ b/lib/cinder
@@ -84,20 +84,6 @@
# CINDER_ENABLED_BACKENDS=${CINDER_ENABLED_BACKENDS:-lvm:lvmdriver-1,lvm:lvmdriver-2}
CINDER_ENABLED_BACKENDS=${CINDER_ENABLED_BACKENDS:-lvm:lvmdriver-1}
-
-# Should cinder perform secure deletion of volumes?
-# Defaults to zero. Can also be set to none or shred.
-# This was previously CINDER_SECURE_DELETE (True or False).
-# Equivalents using CINDER_VOLUME_CLEAR are zero and none, respectively.
-# Set to none to avoid this bug when testing:
-# https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1023755
-if [[ -n $CINDER_SECURE_DELETE ]]; then
- CINDER_SECURE_DELETE=$(trueorfalse True CINDER_SECURE_DELETE)
- if [[ $CINDER_SECURE_DELETE == "False" ]]; then
- CINDER_VOLUME_CLEAR_DEFAULT="none"
- fi
- deprecated "Configure secure Cinder volume deletion using CINDER_VOLUME_CLEAR instead of CINDER_SECURE_DELETE."
-fi
CINDER_VOLUME_CLEAR=${CINDER_VOLUME_CLEAR:-${CINDER_VOLUME_CLEAR_DEFAULT:-zero}}
CINDER_VOLUME_CLEAR=$(echo ${CINDER_VOLUME_CLEAR} | tr '[:upper:]' '[:lower:]')
@@ -143,6 +129,7 @@
# Test if any Cinder services are enabled
# is_cinder_enabled
function is_cinder_enabled {
+ [[ ,${DISABLED_SERVICES} =~ ,"cinder" ]] && return 1
[[ ,${ENABLED_SERVICES} =~ ,"c-" ]] && return 0
return 1
}
@@ -244,6 +231,10 @@
configure_rootwrap cinder
+ if [[ -f "$CINDER_DIR/etc/cinder/resource_filters.json" ]]; then
+ cp -p "$CINDER_DIR/etc/cinder/resource_filters.json" "$CINDER_CONF_DIR/resource_filters.json"
+ fi
+
cp $CINDER_DIR/etc/cinder/api-paste.ini $CINDER_API_PASTE_INI
inicomment $CINDER_API_PASTE_INI filter:authtoken auth_host
@@ -299,6 +290,9 @@
default_name=$be_name
fi
enabled_backends+=$be_name,
+
+ iniset $CINDER_CONF $be_name volume_clear $CINDER_VOLUME_CLEAR
+
done
iniset $CINDER_CONF DEFAULT enabled_backends ${enabled_backends%,*}
if [[ -n "$default_name" ]]; then
@@ -327,8 +321,6 @@
iniset_rpc_backend cinder $CINDER_CONF
- iniset $CINDER_CONF DEFAULT volume_clear $CINDER_VOLUME_CLEAR
-
# Format logging
setup_logging $CINDER_CONF $CINDER_USE_MOD_WSGI
@@ -342,7 +334,7 @@
iniset $CINDER_CONF DEFAULT osapi_volume_workers "$API_WORKERS"
- iniset $CINDER_CONF DEFAULT glance_api_servers "${GLANCE_SERVICE_PROTOCOL}://${GLANCE_HOSTPORT}"
+ iniset $CINDER_CONF DEFAULT glance_api_servers "$GLANCE_URL"
if is_service_enabled tls-proxy; then
iniset $CINDER_CONF DEFAULT glance_protocol https
iniset $CINDER_CONF DEFAULT glance_ca_certificates_file $SSL_BUNDLE_FILE
@@ -358,11 +350,10 @@
iniset $CINDER_CONF DEFAULT os_privileged_user_tenant "$SERVICE_PROJECT_NAME"
iniset $CINDER_CONF DEFAULT graceful_shutdown_timeout "$SERVICE_GRACEFUL_SHUTDOWN_TIMEOUT"
- # Set the backend url according to the configured dlm backend
- if is_dlm_enabled; then
- if [[ "$(dlm_backend)" == "zookeeper" ]]; then
- iniset $CINDER_CONF coordination backend_url "zookeeper://${SERVICE_HOST}:2181"
- fi
+ if [[ ! -z "$CINDER_COORDINATION_URL" ]]; then
+ iniset $CINDER_CONF coordination backend_url "$CINDER_COORDINATION_URL"
+ elif is_service_enabled etcd3; then
+ iniset $CINDER_CONF coordination backend_url "etcd3+http://${SERVICE_HOST}:2379"
fi
}
@@ -415,8 +406,10 @@
# (Re)create cinder database
recreate_database cinder
+ time_start "dbsync"
# Migrate cinder database
$CINDER_BIN_DIR/cinder-manage --config-file $CINDER_CONF db sync
+ time_stop "dbsync"
fi
if is_service_enabled c-vol && [[ -n "$CINDER_ENABLED_BACKENDS" ]]; then
@@ -507,17 +500,24 @@
fi
fi
- if [ "$CINDER_USE_MOD_WSGI" == "True" ]; then
- enable_apache_site osapi-volume
- restart_apache_server
- tail_log c-api /var/log/$APACHE_NAME/c-api.log
- else
- run_process c-api "$CINDER_BIN_DIR/cinder-api --config-file $CINDER_CONF"
- fi
+ if is_service_enabled c-api ; then
+ if [ "$CINDER_USE_MOD_WSGI" == "True" ]; then
+ enable_apache_site osapi-volume
+ restart_apache_server
+ tail_log c-api /var/log/$APACHE_NAME/c-api.log
+ else
+ run_process c-api "$CINDER_BIN_DIR/cinder-api --config-file $CINDER_CONF"
+ fi
- echo "Waiting for Cinder API to start..."
- if ! wait_for_service $SERVICE_TIMEOUT $service_protocol://$CINDER_SERVICE_HOST:$service_port; then
- die $LINENO "c-api did not start"
+ echo "Waiting for Cinder API to start..."
+ if ! wait_for_service $SERVICE_TIMEOUT $service_protocol://$CINDER_SERVICE_HOST:$service_port; then
+ die $LINENO "c-api did not start"
+ fi
+
+ # Start proxies if enabled
+ if is_service_enabled tls-proxy; then
+ start_tls_proxy cinder '*' $CINDER_SERVICE_PORT $CINDER_SERVICE_HOST $CINDER_SERVICE_PORT_INT
+ fi
fi
run_process c-sch "$CINDER_BIN_DIR/cinder-scheduler --config-file $CINDER_CONF"
@@ -527,11 +527,6 @@
# NOTE(jdg): For cinder, startup order matters. To ensure that repor_capabilities is received
# by the scheduler start the cinder-volume service last (or restart it) after the scheduler
# has started. This is a quick fix for lp bug/1189595
-
- # Start proxies if enabled
- if is_service_enabled c-api && is_service_enabled tls-proxy; then
- start_tls_proxy cinder '*' $CINDER_SERVICE_PORT $CINDER_SERVICE_HOST $CINDER_SERVICE_PORT_INT
- fi
}
# stop_cinder() - Stop running processes
diff --git a/lib/cinder_backends/lvm b/lib/cinder_backends/lvm
index d927f9c..03e1880 100644
--- a/lib/cinder_backends/lvm
+++ b/lib/cinder_backends/lvm
@@ -53,9 +53,6 @@
iniset $CINDER_CONF $be_name iscsi_helper "$CINDER_ISCSI_HELPER"
iniset $CINDER_CONF $be_name lvm_type "$CINDER_LVM_TYPE"
- if [[ "$CINDER_SECURE_DELETE" == "False" ]]; then
- iniset $CINDER_CONF $be_name volume_clear none
- fi
}
# init_cinder_backend_lvm - Initialize volume group
diff --git a/lib/databases/mysql b/lib/databases/mysql
index 7bbcace..a0cf7a4 100644
--- a/lib/databases/mysql
+++ b/lib/databases/mysql
@@ -71,6 +71,10 @@
elif is_fedora; then
mysql=mariadb
my_conf=/etc/my.cnf
+ local cracklib_conf=/etc/my.cnf.d/cracklib_password_check.cnf
+ if [ -f "$cracklib_conf" ]; then
+ inicomment -sudo "$cracklib_conf" "mariadb" "plugin-load-add"
+ fi
else
exit_distro_not_supported "mysql configuration"
fi
diff --git a/lib/dlm b/lib/dlm
deleted file mode 100644
index b5ac0f5..0000000
--- a/lib/dlm
+++ /dev/null
@@ -1,111 +0,0 @@
-#!/bin/bash
-#
-# lib/dlm
-#
-# Functions to control the installation and configuration of software
-# that provides a dlm (and possibly other functions). The default is
-# **zookeeper**, and is going to be the only backend supported in the
-# devstack tree.
-
-# Dependencies:
-#
-# - ``functions`` file
-
-# ``stack.sh`` calls the entry points in this order:
-#
-# - is_dlm_enabled
-# - install_dlm
-# - configure_dlm
-# - cleanup_dlm
-
-# Save trace setting
-_XTRACE_DLM=$(set +o | grep xtrace)
-set +o xtrace
-
-
-# Defaults
-# --------
-
-# <define global variables here that belong to this project>
-
-# Set up default directories
-ZOOKEEPER_DATA_DIR=$DEST/data/zookeeper
-ZOOKEEPER_CONF_DIR=/etc/zookeeper
-
-
-# Entry Points
-# ------------
-#
-# NOTE(sdague): it is expected that when someone wants to implement
-# another one of these out of tree, they'll implement the following
-# functions:
-#
-# - dlm_backend
-# - install_dlm
-# - configure_dlm
-# - cleanup_dlm
-
-# This should be declared in the settings file of any plugin or
-# service that needs to have a dlm in their environment.
-function use_dlm {
- enable_service $(dlm_backend)
-}
-
-# A function to return the name of the backend in question, some users
-# are going to need to know this.
-function dlm_backend {
- echo "zookeeper"
-}
-
-# Test if a dlm is enabled (defaults to a zookeeper specific check)
-function is_dlm_enabled {
- [[ ,${ENABLED_SERVICES}, =~ ,"$(dlm_backend)", ]] && return 0
- return 1
-}
-
-# cleanup_dlm() - Remove residual data files, anything left over from previous
-# runs that a clean run would need to clean up
-function cleanup_dlm {
- # NOTE(sdague): we don't check for is_enabled here because we
- # should just delete this regardless. Some times users updated
- # their service list before they run cleanup.
- sudo rm -rf $ZOOKEEPER_DATA_DIR
-}
-
-# configure_dlm() - Set config files, create data dirs, etc
-function configure_dlm {
- if is_dlm_enabled; then
- sudo cp $FILES/zookeeper/* $ZOOKEEPER_CONF_DIR
- sudo sed -i -e 's|.*dataDir.*|dataDir='$ZOOKEEPER_DATA_DIR'|' $ZOOKEEPER_CONF_DIR/zoo.cfg
- # clean up from previous (possibly aborted) runs
- # create required data files
- sudo rm -rf $ZOOKEEPER_DATA_DIR
- sudo mkdir -p $ZOOKEEPER_DATA_DIR
- # restart after configuration, there is no reason to make this
- # another step, because having data files that don't match the
- # zookeeper running is just going to cause tears.
- restart_service zookeeper
- fi
-}
-
-# install_dlm() - Collect source and prepare
-function install_dlm {
- if is_dlm_enabled; then
- pip_install_gr_extras tooz zookeeper
- if is_ubuntu; then
- install_package zookeeperd
- elif is_fedora; then
- install_package zookeeper
- else
- die $LINENO "Don't know how to install zookeeper on this platform"
- fi
- fi
-}
-
-# Restore xtrace
-$_XTRACE_DLM
-
-# Tell emacs to use shell-script-mode
-## Local variables:
-## mode: shell-script
-## End:
diff --git a/lib/etcd3 b/lib/etcd3
new file mode 100644
index 0000000..0e1fbd5
--- /dev/null
+++ b/lib/etcd3
@@ -0,0 +1,139 @@
+#!/bin/bash
+#
+# lib/etcd3
+#
+# Functions to control the installation and configuration of etcd 3.x
+# that provides a key-value store (and possibly other functions).
+
+# Dependencies:
+#
+# - ``functions`` file
+
+# ``stack.sh`` calls the entry points in this order:
+#
+# - start_etcd3
+# - stop_etcd3
+# - cleanup_etcd3
+
+# Save trace setting
+_XTRACE_ETCD3=$(set +o | grep xtrace)
+set +o xtrace
+
+
+# Defaults
+# --------
+
+# Set up default values for etcd
+ETCD_DOWNLOAD_URL=${ETCD_DOWNLOAD_URL:-https://github.com/coreos/etcd/releases/download}
+ETCD_VERSION=${ETCD_VERSION:-v3.1.7}
+ETCD_DATA_DIR="$DEST/data/etcd"
+ETCD_SYSTEMD_SERVICE="devstack@etcd.service"
+ETCD_BIN_DIR="$DEST/bin"
+ETCD_SHA256_AMD64="4fde194bbcd259401e2b5c462dfa579ee7f6af539f13f130b8f5b4f52e3b3c52"
+# NOTE(sdague): etcd v3.1.7 doesn't have anything for these architectures, though 3.2.0 does.
+ETCD_SHA256_ARM64=""
+ETCD_SHA256_PPC64=""
+ETCD_PORT=2379
+
+if is_ubuntu ; then
+ UBUNTU_RELEASE_BASE_NUM=`lsb_release -r | awk '{print $2}' | cut -d '.' -f 1`
+fi
+
+# start_etcd3() - Starts to run the etcd process
+function start_etcd3 {
+ local cmd="$ETCD_BIN_DIR/etcd"
+ cmd+=" --name $HOSTNAME --data-dir $ETCD_DATA_DIR"
+ cmd+=" --initial-cluster-state new --initial-cluster-token etcd-cluster-01"
+ cmd+=" --initial-cluster $HOSTNAME=http://$SERVICE_HOST:2380"
+ cmd+=" --initial-advertise-peer-urls http://$SERVICE_HOST:2380"
+ cmd+=" --advertise-client-urls http://${HOST_IP}:$ETCD_PORT"
+ cmd+=" --listen-peer-urls http://0.0.0.0:2380 "
+ cmd+=" --listen-client-urls http://${HOST_IP}:$ETCD_PORT"
+
+ local unitfile="$SYSTEMD_DIR/$ETCD_SYSTEMD_SERVICE"
+ write_user_unit_file $ETCD_SYSTEMD_SERVICE "$cmd" "" "root"
+
+ iniset -sudo $unitfile "Unit" "After" "network.target"
+ iniset -sudo $unitfile "Service" "Type" "notify"
+ iniset -sudo $unitfile "Service" "Restart" "on-failure"
+ iniset -sudo $unitfile "Service" "LimitNOFILE" "65536"
+
+ $SYSTEMCTL daemon-reload
+ $SYSTEMCTL enable $ETCD_SYSTEMD_SERVICE
+ $SYSTEMCTL start $ETCD_SYSTEMD_SERVICE
+}
+
+# stop_etcd3() stops the etcd3 process
+function stop_etcd3 {
+ # Don't install in sub nodes (multinode scenario)
+ if [ "$SERVICE_HOST" != "$HOST_IP" ]; then
+ return
+ fi
+
+ $SYSTEMCTL stop $ETCD_SYSTEMD_SERVICE
+}
+
+function cleanup_etcd3 {
+ # Don't install in sub nodes (multinode scenario)
+ if [ "$SERVICE_HOST" != "$HOST_IP" ]; then
+ return
+ fi
+
+ $SYSTEMCTL disable $ETCD_SYSTEMD_SERVICE
+
+ local unitfile="$SYSTEMD_DIR/$ETCD_SYSTEMD_SERVICE"
+ sudo rm -f $unitfile
+
+ $SYSTEMCTL daemon-reload
+
+ sudo rm -rf $ETCD_DATA_DIR
+}
+
+function install_etcd3 {
+ echo "Installing etcd"
+
+ # Make sure etcd3 downloads the correct architecture
+ if is_arch "x86_64"; then
+ ETCD_ARCH="amd64"
+ ETCD_SHA256=${ETCD_SHA256:-$ETCD_SHA256_AMD64}
+ elif is_arch "aarch64"; then
+ ETCD_ARCH="arm64"
+ ETCD_SHA256=${ETCD_SHA256:-$ETCD_SHA256_ARM64}
+ elif is_arch "ppc64le"; then
+ ETCD_ARCH="ppc64le"
+ ETCD_SHA256=${ETCD_SHA256:-$ETCD_SHA256_PPC64}
+ else
+ exit_distro_not_supported "invalid hardware type - $ETCD_ARCH"
+ fi
+
+ ETCD_NAME=etcd-$ETCD_VERSION-linux-$ETCD_ARCH
+
+ # Create the necessary directories
+ sudo mkdir -p $ETCD_BIN_DIR
+ sudo mkdir -p $ETCD_DATA_DIR
+
+ # Download and cache the etcd tgz for subsequent use
+ if [ ! -f "$FILES/etcd-$ETCD_VERSION-linux-$ETCD_ARCH/etcd" ]; then
+ ETCD_DOWNLOAD_FILE=$ETCD_NAME.tar.gz
+ if [ ! -f "$FILES/$ETCD_DOWNLOAD_FILE" ]; then
+ wget $ETCD_DOWNLOAD_URL/$ETCD_VERSION/$ETCD_DOWNLOAD_FILE -O $FILES/$ETCD_DOWNLOAD_FILE
+ fi
+ echo "${ETCD_SHA256} $FILES/${ETCD_DOWNLOAD_FILE}" > $FILES/etcd.sha256sum
+ # NOTE(sdague): this should go fatal if this fails
+ sha256sum -c $FILES/etcd.sha256sum
+
+ tar xzvf $FILES/$ETCD_DOWNLOAD_FILE -C $FILES
+ sudo cp $FILES/$ETCD_NAME/etcd $ETCD_BIN_DIR/etcd
+ fi
+ if [ ! -f "$ETCD_BIN_DIR/etcd" ]; then
+ sudo cp $FILES/$ETCD_NAME/etcd $ETCD_BIN_DIR/etcd
+ fi
+}
+
+# Restore xtrace
+$_XTRACE_ETCD3
+
+# Tell emacs to use shell-script-mode
+## Local variables:
+## mode: shell-script
+## End:
diff --git a/lib/glance b/lib/glance
index d6438a6..41145f9 100644
--- a/lib/glance
+++ b/lib/glance
@@ -71,6 +71,16 @@
GLANCE_SERVICE_PROTOCOL=${GLANCE_SERVICE_PROTOCOL:-$SERVICE_PROTOCOL}
GLANCE_REGISTRY_PORT=${GLANCE_REGISTRY_PORT:-9191}
GLANCE_REGISTRY_PORT_INT=${GLANCE_REGISTRY_PORT_INT:-19191}
+GLANCE_UWSGI=$GLANCE_BIN_DIR/glance-wsgi-api
+GLANCE_UWSGI_CONF=$GLANCE_CONF_DIR/glance-uswgi.ini
+# If wsgi mode is uwsgi run glance under uwsgi, else default to eventlet
+# TODO(mtreinish): Remove the eventlet path here and in all the similar
+# conditionals below after the Pike release
+if [[ "$WSGI_MODE" == "uwsgi" ]]; then
+ GLANCE_URL="$GLANCE_SERVICE_PROTOCOL://$GLANCE_SERVICE_HOST/image"
+else
+ GLANCE_URL="$GLANCE_SERVICE_PROTOCOL://$GLANCE_HOSTPORT"
+fi
# Functions
# ---------
@@ -78,6 +88,7 @@
# Test if any Glance services are enabled
# is_glance_enabled
function is_glance_enabled {
+ [[ ,${DISABLED_SERVICES} =~ ,"glance" ]] && return 1
[[ ,${ENABLED_SERVICES} =~ ,"g-" ]] && return 0
return 1
}
@@ -103,16 +114,13 @@
dburl=`database_connection_url glance`
iniset $GLANCE_REGISTRY_CONF database connection $dburl
iniset $GLANCE_REGISTRY_CONF DEFAULT use_syslog $SYSLOG
- iniset $GLANCE_REGISTRY_CONF DEFAULT workers "$API_WORKERS"
iniset $GLANCE_REGISTRY_CONF paste_deploy flavor keystone
configure_auth_token_middleware $GLANCE_REGISTRY_CONF glance $GLANCE_AUTH_CACHE_DIR/registry
iniset $GLANCE_REGISTRY_CONF oslo_messaging_notifications driver messagingv2
iniset_rpc_backend glance $GLANCE_REGISTRY_CONF
iniset $GLANCE_REGISTRY_CONF DEFAULT graceful_shutdown_timeout "$SERVICE_GRACEFUL_SHUTDOWN_TIMEOUT"
- cp $GLANCE_DIR/etc/glance-api.conf $GLANCE_API_CONF
iniset $GLANCE_API_CONF DEFAULT debug $ENABLE_DEBUG_LOG_LEVEL
- iniset $GLANCE_API_CONF DEFAULT bind_host $GLANCE_SERVICE_LISTEN_ADDRESS
inicomment $GLANCE_API_CONF DEFAULT log_file
iniset $GLANCE_API_CONF database connection $dburl
iniset $GLANCE_API_CONF DEFAULT use_syslog $SYSLOG
@@ -140,8 +148,6 @@
iniset $GLANCE_API_CONF glance_store filesystem_store_datadir $GLANCE_IMAGE_DIR/
iniset $GLANCE_API_CONF DEFAULT registry_host $GLANCE_SERVICE_HOST
- iniset $GLANCE_API_CONF DEFAULT workers "$API_WORKERS"
-
# CORS feature support - to allow calls from Horizon by default
if [ -n "$GLANCE_CORS_ALLOWED_ORIGIN" ]; then
iniset $GLANCE_API_CONF cors allowed_origin "$GLANCE_CORS_ALLOWED_ORIGIN"
@@ -180,9 +186,12 @@
inicomment $GLANCE_API_CONF glance_store swift_store_auth_address
fi
+ # We need to tell glance what it's public endpoint is so that the version
+ # discovery document will be correct
+ iniset $GLANCE_API_CONF DEFAULT public_endpoint $GLANCE_URL
+
if is_service_enabled tls-proxy; then
iniset $GLANCE_API_CONF DEFAULT bind_port $GLANCE_SERVICE_PORT_INT
- iniset $GLANCE_API_CONF DEFAULT public_endpoint $GLANCE_SERVICE_PROTOCOL://$GLANCE_HOSTPORT
iniset $GLANCE_REGISTRY_CONF DEFAULT bind_port $GLANCE_REGISTRY_PORT_INT
iniset $GLANCE_API_CONF keystone_authtoken identity_uri $KEYSTONE_AUTH_URI
@@ -198,7 +207,6 @@
setup_logging $GLANCE_REGISTRY_CONF
cp -p $GLANCE_DIR/etc/glance-registry-paste.ini $GLANCE_REGISTRY_PASTE_INI
-
cp -p $GLANCE_DIR/etc/glance-api-paste.ini $GLANCE_API_PASTE_INI
cp $GLANCE_DIR/etc/glance-cache.conf $GLANCE_CACHE_CONF
@@ -231,6 +239,13 @@
iniset $GLANCE_API_CONF DEFAULT cinder_endpoint_template "https://$CINDER_SERVICE_HOST:$CINDER_SERVICE_PORT/v1/%(project_id)s"
iniset $GLANCE_CACHE_CONF DEFAULT cinder_endpoint_template "https://$CINDER_SERVICE_HOST:$CINDER_SERVICE_PORT/v1/%(project_id)s"
fi
+
+ if [[ "$WSGI_MODE" == "uwsgi" ]]; then
+ write_local_uwsgi_http_config "$GLANCE_UWSGI_CONF" "$GLANCE_UWSGI" "/image"
+ else
+ iniset $GLANCE_API_CONF DEFAULT bind_host $GLANCE_SERVICE_LISTEN_ADDRESS
+ iniset $GLANCE_API_CONF DEFAULT workers "$API_WORKERS"
+ fi
}
# create_glance_accounts() - Set up common required glance accounts
@@ -255,7 +270,7 @@
get_or_create_endpoint \
"image" \
"$REGION_NAME" \
- "$GLANCE_SERVICE_PROTOCOL://$GLANCE_HOSTPORT"
+ "$GLANCE_URL"
# Note(frickler): Crude workaround for https://bugs.launchpad.net/glance-store/+bug/1620999
service_domain_id=$(get_or_create_domain $SERVICE_DOMAIN_NAME)
@@ -284,11 +299,13 @@
# (Re)create glance database
recreate_database glance
+ time_start "dbsync"
# Migrate glance database
$GLANCE_BIN_DIR/glance-manage --config-file $GLANCE_CONF_DIR/glance-api.conf db_sync
# Load metadata definitions
$GLANCE_BIN_DIR/glance-manage --config-file $GLANCE_CONF_DIR/glance-api.conf db_load_metadefs
+ time_stop "dbsync"
create_glance_cache_dir
}
@@ -320,15 +337,21 @@
function start_glance {
local service_protocol=$GLANCE_SERVICE_PROTOCOL
if is_service_enabled tls-proxy; then
- start_tls_proxy glance-service '*' $GLANCE_SERVICE_PORT $GLANCE_SERVICE_HOST $GLANCE_SERVICE_PORT_INT
+ if [[ "$WSGI_MODE" != "uwsgi" ]]; then
+ start_tls_proxy glance-service '*' $GLANCE_SERVICE_PORT $GLANCE_SERVICE_HOST $GLANCE_SERVICE_PORT_INT
+ fi
start_tls_proxy glance-registry '*' $GLANCE_REGISTRY_PORT $GLANCE_SERVICE_HOST $GLANCE_REGISTRY_PORT_INT
fi
run_process g-reg "$GLANCE_BIN_DIR/glance-registry --config-file=$GLANCE_CONF_DIR/glance-registry.conf"
- run_process g-api "$GLANCE_BIN_DIR/glance-api --config-file=$GLANCE_CONF_DIR/glance-api.conf"
+ if [[ "$WSGI_MODE" == "uwsgi" ]]; then
+ run_process g-api "$GLANCE_BIN_DIR/uwsgi --ini $GLANCE_UWSGI_CONF"
+ else
+ run_process g-api "$GLANCE_BIN_DIR/glance-api --config-file=$GLANCE_CONF_DIR/glance-api.conf"
+ fi
- echo "Waiting for g-api ($GLANCE_HOSTPORT) to start..."
- if ! wait_for_service $SERVICE_TIMEOUT $GLANCE_SERVICE_PROTOCOL://$GLANCE_HOSTPORT; then
+ echo "Waiting for g-api ($GLANCE_SERVICE_HOST) to start..."
+ if ! wait_for_service $SERVICE_TIMEOUT $GLANCE_URL; then
die $LINENO "g-api did not start"
fi
}
diff --git a/lib/keystone b/lib/keystone
index 61b0a0c..eb46526 100644
--- a/lib/keystone
+++ b/lib/keystone
@@ -124,9 +124,15 @@
# Security compliance
KEYSTONE_SECURITY_COMPLIANCE_ENABLED=${KEYSTONE_SECURITY_COMPLIANCE_ENABLED:-True}
KEYSTONE_LOCKOUT_FAILURE_ATTEMPTS=${KEYSTONE_LOCKOUT_FAILURE_ATTEMPTS:-2}
-KEYSTONE_LOCKOUT_DURATION=${KEYSTONE_LOCKOUT_DURATION:-5}
+KEYSTONE_LOCKOUT_DURATION=${KEYSTONE_LOCKOUT_DURATION:-10}
KEYSTONE_UNIQUE_LAST_PASSWORD_COUNT=${KEYSTONE_UNIQUE_LAST_PASSWORD_COUNT:-2}
+# Number of bcrypt hashing rounds, increasing number exponentially increases required
+# resources to generate password hash. This is very effective way to protect from
+# bruteforce attacks. 4 is minimal value that can be specified for bcrypt and
+# it works way faster than default 12. Minimal value is great for CI and development
+# however may not be suitable for real production.
+KEYSTONE_PASSWORD_HASH_ROUNDS=${KEYSTONE_PASSWORD_HASH_ROUNDS:-4}
# Functions
# ---------
@@ -134,6 +140,7 @@
# Test if Keystone is enabled
# is_keystone_enabled
function is_keystone_enabled {
+ [[ ,${DISABLED_SERVICES} =~ ,"keystone" ]] && return 1
[[ ,${ENABLED_SERVICES}, =~ ,"key", ]] && return 0
return 1
}
@@ -224,6 +231,7 @@
fi
iniset $KEYSTONE_CONF identity driver "$KEYSTONE_IDENTITY_BACKEND"
+ iniset $KEYSTONE_CONF identity password_hash_rounds $KEYSTONE_PASSWORD_HASH_ROUNDS
iniset $KEYSTONE_CONF assignment driver "$KEYSTONE_ASSIGNMENT_BACKEND"
iniset $KEYSTONE_CONF role driver "$KEYSTONE_ROLE_BACKEND"
iniset $KEYSTONE_CONF resource driver "$KEYSTONE_RESOURCE_BACKEND"
@@ -267,9 +275,7 @@
fi
# Format logging
- if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ] && [ "$KEYSTONE_DEPLOY" != "mod_wsgi" ] ; then
- setup_colorized_logging $KEYSTONE_CONF
- fi
+ setup_logging $KEYSTONE_CONF
iniset $KEYSTONE_CONF DEFAULT debug $ENABLE_DEBUG_LOG_LEVEL
@@ -459,8 +465,10 @@
recreate_database keystone
fi
+ time_start "dbsync"
# Initialize keystone database
$KEYSTONE_BIN_DIR/keystone-manage --config-file $KEYSTONE_CONF db_sync
+ time_stop "dbsync"
if [[ "$KEYSTONE_TOKEN_FORMAT" == "pki" || "$KEYSTONE_TOKEN_FORMAT" == "pkiz" ]]; then
# Set up certificates
diff --git a/lib/libraries b/lib/libraries
new file mode 100644
index 0000000..4ceb804
--- /dev/null
+++ b/lib/libraries
@@ -0,0 +1,143 @@
+#!/bin/bash
+#
+# lib/oslo
+#
+# Functions to install libraries from git
+#
+# We need this to handle the fact that projects would like to use
+# pre-released versions of oslo libraries.
+
+# Dependencies:
+#
+# - ``functions`` file
+
+# ``stack.sh`` calls the entry points in this order:
+#
+# - install_libraries
+
+# Save trace setting
+_XTRACE_LIB_LIBRARIES=$(set +o | grep xtrace)
+set +o xtrace
+
+
+# Defaults
+# --------
+GITDIR["automaton"]=$DEST/automaton
+GITDIR["castellan"]=$DEST/castellan
+GITDIR["cliff"]=$DEST/cliff
+GITDIR["cursive"]=$DEST/cursive
+GITDIR["debtcollector"]=$DEST/debtcollector
+GITDIR["futurist"]=$DEST/futurist
+GITDIR["os-client-config"]=$DEST/os-client-config
+GITDIR["osc-lib"]=$DEST/osc-lib
+GITDIR["oslo.cache"]=$DEST/oslo.cache
+GITDIR["oslo.concurrency"]=$DEST/oslo.concurrency
+GITDIR["oslo.config"]=$DEST/oslo.config
+GITDIR["oslo.context"]=$DEST/oslo.context
+GITDIR["oslo.db"]=$DEST/oslo.db
+GITDIR["oslo.i18n"]=$DEST/oslo.i18n
+GITDIR["oslo.log"]=$DEST/oslo.log
+GITDIR["oslo.messaging"]=$DEST/oslo.messaging
+GITDIR["oslo.middleware"]=$DEST/oslo.middleware
+GITDIR["oslo.policy"]=$DEST/oslo.policy
+GITDIR["oslo.privsep"]=$DEST/oslo.privsep
+GITDIR["oslo.reports"]=$DEST/oslo.reports
+GITDIR["oslo.rootwrap"]=$DEST/oslo.rootwrap
+GITDIR["oslo.serialization"]=$DEST/oslo.serialization
+GITDIR["oslo.service"]=$DEST/oslo.service
+GITDIR["oslo.utils"]=$DEST/oslo.utils
+GITDIR["oslo.versionedobjects"]=$DEST/oslo.versionedobjects
+GITDIR["oslo.vmware"]=$DEST/oslo.vmware
+GITDIR["osprofiler"]=$DEST/osprofiler
+GITDIR["pycadf"]=$DEST/pycadf
+GITDIR["python-openstacksdk"]=$DEST/python-openstacksdk
+GITDIR["stevedore"]=$DEST/stevedore
+GITDIR["taskflow"]=$DEST/taskflow
+GITDIR["tooz"]=$DEST/tooz
+
+# Non oslo libraries are welcomed below as well, this prevents
+# duplication of this code.
+GITDIR["os-brick"]=$DEST/os-brick
+GITDIR["os-traits"]=$DEST/os-traits
+
+# Support entry points installation of console scripts
+OSLO_BIN_DIR=$(get_python_exec_prefix)
+
+
+# Functions
+# ---------
+
+function _install_lib_from_source {
+ local name=$1
+ if use_library_from_git "$name"; then
+ git_clone_by_name "$name"
+ setup_dev_lib "$name"
+ fi
+}
+
+# install_oslo - install libraries that oslo needs
+function install_oslo {
+ install_libs
+}
+
+# install_libs() - Install additional libraries that we need and want
+# on all environments. Some will only install here if from source,
+# others will always install.
+function install_libs {
+ _install_lib_from_source "automaton"
+ _install_lib_from_source "castellan"
+ _install_lib_from_source "cliff"
+ _install_lib_from_source "cursive"
+ _install_lib_from_source "debtcollector"
+ _install_lib_from_source "futurist"
+ _install_lib_from_source "osc-lib"
+ _install_lib_from_source "os-client-config"
+ _install_lib_from_source "oslo.cache"
+ _install_lib_from_source "oslo.concurrency"
+ _install_lib_from_source "oslo.config"
+ _install_lib_from_source "oslo.context"
+ _install_lib_from_source "oslo.db"
+ _install_lib_from_source "oslo.i18n"
+ _install_lib_from_source "oslo.log"
+ _install_lib_from_source "oslo.messaging"
+ _install_lib_from_source "oslo.middleware"
+ _install_lib_from_source "oslo.policy"
+ _install_lib_from_source "oslo.privsep"
+ _install_lib_from_source "oslo.reports"
+ _install_lib_from_source "oslo.rootwrap"
+ _install_lib_from_source "oslo.serialization"
+ _install_lib_from_source "oslo.service"
+ _install_lib_from_source "oslo.utils"
+ _install_lib_from_source "oslo.versionedobjects"
+ _install_lib_from_source "oslo.vmware"
+ _install_lib_from_source "osprofiler"
+ _install_lib_from_source "pycadf"
+ _install_lib_from_source "python-openstacksdk"
+ _install_lib_from_source "stevedore"
+ _install_lib_from_source "taskflow"
+ _install_lib_from_source "tooz"
+ # installation of additional libraries
+ #
+ # os-traits for nova
+ _install_lib_from_source "os-brick"
+ _install_lib_from_source "os-traits"
+ #
+ # python client libraries we might need from git can go here
+ _install_lib_from_source "python-barbicanclient"
+
+
+ # etcd (because tooz does not have a hard dependency on these)
+ #
+ # NOTE(sdague): this is currently a work around because tooz
+ # doesn't pull in etcd3.
+ pip_install etcd3
+ pip_install etcd3gw
+}
+
+# Restore xtrace
+$_XTRACE_LIB_LIBRARIES
+
+# Tell emacs to use shell-script-mode
+## Local variables:
+## mode: shell-script
+## End:
diff --git a/lib/neutron b/lib/neutron
index c7f62ed..2a660ec 100644
--- a/lib/neutron
+++ b/lib/neutron
@@ -91,6 +91,7 @@
# Test if any Neutron services are enabled
# is_neutron_enabled
function is_neutron_enabled {
+ [[ ,${DISABLED_SERVICES} =~ ,"neutron" ]] && return 1
[[ ,${ENABLED_SERVICES} =~ ,"neutron-" || ,${ENABLED_SERVICES} =~ ,"q-" ]] && return 0
return 1
}
@@ -98,6 +99,7 @@
# Test if any Neutron services are enabled
# is_neutron_enabled
function is_neutron_legacy_enabled {
+ [[ ,${DISABLED_SERVICES} =~ ,"neutron" ]] && return 1
[[ ,${ENABLED_SERVICES} =~ ,"q-" ]] && return 0
return 1
}
@@ -135,7 +137,11 @@
mkdir -p $NEUTRON_CORE_PLUGIN_CONF_PATH
- cp $NEUTRON_DIR/etc/neutron/plugins/$NEUTRON_CORE_PLUGIN/$NEUTRON_CORE_PLUGIN_CONF_FILENAME.sample $NEUTRON_CORE_PLUGIN_CONF
+ # NOTE(yamamoto): A decomposed plugin should prepare the config file in
+ # its devstack plugin.
+ if [ -f $NEUTRON_DIR/etc/neutron/plugins/$NEUTRON_CORE_PLUGIN/$NEUTRON_CORE_PLUGIN_CONF_FILENAME.sample ]; then
+ cp $NEUTRON_DIR/etc/neutron/plugins/$NEUTRON_CORE_PLUGIN/$NEUTRON_CORE_PLUGIN_CONF_FILENAME.sample $NEUTRON_CORE_PLUGIN_CONF
+ fi
iniset $NEUTRON_CONF database connection `database_connection_url neutron`
iniset $NEUTRON_CONF DEFAULT state_path $NEUTRON_STATE_PATH
@@ -231,12 +237,7 @@
fi
# Format logging
- if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ]; then
- setup_colorized_logging $NEUTRON_CONF DEFAULT project_id
- else
- # Show user_name and project_name by default
- iniset $NEUTRON_CONF DEFAULT logging_context_format_string "%(asctime)s.%(msecs)03d %(levelname)s %(name)s [%(request_id)s %(user_name)s %(project_name)s] %(instance)s%(message)s"
- fi
+ setup_logging $NEUTRON_CONF
if is_service_enabled tls-proxy; then
# Set the service port for a proxy to take the original
@@ -335,8 +336,10 @@
recreate_database neutron
+ time_start "dbsync"
# Run Neutron db migrations
$NEUTRON_BIN_DIR/neutron-db-manage upgrade heads
+ time_stop "dbsync"
create_neutron_cache_dir
}
diff --git a/lib/neutron-legacy b/lib/neutron-legacy
index fa508f8..784f3a8 100644
--- a/lib/neutron-legacy
+++ b/lib/neutron-legacy
@@ -417,8 +417,10 @@
# init_mutnauq() - Initialize databases, etc.
function init_mutnauq {
recreate_database $Q_DB_NAME
+ time_start "dbsync"
# Run Neutron db migrations
$NEUTRON_BIN_DIR/neutron-db-manage --config-file $NEUTRON_CONF --config-file /$Q_PLUGIN_CONF_FILE upgrade head
+ time_stop "dbsync"
}
# install_mutnauq() - Collect source and prepare
@@ -577,7 +579,7 @@
local IP_DEL=""
local IP_UP=""
local DEFAULT_ROUTE_GW
- DEFAULT_ROUTE_GW=$(ip -f $af r | awk "/default.+$from_intf/ { print \$3; exit }")
+ DEFAULT_ROUTE_GW=$(ip -f $af r | awk "/default.+$from_intf\s/ { print \$3; exit }")
local ADD_OVS_PORT=""
local DEL_OVS_PORT=""
local ARP_CMD=""
diff --git a/lib/nova b/lib/nova
index de053ab..f56509a 100644
--- a/lib/nova
+++ b/lib/nova
@@ -175,6 +175,7 @@
# Test if any Nova services are enabled
# is_nova_enabled
function is_nova_enabled {
+ [[ ,${DISABLED_SERVICES} =~ ,"nova" ]] && return 1
[[ ,${ENABLED_SERVICES} =~ ,"n-" ]] && return 0
return 1
}
@@ -453,8 +454,8 @@
fi
iniset $NOVA_CONF wsgi api_paste_config "$NOVA_API_PASTE_INI"
iniset $NOVA_CONF DEFAULT rootwrap_config "$NOVA_CONF_DIR/rootwrap.conf"
- iniset $NOVA_CONF DEFAULT scheduler_driver "$SCHEDULER"
- iniset $NOVA_CONF DEFAULT scheduler_default_filters "$FILTERS"
+ iniset $NOVA_CONF scheduler driver "$SCHEDULER"
+ iniset $NOVA_CONF filter_scheduler enabled_filters "$FILTERS"
iniset $NOVA_CONF DEFAULT default_floating_pool "$PUBLIC_NETWORK_NAME"
if [[ $SERVICE_IP_VERSION == 6 ]]; then
iniset $NOVA_CONF DEFAULT my_ip "$HOST_IPV6"
@@ -574,7 +575,7 @@
# enable notifications, but it will allow them to function when enabled.
iniset $NOVA_CONF oslo_messaging_notifications driver "messagingv2"
iniset_rpc_backend nova $NOVA_CONF
- iniset $NOVA_CONF glance api_servers "${GLANCE_SERVICE_PROTOCOL}://${GLANCE_HOSTPORT}"
+ iniset $NOVA_CONF glance api_servers "$GLANCE_URL"
iniset $NOVA_CONF DEFAULT osapi_compute_workers "$API_WORKERS"
iniset $NOVA_CONF DEFAULT metadata_workers "$API_WORKERS"
@@ -648,7 +649,9 @@
iniset $NOVA_CELLS_CONF DEFAULT enabled_apis metadata
fi
+ time_start "dbsync"
$NOVA_BIN_DIR/nova-manage --config-file $NOVA_CELLS_CONF db sync
+ time_stop "dbsync"
$NOVA_BIN_DIR/nova-manage --config-file $NOVA_CELLS_CONF cell create --name=region --cell_type=parent --username=$RABBIT_USERID --hostname=$RABBIT_HOST --port=5672 --password=$RABBIT_PASSWORD --virtual_host=/ --woffset=0 --wscale=1
$NOVA_BIN_DIR/nova-manage cell create --name=child --cell_type=child --username=$RABBIT_USERID --hostname=$RABBIT_HOST --port=5672 --password=$RABBIT_PASSWORD --virtual_host=child_cell --woffset=0 --wscale=1
@@ -878,7 +881,6 @@
run_process n-cond "$NOVA_BIN_DIR/nova-conductor --config-file $compute_cell_conf"
run_process n-cell-region "$NOVA_BIN_DIR/nova-cells --config-file $api_cell_conf"
run_process n-cell-child "$NOVA_BIN_DIR/nova-cells --config-file $compute_cell_conf"
- run_process n-crt "$NOVA_BIN_DIR/nova-cert --config-file $api_cell_conf"
if is_service_enabled n-net; then
if ! running_in_container; then
@@ -929,7 +931,7 @@
# Kill the nova screen windows
# Some services are listed here twice since more than one instance
# of a service may be running in certain configs.
- for serv in n-api n-crt n-net n-sch n-novnc n-xvnc n-cauth n-spice n-cond n-cell n-cell n-api-meta n-sproxy; do
+ for serv in n-api n-net n-sch n-novnc n-xvnc n-cauth n-spice n-cond n-cell n-cell n-api-meta n-sproxy; do
stop_process $serv
done
}
diff --git a/lib/nova_plugins/functions-libvirt b/lib/nova_plugins/functions-libvirt
index 47605af..3e38b89 100644
--- a/lib/nova_plugins/functions-libvirt
+++ b/lib/nova_plugins/functions-libvirt
@@ -67,6 +67,8 @@
else
install_package libvirt-clients libvirt-daemon-system libvirt-dev
fi
+ # uninstall in case the libvirt version changed
+ pip_uninstall libvirt-python
pip_install_gr libvirt-python
#pip_install_gr <there-si-no-guestfs-in-pypi>
elif is_fedora || is_suse; then
@@ -84,6 +86,7 @@
fi
install_package libvirt libvirt-devel
+ pip_uninstall libvirt-python
pip_install_gr libvirt-python
fi
@@ -95,7 +98,7 @@
# Configures the installed libvirt system so that is accessible by
# STACK_USER via qemu:///system with management capabilities.
function configure_libvirt {
- if is_service_enabled neutron && is_neutron_ovs_base_plugin && ! sudo grep -q '^cgroup_device_acl' $QEMU_CONF; then
+ if is_service_enabled neutron && ! sudo grep -q '^cgroup_device_acl' $QEMU_CONF; then
# Add /dev/net/tun to cgroup_device_acls, needed for type=ethernet interfaces
cat <<EOF | sudo tee -a $QEMU_CONF
cgroup_device_acl = [
diff --git a/lib/nova_plugins/hypervisor-fake b/lib/nova_plugins/hypervisor-fake
index f9b95c1..49c8dee 100644
--- a/lib/nova_plugins/hypervisor-fake
+++ b/lib/nova_plugins/hypervisor-fake
@@ -49,7 +49,7 @@
iniset $NOVA_CONF DEFAULT quota_security_groups -1
iniset $NOVA_CONF DEFAULT quota_security_group_rules -1
iniset $NOVA_CONF DEFAULT quota_key_pairs -1
- iniset $NOVA_CONF DEFAULT scheduler_default_filters "RetryFilter,AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,CoreFilter,RamFilter,DiskFilter"
+ iniset $NOVA_CONF filter_scheduler enabled_filters "RetryFilter,AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,CoreFilter,RamFilter,DiskFilter"
}
# install_nova_hypervisor() - Install external components
diff --git a/lib/nova_plugins/hypervisor-xenserver b/lib/nova_plugins/hypervisor-xenserver
index 880b87f..6f79e4f 100644
--- a/lib/nova_plugins/hypervisor-xenserver
+++ b/lib/nova_plugins/hypervisor-xenserver
@@ -84,14 +84,6 @@
* * * * * /root/rotate_xen_guest_logs.sh >/dev/null 2>&1
CRONTAB
- # Create directories for kernels and images
- {
- echo "set -eux"
- cat $TOP_DIR/tools/xen/functions
- echo "create_directory_for_images"
- echo "create_directory_for_kernels"
- echo "install_conntrack_tools"
- } | $ssh_dom0
}
# install_nova_hypervisor() - Install external components
diff --git a/lib/os_brick b/lib/os_brick
deleted file mode 100644
index d1cca4a..0000000
--- a/lib/os_brick
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/bin/bash
-#
-# lib/os_brick
-# Install **os-brick** python module from source
-
-# Dependencies:
-#
-# - functions
-# - DEST, DATA_DIR must be defined
-
-# stack.sh
-# ---------
-# - install_os_brick
-
-# Save trace setting
-_XTRACE_OS_BRICK=$(set +o | grep xtrace)
-set +o xtrace
-
-
-GITDIR["os-brick"]=$DEST/os-brick
-
-# Install os_brick from git only if requested, otherwise it will be pulled from
-# pip repositories by requirements of projects that need it.
-function install_os_brick {
- if use_library_from_git "os-brick"; then
- git_clone_by_name "os-brick"
- setup_dev_lib "os-brick"
- fi
-}
-
-# Restore xtrace
-$_XTRACE_OS_BRICK
\ No newline at end of file
diff --git a/lib/oslo b/lib/oslo
index 2895503..3ae64c8 100644
--- a/lib/oslo
+++ b/lib/oslo
@@ -6,110 +6,6 @@
#
# We need this to handle the fact that projects would like to use
# pre-released versions of oslo libraries.
-
-# Dependencies:
#
-# - ``functions`` file
-
-# ``stack.sh`` calls the entry points in this order:
-#
-# - install_oslo
-
-# Save trace setting
-_XTRACE_LIB_OSLO=$(set +o | grep xtrace)
-set +o xtrace
-
-
-# Defaults
-# --------
-GITDIR["automaton"]=$DEST/automaton
-GITDIR["castellan"]=$DEST/castellan
-GITDIR["cliff"]=$DEST/cliff
-GITDIR["cursive"]=$DEST/cursive
-GITDIR["debtcollector"]=$DEST/debtcollector
-GITDIR["futurist"]=$DEST/futurist
-GITDIR["os-client-config"]=$DEST/os-client-config
-GITDIR["osc-lib"]=$DEST/osc-lib
-GITDIR["oslo.cache"]=$DEST/oslo.cache
-GITDIR["oslo.concurrency"]=$DEST/oslo.concurrency
-GITDIR["oslo.config"]=$DEST/oslo.config
-GITDIR["oslo.context"]=$DEST/oslo.context
-GITDIR["oslo.db"]=$DEST/oslo.db
-GITDIR["oslo.i18n"]=$DEST/oslo.i18n
-GITDIR["oslo.log"]=$DEST/oslo.log
-GITDIR["oslo.messaging"]=$DEST/oslo.messaging
-GITDIR["oslo.middleware"]=$DEST/oslo.middleware
-GITDIR["oslo.policy"]=$DEST/oslo.policy
-GITDIR["oslo.privsep"]=$DEST/oslo.privsep
-GITDIR["oslo.reports"]=$DEST/oslo.reports
-GITDIR["oslo.rootwrap"]=$DEST/oslo.rootwrap
-GITDIR["oslo.serialization"]=$DEST/oslo.serialization
-GITDIR["oslo.service"]=$DEST/oslo.service
-GITDIR["oslo.utils"]=$DEST/oslo.utils
-GITDIR["oslo.versionedobjects"]=$DEST/oslo.versionedobjects
-GITDIR["oslo.vmware"]=$DEST/oslo.vmware
-GITDIR["osprofiler"]=$DEST/osprofiler
-GITDIR["pycadf"]=$DEST/pycadf
-GITDIR["python-openstacksdk"]=$DEST/python-openstacksdk
-GITDIR["stevedore"]=$DEST/stevedore
-GITDIR["taskflow"]=$DEST/taskflow
-GITDIR["tooz"]=$DEST/tooz
-
-# Support entry points installation of console scripts
-OSLO_BIN_DIR=$(get_python_exec_prefix)
-
-
-# Functions
-# ---------
-
-function _do_install_oslo_lib {
- local name=$1
- if use_library_from_git "$name"; then
- git_clone_by_name "$name"
- setup_dev_lib "$name"
- fi
-}
-
-# install_oslo() - Collect source and prepare
-function install_oslo {
- _do_install_oslo_lib "automaton"
- _do_install_oslo_lib "castellan"
- _do_install_oslo_lib "cliff"
- _do_install_oslo_lib "cursive"
- _do_install_oslo_lib "debtcollector"
- _do_install_oslo_lib "futurist"
- _do_install_oslo_lib "osc-lib"
- _do_install_oslo_lib "os-client-config"
- _do_install_oslo_lib "oslo.cache"
- _do_install_oslo_lib "oslo.concurrency"
- _do_install_oslo_lib "oslo.config"
- _do_install_oslo_lib "oslo.context"
- _do_install_oslo_lib "oslo.db"
- _do_install_oslo_lib "oslo.i18n"
- _do_install_oslo_lib "oslo.log"
- _do_install_oslo_lib "oslo.messaging"
- _do_install_oslo_lib "oslo.middleware"
- _do_install_oslo_lib "oslo.policy"
- _do_install_oslo_lib "oslo.privsep"
- _do_install_oslo_lib "oslo.reports"
- _do_install_oslo_lib "oslo.rootwrap"
- _do_install_oslo_lib "oslo.serialization"
- _do_install_oslo_lib "oslo.service"
- _do_install_oslo_lib "oslo.utils"
- _do_install_oslo_lib "oslo.versionedobjects"
- _do_install_oslo_lib "oslo.vmware"
- _do_install_oslo_lib "osprofiler"
- _do_install_oslo_lib "pycadf"
- _do_install_oslo_lib "python-openstacksdk"
- _do_install_oslo_lib "stevedore"
- _do_install_oslo_lib "taskflow"
- _do_install_oslo_lib "tooz"
-}
-
-# Restore xtrace
-$_XTRACE_LIB_OSLO
-
-# Tell emacs to use shell-script-mode
-## Local variables:
-## mode: shell-script
-## End:
+# Included for compatibility with grenade, remove in Queens
+source $TOP_DIR/lib/libraries
diff --git a/lib/placement b/lib/placement
index ad12824..8adbbde 100644
--- a/lib/placement
+++ b/lib/placement
@@ -149,7 +149,9 @@
function init_placement {
if [ "$PLACEMENT_DB_ENABLED" != False ]; then
recreate_database placement
+ time_start "dbsync"
$NOVA_BIN_DIR/nova-manage --config-file $NOVA_CONF api_db sync
+ time_stop "dbsync"
fi
create_placement_accounts
}
diff --git a/lib/swift b/lib/swift
index 8fad6b8..e247f15 100644
--- a/lib/swift
+++ b/lib/swift
@@ -174,6 +174,7 @@
# Test if any Swift services are enabled
# is_swift_enabled
function is_swift_enabled {
+ [[ ,${DISABLED_SERVICES} =~ ,"swift" ]] && return 1
[[ ,${ENABLED_SERVICES} =~ ,"s-" ]] && return 0
return 1
}
@@ -541,6 +542,10 @@
else
iniset ${testfile} func_test auth_prefix /identity/v2.0/
fi
+ if is_service_enabled tls-proxy; then
+ iniset ${testfile} func_test cafile ${SSL_BUNDLE_FILE}
+ iniset ${testfile} func_test web_front_end apache2
+ fi
fi
local user_group
diff --git a/lib/tempest b/lib/tempest
index fd8e3e7..cc65ec7 100644
--- a/lib/tempest
+++ b/lib/tempest
@@ -394,24 +394,6 @@
iniset $TEMPEST_CONFIG network-feature-enabled ipv6_subnet_attributes "$IPV6_SUBNET_ATTRIBUTES_ENABLED"
iniset $TEMPEST_CONFIG network-feature-enabled port_security $NEUTRON_PORT_SECURITY
- # Orchestration Tests
- if is_service_enabled heat; then
- if [[ ! -z "$HEAT_CFN_IMAGE_URL" ]]; then
- iniset $TEMPEST_CONFIG orchestration image_ref $(basename "${HEAT_CFN_IMAGE_URL%.*}")
- fi
- # Nova might not be enabled, especially when we want to test tempest scenario/API that only create Neutron resources
- if is_service_enabled nova; then
- # build a specialized heat flavor
- available_flavors=$(nova flavor-list)
- if [[ ! ( $available_flavors =~ 'm1.heat' ) ]]; then
- openstack flavor create --id 451 --ram 512 --disk 0 --vcpus 1 m1.heat
- fi
- iniset $TEMPEST_CONFIG orchestration instance_type "m1.heat"
- fi
- iniset $TEMPEST_CONFIG orchestration build_timeout 900
- iniset $TEMPEST_CONFIG orchestration stack_owner_role Member
- fi
-
# Scenario
if [ "$VIRT_DRIVER" = "xenserver" ]; then
SCENARIO_IMAGE_DIR=${SCENARIO_IMAGE_DIR:-$FILES}
@@ -431,7 +413,7 @@
TEMPEST_SSH_NETWORK_NAME=$PHYSICAL_NETWORK
fi
# Validation
- iniset $TEMPEST_CONFIG validation run_validation ${TEMPEST_RUN_VALIDATION:-False}
+ iniset $TEMPEST_CONFIG validation run_validation ${TEMPEST_RUN_VALIDATION:-True}
iniset $TEMPEST_CONFIG validation ip_version_for_ssh 4
iniset $TEMPEST_CONFIG validation ssh_timeout $BUILD_TIMEOUT
iniset $TEMPEST_CONFIG validation image_ssh_user ${DEFAULT_INSTANCE_USER:-cirros}
diff --git a/lib/template b/lib/template
index b92fb40..25d653c 100644
--- a/lib/template
+++ b/lib/template
@@ -41,6 +41,7 @@
# Test if any XXXX services are enabled
# is_XXXX_enabled
function is_XXXX_enabled {
+ [[ ,${DISABLED_SERVICES} =~ ,"XXXX" ]] && return 1
[[ ,${ENABLED_SERVICES} =~ ,"XX-" ]] && return 0
return 1
}
diff --git a/lib/tls b/lib/tls
index 238687c..6d67c90 100644
--- a/lib/tls
+++ b/lib/tls
@@ -113,11 +113,11 @@
certificate = \$dir/cacert.pem
private_key = \$dir/private/cacert.key
RANDFILE = \$dir/private/.rand
-default_md = default
+default_md = sha256
[ req ]
-default_bits = 1024
-default_md = sha1
+default_bits = 2048
+default_md = sha256
prompt = no
distinguished_name = ca_distinguished_name
@@ -212,6 +212,9 @@
if is_fedora; then
sudo cp $INT_CA_DIR/ca-chain.pem /usr/share/pki/ca-trust-source/anchors/devstack-chain.pem
sudo update-ca-trust
+ elif is_suse; then
+ sudo cp $INT_CA_DIR/ca-chain.pem /usr/share/pki/trust/anchors/devstack-chain.pem
+ sudo update-ca-certificates
elif is_ubuntu; then
sudo cp $INT_CA_DIR/ca-chain.pem /usr/local/share/ca-certificates/devstack-int.crt
sudo cp $ROOT_CA_DIR/cacert.pem /usr/local/share/ca-certificates/devstack-root.crt
@@ -345,7 +348,8 @@
function fix_system_ca_bundle_path {
if is_service_enabled tls-proxy; then
local capath
- capath=$(python -c $'try:\n from requests import certs\n print certs.where()\nexcept ImportError: pass')
+ local python_cmd=${1:-python}
+ capath=$($python_cmd -c $'try:\n from requests import certs\n print (certs.where())\nexcept ImportError: pass')
if [[ ! $capath == "" && ! $capath =~ ^/etc/.* && ! -L $capath ]]; then
if is_fedora; then
@@ -354,6 +358,9 @@
elif is_ubuntu; then
sudo rm -f $capath
sudo ln -s /etc/ssl/certs/ca-certificates.crt $capath
+ elif is_suse; then
+ sudo rm -f $capath
+ sudo ln -s /etc/ssl/ca-bundle.pem $capath
else
echo "Don't know how to set the CA bundle, expect the install to fail."
fi
@@ -416,6 +423,9 @@
if is_ubuntu; then
sudo a2enmod ssl
+ elif is_suse; then
+ sudo a2enmod ssl
+ sudo a2enflag SSL
elif is_fedora; then
# Fedora enables mod_ssl by default
:
@@ -522,6 +532,9 @@
LogFormat "%v %h %l %u %t \"%r\" %>s %b"
</VirtualHost>
EOF
+ if is_suse ; then
+ sudo a2enflag SSL
+ fi
for mod in ssl proxy proxy_http; do
enable_apache_mod $mod
done
diff --git a/openrc b/openrc
index 38f5c47..23c173c 100644
--- a/openrc
+++ b/openrc
@@ -72,8 +72,6 @@
GLANCE_HOST=${GLANCE_HOST:-$HOST_IP}
fi
-SERVICE_PROTOCOL=${SERVICE_PROTOCOL:-http}
-
# Identity API version
export OS_IDENTITY_API_VERSION=${IDENTITY_API_VERSION:-3}
@@ -85,7 +83,12 @@
# the user/project has access to - including nova, glance, keystone, swift, ...
# We currently recommend using the version 3 *identity api*.
#
-export OS_AUTH_URL=$KEYSTONE_AUTH_URI
+
+# If you don't have a working .stackenv, this is the backup possition
+KEYSTONE_BACKUP=$SERVICE_PROTOCOL://$SERVICE_HOST:5000
+KEYSTONE_AUTH_URI=${KEYSTONE_AUTH_URI:-$KEYSTONE_BACKUP}
+
+export OS_AUTH_URL=${OS_AUTH_URL:-$KEYSTONE_AUTH_URI}
# Currently, in order to use openstackclient with Identity API v3,
# we need to set the domain which the user and project belong to.
diff --git a/setup.cfg b/setup.cfg
index e4b2888..73d22b5 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -2,7 +2,7 @@
name = DevStack
summary = OpenStack DevStack
description-file =
- README.md
+ README.rst
author = OpenStack
author-email = openstack-dev@lists.openstack.org
home-page = http://docs.openstack.org/developer/devstack
@@ -15,6 +15,7 @@
all_files = 1
build-dir = doc/build
source-dir = doc/source
+warning-is-error = 1
[pbr]
warnerrors = True
diff --git a/stack.sh b/stack.sh
index 2c10200..59f2597 100755
--- a/stack.sh
+++ b/stack.sh
@@ -2,7 +2,7 @@
# ``stack.sh`` is an opinionated OpenStack developer installation. It
# installs and configures various combinations of **Cinder**, **Glance**,
-# **Heat**, **Horizon**, **Keystone**, **Nova**, **Neutron**, and **Swift**
+# **Horizon**, **Keystone**, **Nova**, **Neutron**, and **Swift**
# This script's options can be changed by setting appropriate environment
# variables. You can configure things like which git repositories to use,
@@ -27,11 +27,37 @@
# Make sure custom grep options don't get in the way
unset GREP_OPTIONS
-# Sanitize language settings to avoid commands bailing out
-# with "unsupported locale setting" errors.
+# NOTE(sdague): why do we explicitly set locale when running stack.sh?
+#
+# Devstack is written in bash, and many functions used throughout
+# devstack process text comming off a command (like the ip command)
+# and do transforms using grep, sed, cut, awk on the strings that are
+# returned. Many of these programs are interationalized, which is
+# great for end users, but means that the strings that devstack
+# functions depend upon might not be there in other locales. We thus
+# need to pin the world to an english basis during the runs.
+#
+# Previously we used the C locale for this, every system has it, and
+# it gives us a stable sort order. It does however mean that we
+# effectively drop unicode support.... boo! :(
+#
+# With python3 being more unicode aware by default, that's not the
+# right option. While there is a C.utf8 locale, some distros are
+# shipping it as C.UTF8 for extra confusingness. And it's support
+# isn't super clear across distros. This is made more challenging when
+# trying to support both out of the box distros, and the gate which
+# uses diskimage builder to build disk images in a different way than
+# the distros do.
+#
+# So... en_US.utf8 it is. That's existed for a very long time. It is a
+# compromise position, but it is the least worse idea at the time of
+# this comment.
+#
+# We also have to unset other variables that might impact LC_ALL
+# taking effect.
unset LANG
unset LANGUAGE
-LC_ALL=C
+LC_ALL=en_US.utf8
export LC_ALL
# Make sure umask is sane
@@ -192,7 +218,7 @@
# Warn users who aren't on an explicitly supported distro, but allow them to
# override check and attempt installation with ``FORCE=yes ./stack``
-if [[ ! ${DISTRO} =~ (xenial|yakkety|zesty|stretch|jessie|f24|f25|rhel7|kvmibm1) ]]; then
+if [[ ! ${DISTRO} =~ (xenial|yakkety|zesty|stretch|jessie|f24|f25|opensuse-42.2|rhel7|kvmibm1) ]]; then
echo "WARNING: this script has not been tested on $DISTRO"
if [[ "$FORCE" != "yes" ]]; then
die $LINENO "If you wish to run this script anyway run with FORCE=yes"
@@ -498,6 +524,11 @@
kill 2>&1 $jobs
fi
+ #Remove timing data file
+ if [ -f "$OSCWRAP_TIMER_FILE" ] ; then
+ rm "$OSCWRAP_TIMER_FILE"
+ fi
+
# Kill the last spinner process
kill_spinner
@@ -561,7 +592,7 @@
# Source project function libraries
source $TOP_DIR/lib/infra
-source $TOP_DIR/lib/oslo
+source $TOP_DIR/lib/libraries
source $TOP_DIR/lib/lvm
source $TOP_DIR/lib/horizon
source $TOP_DIR/lib/keystone
@@ -573,8 +604,7 @@
source $TOP_DIR/lib/neutron
source $TOP_DIR/lib/ldap
source $TOP_DIR/lib/dstat
-source $TOP_DIR/lib/dlm
-source $TOP_DIR/lib/os_brick
+source $TOP_DIR/lib/etcd3
# Extras Source
# --------------
@@ -748,6 +778,10 @@
if [[ "$USE_SYSTEMD" == "True" ]]; then
pip_install_gr systemd-python
+ # the default rate limit of 1000 messages / 30 seconds is not
+ # sufficient given how verbose our logging is.
+ iniset -sudo /etc/systemd/journald.conf "Journal" "RateLimitBurst" "0"
+ sudo systemctl restart systemd-journald
fi
# Virtual Environment
@@ -761,13 +795,12 @@
# Phase: pre-install
run_phase stack pre-install
+# NOTE(danms): Set global limits before installing anything
+set_systemd_override DefaultLimitNOFILE ${ULIMIT_NOFILE}
+
install_rpc_backend
restart_rpc_backend
-# NOTE(sdague): dlm install is conditional on one being enabled by configuration
-install_dlm
-configure_dlm
-
if is_service_enabled $DATABASE_BACKENDS; then
install_database
fi
@@ -779,13 +812,17 @@
install_neutron_agent_packages
fi
+if is_service_enabled etcd3; then
+ install_etcd3
+fi
+
# Check Out and Install Source
# ----------------------------
echo_summary "Installing OpenStack project source"
-# Install Oslo libraries
-install_oslo
+# Install additional libraries
+install_libs
# Install uwsgi
install_apache_uwsgi
@@ -803,11 +840,6 @@
install_neutronclient
fi
-# Install shared libraries
-if is_service_enabled cinder nova; then
- install_os_brick
-fi
-
# Setup TLS certs
if is_service_enabled tls-proxy; then
configure_CA
@@ -891,6 +923,9 @@
if is_service_enabled tls-proxy; then
fix_system_ca_bundle_path
+ if python3_enabled ; then
+ fix_system_ca_bundle_path python3
+ fi
fi
# Extras Install
@@ -907,6 +942,10 @@
pip_install_gr python-openstackclient
fi
+# Installs alias for osc so that we can collect timing for all
+# osc commands. Alias dies with stack.sh.
+install_oscwrap
+
if [[ $TRACK_DEPENDS = True ]]; then
$DEST/.venv/bin/pip freeze > $DEST/requires-post-pip
if ! diff -Nru $DEST/requires-pre-pip $DEST/requires-post-pip > $DEST/requires.diff; then
@@ -1036,6 +1075,13 @@
# A better kind of sysstat, with the top process per time slice
start_dstat
+# Etcd
+# -----
+
+# etcd is a distributed key value store that provides a reliable way to store data across a cluster of machines
+if is_service_enabled etcd3; then
+ start_etcd3
+fi
# Keystone
# --------
@@ -1249,8 +1295,13 @@
fi
# Create a randomized default value for the key manager's fixed_key
+# NOTE(lyarwood): This is currently set to 36 as a workaround to the following
+# libvirt bug that incorrectly pads passphrases that are a multiple of 16 bytes
+# in length.
+# Unable to use LUKS passphrase that is exactly 16 bytes long
+# https://bugzilla.redhat.com/show_bug.cgi?id=1447297
if is_service_enabled nova; then
- iniset $NOVA_CONF key_manager fixed_key $(generate_hex_string 32)
+ iniset $NOVA_CONF key_manager fixed_key $(generate_hex_string 36)
fi
# Launch the nova-api and wait for it to answer before continuing
@@ -1282,6 +1333,13 @@
$NOVA_BIN_DIR/nova-manage --config-file $NM_CONF floating create --ip_range=$TEST_FLOATING_RANGE --pool=$TEST_FLOATING_POOL
fi
+# Start placement before any of the service that are likely to want
+# to use it to manage resource providers.
+if is_service_enabled placement; then
+ echo_summary "Starting Placement"
+ start_placement
+fi
+
if is_service_enabled neutron; then
start_neutron
fi
@@ -1296,10 +1354,6 @@
start_nova
create_flavors
fi
-if is_service_enabled placement; then
- echo_summary "Starting Placement"
- start_placement
-fi
if is_service_enabled cinder; then
echo_summary "Starting Cinder"
start_cinder
@@ -1328,10 +1382,6 @@
USERRC_PARAMS="$USERRC_PARAMS --os-cacert $SSL_BUNDLE_FILE"
fi
- if [[ "$HEAT_STANDALONE" = "True" ]]; then
- USERRC_PARAMS="$USERRC_PARAMS --heat-url http://$HEAT_API_HOST:$HEAT_API_PORT/v1"
- fi
-
$TOP_DIR/tools/create_userrc.sh $USERRC_PARAMS
fi
diff --git a/stackrc b/stackrc
index 1300f45..50f7c89 100644
--- a/stackrc
+++ b/stackrc
@@ -65,7 +65,7 @@
# Dashboard
ENABLED_SERVICES+=,horizon
# Additional services
- ENABLED_SERVICES+=,rabbit,tempest,mysql,dstat
+ ENABLED_SERVICES+=,rabbit,tempest,mysql,etcd3,dstat
fi
# Global toggle for enabling services under mod_wsgi. If this is set to
@@ -100,11 +100,9 @@
if [[ "$USER_UNITS" == "True" ]]; then
SYSTEMD_DIR="$HOME/.local/share/systemd/user"
SYSTEMCTL="systemctl --user"
- JOURNALCTL_F="journalctl -f -o short-precise --user-unit"
else
SYSTEMD_DIR="/etc/systemd/system"
SYSTEMCTL="sudo systemctl"
- JOURNALCTL_F="journalctl -f -o short-precise --unit"
fi
@@ -343,6 +341,11 @@
GITREPO["python-brick-cinderclient-ext"]=${BRICK_CINDERCLIENT_REPO:-${GIT_BASE}/openstack/python-brick-cinderclient-ext.git}
GITBRANCH["python-brick-cinderclient-ext"]=${BRICK_CINDERCLIENT_BRANCH:-master}
+# python barbican client library
+GITREPO["python-barbicanclient"]=${BARBICANCLIENT_REPO:-${GIT_BASE}/openstack/python-barbicanclient.git}
+GITBRANCH["python-barbicanclient"]=${BARBICANCLIENT_BRANCH:-master}
+GITDIR["python-barbicanclient"]=$DEST/python-barbicanclient
+
# python glance client library
GITREPO["python-glanceclient"]=${GLANCECLIENT_REPO:-${GIT_BASE}/openstack/python-glanceclient.git}
GITBRANCH["python-glanceclient"]=${GLANCECLIENT_BRANCH:-master}
@@ -518,14 +521,6 @@
GITREPO["glance_store"]=${GLANCE_STORE_REPO:-${GIT_BASE}/openstack/glance_store.git}
GITBRANCH["glance_store"]=${GLANCE_STORE_BRANCH:-master}
-# heat-cfntools server agent
-HEAT_CFNTOOLS_REPO=${HEAT_CFNTOOLS_REPO:-${GIT_BASE}/openstack/heat-cfntools.git}
-HEAT_CFNTOOLS_BRANCH=${HEAT_CFNTOOLS_BRANCH:-master}
-
-# heat example templates and elements
-HEAT_TEMPLATES_REPO=${HEAT_TEMPLATES_REPO:-${GIT_BASE}/openstack/heat-templates.git}
-HEAT_TEMPLATES_BRANCH=${HEAT_TEMPLATES_BRANCH:-master}
-
# django openstack_auth library
GITREPO["django_openstack_auth"]=${HORIZONAUTH_REPO:-${GIT_BASE}/openstack/django_openstack_auth.git}
GITBRANCH["django_openstack_auth"]=${HORIZONAUTH_BRANCH:-master}
@@ -580,6 +575,10 @@
GITBRANCH["neutron-lib"]=${NEUTRON_LIB_BRANCH:-master}
GITDIR["neutron-lib"]=$DEST/neutron-lib
+# os-traits library for resource provider traits in the placement service
+GITREPO["os-traits"]=${OS_TRAITS_REPO:-${GIT_BASE}/openstack/os-traits.git}
+GITBRANCH["os-traits"]=${OS_TRAITS_BRANCH:-master}
+
##################
#
# TripleO / Heat Agent Components
@@ -617,7 +616,7 @@
# a websockets/html5 or flash powered VNC console for vm instances
NOVNC_REPO=${NOVNC_REPO:-https://github.com/kanaka/noVNC.git}
-NOVNC_BRANCH=${NOVNC_BRANCH:-master}
+NOVNC_BRANCH=${NOVNC_BRANCH:-stable/v0.6}
# a websockets/html5 or flash powered SPICE console for vm instances
SPICE_REPO=${SPICE_REPO:-http://anongit.freedesktop.org/git/spice/spice-html5.git}
@@ -665,6 +664,8 @@
;;
esac
+# By default, devstack will use Ubuntu Cloud Archive.
+ENABLE_UBUNTU_CLOUD_ARCHIVE=$(trueorfalse True ENABLE_UBUNTU_CLOUD_ARCHIVE)
# Images
# ------
@@ -933,6 +934,9 @@
# ``LOGDIR`` is always set at this point so it is not useful as a 'enable' for service logs
# ``SCREEN_LOGDIR`` may be set, it is useful to enable the compat symlinks
+# System-wide ulimit file descriptors override
+ULIMIT_NOFILE=${ULIMIT_NOFILE:-2048}
+
# Local variables:
# mode: shell-script
# End:
diff --git a/tests/test_libs_from_pypi.sh b/tests/test_libs_from_pypi.sh
index 608ef6a..5b4ff32 100755
--- a/tests/test_libs_from_pypi.sh
+++ b/tests/test_libs_from_pypi.sh
@@ -39,11 +39,11 @@
ALL_LIBS+=" python-openstackclient osc-lib os-client-config oslo.rootwrap"
ALL_LIBS+=" oslo.i18n oslo.utils python-openstacksdk python-swiftclient"
ALL_LIBS+=" python-neutronclient tooz ceilometermiddleware oslo.policy"
-ALL_LIBS+=" debtcollector os-brick automaton futurist oslo.service"
+ALL_LIBS+=" debtcollector os-brick os-traits automaton futurist oslo.service"
ALL_LIBS+=" oslo.cache oslo.reports osprofiler cursive"
ALL_LIBS+=" keystoneauth ironic-lib neutron-lib oslo.privsep"
ALL_LIBS+=" diskimage-builder os-vif python-brick-cinderclient-ext"
-ALL_LIBS+=" castellan"
+ALL_LIBS+=" castellan python-barbicanclient"
# Generate the above list with
# echo ${!GITREPO[@]}
diff --git a/tests/test_refs.sh b/tests/test_refs.sh
index bccca5d..65848cd 100755
--- a/tests/test_refs.sh
+++ b/tests/test_refs.sh
@@ -15,7 +15,7 @@
echo "Ensuring we don't have crazy refs"
-REFS=`grep BRANCH stackrc | grep -v -- '-master'`
+REFS=`grep BRANCH stackrc | grep -v -- '-master' | grep -v 'NOVNC_BRANCH'`
rc=$?
if [[ $rc -eq 0 ]]; then
echo "Branch defaults must be master. Found:"
diff --git a/tools/dstat.sh b/tools/dstat.sh
index ae7306e..01c6d9b 100755
--- a/tools/dstat.sh
+++ b/tools/dstat.sh
@@ -13,10 +13,10 @@
LOGDIR=$1
# Command line arguments for primary DStat process.
-DSTAT_OPTS="-tcmndrylpg --top-cpu-adv --top-io-adv --top-mem --swap"
+DSTAT_OPTS="-tcmndrylpg --top-cpu-adv --top-io-adv --top-mem --swap --tcp"
# Command-line arguments for secondary background DStat process.
-DSTAT_CSV_OPTS="-tcmndrylpg --output $LOGDIR/dstat-csv.log"
+DSTAT_CSV_OPTS="-tcmndrylpg --tcp --output $LOGDIR/dstat-csv.log"
# Execute and background the secondary dstat process and discard its output.
dstat $DSTAT_CSV_OPTS >& /dev/null &
diff --git a/tools/fixup_stuff.sh b/tools/fixup_stuff.sh
index f3ba702..0b78bde 100755
--- a/tools/fixup_stuff.sh
+++ b/tools/fixup_stuff.sh
@@ -72,7 +72,10 @@
# We've found that Libvirt on Xenial is flaky and crashes enough to be
# a regular top e-r bug. Opt into Ubuntu Cloud Archive if on Xenial to
# get newer Libvirt.
-if [[ "$DISTRO" = "xenial" ]]; then
+# Make it possible to switch this based on an environment variable as
+# libvirt 2.5.0 doesn't handle nested virtualization quite well and this
+# is required for the trove development environment.
+if [[ "${ENABLE_UBUNTU_CLOUD_ARCHIVE}" == "True" && "$DISTRO" = "xenial" ]]; then
# This pulls in apt-add-repository
install_package "software-properties-common"
# Use UCA for newer libvirt. Should give us libvirt 2.5.0.
@@ -82,15 +85,17 @@
source /etc/ci/mirror_info.sh
sudo apt-add-repository -y "deb $NODEPOOL_UCA_MIRROR xenial-updates/ocata main"
-
- # Disable use of libvirt wheel here as presence of mirror implies
- # presence of cached wheel build against older libvirt binary.
- # TODO(clarkb) figure out how to use wheel again.
- sudo bash -c 'echo "no-binary = libvirt-python" >> /etc/pip.conf'
else
# Otherwise use upstream UCA
sudo add-apt-repository -y cloud-archive:ocata
fi
+
+ # Disable use of libvirt wheel since a cached wheel build might be
+ # against older libvirt binary. Particularly a problem if using
+ # the openstack wheel mirrors, but can hit locally too.
+ # TODO(clarkb) figure out how to use upstream wheel again.
+ iniset -sudo /etc/pip.conf "global" "no-binary" "libvirt-python"
+
# Force update our APT repos, since we added UCA above.
REPOS_UPDATED=False
apt_get_update
diff --git a/tox.ini b/tox.ini
index 55a06d0..cc7c544 100644
--- a/tox.ini
+++ b/tox.ini
@@ -37,8 +37,8 @@
deps =
Pygments
docutils
- sphinx>=1.1.2,<1.2
- pbr>=0.6,!=0.7,<1.0
+ sphinx>=1.5.1,<1.6.1
+ pbr>=2.0.0,!=2.1.0
oslosphinx
nwdiag
blockdiag
@@ -52,8 +52,8 @@
[testenv:venv]
deps =
- pbr>=0.6,!=0.7,<1.0
- sphinx>=1.1.2,<1.2
+ pbr>=2.0.0,!=2.1.0
+ sphinx>=1.5.1,<1.6.1
oslosphinx
blockdiag
sphinxcontrib-blockdiag
diff --git a/unstack.sh b/unstack.sh
index 485fed7..77a151f 100755
--- a/unstack.sh
+++ b/unstack.sh
@@ -69,7 +69,7 @@
source $TOP_DIR/lib/neutron
source $TOP_DIR/lib/ldap
source $TOP_DIR/lib/dstat
-source $TOP_DIR/lib/dlm
+source $TOP_DIR/lib/etcd3
# Extras Source
# --------------
@@ -162,6 +162,11 @@
cleanup_neutron
fi
+if is_service_enabled etcd3; then
+ stop_etcd3
+ cleanup_etcd3
+fi
+
if is_service_enabled dstat; then
stop_dstat
fi