Merge "Use proper python when configuring certs"
diff --git a/.gitignore b/.gitignore
index d1781bc..7967e14 100644
--- a/.gitignore
+++ b/.gitignore
@@ -23,6 +23,7 @@
files/pip-*
files/get-pip.py*
files/ir-deploy*
+files/ironic-inspector*
local.conf
local.sh
localrc
diff --git a/HACKING.rst b/HACKING.rst
index b76cb6c..fc67f09 100644
--- a/HACKING.rst
+++ b/HACKING.rst
@@ -322,7 +322,7 @@
Review Criteria
-===============
+---------------
There are some broad criteria that will be followed when reviewing
your change
@@ -364,3 +364,26 @@
* **Reviewers** -- please see ``MAINTAINERS.rst`` for a list of people
that should be added to reviews of various sub-systems.
+
+
+Making Changes, Testing, and CI
+-------------------------------
+
+Changes to Devstack are tested by automated continuous integration jobs
+that run on a variety of Linux Distros using a handful of common
+configurations. What this means is that every change to Devstack is
+self testing. One major benefit of this is that developers do not
+typically need to add new non voting test jobs to add features to
+Devstack. Instead the features can be added, then if testing passes
+with the feature enabled the change is ready to merge (pending code
+review).
+
+A concrete example of this was the switch from screen based service
+management to systemd based service management. No new jobs were
+created for this. Instead the features were added to devstack, tested
+locally and in CI using a change that enabled the feature, then once
+the enabling change was passing and the new behavior communicated and
+documented it was merged.
+
+Using this process has been proven to be effective and leads to
+quicker implementation of desired features.
diff --git a/README.md b/README.rst
similarity index 88%
rename from README.md
rename to README.rst
index ff5598b..dfa68b9 100644
--- a/README.md
+++ b/README.rst
@@ -1,6 +1,7 @@
DevStack is a set of scripts and utilities to quickly deploy an OpenStack cloud.
-# Goals
+Goals
+=====
* To quickly build dev OpenStack environments in a clean Ubuntu or Fedora
environment
@@ -20,12 +21,13 @@
networking configuration. We strongly recommend that you run `stack.sh`
in a clean and disposable vm when you are first getting started.
-# Versions
+Versions
+========
The DevStack master branch generally points to trunk versions of OpenStack
components. For older, stable versions, look for branches named
stable/[release] in the DevStack repo. For example, you can do the
-following to create a Newton OpenStack cloud:
+following to create a Newton OpenStack cloud::
git checkout stable/newton
./stack.sh
@@ -38,7 +40,8 @@
GLANCE_REPO=git://git.openstack.org/openstack/glance.git
GLANCE_BRANCH=milestone-proposed
-# Start A Dev Cloud
+Start A Dev Cloud
+=================
Installing in a dedicated disposable VM is safer than installing on your
dev machine! Plus you can pick one of the supported Linux distros for
@@ -54,14 +57,15 @@
* Keystone: http://myhost:5000/v2.0/
We also provide an environment file that you can use to interact with your
-cloud via CLI:
+cloud via CLI::
# source openrc file to load your environment with OpenStack CLI creds
. openrc
# list instances
- nova list
+ openstack server list
-# DevStack Execution Environment
+DevStack Execution Environment
+==============================
DevStack runs rampant over the system it runs on, installing things and
uninstalling other things. Running this on a system you care about is a recipe
@@ -81,10 +85,12 @@
it runs under. Many people simply use their usual login (the default
'ubuntu' login on a UEC image for example).
-# Customizing
+Customizing
+===========
DevStack can be extensively configured via the configuration file
`local.conf`. It is likely that you will need to provide and modify
this file if you want anything other than the most basic setup. Start
-by reading the [configuration guide](doc/source/configuration.rst) for
-details of the configuration file and the many available options.
+by reading the `configuration guide
+<https://docs.openstack.org/developer/devstack/configuration.html>_`
+for details of the configuration file and the many available options.
diff --git a/clean.sh b/clean.sh
index 90b21eb..ef38fbf 100755
--- a/clean.sh
+++ b/clean.sh
@@ -64,13 +64,8 @@
done
fi
-# See if there is anything running...
-# need to adapt when run_service is merged
-SESSION=$(screen -ls | awk '/[0-9].stack/ { print $1 }')
-if [[ -n "$SESSION" ]]; then
- # Let unstack.sh do its thing first
- $TOP_DIR/unstack.sh --all
-fi
+# Let unstack.sh do its thing first
+$TOP_DIR/unstack.sh --all
# Run extras
# ==========
diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst
index 53ae82f..66b8702 100644
--- a/doc/source/configuration.rst
+++ b/doc/source/configuration.rst
@@ -278,43 +278,22 @@
LOGDAYS=1
-The some of the project logs (Nova, Cinder, etc) will be colorized by
-default (if ``SYSLOG`` is not set below); this can be turned off by
-setting ``LOG_COLOR`` to ``False``.
-
- ::
+Some coloring is used during the DevStack runs to make it easier to
+see what is going on. This can be disabled with::
LOG_COLOR=False
Logging the Service Output
~~~~~~~~~~~~~~~~~~~~~~~~~~
-DevStack will log the ``stdout`` output of the services it starts.
-When using ``screen`` this logs the output in the screen windows to a
-file. Without ``screen`` this simply redirects stdout of the service
-process to a file in ``LOGDIR``.
+By default, services run under ``systemd`` and are natively logging to
+the systemd journal.
- ::
+To query the logs use the ``journalctl`` command, such as::
- LOGDIR=$DEST/logs
+ journalctl --unit devstack@*
-Note the use of ``DEST`` to locate the main install directory; this
-is why we suggest setting it in ``local.conf``.
-
-Enabling Syslog
-~~~~~~~~~~~~~~~
-
-Logging all services to a single syslog can be convenient. Enable
-syslogging by setting ``SYSLOG`` to ``True``. If the destination log
-host is not localhost ``SYSLOG_HOST`` and ``SYSLOG_PORT`` can be used
-to direct the message stream to the log host.
-
- ::
-
- SYSLOG=True
- SYSLOG_HOST=$HOST_IP
- SYSLOG_PORT=516
-
+More examples can be found in :ref:`journalctl-examples`.
Example Logging Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -326,7 +305,6 @@
[[local|localrc]]
DEST=/opt/stack/
- LOGDIR=$DEST/logs
LOGFILE=$LOGDIR/stack.sh.log
LOG_COLOR=False
@@ -587,9 +565,7 @@
Swift is disabled by default. When enabled, it is configured with
only one replica to avoid being IO/memory intensive on a small
-VM. When running with only one replica the account, container and
-object services will run directly in screen. The others services like
-replicator, updaters or auditor runs in background.
+VM.
If you would like to enable Swift you can add this to your ``localrc``
section:
@@ -630,32 +606,9 @@
act as a S3 endpoint for Keystone so effectively replacing the
``nova-objectstore``.
-Only Swift proxy server is launched in the screen session all other
+Only Swift proxy server is launched in the systemd system all other
services are started in background and managed by ``swift-init`` tool.
-Heat
-~~~~
-
-Heat is disabled by default (see ``stackrc`` file). To enable it
-explicitly you'll need the following settings in your ``localrc``
-section
-
-::
-
- enable_service heat h-api h-api-cfn h-api-cw h-eng
-
-Heat can also run in standalone mode, and be configured to orchestrate
-on an external OpenStack cloud. To launch only Heat in standalone mode
-you'll need the following settings in your ``localrc`` section
-
-::
-
- disable_all_services
- enable_service rabbit mysql heat h-api h-api-cfn h-api-cw h-eng
- HEAT_STANDALONE=True
- KEYSTONE_SERVICE_HOST=...
- KEYSTONE_AUTH_HOST=...
-
Tempest
~~~~~~~
diff --git a/doc/source/development.rst b/doc/source/development.rst
index 776ac6c..957de9b 100644
--- a/doc/source/development.rst
+++ b/doc/source/development.rst
@@ -8,56 +8,33 @@
Inspecting Services
===================
-By default most services in DevStack are running in a `screen
-<https://www.gnu.org/software/screen/manual/screen.html>`_
-session.
+By default most services in DevStack are running as `systemd` units
+named `devstack@$servicename.service`. You can see running services
+with.
.. code-block:: bash
- os3:~> screen -list
- There is a screen on:
- 28994.stack (08/10/2016 09:01:33 PM) (Detached)
- 1 Socket in /var/run/screen/S-sdague.
+ sudo systemctl status "devstack@*"
-You can attach to this screen session using ``screen -r`` which gives
-you a view of the services in action.
-
-.. image:: assets/images/screen_session_1.png
- :width: 100%
-
-Basic Screen Commands
----------------------
-
-The following minimal commands will be useful to using screen:
-
-* ``ctrl-a n`` - go to next window. Next is assumed to be right of
- current window.
-* ``ctrl-a p`` - go to previous window. Previous is assumed to be left
- of current window.
-* ``ctrl-a [`` - entry copy/scrollback mode. This allows you to
- navigate back through the logs with the up arrow.
-* ``ctrl-a d`` - detach from screen. Gets you back to a normal
- terminal, while leaving everything running.
-
-For more about using screen, see the excellent `screen manual
-<https://www.gnu.org/software/screen/manual/screen.html>`_.
+To learn more about the basics of systemd, see :doc:`/systemd`
Patching a Service
==================
If you want to make a quick change to a running service the easiest
-way to do this is:
+way to do that is to change the code directly in /opt/stack/$service
+and then restart the affected daemons.
-* attach to screen
-* navigate to the window in question
-* ``ctrl-c`` to kill the service
-* make appropriate changes to the code
-* ``up arrow`` in the screen window to display the command used to run
- that service
-* ``enter`` to restart the service
+.. code-block:: bash
-This works for services, except those running under Apache (currently
-just ``keystone`` by default).
+ sudo systemctl restart devstack@n-cpu.service
+
+If your change impacts more than one daemon you can restart by
+wildcard as well.
+
+.. code-block:: bash
+
+ sudo systemctl restart "devstack@n-*"
.. warning::
@@ -102,14 +79,6 @@
NOVA_BRANCH=refs/changes/10/353710/1
-Testing Changes to Apache Based Services
-========================================
-
-When testing changes to Apache based services, such as ``keystone``,
-you can either use the Testing a Patch Series approach above, or make
-changes in the code tree and issue an apache restart.
-
-
Testing Changes to Libraries
============================
@@ -132,9 +101,17 @@
OSLOPOLICY_REPO=/home/sdague/oslo.policy
OSLOPOLICY_BRANCH=better_exception
-Because libraries are used by many services, library changes really
-need to go through a full ``./unstack.sh && ./stack.sh`` to see your
-changes in action.
+As libraries are not installed `editable` by pip, after you make any
+local changes you will need to:
-To figure out the repo / branch names for every library that's
-supported, you'll need to read the devstack source.
+* cd to top of library path
+* sudo pip install -U .
+* restart all services you want to use the new library
+
+You can do that with wildcards such as
+
+.. code-block:: bash
+
+ sudo systemctl restart "devstack@n-*"
+
+which will restart all nova services.
diff --git a/doc/source/faq.rst b/doc/source/faq.rst
index f03304f..a186336 100644
--- a/doc/source/faq.rst
+++ b/doc/source/faq.rst
@@ -41,8 +41,9 @@
~~~~~~~~~~~~~~~~~~~~~
Unlike packages, DevStack leaves your cloud ready to develop -
-checkouts of the code and services running in screen. However, many
-people are doing the hard work of packaging and recipes for production
+checkouts of the code and services running locally under systemd,
+making it easy to hack on and test new patches. However, many people
+are doing the hard work of packaging and recipes for production
deployments.
Why isn't $MY\_FAVORITE\_DISTRO supported?
@@ -158,16 +159,6 @@
often good enough for a single-node installation. And in an extreme
case, use ``clean.sh`` to eradicate it and try again.
-Configure ``local.conf`` thusly:
-
- ::
-
- [[local|localrc]]
- HEAT_STANDALONE=True
- ENABLED_SERVICES=rabbit,mysql,heat,h-api,h-api-cfn,h-api-cw,h-eng
- KEYSTONE_SERVICE_HOST=<keystone-host>
- KEYSTONE_AUTH_HOST=<keystone-host>
-
Why are my configuration changes ignored?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/source/guides/devstack-with-lbaas-v2.rst b/doc/source/guides/devstack-with-lbaas-v2.rst
index 21bea99..4ed64bf 100644
--- a/doc/source/guides/devstack-with-lbaas-v2.rst
+++ b/doc/source/guides/devstack-with-lbaas-v2.rst
@@ -45,7 +45,7 @@
# Horizon
ENABLED_SERVICES+=,horizon
# Nova
- ENABLED_SERVICES+=,n-api,n-crt,n-cpu,n-cond,n-sch
+ ENABLED_SERVICES+=,n-api,n-cpu,n-cond,n-sch
# Glance
ENABLED_SERVICES+=,g-api,g-reg
# Neutron
diff --git a/doc/source/index.rst b/doc/source/index.rst
index cbd6971..47087c5 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -142,3 +142,12 @@
Get :doc:`the big picture <overview>` of what we are trying to do
with devstack, and help us by :doc:`contributing to the project
<hacking>`.
+
+Contents
+--------
+
+.. toctree::
+ :glob:
+ :maxdepth: 2
+
+ *
diff --git a/doc/source/overview.rst b/doc/source/overview.rst
index d245035..c07a8e6 100644
--- a/doc/source/overview.rst
+++ b/doc/source/overview.rst
@@ -20,11 +20,11 @@
*The OpenStack Technical Committee (TC) has defined the current CI
strategy to include the latest Ubuntu release and the latest RHEL
-release (for Python 2.6 testing).*
+release.*
- Ubuntu: current LTS release plus current development release
- Fedora: current release plus previous release
-- RHEL: current major release
+- RHEL/Centos: current major release
- Other OS platforms may continue to be included but the maintenance of
those platforms shall not be assumed simply due to their presence.
Having a listed point-of-contact for each additional OS will greatly
@@ -38,7 +38,6 @@
*As packaged by the host OS*
- MySQL
-- PostgreSQL
Queues
------
@@ -46,7 +45,6 @@
*As packaged by the host OS*
- Rabbit
-- Qpid
Web Server
----------
@@ -58,9 +56,6 @@
OpenStack Network
-----------------
-*Defaults to nova network, optionally use neutron*
-
-- Nova Network: FlatDHCP
- Neutron: A basic configuration approximating the original FlatDHCP
mode using linuxbridge or OpenVSwitch.
@@ -68,9 +63,8 @@
--------
The default services configured by DevStack are Identity (keystone),
-Object Storage (swift), Image Service (glance), Block Storage (cinder),
-Compute (nova), Networking (nova), Dashboard (horizon), Orchestration
-(heat)
+Object Storage (swift), Image Service (glance), Block Storage
+(cinder), Compute (nova), Networking (neutron), Dashboard (horizon)
Additional services not included directly in DevStack can be tied in to
``stack.sh`` using the :doc:`plugin mechanism <plugins>` to call
@@ -80,8 +74,7 @@
-------------------
- single node
-- multi-node is not tested regularly by the core team, and even then
- only minimal configurations are reviewed
+- multi-node configurations as are tested by the gate
Exercises
---------
diff --git a/doc/source/site-map.rst b/doc/source/site-map.rst
deleted file mode 100644
index 801fc66..0000000
--- a/doc/source/site-map.rst
+++ /dev/null
@@ -1,23 +0,0 @@
-:orphan:
-
-.. the TOC on the front page actually makes the document a lot more
- confusing. This lets us bury a toc which we can link in when
- appropriate.
-
-==========
- Site Map
-==========
-
-.. toctree::
- :glob:
- :maxdepth: 3
-
- overview
- configuration
- networking
- plugins
- plugin-registry
- faq
- development
- hacking
- guides
diff --git a/SYSTEMD.rst b/doc/source/systemd.rst
similarity index 81%
rename from SYSTEMD.rst
rename to doc/source/systemd.rst
index 729fdf4..60a7719 100644
--- a/SYSTEMD.rst
+++ b/doc/source/systemd.rst
@@ -2,12 +2,7 @@
Using Systemd in DevStack
===========================
-.. note::
-
- This is an in progress document as we work out the way forward here
- with DevStack and systemd.
-
-DevStack can be run with all the services as systemd unit
+By default DevStack is run with all the services as systemd unit
files. Systemd is now the default init system for nearly every Linux
distro, and systemd encodes and solves many of the problems related to
poorly running processes.
@@ -25,12 +20,6 @@
than one service, and needing to restart a bunch of services for that
to take effect.
-To enable this add the following to your local.conf::
-
- USE_SYSTEMD=True
-
-
-
Unit Structure
==============
@@ -42,8 +31,9 @@
code is left in place in case we can switch back later.
All DevStack user units are created as a part of the DevStack slice
-given the name ``devstack@$servicename.service``. This lets us do
-certain operations at the slice level.
+given the name ``devstack@$servicename.service``. This makes it easy
+to understand which services are part of the devstack run, and lets us
+disable / stop them in a single command.
Manipulating Units
==================
@@ -89,6 +79,8 @@
We'll eventually make the unit names a bit more meaningful so that
it's easier to understand what you are restarting.
+.. _journalctl-examples:
+
Querying Logs
=============
@@ -117,6 +109,17 @@
journalctl -f -o short-precise --unit devstack@n-cpu.service
+By default, journalctl strips out "unprintable" characters, including
+ASCII color codes. To keep the color codes (which can be interpreted by
+an appropriate terminal/pager - e.g. ``less``, the default)::
+
+ journalctl -a --unit devstack@n-cpu.service
+
+When outputting to the terminal using the default pager, long lines
+appear to be truncated, but horizontal scrolling is supported via the
+left/right arrow keys.
+
+See ``man 1 journalctl`` for more.
Known Issues
============
@@ -126,7 +129,7 @@
into the ``systemd`` namespace, which can cause some issues.
- ``systemd-python`` - this is the upstream maintained library, it has
- a version number like systemd itself (currently ``233``). This is
+ a version number like systemd itself (currently ``234``). This is
the one you want.
- ``systemd`` - a python 3 only library, not what you want.
- ``python-systemd`` - another library you don't want. Installing it
@@ -145,27 +148,6 @@
Future Work
===========
-oslo.log journald
------------------
-
-Journald has an extremely rich mechanism for direct logging including
-structured metadata. We should enhance oslo.log to take advantage of
-that. It would let us do things like::
-
- journalctl REQUEST_ID=......
-
- journalctl INSTANCE_ID=......
-
-And get all lines related to the request id or instance id. (Note:
-this work has been started at https://review.openstack.org/#/c/451525/)
-
-log colorizing
---------------
-
-We lose log colorization through this process. We might want to build
-a custom colorizer that we could run journalctl output through
-optionally for people.
-
user units
----------
diff --git a/files/debs/general b/files/debs/general
index 20490c6..1dde03b 100644
--- a/files/debs/general
+++ b/files/debs/general
@@ -17,6 +17,7 @@
libmysqlclient-dev # MySQL-python
libpq-dev # psycopg2
libssl-dev # for pyOpenSSL
+libsystemd-dev # for systemd-python
libxml2-dev # lxml
libxslt1-dev # lxml
libyaml-dev
@@ -26,10 +27,8 @@
pkg-config
psmisc
python2.7
-python3-systemd
python-dev
python-gdbm # needed for testr
-python-systemd
screen
tar
tcpdump
diff --git a/files/debs/n-api b/files/debs/n-api
deleted file mode 100644
index 0928cd5..0000000
--- a/files/debs/n-api
+++ /dev/null
@@ -1 +0,0 @@
-fping
diff --git a/files/debs/zookeeper b/files/debs/zookeeper
deleted file mode 100644
index f41b559..0000000
--- a/files/debs/zookeeper
+++ /dev/null
@@ -1 +0,0 @@
-zookeeperd
diff --git a/files/rpms-suse/n-api b/files/rpms-suse/n-api
index af5ac2f..0f08daa 100644
--- a/files/rpms-suse/n-api
+++ b/files/rpms-suse/n-api
@@ -1,2 +1 @@
-fping
python-dateutil
diff --git a/files/rpms/general b/files/rpms/general
index 106aa6a..1393d18 100644
--- a/files/rpms/general
+++ b/files/rpms/general
@@ -29,7 +29,7 @@
python-devel
redhat-rpm-config # missing dep for gcc hardening flags, see rhbz#1217376
screen
-systemd-python
+systemd-devel # for systemd-python
tar
tcpdump
unzip
diff --git a/files/rpms/n-api b/files/rpms/n-api
deleted file mode 100644
index 0928cd5..0000000
--- a/files/rpms/n-api
+++ /dev/null
@@ -1 +0,0 @@
-fping
diff --git a/files/rpms/zookeeper b/files/rpms/zookeeper
deleted file mode 100644
index 1bfac53..0000000
--- a/files/rpms/zookeeper
+++ /dev/null
@@ -1 +0,0 @@
-zookeeper
diff --git a/files/zookeeper/environment b/files/zookeeper/environment
deleted file mode 100644
index afa2d2f..0000000
--- a/files/zookeeper/environment
+++ /dev/null
@@ -1,36 +0,0 @@
-#
-# (C) Copyright 2015 Hewlett Packard Enterprise Development Company LP
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-# implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Modified from http://packages.ubuntu.com/saucy/zookeeperd
-NAME=zookeeper
-ZOOCFGDIR=/etc/zookeeper/conf
-
-# seems, that log4j requires the log4j.properties file to be in the classpath
-CLASSPATH="$ZOOCFGDIR:/usr/share/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/zookeeper.jar"
-
-ZOOCFG="$ZOOCFGDIR/zoo.cfg"
-ZOO_LOG_DIR=/var/log/zookeeper
-USER=$NAME
-GROUP=$NAME
-PIDDIR=/var/run/$NAME
-PIDFILE=$PIDDIR/$NAME.pid
-SCRIPTNAME=/etc/init.d/$NAME
-JAVA=/usr/bin/java
-ZOOMAIN="org.apache.zookeeper.server.quorum.QuorumPeerMain"
-ZOO_LOG4J_PROP="INFO,ROLLINGFILE"
-JMXLOCALONLY=false
-JAVA_OPTS=""
diff --git a/files/zookeeper/log4j.properties b/files/zookeeper/log4j.properties
deleted file mode 100644
index 6c45a4a..0000000
--- a/files/zookeeper/log4j.properties
+++ /dev/null
@@ -1,69 +0,0 @@
-#
-# (C) Copyright 2015 Hewlett Packard Enterprise Development Company LP
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-# implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# From http://packages.ubuntu.com/saucy/zookeeperd
-
-# ZooKeeper Logging Configuration
-#
-
-# Format is "<default threshold> (, <appender>)+
-
-log4j.rootLogger=${zookeeper.root.logger}
-
-# Example: console appender only
-# log4j.rootLogger=INFO, CONSOLE
-
-# Example with rolling log file
-#log4j.rootLogger=DEBUG, CONSOLE, ROLLINGFILE
-
-# Example with rolling log file and tracing
-#log4j.rootLogger=TRACE, CONSOLE, ROLLINGFILE, TRACEFILE
-
-#
-# Log INFO level and above messages to the console
-#
-log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
-log4j.appender.CONSOLE.Threshold=INFO
-log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
-log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n
-
-#
-# Add ROLLINGFILE to rootLogger to get log file output
-# Log DEBUG level and above messages to a log file
-log4j.appender.ROLLINGFILE=org.apache.log4j.RollingFileAppender
-log4j.appender.ROLLINGFILE.Threshold=WARN
-log4j.appender.ROLLINGFILE.File=${zookeeper.log.dir}/zookeeper.log
-
-# Max log file size of 10MB
-log4j.appender.ROLLINGFILE.MaxFileSize=10MB
-# uncomment the next line to limit number of backup files
-#log4j.appender.ROLLINGFILE.MaxBackupIndex=10
-
-log4j.appender.ROLLINGFILE.layout=org.apache.log4j.PatternLayout
-log4j.appender.ROLLINGFILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n
-
-
-#
-# Add TRACEFILE to rootLogger to get log file output
-# Log DEBUG level and above messages to a log file
-log4j.appender.TRACEFILE=org.apache.log4j.FileAppender
-log4j.appender.TRACEFILE.Threshold=TRACE
-log4j.appender.TRACEFILE.File=${zookeeper.log.dir}/zookeeper_trace.log
-
-log4j.appender.TRACEFILE.layout=org.apache.log4j.PatternLayout
-### Notice we are including log4j's NDC here (%x)
-log4j.appender.TRACEFILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L][%x] - %m%n
diff --git a/files/zookeeper/myid b/files/zookeeper/myid
deleted file mode 100644
index c227083..0000000
--- a/files/zookeeper/myid
+++ /dev/null
@@ -1 +0,0 @@
-0
\ No newline at end of file
diff --git a/files/zookeeper/zoo.cfg b/files/zookeeper/zoo.cfg
deleted file mode 100644
index b8f5582..0000000
--- a/files/zookeeper/zoo.cfg
+++ /dev/null
@@ -1,74 +0,0 @@
-#
-# (C) Copyright 2015 Hewlett Packard Enterprise Development Company LP
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-# implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html
-
-# The number of milliseconds of each tick
-tickTime=2000
-# The number of ticks that the initial
-# synchronization phase can take
-initLimit=10
-# The number of ticks that can pass between
-# sending a request and getting an acknowledgement
-syncLimit=5
-# the directory where the snapshot is stored.
-dataDir=/var/lib/zookeeper
-# Place the dataLogDir to a separate physical disc for better performance
-# dataLogDir=/disk2/zookeeper
-
-# the port at which the clients will connect
-clientPort=2181
-
-# Maximum number of clients that can connect from one client
-maxClientCnxns=60
-
-# specify all zookeeper servers
-# The fist port is used by followers to connect to the leader
-# The second one is used for leader election
-
-server.0=127.0.0.1:2888:3888
-
-# To avoid seeks ZooKeeper allocates space in the transaction log file in
-# blocks of preAllocSize kilobytes. The default block size is 64M. One reason
-# for changing the size of the blocks is to reduce the block size if snapshots
-# are taken more often. (Also, see snapCount).
-#preAllocSize=65536
-
-# Clients can submit requests faster than ZooKeeper can process them,
-# especially if there are a lot of clients. To prevent ZooKeeper from running
-# out of memory due to queued requests, ZooKeeper will throttle clients so that
-# there is no more than globalOutstandingLimit outstanding requests in the
-# system. The default limit is 1,000.ZooKeeper logs transactions to a
-# transaction log. After snapCount transactions are written to a log file a
-# snapshot is started and a new transaction log file is started. The default
-# snapCount is 10,000.
-#snapCount=1000
-
-# If this option is defined, requests will be will logged to a trace file named
-# traceFile.year.month.day.
-#traceFile=
-
-# Leader accepts client connections. Default value is "yes". The leader machine
-# coordinates updates. For higher update throughput at thes slight expense of
-# read throughput the leader can be configured to not accept clients and focus
-# on coordination.
-#leaderServes=yes
-
-# Autopurge every hour to avoid using lots of disk in bursts
-# Order of the next 2 properties matters.
-# autopurge.snapRetainCount must be before autopurge.purgeInterval.
-autopurge.snapRetainCount=3
-autopurge.purgeInterval=1
\ No newline at end of file
diff --git a/functions b/functions
index c99e435..e497e45 100644
--- a/functions
+++ b/functions
@@ -310,6 +310,11 @@
disk_format=qcow2
container_format=bare
;;
+ *.raw)
+ image_name=$(basename "$image" ".raw")
+ disk_format=raw
+ container_format=bare
+ ;;
*.iso)
image_name=$(basename "$image" ".iso")
disk_format=iso
@@ -318,7 +323,7 @@
*.vhd|*.vhdx|*.vhd.gz|*.vhdx.gz)
local extension="${image_fname#*.}"
image_name=$(basename "$image" ".$extension")
- disk_format=vhd
+ disk_format=$(echo $image_fname | grep -oP '(?<=\.)vhdx?(?=\.|$)')
container_format=bare
if [ "${image_fname##*.}" == "gz" ]; then
unpack=zcat
@@ -606,14 +611,23 @@
function setup_systemd_logging {
local conf_file=$1
local conf_section="DEFAULT"
- iniset $conf_file $conf_section use_journal "True"
- iniset $conf_file $conf_section logging_context_format_string \
- "%(levelname)s %(name)s [%(request_id)s %(project_name)s %(user_name)s] %(instance)s%(message)s"
- iniset $conf_file $conf_section logging_default_format_string \
- "%(levelname)s %(name)s [-] %(instance)s%(color)s%(message)s"
- iniset $conf_file $conf_section logging_debug_format_suffix \
- "from (pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d"
- iniset $conf_file $conf_section logging_exception_prefix "ERROR %(name)s %(instance)s"
+ # NOTE(sdague): this is a nice to have, and means we're using the
+ # native systemd path, which provides for things like search on
+ # request-id. However, there may be an eventlet interaction here,
+ # so going off for now.
+ USE_JOURNAL=$(trueorfalse USE_JOURNAL False)
+ local pidstr=""
+ if [[ "$USE_JOURNAL" == "True" ]]; then
+ iniset $conf_file $conf_section use_journal "True"
+ # if we are using the journal directly, our process id is already correct
+ else
+ pidstr="(pid=%(process)d) "
+ fi
+ iniset $conf_file $conf_section logging_debug_format_suffix "[00;33m{{${pidstr}%(funcName)s %(pathname)s:%(lineno)d}}[00m"
+
+ iniset $conf_file $conf_section logging_context_format_string "%(color)s%(levelname)s %(name)s [[01;36m%(global_request_id)s %(request_id)s [00;36m%(project_name)s %(user_name)s%(color)s] [01;35m%(instance)s%(color)s%(message)s[00m"
+ iniset $conf_file $conf_section logging_default_format_string "%(color)s%(levelname)s %(name)s [[00;36m-%(color)s] [01;35m%(instance)s%(color)s%(message)s[00m"
+ iniset $conf_file $conf_section logging_exception_prefix "ERROR %(name)s [01;35m%(instance)s[00m"
}
function setup_standard_logging_identity {
@@ -703,6 +717,22 @@
}
+# Set a systemd system override
+#
+# This sets a system-side override in system.conf. A per-service
+# override would be /etc/systemd/system/${service}.service/override.conf
+function set_systemd_override {
+ local key="$1"
+ local value="$2"
+
+ local sysconf="/etc/systemd/system.conf"
+ iniset -sudo "${sysconf}" "Manager" "$key" "$value"
+ echo "Set systemd system override for ${key}=${value}"
+
+ sudo systemctl daemon-reload
+}
+
+
# Restore xtrace
$_XTRACE_FUNCTIONS
diff --git a/functions-common b/functions-common
index 35b4860..30933ea 100644
--- a/functions-common
+++ b/functions-common
@@ -48,8 +48,8 @@
# Save these variables to .stackenv
STACK_ENV_VARS="BASE_SQL_CONN DATA_DIR DEST ENABLED_SERVICES HOST_IP \
- KEYSTONE_AUTH_PROTOCOL KEYSTONE_AUTH_URI KEYSTONE_SERVICE_URI \
- LOGFILE OS_CACERT SERVICE_HOST SERVICE_PROTOCOL STACK_USER TLS_IP \
+ KEYSTONE_AUTH_URI KEYSTONE_SERVICE_URI \
+ LOGFILE OS_CACERT SERVICE_HOST STACK_USER TLS_IP \
HOST_IPV6 SERVICE_IP_VERSION"
@@ -93,7 +93,7 @@
--os-region-name $REGION_NAME \
--os-identity-api-version 3 \
$CA_CERT_ARG \
- --os-auth-url $KEYSTONE_AUTH_URI \
+ --os-auth-url $KEYSTONE_SERVICE_URI \
--os-username demo \
--os-password $ADMIN_PASSWORD \
--os-project-name demo
@@ -105,7 +105,7 @@
--os-region-name $REGION_NAME \
--os-identity-api-version 3 \
$CA_CERT_ARG \
- --os-auth-url $KEYSTONE_AUTH_URI \
+ --os-auth-url $KEYSTONE_SERVICE_URI \
--os-username alt_demo \
--os-password $ADMIN_PASSWORD \
--os-project-name alt_demo
@@ -117,7 +117,7 @@
--os-region-name $REGION_NAME \
--os-identity-api-version 3 \
$CA_CERT_ARG \
- --os-auth-url $KEYSTONE_AUTH_URI \
+ --os-auth-url $KEYSTONE_SERVICE_URI \
--os-username admin \
--os-password $ADMIN_PASSWORD \
--os-project-name admin
@@ -1463,6 +1463,7 @@
mkdir -p $SYSTEMD_DIR
iniset -sudo $unitfile "Unit" "Description" "Devstack $service"
+ iniset -sudo $unitfile "Service" "SyslogIdentifier" "$service"
iniset -sudo $unitfile "Service" "User" "$user"
iniset -sudo $unitfile "Service" "ExecStart" "$command"
iniset -sudo $unitfile "Service" "Type" "notify"
@@ -1480,10 +1481,41 @@
$SYSTEMCTL daemon-reload
}
+function _common_systemd_pitfalls {
+ local cmd=$1
+ # do some sanity checks on $cmd to see things we don't expect to work
+
+ if [[ "$cmd" =~ "sudo" ]]; then
+ local msg=<<EOF
+You are trying to use run_process with sudo, this is not going to work under systemd.
+
+If you need to run a service as a user other than $STACK_USER call it with:
+
+ run_process \$name \$cmd \$group \$user
+EOF
+ die $LINENO $msg
+ fi
+
+ if [[ ! "$cmd" =~ ^/ ]]; then
+ local msg=<<EOF
+The cmd="$cmd" does not start with an absolute path. It will fail to
+start under systemd.
+
+Please update your run_process stanza to have an absolute path.
+EOF
+ die $LINENO $msg
+ fi
+
+}
+
+# Helper function to build a basic unit file and run it under systemd.
function _run_under_systemd {
local service=$1
local command="$2"
local cmd=$command
+ # sanity check the command
+ _common_systemd_pitfalls "$cmd"
+
local systemd_service="devstack@$service.service"
local group=$3
local user=${4:-$STACK_USER}
@@ -1495,22 +1527,6 @@
$SYSTEMCTL enable $systemd_service
$SYSTEMCTL start $systemd_service
- _journal_log $service $systemd_service
-}
-
-function _journal_log {
- local service=$1
- local unit=$2
- local logfile="${service}.log.${CURRENT_LOG_TIME}"
- local real_logfile="${LOGDIR}/${logfile}"
- if [[ -n ${LOGDIR} ]]; then
- $JOURNALCTL_F $2 > "$real_logfile" &
- bash -c "cd '$LOGDIR' && ln -sf '$logfile' ${service}.log"
- if [[ -n ${SCREEN_LOGDIR} ]]; then
- # Drop the backward-compat symlink
- ln -sf "$real_logfile" ${SCREEN_LOGDIR}/screen-${service}.log
- fi
- fi
}
# Helper to remove the ``*.failure`` files under ``$SERVICE_DIR/$SCREEN_NAME``.
@@ -1543,7 +1559,7 @@
# If an optional group is provided sg will be used to run the
# command as that group.
# Uses globals ``USE_SCREEN``
-# run_process service "command-line" [group]
+# run_process service "command-line" [group] [user]
function run_process {
local service=$1
local command="$2"
@@ -1695,11 +1711,10 @@
SERVICE_DIR=${SERVICE_DIR:-${DEST}/status}
if is_service_enabled $service; then
- # Kill via pid if we have one available
- if [[ "$USE_SYSTEMD" == "True" ]]; then
- # Only do this for units which appear enabled, this also
- # catches units that don't really exist for cases like
- # keystone without a failure.
+ # Only do this for units which appear enabled, this also
+ # catches units that don't really exist for cases like
+ # keystone without a failure.
+ if $SYSTEMCTL is-enabled devstack@$service.service; then
$SYSTEMCTL stop devstack@$service.service
$SYSTEMCTL disable devstack@$service.service
fi
diff --git a/inc/python b/inc/python
index 2443c4d..4c443d6 100644
--- a/inc/python
+++ b/inc/python
@@ -351,6 +351,20 @@
return $result
}
+function pip_uninstall {
+ local name=$1
+ if [[ -n ${PIP_VIRTUAL_ENV:=} && -d ${PIP_VIRTUAL_ENV} ]]; then
+ local cmd_pip=$PIP_VIRTUAL_ENV/bin/pip
+ local sudo_pip="env"
+ else
+ local cmd_pip
+ cmd_pip=$(get_pip_command $PYTHON2_VERSION)
+ local sudo_pip="sudo -H"
+ fi
+ # don't error if we can't uninstall, it might not be there
+ $sudo_pip $cmd_pip uninstall -y $name || /bin/true
+}
+
# get version of a package from global requirements file
# get_from_global_requirements <package>
function get_from_global_requirements {
diff --git a/lib/cinder b/lib/cinder
index 9fc25c7..762edc4 100644
--- a/lib/cinder
+++ b/lib/cinder
@@ -84,20 +84,6 @@
# CINDER_ENABLED_BACKENDS=${CINDER_ENABLED_BACKENDS:-lvm:lvmdriver-1,lvm:lvmdriver-2}
CINDER_ENABLED_BACKENDS=${CINDER_ENABLED_BACKENDS:-lvm:lvmdriver-1}
-
-# Should cinder perform secure deletion of volumes?
-# Defaults to zero. Can also be set to none or shred.
-# This was previously CINDER_SECURE_DELETE (True or False).
-# Equivalents using CINDER_VOLUME_CLEAR are zero and none, respectively.
-# Set to none to avoid this bug when testing:
-# https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1023755
-if [[ -n $CINDER_SECURE_DELETE ]]; then
- CINDER_SECURE_DELETE=$(trueorfalse True CINDER_SECURE_DELETE)
- if [[ $CINDER_SECURE_DELETE == "False" ]]; then
- CINDER_VOLUME_CLEAR_DEFAULT="none"
- fi
- deprecated "Configure secure Cinder volume deletion using CINDER_VOLUME_CLEAR instead of CINDER_SECURE_DELETE."
-fi
CINDER_VOLUME_CLEAR=${CINDER_VOLUME_CLEAR:-${CINDER_VOLUME_CLEAR_DEFAULT:-zero}}
CINDER_VOLUME_CLEAR=$(echo ${CINDER_VOLUME_CLEAR} | tr '[:upper:]' '[:lower:]')
@@ -244,6 +230,10 @@
configure_rootwrap cinder
+ if [[ -f "$CINDER_DIR/etc/cinder/resource_filters.json" ]]; then
+ cp -p "$CINDER_DIR/etc/cinder/resource_filters.json" "$CINDER_CONF_DIR/resource_filters.json"
+ fi
+
cp $CINDER_DIR/etc/cinder/api-paste.ini $CINDER_API_PASTE_INI
inicomment $CINDER_API_PASTE_INI filter:authtoken auth_host
@@ -299,6 +289,9 @@
default_name=$be_name
fi
enabled_backends+=$be_name,
+
+ iniset $CINDER_CONF $be_name volume_clear $CINDER_VOLUME_CLEAR
+
done
iniset $CINDER_CONF DEFAULT enabled_backends ${enabled_backends%,*}
if [[ -n "$default_name" ]]; then
@@ -328,8 +321,6 @@
iniset_rpc_backend cinder $CINDER_CONF
- iniset $CINDER_CONF DEFAULT volume_clear $CINDER_VOLUME_CLEAR
-
# Format logging
setup_logging $CINDER_CONF $CINDER_USE_MOD_WSGI
@@ -359,11 +350,10 @@
iniset $CINDER_CONF DEFAULT os_privileged_user_tenant "$SERVICE_PROJECT_NAME"
iniset $CINDER_CONF DEFAULT graceful_shutdown_timeout "$SERVICE_GRACEFUL_SHUTDOWN_TIMEOUT"
- # Set the backend url according to the configured dlm backend
- if is_dlm_enabled; then
- if [[ "$(dlm_backend)" == "zookeeper" ]]; then
- iniset $CINDER_CONF coordination backend_url "zookeeper://${SERVICE_HOST}:2181"
- fi
+ if [[ ! -z "$CINDER_COORDINATION_URL" ]]; then
+ iniset $CINDER_CONF coordination backend_url "$CINDER_COORDINATION_URL"
+ elif is_service_enabled etcd3; then
+ iniset $CINDER_CONF coordination backend_url "etcd3+http://${SERVICE_HOST}:2379"
fi
}
diff --git a/lib/cinder_backends/lvm b/lib/cinder_backends/lvm
index d927f9c..03e1880 100644
--- a/lib/cinder_backends/lvm
+++ b/lib/cinder_backends/lvm
@@ -53,9 +53,6 @@
iniset $CINDER_CONF $be_name iscsi_helper "$CINDER_ISCSI_HELPER"
iniset $CINDER_CONF $be_name lvm_type "$CINDER_LVM_TYPE"
- if [[ "$CINDER_SECURE_DELETE" == "False" ]]; then
- iniset $CINDER_CONF $be_name volume_clear none
- fi
}
# init_cinder_backend_lvm - Initialize volume group
diff --git a/lib/dlm b/lib/dlm
deleted file mode 100644
index b5ac0f5..0000000
--- a/lib/dlm
+++ /dev/null
@@ -1,111 +0,0 @@
-#!/bin/bash
-#
-# lib/dlm
-#
-# Functions to control the installation and configuration of software
-# that provides a dlm (and possibly other functions). The default is
-# **zookeeper**, and is going to be the only backend supported in the
-# devstack tree.
-
-# Dependencies:
-#
-# - ``functions`` file
-
-# ``stack.sh`` calls the entry points in this order:
-#
-# - is_dlm_enabled
-# - install_dlm
-# - configure_dlm
-# - cleanup_dlm
-
-# Save trace setting
-_XTRACE_DLM=$(set +o | grep xtrace)
-set +o xtrace
-
-
-# Defaults
-# --------
-
-# <define global variables here that belong to this project>
-
-# Set up default directories
-ZOOKEEPER_DATA_DIR=$DEST/data/zookeeper
-ZOOKEEPER_CONF_DIR=/etc/zookeeper
-
-
-# Entry Points
-# ------------
-#
-# NOTE(sdague): it is expected that when someone wants to implement
-# another one of these out of tree, they'll implement the following
-# functions:
-#
-# - dlm_backend
-# - install_dlm
-# - configure_dlm
-# - cleanup_dlm
-
-# This should be declared in the settings file of any plugin or
-# service that needs to have a dlm in their environment.
-function use_dlm {
- enable_service $(dlm_backend)
-}
-
-# A function to return the name of the backend in question, some users
-# are going to need to know this.
-function dlm_backend {
- echo "zookeeper"
-}
-
-# Test if a dlm is enabled (defaults to a zookeeper specific check)
-function is_dlm_enabled {
- [[ ,${ENABLED_SERVICES}, =~ ,"$(dlm_backend)", ]] && return 0
- return 1
-}
-
-# cleanup_dlm() - Remove residual data files, anything left over from previous
-# runs that a clean run would need to clean up
-function cleanup_dlm {
- # NOTE(sdague): we don't check for is_enabled here because we
- # should just delete this regardless. Some times users updated
- # their service list before they run cleanup.
- sudo rm -rf $ZOOKEEPER_DATA_DIR
-}
-
-# configure_dlm() - Set config files, create data dirs, etc
-function configure_dlm {
- if is_dlm_enabled; then
- sudo cp $FILES/zookeeper/* $ZOOKEEPER_CONF_DIR
- sudo sed -i -e 's|.*dataDir.*|dataDir='$ZOOKEEPER_DATA_DIR'|' $ZOOKEEPER_CONF_DIR/zoo.cfg
- # clean up from previous (possibly aborted) runs
- # create required data files
- sudo rm -rf $ZOOKEEPER_DATA_DIR
- sudo mkdir -p $ZOOKEEPER_DATA_DIR
- # restart after configuration, there is no reason to make this
- # another step, because having data files that don't match the
- # zookeeper running is just going to cause tears.
- restart_service zookeeper
- fi
-}
-
-# install_dlm() - Collect source and prepare
-function install_dlm {
- if is_dlm_enabled; then
- pip_install_gr_extras tooz zookeeper
- if is_ubuntu; then
- install_package zookeeperd
- elif is_fedora; then
- install_package zookeeper
- else
- die $LINENO "Don't know how to install zookeeper on this platform"
- fi
- fi
-}
-
-# Restore xtrace
-$_XTRACE_DLM
-
-# Tell emacs to use shell-script-mode
-## Local variables:
-## mode: shell-script
-## End:
diff --git a/lib/etcd3 b/lib/etcd3
new file mode 100644
index 0000000..5cab3f5
--- /dev/null
+++ b/lib/etcd3
@@ -0,0 +1,147 @@
+#!/bin/bash
+#
+# lib/etcd3
+#
+# Functions to control the installation and configuration of etcd 3.x
+# that provides a key-value store (and possibly other functions).
+
+# Dependencies:
+#
+# - ``functions`` file
+
+# ``stack.sh`` calls the entry points in this order:
+#
+# - start_etcd3
+# - stop_etcd3
+# - cleanup_etcd3
+
+# Save trace setting
+_XTRACE_ETCD3=$(set +o | grep xtrace)
+set +o xtrace
+
+
+# Defaults
+# --------
+
+# Set up default values for etcd
+ETCD_DOWNLOAD_URL=${ETCD_DOWNLOAD_URL:-https://github.com/coreos/etcd/releases/download}
+ETCD_VERSION=${ETCD_VERSION:-v3.1.7}
+ETCD_DATA_DIR="$DEST/data/etcd"
+ETCD_SYSTEMD_SERVICE="devstack@etcd.service"
+ETCD_BIN_DIR="$DEST/bin"
+ETCD_SHA256_AMD64="4fde194bbcd259401e2b5c462dfa579ee7f6af539f13f130b8f5b4f52e3b3c52"
+# NOTE(sdague): etcd v3.1.7 doesn't have anything for these architectures, though 3.2.0 does.
+ETCD_SHA256_ARM64=""
+ETCD_SHA256_PPC64=""
+
+if is_ubuntu ; then
+ UBUNTU_RELEASE_BASE_NUM=`lsb_release -r | awk '{print $2}' | cut -d '.' -f 1`
+fi
+
+# start_etcd3() - Starts to run the etcd process
+function start_etcd3 {
+ # Don't install in sub nodes (multinode scenario)
+ if [ "$SERVICE_HOST" != "$HOST_IP" ]; then
+ return
+ fi
+
+ _install_etcd
+
+ local cmd="$ETCD_BIN_DIR/etcd"
+ cmd+=" --name $HOSTNAME --data-dir $ETCD_DATA_DIR"
+ cmd+=" --initial-cluster-state new --initial-cluster-token etcd-cluster-01"
+ cmd+=" --initial-cluster $HOSTNAME=http://$SERVICE_HOST:2380"
+ cmd+=" --initial-advertise-peer-urls http://$SERVICE_HOST:2380"
+ cmd+=" --advertise-client-urls http://$SERVICE_HOST:2379"
+ cmd+=" --listen-peer-urls http://0.0.0.0:2380 "
+ cmd+=" --listen-client-urls http://$SERVICE_HOST:2379"
+
+ local unitfile="$SYSTEMD_DIR/$ETCD_SYSTEMD_SERVICE"
+ write_user_unit_file $ETCD_SYSTEMD_SERVICE "$cmd" "" "root"
+
+ iniset -sudo $unitfile "Unit" "After" "network.target"
+ iniset -sudo $unitfile "Service" "Type" "notify"
+ iniset -sudo $unitfile "Service" "Restart" "on-failure"
+ iniset -sudo $unitfile "Service" "LimitNOFILE" "65536"
+
+ $SYSTEMCTL daemon-reload
+ $SYSTEMCTL enable $ETCD_SYSTEMD_SERVICE
+ $SYSTEMCTL start $ETCD_SYSTEMD_SERVICE
+}
+
+# stop_etcd3() stops the etcd3 process
+function stop_etcd3 {
+ # Don't install in sub nodes (multinode scenario)
+ if [ "$SERVICE_HOST" != "$HOST_IP" ]; then
+ return
+ fi
+
+ $SYSTEMCTL stop $ETCD_SYSTEMD_SERVICE
+}
+
+function cleanup_etcd3 {
+ # Don't install in sub nodes (multinode scenario)
+ if [ "$SERVICE_HOST" != "$HOST_IP" ]; then
+ return
+ fi
+
+ $SYSTEMCTL disable $ETCD_SYSTEMD_SERVICE
+
+ local unitfile="$SYSTEMD_DIR/$ETCD_SYSTEMD_SERVICE"
+ sudo rm -f $unitfile
+
+ $SYSTEMCTL daemon-reload
+
+ sudo rm -rf $ETCD_DATA_DIR
+}
+
+function _install_etcd {
+ echo "Installing etcd"
+
+ # Make sure etcd3 downloads the correct architecture
+ if is_arch "x86_64"; then
+ ETCD_ARCH="amd64"
+ ETCD_SHA256=${ETCD_SHA256:-$ETCD_SHA256_AMD64}
+ elif is_arch "aarch64"; then
+ ETCD_ARCH="arm64"
+ ETCD_SHA256=${ETCD_SHA256:-$ETCD_SHA256_ARM64}
+ elif is_arch "ppc64le"; then
+ ETCD_ARCH="ppc64le"
+ ETCD_SHA256=${ETCD_SHA256:-$ETCD_SHA256_PPC64}
+ else
+ exit_distro_not_supported "invalid hardware type - $ETCD_ARCH"
+ fi
+
+ ETCD_NAME=etcd-$ETCD_VERSION-linux-$ETCD_ARCH
+
+ # Install the libraries needed. Note: tooz for example does not have a hard dependency on these libraries
+ pip_install etcd3
+ pip_install etcd3gw
+
+ # Create the necessary directories
+ sudo mkdir -p $ETCD_BIN_DIR
+ sudo mkdir -p $ETCD_DATA_DIR
+
+ # Download and cache the etcd tgz for subsequent use
+ if [ ! -f "files/etcd-$ETCD_VERSION-linux-$ETCD_ARCH/etcd" ]; then
+ ETCD_DOWNLOAD_FILE=$ETCD_NAME.tar.gz
+ wget $ETCD_DOWNLOAD_URL/$ETCD_VERSION/$ETCD_DOWNLOAD_FILE -O files/$ETCD_DOWNLOAD_FILE
+ echo "${ETCD_SHA256} files/${ETCD_DOWNLOAD_FILE}" > files/etcd.sha256sum
+ # NOTE(sdague): this should go fatal if this fails
+ sha256sum -c files/etcd.sha256sum
+
+ tar xzvf files/$ETCD_DOWNLOAD_FILE -C files
+ sudo cp files/$ETCD_NAME/etcd $ETCD_BIN_DIR/etcd
+ fi
+ if [ ! -f "$ETCD_BIN_DIR/etcd" ]; then
+ sudo cp files/$ETCD_NAME/etcd $ETCD_BIN_DIR/etcd
+ fi
+}
+
+# Restore xtrace
+$_XTRACE_ETCD3
+
+# Tell emacs to use shell-script-mode
+## Local variables:
+## mode: shell-script
+## End:
diff --git a/lib/glance b/lib/glance
index 23a1cbf..d6438a6 100644
--- a/lib/glance
+++ b/lib/glance
@@ -43,6 +43,7 @@
GLANCE_CACHE_DIR=${GLANCE_CACHE_DIR:=$DATA_DIR/glance/cache}
GLANCE_IMAGE_DIR=${GLANCE_IMAGE_DIR:=$DATA_DIR/glance/images}
+GLANCE_LOCK_DIR=${GLANCE_LOCK_DIR:=$DATA_DIR/glance/locks}
GLANCE_AUTH_CACHE_DIR=${GLANCE_AUTH_CACHE_DIR:-/var/cache/glance}
GLANCE_CONF_DIR=${GLANCE_CONF_DIR:-/etc/glance}
@@ -116,6 +117,7 @@
iniset $GLANCE_API_CONF database connection $dburl
iniset $GLANCE_API_CONF DEFAULT use_syslog $SYSLOG
iniset $GLANCE_API_CONF DEFAULT image_cache_dir $GLANCE_CACHE_DIR/
+ iniset $GLANCE_API_CONF DEFAULT lock_path $GLANCE_LOCK_DIR
iniset $GLANCE_API_CONF paste_deploy flavor keystone+cachemanagement
configure_auth_token_middleware $GLANCE_API_CONF glance $GLANCE_AUTH_CACHE_DIR/api
iniset $GLANCE_API_CONF oslo_messaging_notifications driver messagingv2
@@ -205,7 +207,7 @@
iniset $GLANCE_CACHE_CONF DEFAULT use_syslog $SYSLOG
iniset $GLANCE_CACHE_CONF DEFAULT image_cache_dir $GLANCE_CACHE_DIR/
iniuncomment $GLANCE_CACHE_CONF DEFAULT auth_url
- iniset $GLANCE_CACHE_CONF DEFAULT auth_url $KEYSTONE_AUTH_URI/v3
+ iniset $GLANCE_CACHE_CONF DEFAULT auth_url $KEYSTONE_AUTH_URI
iniuncomment $GLANCE_CACHE_CONF DEFAULT auth_tenant_name
iniset $GLANCE_CACHE_CONF DEFAULT admin_tenant_name $SERVICE_PROJECT_NAME
iniuncomment $GLANCE_CACHE_CONF DEFAULT auth_user
diff --git a/lib/keystone b/lib/keystone
index a26ef8a..4bb6893 100644
--- a/lib/keystone
+++ b/lib/keystone
@@ -113,8 +113,9 @@
KEYSTONE_SERVICE_PROTOCOL="https"
fi
-KEYSTONE_AUTH_URI=${KEYSTONE_AUTH_PROTOCOL}://${KEYSTONE_AUTH_HOST}/identity_admin
KEYSTONE_SERVICE_URI=${KEYSTONE_SERVICE_PROTOCOL}://${KEYSTONE_SERVICE_HOST}/identity
+# for compat
+KEYSTONE_AUTH_URI=$KEYSTONE_SERVICE_URI
# V3 URIs
KEYSTONE_AUTH_URI_V3=$KEYSTONE_AUTH_URI/v3
@@ -140,15 +141,16 @@
# cleanup_keystone() - Remove residual data files, anything left over from previous
# runs that a clean run would need to clean up
function cleanup_keystone {
- if [[ "$WSGI_MODE" == "uwsgi" ]]; then
- remove_uwsgi_config "$KEYSTONE_PUBLIC_UWSGI_CONF" "$KEYSTONE_PUBLIC_UWSGI"
- remove_uwsgi_config "$KEYSTONE_ADMIN_UWSGI_CONF" "$KEYSTONE_ADMIN_UWSGI"
- sudo rm -f $(apache_site_config_for keystone-wsgi-public)
- sudo rm -f $(apache_site_config_for keystone-wsgi-admin)
- else
- disable_apache_site keystone
- sudo rm -f $(apache_site_config_for keystone)
- fi
+ # TODO: remove admin at pike-2
+ # These files will be created if we are running WSGI_MODE="uwsgi"
+ remove_uwsgi_config "$KEYSTONE_PUBLIC_UWSGI_CONF" "$KEYSTONE_PUBLIC_UWSGI"
+ remove_uwsgi_config "$KEYSTONE_ADMIN_UWSGI_CONF" "$KEYSTONE_ADMIN_UWSGI"
+ sudo rm -f $(apache_site_config_for keystone-wsgi-public)
+ sudo rm -f $(apache_site_config_for keystone-wsgi-admin)
+
+ # These files will be created if we are running WSGI_MODE="mod_wsgi"
+ disable_apache_site keystone
+ sudo rm -f $(apache_site_config_for keystone)
}
# _config_keystone_apache_wsgi() - Set WSGI config files of Keystone
@@ -265,9 +267,7 @@
fi
# Format logging
- if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ] && [ "$KEYSTONE_DEPLOY" != "mod_wsgi" ] ; then
- setup_colorized_logging $KEYSTONE_CONF
- fi
+ setup_logging $KEYSTONE_CONF
iniset $KEYSTONE_CONF DEFAULT debug $ENABLE_DEBUG_LOG_LEVEL
@@ -434,14 +434,13 @@
local section=${4:-keystone_authtoken}
iniset $conf_file $section auth_type password
- iniset $conf_file $section auth_url $KEYSTONE_AUTH_URI
+ iniset $conf_file $section auth_url $KEYSTONE_SERVICE_URI
iniset $conf_file $section username $admin_user
iniset $conf_file $section password $SERVICE_PASSWORD
iniset $conf_file $section user_domain_name "$SERVICE_DOMAIN_NAME"
iniset $conf_file $section project_name $SERVICE_PROJECT_NAME
iniset $conf_file $section project_domain_name "$SERVICE_DOMAIN_NAME"
- iniset $conf_file $section auth_uri $KEYSTONE_SERVICE_URI
iniset $conf_file $section cafile $SSL_BUNDLE_FILE
iniset $conf_file $section signing_dir $signing_dir
iniset $conf_file $section memcached_servers $SERVICE_HOST:11211
@@ -544,11 +543,7 @@
tail_log key /var/log/$APACHE_NAME/keystone.log
tail_log key-access /var/log/$APACHE_NAME/keystone_access.log
else # uwsgi
- # TODO(sdague): we should really get down to a single keystone here
- enable_service key-p
- enable_service key-a
- run_process key-p "$KEYSTONE_BIN_DIR/uwsgi --ini $KEYSTONE_PUBLIC_UWSGI_CONF" ""
- run_process key-a "$KEYSTONE_BIN_DIR/uwsgi --ini $KEYSTONE_ADMIN_UWSGI_CONF" ""
+ run_process keystone "$KEYSTONE_BIN_DIR/uwsgi --ini $KEYSTONE_PUBLIC_UWSGI_CONF" ""
fi
echo "Waiting for keystone to start..."
@@ -579,9 +574,9 @@
disable_apache_site keystone
restart_apache_server
else
- stop_process key-p
- stop_process key-a
+ stop_process keystone
remove_uwsgi_config "$KEYSTONE_PUBLIC_UWSGI_CONF" "$KEYSTONE_PUBLIC_UWSGI"
+ # TODO(remove in at pike-2)
remove_uwsgi_config "$KEYSTONE_ADMIN_UWSGI_CONF" "$KEYSTONE_ADMIN_UWSGI"
fi
# Kill the Keystone screen window
diff --git a/lib/neutron b/lib/neutron
index 492a0ee..efca880 100644
--- a/lib/neutron
+++ b/lib/neutron
@@ -135,7 +135,11 @@
mkdir -p $NEUTRON_CORE_PLUGIN_CONF_PATH
- cp $NEUTRON_DIR/etc/neutron/plugins/$NEUTRON_CORE_PLUGIN/$NEUTRON_CORE_PLUGIN_CONF_FILENAME.sample $NEUTRON_CORE_PLUGIN_CONF
+ # NOTE(yamamoto): A decomposed plugin should prepare the config file in
+ # its devstack plugin.
+ if [ -f $NEUTRON_DIR/etc/neutron/plugins/$NEUTRON_CORE_PLUGIN/$NEUTRON_CORE_PLUGIN_CONF_FILENAME.sample ]; then
+ cp $NEUTRON_DIR/etc/neutron/plugins/$NEUTRON_CORE_PLUGIN/$NEUTRON_CORE_PLUGIN_CONF_FILENAME.sample $NEUTRON_CORE_PLUGIN_CONF
+ fi
iniset $NEUTRON_CONF database connection `database_connection_url neutron`
iniset $NEUTRON_CONF DEFAULT state_path $NEUTRON_STATE_PATH
@@ -226,17 +230,12 @@
iniset $NEUTRON_META_CONF agent root_helper_daemon "$NEUTRON_ROOTWRAP_DAEMON_CMD"
# TODO(dtroyer): remove the v2.0 hard code below
- iniset $NEUTRON_META_CONF DEFAULT auth_url $KEYSTONE_SERVICE_URI/v2.0
+ iniset $NEUTRON_META_CONF DEFAULT auth_url $KEYSTONE_SERVICE_URI
configure_auth_token_middleware $NEUTRON_META_CONF neutron $NEUTRON_AUTH_CACHE_DIR DEFAULT
fi
# Format logging
- if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ]; then
- setup_colorized_logging $NEUTRON_CONF DEFAULT project_id
- else
- # Show user_name and project_name by default
- iniset $NEUTRON_CONF DEFAULT logging_context_format_string "%(asctime)s.%(msecs)03d %(levelname)s %(name)s [%(request_id)s %(user_name)s %(project_name)s] %(instance)s%(message)s"
- fi
+ setup_logging $NEUTRON_CONF
if is_service_enabled tls-proxy; then
# Set the service port for a proxy to take the original
@@ -284,7 +283,7 @@
function configure_neutron_nova_new {
iniset $NOVA_CONF DEFAULT use_neutron True
iniset $NOVA_CONF neutron auth_type "password"
- iniset $NOVA_CONF neutron auth_url "$KEYSTONE_SERVICE_URI/v3"
+ iniset $NOVA_CONF neutron auth_url "$KEYSTONE_SERVICE_URI"
iniset $NOVA_CONF neutron username neutron
iniset $NOVA_CONF neutron password "$SERVICE_PASSWORD"
iniset $NOVA_CONF neutron user_domain_name "Default"
diff --git a/lib/neutron-legacy b/lib/neutron-legacy
index af9a3d2..67cf110 100644
--- a/lib/neutron-legacy
+++ b/lib/neutron-legacy
@@ -368,7 +368,7 @@
function create_nova_conf_neutron {
iniset $NOVA_CONF DEFAULT use_neutron True
iniset $NOVA_CONF neutron auth_type "password"
- iniset $NOVA_CONF neutron auth_url "$KEYSTONE_AUTH_URI/v3"
+ iniset $NOVA_CONF neutron auth_url "$KEYSTONE_AUTH_URI"
iniset $NOVA_CONF neutron username "$Q_ADMIN_USERNAME"
iniset $NOVA_CONF neutron password "$SERVICE_PASSWORD"
iniset $NOVA_CONF neutron user_domain_name "$SERVICE_DOMAIN_NAME"
@@ -577,7 +577,7 @@
local IP_DEL=""
local IP_UP=""
local DEFAULT_ROUTE_GW
- DEFAULT_ROUTE_GW=$(ip -f $af r | awk "/default.+$from_intf/ { print \$3; exit }")
+ DEFAULT_ROUTE_GW=$(ip -f $af r | awk "/default.+$from_intf\s/ { print \$3; exit }")
local ADD_OVS_PORT=""
local DEL_OVS_PORT=""
local ARP_CMD=""
diff --git a/lib/nova b/lib/nova
index de053ab..e580abb 100644
--- a/lib/nova
+++ b/lib/nova
@@ -878,7 +878,6 @@
run_process n-cond "$NOVA_BIN_DIR/nova-conductor --config-file $compute_cell_conf"
run_process n-cell-region "$NOVA_BIN_DIR/nova-cells --config-file $api_cell_conf"
run_process n-cell-child "$NOVA_BIN_DIR/nova-cells --config-file $compute_cell_conf"
- run_process n-crt "$NOVA_BIN_DIR/nova-cert --config-file $api_cell_conf"
if is_service_enabled n-net; then
if ! running_in_container; then
@@ -929,7 +928,7 @@
# Kill the nova screen windows
# Some services are listed here twice since more than one instance
# of a service may be running in certain configs.
- for serv in n-api n-crt n-net n-sch n-novnc n-xvnc n-cauth n-spice n-cond n-cell n-cell n-api-meta n-sproxy; do
+ for serv in n-api n-net n-sch n-novnc n-xvnc n-cauth n-spice n-cond n-cell n-cell n-api-meta n-sproxy; do
stop_process $serv
done
}
diff --git a/lib/nova_plugins/functions-libvirt b/lib/nova_plugins/functions-libvirt
index 47605af..3e38b89 100644
--- a/lib/nova_plugins/functions-libvirt
+++ b/lib/nova_plugins/functions-libvirt
@@ -67,6 +67,8 @@
else
install_package libvirt-clients libvirt-daemon-system libvirt-dev
fi
+ # uninstall in case the libvirt version changed
+ pip_uninstall libvirt-python
pip_install_gr libvirt-python
#pip_install_gr <there-si-no-guestfs-in-pypi>
elif is_fedora || is_suse; then
@@ -84,6 +86,7 @@
fi
install_package libvirt libvirt-devel
+ pip_uninstall libvirt-python
pip_install_gr libvirt-python
fi
@@ -95,7 +98,7 @@
# Configures the installed libvirt system so that is accessible by
# STACK_USER via qemu:///system with management capabilities.
function configure_libvirt {
- if is_service_enabled neutron && is_neutron_ovs_base_plugin && ! sudo grep -q '^cgroup_device_acl' $QEMU_CONF; then
+ if is_service_enabled neutron && ! sudo grep -q '^cgroup_device_acl' $QEMU_CONF; then
# Add /dev/net/tun to cgroup_device_acls, needed for type=ethernet interfaces
cat <<EOF | sudo tee -a $QEMU_CONF
cgroup_device_acl = [
diff --git a/lib/nova_plugins/hypervisor-ironic b/lib/nova_plugins/hypervisor-ironic
index c9544fe..7d47ef0 100644
--- a/lib/nova_plugins/hypervisor-ironic
+++ b/lib/nova_plugins/hypervisor-ironic
@@ -49,7 +49,7 @@
iniset $NOVA_CONF ironic auth_type password
iniset $NOVA_CONF ironic username admin
iniset $NOVA_CONF ironic password $ADMIN_PASSWORD
- iniset $NOVA_CONF ironic auth_url $KEYSTONE_AUTH_URI/v3
+ iniset $NOVA_CONF ironic auth_url $KEYSTONE_AUTH_URI
iniset $NOVA_CONF ironic project_domain_id default
iniset $NOVA_CONF ironic user_domain_id default
iniset $NOVA_CONF ironic project_name demo
diff --git a/lib/placement b/lib/placement
index 4755a58..ad12824 100644
--- a/lib/placement
+++ b/lib/placement
@@ -69,6 +69,7 @@
# cleanup_placement() - Remove residual data files, anything left over from previous
# runs that a clean run would need to clean up
function cleanup_placement {
+ sudo rm -f $(apache_site_config_for nova-placement-api)
sudo rm -f $(apache_site_config_for placement-api)
}
@@ -102,7 +103,7 @@
function configure_placement_nova_compute {
iniset $NOVA_CONF placement auth_type "password"
- iniset $NOVA_CONF placement auth_url "$KEYSTONE_SERVICE_URI/v3"
+ iniset $NOVA_CONF placement auth_url "$KEYSTONE_SERVICE_URI"
iniset $NOVA_CONF placement username placement
iniset $NOVA_CONF placement password "$SERVICE_PASSWORD"
iniset $NOVA_CONF placement user_domain_name "$SERVICE_DOMAIN_NAME"
diff --git a/lib/tempest b/lib/tempest
index f19686a..cc65ec7 100644
--- a/lib/tempest
+++ b/lib/tempest
@@ -11,6 +11,7 @@
# - ``DEST``, ``FILES``
# - ``ADMIN_PASSWORD``
# - ``DEFAULT_IMAGE_NAME``
+# - ``DEFAULT_IMAGE_FILE_NAME``
# - ``S3_SERVICE_PORT``
# - ``SERVICE_HOST``
# - ``BASE_SQL_CONN`` ``lib/database`` declares
@@ -278,6 +279,10 @@
iniset $TEMPEST_CONFIG identity-feature-enabled api_v2 False
fi
iniset $TEMPEST_CONFIG identity auth_version ${TEMPEST_AUTH_VERSION:-v3}
+ if [[ "$TEMPEST_AUTH_VERSION" != "v2.0" ]]; then
+ # we're going to disable v2 admin unless we're using v2.0 by default.
+ iniset $TEMPEST_CONFIG identity-feature-enabled api_v2_admin False
+ fi
if is_service_enabled tls-proxy; then
iniset $TEMPEST_CONFIG identity ca_certificates_file $SSL_BUNDLE_FILE
@@ -389,24 +394,6 @@
iniset $TEMPEST_CONFIG network-feature-enabled ipv6_subnet_attributes "$IPV6_SUBNET_ATTRIBUTES_ENABLED"
iniset $TEMPEST_CONFIG network-feature-enabled port_security $NEUTRON_PORT_SECURITY
- # Orchestration Tests
- if is_service_enabled heat; then
- if [[ ! -z "$HEAT_CFN_IMAGE_URL" ]]; then
- iniset $TEMPEST_CONFIG orchestration image_ref $(basename "${HEAT_CFN_IMAGE_URL%.*}")
- fi
- # Nova might not be enabled, especially when we want to test tempest scenario/API that only create Neutron resources
- if is_service_enabled nova; then
- # build a specialized heat flavor
- available_flavors=$(nova flavor-list)
- if [[ ! ( $available_flavors =~ 'm1.heat' ) ]]; then
- openstack flavor create --id 451 --ram 512 --disk 0 --vcpus 1 m1.heat
- fi
- iniset $TEMPEST_CONFIG orchestration instance_type "m1.heat"
- fi
- iniset $TEMPEST_CONFIG orchestration build_timeout 900
- iniset $TEMPEST_CONFIG orchestration stack_owner_role Member
- fi
-
# Scenario
if [ "$VIRT_DRIVER" = "xenserver" ]; then
SCENARIO_IMAGE_DIR=${SCENARIO_IMAGE_DIR:-$FILES}
@@ -415,7 +402,7 @@
iniset $TEMPEST_CONFIG scenario img_container_format ovf
else
SCENARIO_IMAGE_DIR=${SCENARIO_IMAGE_DIR:-$FILES}
- SCENARIO_IMAGE_FILE=$DEFAULT_IMAGE_NAME
+ SCENARIO_IMAGE_FILE=$DEFAULT_IMAGE_FILE_NAME
fi
iniset $TEMPEST_CONFIG scenario img_dir $SCENARIO_IMAGE_DIR
iniset $TEMPEST_CONFIG scenario img_file $SCENARIO_IMAGE_FILE
@@ -426,7 +413,7 @@
TEMPEST_SSH_NETWORK_NAME=$PHYSICAL_NETWORK
fi
# Validation
- iniset $TEMPEST_CONFIG validation run_validation ${TEMPEST_RUN_VALIDATION:-False}
+ iniset $TEMPEST_CONFIG validation run_validation ${TEMPEST_RUN_VALIDATION:-True}
iniset $TEMPEST_CONFIG validation ip_version_for_ssh 4
iniset $TEMPEST_CONFIG validation ssh_timeout $BUILD_TIMEOUT
iniset $TEMPEST_CONFIG validation image_ssh_user ${DEFAULT_INSTANCE_USER:-cirros}
diff --git a/openrc b/openrc
index 4cdb50e..23c173c 100644
--- a/openrc
+++ b/openrc
@@ -72,17 +72,23 @@
GLANCE_HOST=${GLANCE_HOST:-$HOST_IP}
fi
-SERVICE_PROTOCOL=${SERVICE_PROTOCOL:-http}
-
# Identity API version
export OS_IDENTITY_API_VERSION=${IDENTITY_API_VERSION:-3}
+# Ask keystoneauth1 to use keystone
+export OS_AUTH_TYPE=password
+
# Authenticating against an OpenStack cloud using Keystone returns a **Token**
# and **Service Catalog**. The catalog contains the endpoints for all services
# the user/project has access to - including nova, glance, keystone, swift, ...
# We currently recommend using the version 3 *identity api*.
#
-export OS_AUTH_URL=$KEYSTONE_AUTH_URI
+
+# If you don't have a working .stackenv, this is the backup possition
+KEYSTONE_BACKUP=$SERVICE_PROTOCOL://$SERVICE_HOST:5000
+KEYSTONE_AUTH_URI=${KEYSTONE_AUTH_URI:-$KEYSTONE_BACKUP}
+
+export OS_AUTH_URL=${OS_AUTH_URL:-$KEYSTONE_AUTH_URI}
# Currently, in order to use openstackclient with Identity API v3,
# we need to set the domain which the user and project belong to.
diff --git a/setup.cfg b/setup.cfg
index e4b2888..73d22b5 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -2,7 +2,7 @@
name = DevStack
summary = OpenStack DevStack
description-file =
- README.md
+ README.rst
author = OpenStack
author-email = openstack-dev@lists.openstack.org
home-page = http://docs.openstack.org/developer/devstack
@@ -15,6 +15,7 @@
all_files = 1
build-dir = doc/build
source-dir = doc/source
+warning-is-error = 1
[pbr]
warnerrors = True
diff --git a/stack.sh b/stack.sh
index 1615b1c..048acf8 100755
--- a/stack.sh
+++ b/stack.sh
@@ -2,7 +2,7 @@
# ``stack.sh`` is an opinionated OpenStack developer installation. It
# installs and configures various combinations of **Cinder**, **Glance**,
-# **Heat**, **Horizon**, **Keystone**, **Nova**, **Neutron**, and **Swift**
+# **Horizon**, **Keystone**, **Nova**, **Neutron**, and **Swift**
# This script's options can be changed by setting appropriate environment
# variables. You can configure things like which git repositories to use,
@@ -573,7 +573,7 @@
source $TOP_DIR/lib/neutron
source $TOP_DIR/lib/ldap
source $TOP_DIR/lib/dstat
-source $TOP_DIR/lib/dlm
+source $TOP_DIR/lib/etcd3
source $TOP_DIR/lib/os_brick
# Extras Source
@@ -746,6 +746,13 @@
# Do the ugly hacks for broken packages and distros
source $TOP_DIR/tools/fixup_stuff.sh
+if [[ "$USE_SYSTEMD" == "True" ]]; then
+ pip_install_gr systemd-python
+ # the default rate limit of 1000 messages / 30 seconds is not
+ # sufficient given how verbose our logging is.
+ iniset -sudo /etc/systemd/journald.conf "Journal" "RateLimitBurst" "0"
+ sudo systemctl restart systemd-journald
+fi
# Virtual Environment
# -------------------
@@ -758,13 +765,12 @@
# Phase: pre-install
run_phase stack pre-install
+# NOTE(danms): Set global limits before installing anything
+set_systemd_override DefaultLimitNOFILE ${ULIMIT_NOFILE}
+
install_rpc_backend
restart_rpc_backend
-# NOTE(sdague): dlm install is conditional on one being enabled by configuration
-install_dlm
-configure_dlm
-
if is_service_enabled $DATABASE_BACKENDS; then
install_database
fi
@@ -1036,6 +1042,13 @@
# A better kind of sysstat, with the top process per time slice
start_dstat
+# Etcd
+# -----
+
+# etcd is a distributed key value store that provides a reliable way to store data across a cluster of machines
+if is_service_enabled etcd3; then
+ start_etcd3
+fi
# Keystone
# --------
@@ -1249,8 +1262,13 @@
fi
# Create a randomized default value for the key manager's fixed_key
+# NOTE(lyarwood): This is currently set to 36 as a workaround to the following
+# libvirt bug that incorrectly pads passphrases that are a multiple of 16 bytes
+# in length.
+# Unable to use LUKS passphrase that is exactly 16 bytes long
+# https://bugzilla.redhat.com/show_bug.cgi?id=1447297
if is_service_enabled nova; then
- iniset $NOVA_CONF key_manager fixed_key $(generate_hex_string 32)
+ iniset $NOVA_CONF key_manager fixed_key $(generate_hex_string 36)
fi
# Launch the nova-api and wait for it to answer before continuing
@@ -1328,10 +1346,6 @@
USERRC_PARAMS="$USERRC_PARAMS --os-cacert $SSL_BUNDLE_FILE"
fi
- if [[ "$HEAT_STANDALONE" = "True" ]]; then
- USERRC_PARAMS="$USERRC_PARAMS --heat-url http://$HEAT_API_HOST:$HEAT_API_PORT/v1"
- fi
-
$TOP_DIR/tools/create_userrc.sh $USERRC_PARAMS
fi
@@ -1474,6 +1488,13 @@
echo_summary "WARNING: $DEPRECATED_TEXT"
fi
+# If USE_SYSTEMD is enabled, tell the user about using it.
+if [[ "$USE_SYSTEMD" == "True" ]]; then
+ echo "Services are running under systemd unit files."
+ echo "For more information see: "
+ echo "https://docs.openstack.org/developer/devstack/systemd.html"
+fi
+
# Indicate how long this took to run (bash maintained variable ``SECONDS``)
echo_summary "stack.sh completed in $SECONDS seconds."
diff --git a/stackrc b/stackrc
index ed1cf6e..e9b8df2 100644
--- a/stackrc
+++ b/stackrc
@@ -65,7 +65,7 @@
# Dashboard
ENABLED_SERVICES+=,horizon
# Additional services
- ENABLED_SERVICES+=,rabbit,tempest,mysql,dstat
+ ENABLED_SERVICES+=,rabbit,tempest,mysql,etcd3,dstat
fi
# Global toggle for enabling services under mod_wsgi. If this is set to
@@ -80,12 +80,19 @@
# Set the root URL for Horizon
HORIZON_APACHE_ROOT="/dashboard"
+# TODO(sdague): Queens
+#
+# All the non systemd paths should be removed in queens, they only
+# exist in Pike to support testing from grenade. Ensure that all this
+# is cleaned up and purged, which should dramatically simplify the
+# devstack codebase.
+
# Whether to use 'dev mode' for screen windows. Dev mode works by
# stuffing text into the screen windows so that a developer can use
# ctrl-c, up-arrow, enter to restart the service. Starting services
# this way is slightly unreliable, and a bit slower, so this can
# be disabled for automated testing by setting this value to False.
-USE_SCREEN=$(trueorfalse True USE_SCREEN)
+USE_SCREEN=$(trueorfalse False USE_SCREEN)
# Whether to use SYSTEMD to manage services
USE_SYSTEMD=$(trueorfalse False USE_SYSTEMD)
@@ -93,16 +100,11 @@
if [[ "$USER_UNITS" == "True" ]]; then
SYSTEMD_DIR="$HOME/.local/share/systemd/user"
SYSTEMCTL="systemctl --user"
- JOURNALCTL_F="journalctl -f -o short-precise --user-unit"
else
SYSTEMD_DIR="/etc/systemd/system"
SYSTEMCTL="sudo systemctl"
- JOURNALCTL_F="journalctl -f -o short-precise --unit"
fi
-if [[ "$USE_SYSTEMD" == "True" ]]; then
- USE_SCREEN=False
-fi
# Whether or not to enable Kernel Samepage Merging (KSM) if available.
# This allows programs that mark their memory as mergeable to share
@@ -157,6 +159,19 @@
source $RC_DIR/.localrc.auto
fi
+# TODO(sdague): Delete all this in Queens.
+if [[ "$USE_SYSTEMD" == "True" ]]; then
+ USE_SCREEN=False
+fi
+# if we are forcing off USE_SCREEN (as we do in the gate), force on
+# systemd. This allows us to drop one of 3 paths through the code.
+if [[ "$USE_SCREEN" == "False" ]]; then
+ # Remove in Pike: this gets us through grenade upgrade
+ if [[ "$GRENADE_PHASE" != "target" ]]; then
+ USE_SYSTEMD="True"
+ fi
+fi
+
# Default for log coloring is based on interactive-or-not.
# Baseline assumption is that non-interactive invocations are for CI,
# where logs are to be presented as browsable text files; hence color
@@ -501,14 +516,6 @@
GITREPO["glance_store"]=${GLANCE_STORE_REPO:-${GIT_BASE}/openstack/glance_store.git}
GITBRANCH["glance_store"]=${GLANCE_STORE_BRANCH:-master}
-# heat-cfntools server agent
-HEAT_CFNTOOLS_REPO=${HEAT_CFNTOOLS_REPO:-${GIT_BASE}/openstack/heat-cfntools.git}
-HEAT_CFNTOOLS_BRANCH=${HEAT_CFNTOOLS_BRANCH:-master}
-
-# heat example templates and elements
-HEAT_TEMPLATES_REPO=${HEAT_TEMPLATES_REPO:-${GIT_BASE}/openstack/heat-templates.git}
-HEAT_TEMPLATES_BRANCH=${HEAT_TEMPLATES_BRANCH:-master}
-
# django openstack_auth library
GITREPO["django_openstack_auth"]=${HORIZONAUTH_REPO:-${GIT_BASE}/openstack/django_openstack_auth.git}
GITBRANCH["django_openstack_auth"]=${HORIZONAUTH_BRANCH:-master}
@@ -600,7 +607,7 @@
# a websockets/html5 or flash powered VNC console for vm instances
NOVNC_REPO=${NOVNC_REPO:-https://github.com/kanaka/noVNC.git}
-NOVNC_BRANCH=${NOVNC_BRANCH:-master}
+NOVNC_BRANCH=${NOVNC_BRANCH:-stable/v0.6}
# a websockets/html5 or flash powered SPICE console for vm instances
SPICE_REPO=${SPICE_REPO:-http://anongit.freedesktop.org/git/spice/spice-html5.git}
@@ -687,17 +694,21 @@
case "$LIBVIRT_TYPE" in
lxc) # the cirros root disk in the uec tarball is empty, so it will not work for lxc
DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-rootfs}
- IMAGE_URLS+="http://download.cirros-cloud.net/${CIRROS_VERSION}/cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-rootfs.img.gz";;
+ DEFAULT_IMAGE_FILE_NAME=${DEFAULT_IMAGE_FILE_NAME:-cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-rootfs.img.gz}
+ IMAGE_URLS+="http://download.cirros-cloud.net/${CIRROS_VERSION}/${DEFAULT_IMAGE_FILE_NAME}";;
*) # otherwise, use the qcow image
- DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-disk.img}
- IMAGE_URLS+="http://download.cirros-cloud.net/${CIRROS_VERSION}/cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-disk.img";;
+ DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-disk}
+ DEFAULT_IMAGE_FILE_NAME=${DEFAULT_IMAGE_FILE_NAME:-cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-disk.img}
+ IMAGE_URLS+="http://download.cirros-cloud.net/${CIRROS_VERSION}/${DEFAULT_IMAGE_FILE_NAME}";;
esac
;;
vsphere)
DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-0.3.2-i386-disk.vmdk}
- IMAGE_URLS+="http://partnerweb.vmware.com/programs/vmdkimage/cirros-0.3.2-i386-disk.vmdk";;
+ DEFAULT_IMAGE_FILE_NAME=${DEFAULT_IMAGE_FILE_NAME:-$DEFAULT_IMAGE_NAME}
+ IMAGE_URLS+="http://partnerweb.vmware.com/programs/vmdkimage/${DEFAULT_IMAGE_FILE_NAME}";;
xenserver)
DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-0.3.4-x86_64-disk}
+ DEFAULT_IMAGE_FILE_NAME=${DEFAULT_IMAGE_NAME:-cirros-0.3.4-x86_64-disk.vhd.tgz}
IMAGE_URLS+="http://ca.downloads.xensource.com/OpenStack/cirros-0.3.4-x86_64-disk.vhd.tgz"
IMAGE_URLS+=",http://download.cirros-cloud.net/${CIRROS_VERSION}/cirros-${CIRROS_VERSION}-x86_64-uec.tar.gz";;
esac
@@ -912,6 +923,9 @@
# ``LOGDIR`` is always set at this point so it is not useful as a 'enable' for service logs
# ``SCREEN_LOGDIR`` may be set, it is useful to enable the compat symlinks
+# System-wide ulimit file descriptors override
+ULIMIT_NOFILE=${ULIMIT_NOFILE:-2048}
+
# Local variables:
# mode: shell-script
# End:
diff --git a/tests/test_refs.sh b/tests/test_refs.sh
index bccca5d..65848cd 100755
--- a/tests/test_refs.sh
+++ b/tests/test_refs.sh
@@ -15,7 +15,7 @@
echo "Ensuring we don't have crazy refs"
-REFS=`grep BRANCH stackrc | grep -v -- '-master'`
+REFS=`grep BRANCH stackrc | grep -v -- '-master' | grep -v 'NOVNC_BRANCH'`
rc=$?
if [[ $rc -eq 0 ]]; then
echo "Branch defaults must be master. Found:"
diff --git a/tools/dstat.sh b/tools/dstat.sh
index ae7306e..01c6d9b 100755
--- a/tools/dstat.sh
+++ b/tools/dstat.sh
@@ -13,10 +13,10 @@
LOGDIR=$1
# Command line arguments for primary DStat process.
-DSTAT_OPTS="-tcmndrylpg --top-cpu-adv --top-io-adv --top-mem --swap"
+DSTAT_OPTS="-tcmndrylpg --top-cpu-adv --top-io-adv --top-mem --swap --tcp"
# Command-line arguments for secondary background DStat process.
-DSTAT_CSV_OPTS="-tcmndrylpg --output $LOGDIR/dstat-csv.log"
+DSTAT_CSV_OPTS="-tcmndrylpg --tcp --output $LOGDIR/dstat-csv.log"
# Execute and background the secondary dstat process and discard its output.
dstat $DSTAT_CSV_OPTS >& /dev/null &
diff --git a/tools/fixup_stuff.sh b/tools/fixup_stuff.sh
index f3ba702..d07d267 100755
--- a/tools/fixup_stuff.sh
+++ b/tools/fixup_stuff.sh
@@ -82,15 +82,17 @@
source /etc/ci/mirror_info.sh
sudo apt-add-repository -y "deb $NODEPOOL_UCA_MIRROR xenial-updates/ocata main"
-
- # Disable use of libvirt wheel here as presence of mirror implies
- # presence of cached wheel build against older libvirt binary.
- # TODO(clarkb) figure out how to use wheel again.
- sudo bash -c 'echo "no-binary = libvirt-python" >> /etc/pip.conf'
else
# Otherwise use upstream UCA
sudo add-apt-repository -y cloud-archive:ocata
fi
+
+ # Disable use of libvirt wheel since a cached wheel build might be
+ # against older libvirt binary. Particularly a problem if using
+ # the openstack wheel mirrors, but can hit locally too.
+ # TODO(clarkb) figure out how to use upstream wheel again.
+ iniset -sudo /etc/pip.conf "global" "no-binary" "libvirt-python"
+
# Force update our APT repos, since we added UCA above.
REPOS_UPDATED=False
apt_get_update
diff --git a/tox.ini b/tox.ini
index 55a06d0..cc7c544 100644
--- a/tox.ini
+++ b/tox.ini
@@ -37,8 +37,8 @@
deps =
Pygments
docutils
- sphinx>=1.1.2,<1.2
- pbr>=0.6,!=0.7,<1.0
+ sphinx>=1.5.1,<1.6.1
+ pbr>=2.0.0,!=2.1.0
oslosphinx
nwdiag
blockdiag
@@ -52,8 +52,8 @@
[testenv:venv]
deps =
- pbr>=0.6,!=0.7,<1.0
- sphinx>=1.1.2,<1.2
+ pbr>=2.0.0,!=2.1.0
+ sphinx>=1.5.1,<1.6.1
oslosphinx
blockdiag
sphinxcontrib-blockdiag
diff --git a/unstack.sh b/unstack.sh
index 485fed7..77a151f 100755
--- a/unstack.sh
+++ b/unstack.sh
@@ -69,7 +69,7 @@
source $TOP_DIR/lib/neutron
source $TOP_DIR/lib/ldap
source $TOP_DIR/lib/dstat
-source $TOP_DIR/lib/dlm
+source $TOP_DIR/lib/etcd3
# Extras Source
# --------------
@@ -162,6 +162,11 @@
cleanup_neutron
fi
+if is_service_enabled etcd3; then
+ stop_etcd3
+ cleanup_etcd3
+fi
+
if is_service_enabled dstat; then
stop_dstat
fi