Merge "Fix dependency list generation corner cases"
diff --git a/AUTHORS b/AUTHORS
index a3a4b6b..5f75837 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -1,3 +1,4 @@
+Aaron Lee <aaron.lee@rackspace.com>
Adam Gandelman <adamg@canonical.com>
Andy Smith <github@anarkystic.com>
Anthony Young <sleepsonthefloor@gmail.com>
diff --git a/HACKING.rst b/HACKING.rst
new file mode 100644
index 0000000..d91d496
--- /dev/null
+++ b/HACKING.rst
@@ -0,0 +1,153 @@
+Contributing to DevStack
+========================
+
+
+General
+-------
+
+DevStack is written in POSIX shell script. This choice was made because
+it best illustrates the configuration steps that this implementation takes
+on setting up and interacting with OpenStack components. DevStack specifies
+BASH and is compatible with Bash 3.
+
+DevStack's official repository is located on GitHub at
+https://github.com/openstack-dev/devstack.git. Besides the master branch that
+tracks the OpenStack trunk branches a separate branch is maintained for all
+OpenStack releases starting with Diablo (stable/diablo).
+
+The primary script in DevStack is ``stack.sh``, which performs the bulk of the
+work for DevStack's use cases. There is a subscript ``functions`` that contains
+generally useful shell functions and is used by a number of the scripts in
+DevStack.
+
+A number of additional scripts can be found in the ``tools`` directory that may
+be useful in setting up special-case uses of DevStack. These include: bare metal
+deployment, ramdisk deployment and Jenkins integration.
+
+
+Scripts
+-------
+
+DevStack scripts should generally begin by calling ``env(1)`` in the shebang line::
+
+ #!/usr/bin/env bash
+
+Sometimes the script needs to know the location of the DevStack install directory.
+``TOP_DIR`` should always point there, even if the script itself is located in
+a subdirectory::
+
+ # Keep track of the current devstack directory.
+ TOP_DIR=$(cd $(dirname "$0") && pwd)
+
+Many scripts will utilize shared functions from the ``functions`` file. There are
+also rc files (``stackrc`` and ``openrc``) that are often included to set the primary
+configuration of the user environment::
+
+ # Use openrc + stackrc + localrc for settings
+ pushd $(cd $(dirname "$0")/.. && pwd) >/dev/null
+
+ # Import common functions
+ source ./functions
+
+ # Import configuration
+ source ./openrc
+ popd >/dev/null
+
+``stack.sh`` is a rather large monolithic script that flows through from beginning
+to end. There is a proposal to segment it to put the OpenStack projects
+into their own sub-scripts to better document the projects as a unit rather than
+have it scattered throughout ``stack.sh``. Someday.
+
+
+Documentation
+-------------
+
+The official DevStack repo on GitHub does not include a gh-pages branch that
+GitHub uses to create static web sites. That branch is maintained in the
+`CloudBuilders DevStack repo`__ mirror that supports the
+http://devstack.org site. This is the primary DevStack
+documentation along with the DevStack scripts themselves.
+
+__ repo_
+.. _repo: https://github.com/cloudbuilders/devstack
+
+All of the scripts are processed with shocco_ to render them with the comments
+as text describing the script below. For this reason we tend to be a little
+verbose in the comments _ABOVE_ the code they pertain to. Shocco also supports
+Markdown formatting in the comments; use it sparingly. Specifically, ``stack.sh``
+uses Markdown headers to divide the script into logical sections.
+
+.. _shocco: http://rtomayko.github.com/shocco/
+
+
+Exercises
+---------
+
+The scripts in the exercises directory are meant to 1) perform basic operational
+checks on certain aspects of OpenStack; and b) document the use of the
+OpenStack command-line clients.
+
+In addition to the guidelines above, exercise scripts MUST follow the structure
+outlined here. ``swift.sh`` is perhaps the clearest example of these guidelines.
+These scripts are executed serially by ``exercise.sh`` in testing situations.
+
+* Begin and end with a banner that stands out in a sea of script logs to aid
+ in debugging failures, particularly in automated testing situations. If the
+ end banner is not displayed, the script ended prematurely and can be assumed
+ to have failed.
+
+ ::
+
+ echo "**************************************************"
+ echo "Begin DevStack Exercise: $0"
+ echo "**************************************************"
+ ...
+ set +o xtrace
+ echo "**************************************************"
+ echo "End DevStack Exercise: $0"
+ echo "**************************************************"
+
+* The scripts will generally have the shell ``xtrace`` attribute set to display
+ the actual commands being executed, and the ``errexit`` attribute set to exit
+ the script on non-zero exit codes::
+
+ # This script exits on an error so that errors don't compound and you see
+ # only the first error that occured.
+ set -o errexit
+
+ # Print the commands being run so that we can see the command that triggers
+ # an error. It is also useful for following allowing as the install occurs.
+ set -o xtrace
+
+* There are a couple of helper functions in the common ``functions`` sub-script
+ that will check for non-zero exit codes and unset environment variables and
+ print a message and exit the script. These should be called after most client
+ commands that are not otherwise checked to short-circuit long timeouts
+ (instance boot failure, for example)::
+
+ swift post $CONTAINER
+ die_if_error "Failure creating container $CONTAINER"
+
+ FLOATING_IP=`euca-allocate-address | cut -f2`
+ die_if_not_set FLOATING_IP "Failure allocating floating IP"
+
+* The exercise scripts should only use the various OpenStack client binaries to
+ interact with OpenStack. This specifically excludes any ``*-manage`` tools
+ as those assume direct access to configuration and databases, as well as direct
+ database access from the exercise itself.
+
+* If specific configuration needs to be present for the exercise to complete,
+ it should be staged in ``stack.sh``, or called from ``stack.sh`` (see
+ ``files/keystone_data.sh`` for an example of this).
+
+* The ``OS_*`` environment variables should be the only ones used for all
+ authentication to OpenStack clients as documented in the CLIAuth_ wiki page.
+
+.. _CLIAuth: http://wiki.openstack.org/CLIAuth
+
+* The exercise MUST clean up after itself if successful. If it is not successful,
+ it is assumed that state will be left behind; this allows a chance for developers
+ to look around and attempt to debug the problem. The exercise SHOULD clean up
+ or graciously handle possible artifacts left over from previous runs if executed
+ again. It is acceptable to require a reboot or even a re-install of DevStack
+ to restore a clean test environment.
diff --git a/files/keystone.conf b/files/keystone.conf
index d9e639f..5e5bfeb 100644
--- a/files/keystone.conf
+++ b/files/keystone.conf
@@ -1,4 +1,5 @@
[DEFAULT]
+bind_host = 0.0.0.0
public_port = 5000
admin_port = 35357
admin_token = %SERVICE_TOKEN%
@@ -34,7 +35,7 @@
driver = keystone.token.backends.kvs.Token
[policy]
-driver = keystone.policy.backends.simple.SimpleMatch
+driver = keystone.policy.backends.rules.Policy
[ec2]
driver = keystone.contrib.ec2.backends.sql.Ec2
diff --git a/stack.sh b/stack.sh
index fa5652f..cefd635 100755
--- a/stack.sh
+++ b/stack.sh
@@ -179,7 +179,7 @@
# Default Melange Host
M_HOST=${M_HOST:-localhost}
# Melange MAC Address Range
-M_MAC_RANGE=${M_MAC_RANGE:-404040/24}
+M_MAC_RANGE=${M_MAC_RANGE:-FE-EE-DD-00-00-00/24}
# Specify which services to launch. These generally correspond to screen tabs
ENABLED_SERVICES=${ENABLED_SERVICES:-g-api,g-reg,key,n-api,n-crt,n-obj,n-cpu,n-net,n-vol,n-sch,n-novnc,n-xvnc,n-cauth,horizon,mysql,rabbit}
@@ -465,17 +465,22 @@
# Set LOGFILE to turn on logging
# We append '.xxxxxxxx' to the given name to maintain history
# where xxxxxxxx is a representation of the date the file was created
+if [[ -n "$LOGFILE" || -n "$SCREEN_LOGDIR" ]]; then
+ LOGDAYS=${LOGDAYS:-7}
+ TIMESTAMP_FORMAT=${TIMESTAMP_FORMAT:-"%F-%H%M%S"}
+ CURRENT_LOG_TIME=$(date "+$TIMESTAMP_FORMAT")
+fi
+
if [[ -n "$LOGFILE" ]]; then
# First clean up old log files. Use the user-specified LOGFILE
# as the template to search for, appending '.*' to match the date
# we added on earlier runs.
- LOGDAYS=${LOGDAYS:-7}
LOGDIR=$(dirname "$LOGFILE")
LOGNAME=$(basename "$LOGFILE")
+ mkdir -p $LOGDIR
find $LOGDIR -maxdepth 1 -name $LOGNAME.\* -mtime +$LOGDAYS -exec rm {} \;
- TIMESTAMP_FORMAT=${TIMESTAMP_FORMAT:-"%F-%H%M%S"}
- LOGFILE=$LOGFILE.$(date "+$TIMESTAMP_FORMAT")
+ LOGFILE=$LOGFILE.${CURRENT_LOG_TIME}
# Redirect stdout/stderr to tee to write the log file
exec 1> >( tee "${LOGFILE}" ) 2>&1
echo "stack.sh log $LOGFILE"
@@ -483,6 +488,23 @@
ln -sf $LOGFILE $LOGDIR/$LOGNAME
fi
+# Set up logging of screen windows
+# Set SCREEN_LOGDIR to turn on logging of screen windows to the
+# directory specified in SCREEN_LOGDIR, we will log to the the file
+# screen-$SERVICE_NAME-$TIMESTAMP.log in that dir and have a link
+# screen-$SERVICE_NAME.log to the latest log file.
+# Logs are kept for as long specified in LOGDAYS.
+if [[ -n "$SCREEN_LOGDIR" ]]; then
+
+ # We make sure the directory is created.
+ if [[ -d "$SCREEN_LOGDIR" ]]; then
+ # We cleanup the old logs
+ find $SCREEN_LOGDIR -maxdepth 1 -name screen-\*.log -mtime +$LOGDAYS -exec rm {} \;
+ else
+ mkdir -p $SCREEN_LOGDIR
+ fi
+fi
+
# So that errors don't compound we exit on any errors so you see only the
# first error that occurred.
trap failed ERR
@@ -1360,6 +1382,12 @@
# creating a new window in screen and then sends characters, so if
# bash isn't running by the time we send the command, nothing happens
sleep 1.5
+
+ if [[ -n ${SCREEN_LOGDIR} ]]; then
+ screen -S stack -p $1 -X logfile ${SCREEN_LOGDIR}/screen-${1}.${CURRENT_LOG_TIME}.log
+ screen -S stack -p $1 -X log on
+ ln -sf ${SCREEN_LOGDIR}/screen-${1}.${CURRENT_LOG_TIME}.log ${SCREEN_LOGDIR}/screen-${1}.log
+ fi
screen -S stack -p $1 -X stuff "$2$NL"
fi
}