Merge "Moves python-cloudfiles dependency back to PIP."
diff --git a/AUTHORS b/AUTHORS
index a3a4b6b..def453d 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -1,3 +1,4 @@
+Aaron Lee <aaron.lee@rackspace.com>
Adam Gandelman <adamg@canonical.com>
Andy Smith <github@anarkystic.com>
Anthony Young <sleepsonthefloor@gmail.com>
@@ -16,6 +17,7 @@
Jesse Andrews <anotherjesse@gmail.com>
Johannes Erdfelt <johannes.erdfelt@rackspace.com>
Justin Shepherd <galstrom21@gmail.com>
+Ken Pepple <ken.pepple@rabbityard.com>
Kiall Mac Innes <kiall@managedit.ie>
Russell Bryant <rbryant@redhat.com>
Scott Moser <smoser@ubuntu.com>
diff --git a/HACKING.rst b/HACKING.rst
new file mode 100644
index 0000000..d91d496
--- /dev/null
+++ b/HACKING.rst
@@ -0,0 +1,153 @@
+Contributing to DevStack
+========================
+
+
+General
+-------
+
+DevStack is written in POSIX shell script. This choice was made because
+it best illustrates the configuration steps that this implementation takes
+on setting up and interacting with OpenStack components. DevStack specifies
+BASH and is compatible with Bash 3.
+
+DevStack's official repository is located on GitHub at
+https://github.com/openstack-dev/devstack.git. Besides the master branch that
+tracks the OpenStack trunk branches a separate branch is maintained for all
+OpenStack releases starting with Diablo (stable/diablo).
+
+The primary script in DevStack is ``stack.sh``, which performs the bulk of the
+work for DevStack's use cases. There is a subscript ``functions`` that contains
+generally useful shell functions and is used by a number of the scripts in
+DevStack.
+
+A number of additional scripts can be found in the ``tools`` directory that may
+be useful in setting up special-case uses of DevStack. These include: bare metal
+deployment, ramdisk deployment and Jenkins integration.
+
+
+Scripts
+-------
+
+DevStack scripts should generally begin by calling ``env(1)`` in the shebang line::
+
+ #!/usr/bin/env bash
+
+Sometimes the script needs to know the location of the DevStack install directory.
+``TOP_DIR`` should always point there, even if the script itself is located in
+a subdirectory::
+
+ # Keep track of the current devstack directory.
+ TOP_DIR=$(cd $(dirname "$0") && pwd)
+
+Many scripts will utilize shared functions from the ``functions`` file. There are
+also rc files (``stackrc`` and ``openrc``) that are often included to set the primary
+configuration of the user environment::
+
+ # Use openrc + stackrc + localrc for settings
+ pushd $(cd $(dirname "$0")/.. && pwd) >/dev/null
+
+ # Import common functions
+ source ./functions
+
+ # Import configuration
+ source ./openrc
+ popd >/dev/null
+
+``stack.sh`` is a rather large monolithic script that flows through from beginning
+to end. There is a proposal to segment it to put the OpenStack projects
+into their own sub-scripts to better document the projects as a unit rather than
+have it scattered throughout ``stack.sh``. Someday.
+
+
+Documentation
+-------------
+
+The official DevStack repo on GitHub does not include a gh-pages branch that
+GitHub uses to create static web sites. That branch is maintained in the
+`CloudBuilders DevStack repo`__ mirror that supports the
+http://devstack.org site. This is the primary DevStack
+documentation along with the DevStack scripts themselves.
+
+__ repo_
+.. _repo: https://github.com/cloudbuilders/devstack
+
+All of the scripts are processed with shocco_ to render them with the comments
+as text describing the script below. For this reason we tend to be a little
+verbose in the comments _ABOVE_ the code they pertain to. Shocco also supports
+Markdown formatting in the comments; use it sparingly. Specifically, ``stack.sh``
+uses Markdown headers to divide the script into logical sections.
+
+.. _shocco: http://rtomayko.github.com/shocco/
+
+
+Exercises
+---------
+
+The scripts in the exercises directory are meant to 1) perform basic operational
+checks on certain aspects of OpenStack; and b) document the use of the
+OpenStack command-line clients.
+
+In addition to the guidelines above, exercise scripts MUST follow the structure
+outlined here. ``swift.sh`` is perhaps the clearest example of these guidelines.
+These scripts are executed serially by ``exercise.sh`` in testing situations.
+
+* Begin and end with a banner that stands out in a sea of script logs to aid
+ in debugging failures, particularly in automated testing situations. If the
+ end banner is not displayed, the script ended prematurely and can be assumed
+ to have failed.
+
+ ::
+
+ echo "**************************************************"
+ echo "Begin DevStack Exercise: $0"
+ echo "**************************************************"
+ ...
+ set +o xtrace
+ echo "**************************************************"
+ echo "End DevStack Exercise: $0"
+ echo "**************************************************"
+
+* The scripts will generally have the shell ``xtrace`` attribute set to display
+ the actual commands being executed, and the ``errexit`` attribute set to exit
+ the script on non-zero exit codes::
+
+ # This script exits on an error so that errors don't compound and you see
+ # only the first error that occured.
+ set -o errexit
+
+ # Print the commands being run so that we can see the command that triggers
+ # an error. It is also useful for following allowing as the install occurs.
+ set -o xtrace
+
+* There are a couple of helper functions in the common ``functions`` sub-script
+ that will check for non-zero exit codes and unset environment variables and
+ print a message and exit the script. These should be called after most client
+ commands that are not otherwise checked to short-circuit long timeouts
+ (instance boot failure, for example)::
+
+ swift post $CONTAINER
+ die_if_error "Failure creating container $CONTAINER"
+
+ FLOATING_IP=`euca-allocate-address | cut -f2`
+ die_if_not_set FLOATING_IP "Failure allocating floating IP"
+
+* The exercise scripts should only use the various OpenStack client binaries to
+ interact with OpenStack. This specifically excludes any ``*-manage`` tools
+ as those assume direct access to configuration and databases, as well as direct
+ database access from the exercise itself.
+
+* If specific configuration needs to be present for the exercise to complete,
+ it should be staged in ``stack.sh``, or called from ``stack.sh`` (see
+ ``files/keystone_data.sh`` for an example of this).
+
+* The ``OS_*`` environment variables should be the only ones used for all
+ authentication to OpenStack clients as documented in the CLIAuth_ wiki page.
+
+.. _CLIAuth: http://wiki.openstack.org/CLIAuth
+
+* The exercise MUST clean up after itself if successful. If it is not successful,
+ it is assumed that state will be left behind; this allows a chance for developers
+ to look around and attempt to debug the problem. The exercise SHOULD clean up
+ or graciously handle possible artifacts left over from previous runs if executed
+ again. It is acceptable to require a reboot or even a re-install of DevStack
+ to restore a clean test environment.
diff --git a/README.md b/README.md
index 34eb45f..e311bb8 100644
--- a/README.md
+++ b/README.md
@@ -41,6 +41,11 @@
. openrc
# list instances
nova list
+
+If the EC2 API is your cup-o-tea, you can create credentials and use euca2ools:
+
+ # source eucarc to generate EC2 credentials and set up the environment
+ . eucarc
# list instances using ec2 api
euca-describe-instances
diff --git a/eucarc b/eucarc
new file mode 100644
index 0000000..2b0f7dd
--- /dev/null
+++ b/eucarc
@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+#
+# source eucarc [username] [tenantname]
+#
+# Create EC2 credentials for the current user as defined by OS_TENANT_NAME:OS_USERNAME
+# Optionally set the tenant/username via openrc
+
+if [[ -n "$1" ]]; then
+ USERNAME=$1
+fi
+if [[ -n "$2" ]]; then
+ TENANT=$2
+fi
+
+# Find the other rc files
+RC_DIR=$(cd $(dirname "$BASH_SOURCE") && pwd)
+
+# Get user configuration
+source $RC_DIR/openrc
+
+# Set the ec2 url so euca2ools works
+export EC2_URL=$(keystone catalog --service ec2 | awk '/ publicURL / { print $4 }')
+
+# Create EC2 credentials for the current user
+CREDS=$(keystone ec2-credentials-create)
+export EC2_ACCESS_KEY=$(echo "$CREDS" | awk '/ access / { print $4 }')
+export EC2_SECRET_KEY=$(echo "$CREDS" | awk '/ secret / { print $4 }')
+
+# Euca2ools Certificate stuff for uploading bundles
+# See exercises/bundle.sh to see how to get certs using nova cli
+NOVA_KEY_DIR=${NOVA_KEY_DIR:-$RC_DIR}
+export S3_URL=$(keystone catalog --service s3 | awk '/ publicURL / { print $4 }')
+export EC2_USER_ID=42 # nova does not use user id, but bundling requires it
+export EC2_PRIVATE_KEY=${NOVA_KEY_DIR}/pk.pem
+export EC2_CERT=${NOVA_KEY_DIR}/cert.pem
+export NOVA_CERT=${NOVA_KEY_DIR}/cacert.pem
+export EUCALYPTUS_CERT=${NOVA_CERT} # euca-bundle-image seems to require this set
+alias ec2-bundle-image="ec2-bundle-image --cert ${EC2_CERT} --privatekey ${EC2_PRIVATE_KEY} --user ${EC2_USER_ID} --ec2cert ${NOVA_CERT}"
+alias ec2-upload-bundle="ec2-upload-bundle -a ${EC2_ACCESS_KEY} -s ${EC2_SECRET_KEY} --url ${S3_URL} --ec2cert ${NOVA_CERT}"
+
diff --git a/exercises/bundle.sh b/exercises/bundle.sh
index e1c949c..47bacac 100755
--- a/exercises/bundle.sh
+++ b/exercises/bundle.sh
@@ -18,24 +18,24 @@
# Settings
# ========
-# Use openrc + stackrc + localrc for settings
-pushd $(cd $(dirname "$0")/.. && pwd) >/dev/null
+# Keep track of the current directory
+EXERCISE_DIR=$(cd $(dirname "$0") && pwd)
+TOP_DIR=$(cd $EXERCISE_DIR/..; pwd)
# Import common functions
-source ./functions
+source $TOP_DIR/functions
-# Import configuration
-source ./openrc
+# Import EC2 configuration
+source $TOP_DIR/eucarc
# Remove old certificates
-rm -f cacert.pem
-rm -f cert.pem
-rm -f pk.pem
+rm -f $TOP_DIR/cacert.pem
+rm -f $TOP_DIR/cert.pem
+rm -f $TOP_DIR/pk.pem
# Get Certificates
-nova x509-get-root-cert
-nova x509-create-cert
-popd >/dev/null
+nova x509-get-root-cert $TOP_DIR/cacert.pem
+nova x509-create-cert $TOP_DIR/pk.pem $TOP_DIR/cert.pem
# Max time to wait for image to be registered
REGISTER_TIMEOUT=${REGISTER_TIMEOUT:-15}
diff --git a/exercises/client-env.sh b/exercises/client-env.sh
index 28c4d95..d4ba702 100755
--- a/exercises/client-env.sh
+++ b/exercises/client-env.sh
@@ -12,15 +12,15 @@
# Settings
# ========
-# Use openrc + stackrc + localrc for settings
-pushd $(cd $(dirname "$0")/.. && pwd) >/dev/null
+# Keep track of the current directory
+EXERCISE_DIR=$(cd $(dirname "$0") && pwd)
+TOP_DIR=$(cd $EXERCISE_DIR/..; pwd)
# Import common functions
-source ./functions
+source $TOP_DIR/functions
# Import configuration
-source ./openrc
-popd >/dev/null
+source $TOP_DIR/openrc
# Unset all of the known NOVA_ vars
unset NOVA_API_KEY
@@ -53,7 +53,7 @@
STATUS_KEYSTONE="Skipped"
else
echo -e "\nTest Keystone"
- if keystone service-list; then
+ if keystone catalog --service identity; then
STATUS_KEYSTONE="Succeeded"
else
STATUS_KEYSTONE="Failed"
@@ -68,7 +68,9 @@
if [[ "$ENABLED_SERVICES" =~ "n-api" ]]; then
if [[ "$SKIP_EXERCISES" =~ "n-api" ]] ; then
STATUS_NOVA="Skipped"
+ STATUS_EC2="Skipped"
else
+ # Test OSAPI
echo -e "\nTest Nova"
if nova flavor-list; then
STATUS_NOVA="Succeeded"
@@ -76,6 +78,21 @@
STATUS_NOVA="Failed"
RETURN=1
fi
+
+ # Test EC2 API
+ echo -e "\nTest EC2"
+ # Get EC2 creds
+ source $TOP_DIR/eucarc
+
+ if euca-describe-images; then
+ STATUS_EC2="Succeeded"
+ else
+ STATUS_EC2="Failed"
+ RETURN=1
+ fi
+
+ # Clean up side effects
+ unset NOVA_VERSION
fi
fi
@@ -125,6 +142,7 @@
echo -e "\n"
report "Keystone" $STATUS_KEYSTONE
report "Nova" $STATUS_NOVA
+report "EC2" $STATUS_EC2
report "Glance" $STATUS_GLANCE
report "Swift" $STATUS_SWIFT
diff --git a/exercises/euca.sh b/exercises/euca.sh
index b766bab..2be2f62 100755
--- a/exercises/euca.sh
+++ b/exercises/euca.sh
@@ -18,15 +18,15 @@
# Settings
# ========
-# Use openrc + stackrc + localrc for settings
-pushd $(cd $(dirname "$0")/.. && pwd) >/dev/null
+# Keep track of the current directory
+EXERCISE_DIR=$(cd $(dirname "$0") && pwd)
+TOP_DIR=$(cd $EXERCISE_DIR/..; pwd)
# Import common functions
-source ./functions
+source $TOP_DIR/functions
-# Import configuration
-source ./openrc
-popd >/dev/null
+# Import EC2 configuration
+source $TOP_DIR/eucarc
# Max time to wait while vm goes from build to active state
ACTIVE_TIMEOUT=${ACTIVE_TIMEOUT:-30}
diff --git a/files/default_catalog.templates b/files/default_catalog.templates
index b9b1844..0dfd4fc 100644
--- a/files/default_catalog.templates
+++ b/files/default_catalog.templates
@@ -24,6 +24,12 @@
catalog.RegionOne.ec2.name = 'EC2 Service'
+catalog.RegionOne.s3.publicURL = http://%SERVICE_HOST%:3333
+catalog.RegionOne.s3.adminURL = http://%SERVICE_HOST%:3333
+catalog.RegionOne.s3.internalURL = http://%SERVICE_HOST%:3333
+catalog.RegionOne.s3.name = 'S3 Service'
+
+
catalog.RegionOne.image.publicURL = http://%SERVICE_HOST%:9292/v1
catalog.RegionOne.image.adminURL = http://%SERVICE_HOST%:9292/v1
catalog.RegionOne.image.internalURL = http://%SERVICE_HOST%:9292/v1
diff --git a/files/keystone.conf b/files/keystone.conf
index 64f0a66..5e5bfeb 100644
--- a/files/keystone.conf
+++ b/files/keystone.conf
@@ -35,7 +35,7 @@
driver = keystone.token.backends.kvs.Token
[policy]
-driver = keystone.policy.backends.simple.SimpleMatch
+driver = keystone.policy.backends.rules.Policy
[ec2]
driver = keystone.contrib.ec2.backends.sql.Ec2
diff --git a/files/keystone_data.sh b/files/keystone_data.sh
index 958d2af..e97ad73 100755
--- a/files/keystone_data.sh
+++ b/files/keystone_data.sh
@@ -2,9 +2,6 @@
#
# Initial data for Keystone using python-keystoneclient
#
-# A set of EC2-compatible credentials is created for both admin and demo
-# users and placed in $DEVSTACK_DIR/ec2rc.
-#
# Tenant User Roles
# -------------------------------------------------------
# admin admin admin
@@ -48,6 +45,7 @@
--pass="$ADMIN_PASSWORD" \
--email=demo@example.com)
+
# Roles
ADMIN_ROLE=$(get_id keystone role-create --name=admin)
KEYSTONEADMIN_ROLE=$(get_id keystone role-create --name=KeystoneAdmin)
@@ -73,14 +71,7 @@
keystone user-role-add --user $DEMO_USER --role $MEMBER_ROLE --tenant_id $INVIS_TENANT
-# Services
-keystone service-create --name=keystone \
- --type=identity \
- --description="Keystone Identity Service"
-
-keystone service-create --name=nova \
- --type=compute \
- --description="Nova Compute Service"
+# Configure service users/roles
NOVA_USER=$(get_id keystone user-create --name=nova \
--pass="$SERVICE_PASSWORD" \
--tenant_id $SERVICE_TENANT \
@@ -89,13 +80,6 @@
--user $NOVA_USER \
--role $ADMIN_ROLE
-keystone service-create --name=ec2 \
- --type=ec2 \
- --description="EC2 Compatibility Layer"
-
-keystone service-create --name=glance \
- --type=image \
- --description="Glance Image Service"
GLANCE_USER=$(get_id keystone user-create --name=glance \
--pass="$SERVICE_PASSWORD" \
--tenant_id $SERVICE_TENANT \
@@ -104,16 +88,7 @@
--user $GLANCE_USER \
--role $ADMIN_ROLE
-if [[ "$ENABLED_SERVICES" =~ "n-vol" ]]; then
- keystone service-create --name="nova-volume" \
- --type=volume \
- --description="Nova Volume Service"
-fi
-
if [[ "$ENABLED_SERVICES" =~ "swift" ]]; then
- keystone service-create --name=swift \
- --type="object-store" \
- --description="Swift Service"
SWIFT_USER=$(get_id keystone user-create --name=swift \
--pass="$SERVICE_PASSWORD" \
--tenant_id $SERVICE_TENANT \
@@ -124,9 +99,6 @@
fi
if [[ "$ENABLED_SERVICES" =~ "quantum" ]]; then
- keystone service-create --name=quantum \
- --type=network \
- --description="Quantum Service"
QUANTUM_USER=$(get_id keystone user-create --name=quantum \
--pass="$SERVICE_PASSWORD" \
--tenant_id $SERVICE_TENANT \
@@ -135,20 +107,3 @@
--user $QUANTUM_USER \
--role $ADMIN_ROLE
fi
-
-# create ec2 creds and parse the secret and access key returned
-RESULT=$(keystone ec2-credentials-create --tenant_id=$ADMIN_TENANT --user=$ADMIN_USER)
-ADMIN_ACCESS=$(echo "$RESULT" | awk '/ access / { print $4 }')
-ADMIN_SECRET=$(echo "$RESULT" | awk '/ secret / { print $4 }')
-
-RESULT=$(keystone ec2-credentials-create --tenant_id=$DEMO_TENANT --user=$DEMO_USER)
-DEMO_ACCESS=$(echo "$RESULT" | awk '/ access / { print $4 }')
-DEMO_SECRET=$(echo "$RESULT" | awk '/ secret / { print $4 }')
-
-# write the secret and access to ec2rc
-cat > $DEVSTACK_DIR/ec2rc <<EOF
-ADMIN_ACCESS=$ADMIN_ACCESS
-ADMIN_SECRET=$ADMIN_SECRET
-DEMO_ACCESS=$DEMO_ACCESS
-DEMO_SECRET=$DEMO_SECRET
-EOF
diff --git a/functions b/functions
index adcf5bd..c4d56a2 100644
--- a/functions
+++ b/functions
@@ -4,7 +4,7 @@
# apt-get wrapper to set arguments correctly
# apt_get package [package ...]
function apt_get() {
- [[ "$OFFLINE" = "True" ]] && return
+ [[ "$OFFLINE" = "True" || -z "$@" ]] && return
local sudo="sudo"
[[ "$(id -u)" = "0" ]] && sudo="env"
$sudo DEBIAN_FRONTEND=noninteractive \
@@ -124,7 +124,7 @@
# pip install wrapper to set cache and proxy environment variables
# pip_install package [package ...]
function pip_install {
- [[ "$OFFLINE" = "True" ]] && return
+ [[ "$OFFLINE" = "True" || -z "$@" ]] && return
sudo PIP_DOWNLOAD_CACHE=/var/cache/pip \
HTTP_PROXY=$http_proxy \
HTTPS_PROXY=$https_proxy \
diff --git a/openrc b/openrc
index d9e7c92..188ffc5 100644
--- a/openrc
+++ b/openrc
@@ -17,8 +17,11 @@
TENANT=$2
fi
+# Find the other rc files
+RC_DIR=$(cd $(dirname "$BASH_SOURCE") && pwd)
+
# Load local configuration
-source ./stackrc
+source $RC_DIR/stackrc
# The introduction of Keystone to the OpenStack ecosystem has standardized the
# term **tenant** as the entity that owns resources. In some places references
@@ -59,30 +62,8 @@
# In the future this will change names:
export COMPUTE_API_VERSION=${COMPUTE_API_VERSION:-$NOVA_VERSION}
-# Set the ec2 url so euca2ools works
-export EC2_URL=${EC2_URL:-http://$SERVICE_HOST:8773/services/Cloud}
-
-# Access key is set in the initial keystone data to be the same as username
-export EC2_ACCESS_KEY=${DEMO_ACCESS}
-
-# Secret key is set in the initial keystone data to the admin password
-export EC2_SECRET_KEY=${DEMO_SECRET}
-
-# Euca2ools Certificate stuff for uploading bundles
-# See exercises/bundle.sh to see how to get certs using nova cli
-NOVARC=$(readlink -f "${BASH_SOURCE:-${0}}" 2>/dev/null) ||
- NOVARC=$(python -c 'import os,sys; print os.path.abspath(os.path.realpath(sys.argv[1]))' "${BASH_SOURCE:-${0}}")
-NOVA_KEY_DIR=${NOVARC%/*}
-export S3_URL=http://$SERVICE_HOST:3333
-export EC2_USER_ID=42 # nova does not use user id, but bundling requires it
-export EC2_PRIVATE_KEY=${NOVA_KEY_DIR}/pk.pem
-export EC2_CERT=${NOVA_KEY_DIR}/cert.pem
-export NOVA_CERT=${NOVA_KEY_DIR}/cacert.pem
-export EUCALYPTUS_CERT=${NOVA_CERT} # euca-bundle-image seems to require this set
-alias ec2-bundle-image="ec2-bundle-image --cert ${EC2_CERT} --privatekey ${EC2_PRIVATE_KEY} --user 42 --ec2cert ${NOVA_CERT}"
-alias ec2-upload-bundle="ec2-upload-bundle -a ${EC2_ACCESS_KEY} -s ${EC2_SECRET_KEY} --url ${S3_URL} --ec2cert ${NOVA_CERT}"
-
# set log level to DEBUG (helps debug issues)
+# export KEYSTONECLIENT_DEBUG=1
# export NOVACLIENT_DEBUG=1
# Max time till the vm is bootable
diff --git a/stack.sh b/stack.sh
index eead8a1..991ea00 100755
--- a/stack.sh
+++ b/stack.sh
@@ -179,7 +179,7 @@
# Default Melange Host
M_HOST=${M_HOST:-localhost}
# Melange MAC Address Range
-M_MAC_RANGE=${M_MAC_RANGE:-404040/24}
+M_MAC_RANGE=${M_MAC_RANGE:-FE-EE-DD-00-00-00/24}
# Specify which services to launch. These generally correspond to screen tabs
ENABLED_SERVICES=${ENABLED_SERVICES:-g-api,g-reg,key,n-api,n-crt,n-obj,n-cpu,n-net,n-vol,n-sch,n-novnc,n-xvnc,n-cauth,horizon,mysql,rabbit}
@@ -465,17 +465,22 @@
# Set LOGFILE to turn on logging
# We append '.xxxxxxxx' to the given name to maintain history
# where xxxxxxxx is a representation of the date the file was created
+if [[ -n "$LOGFILE" || -n "$SCREEN_LOGDIR" ]]; then
+ LOGDAYS=${LOGDAYS:-7}
+ TIMESTAMP_FORMAT=${TIMESTAMP_FORMAT:-"%F-%H%M%S"}
+ CURRENT_LOG_TIME=$(date "+$TIMESTAMP_FORMAT")
+fi
+
if [[ -n "$LOGFILE" ]]; then
# First clean up old log files. Use the user-specified LOGFILE
# as the template to search for, appending '.*' to match the date
# we added on earlier runs.
- LOGDAYS=${LOGDAYS:-7}
LOGDIR=$(dirname "$LOGFILE")
LOGNAME=$(basename "$LOGFILE")
+ mkdir -p $LOGDIR
find $LOGDIR -maxdepth 1 -name $LOGNAME.\* -mtime +$LOGDAYS -exec rm {} \;
- TIMESTAMP_FORMAT=${TIMESTAMP_FORMAT:-"%F-%H%M%S"}
- LOGFILE=$LOGFILE.$(date "+$TIMESTAMP_FORMAT")
+ LOGFILE=$LOGFILE.${CURRENT_LOG_TIME}
# Redirect stdout/stderr to tee to write the log file
exec 1> >( tee "${LOGFILE}" ) 2>&1
echo "stack.sh log $LOGFILE"
@@ -483,6 +488,23 @@
ln -sf $LOGFILE $LOGDIR/$LOGNAME
fi
+# Set up logging of screen windows
+# Set SCREEN_LOGDIR to turn on logging of screen windows to the
+# directory specified in SCREEN_LOGDIR, we will log to the the file
+# screen-$SERVICE_NAME-$TIMESTAMP.log in that dir and have a link
+# screen-$SERVICE_NAME.log to the latest log file.
+# Logs are kept for as long specified in LOGDAYS.
+if [[ -n "$SCREEN_LOGDIR" ]]; then
+
+ # We make sure the directory is created.
+ if [[ -d "$SCREEN_LOGDIR" ]]; then
+ # We cleanup the old logs
+ find $SCREEN_LOGDIR -maxdepth 1 -name screen-\*.log -mtime +$LOGDAYS -exec rm {} \;
+ else
+ mkdir -p $SCREEN_LOGDIR
+ fi
+fi
+
# So that errors don't compound we exit on any errors so you see only the
# first error that occurred.
trap failed ERR
@@ -516,12 +538,16 @@
# dist:DISTRO1,DISTRO2 it will be installed only for those
# distros (case insensitive).
function get_packages() {
- local file_to_parse="general"
+ local package_dir=$1
+ local file_to_parse
local service
- for service in ${ENABLED_SERVICES//,/ }; do
- # Allow individual services to specify dependencies
- if [[ -e $FILES/apts/${service} ]]; then
+ if [[ -z "$package_dir" ]]; then
+ echo "No package directory supplied"
+ return 1
+ fi
+ for service in general ${ENABLED_SERVICES//,/ }; do # Allow individual services to specify dependencies
+ if [[ -e ${package_dir}/${service} ]]; then
file_to_parse="${file_to_parse} $service"
fi
if [[ $service == n-* ]]; then
@@ -540,9 +566,9 @@
done
for file in ${file_to_parse}; do
- local fname=${FILES}/apts/${file}
+ local fname=${package_dir}/${file}
local OIFS line package distros distro
- [[ -e $fname ]] || { echo "missing: $fname"; exit 1 ;}
+ [[ -e $fname ]] || continue
OIFS=$IFS
IFS=$'\n'
@@ -568,10 +594,10 @@
# install apt requirements
apt_get update
-apt_get install $(get_packages)
+apt_get install $(get_packages $FILES/apts)
# install python requirements
-pip_install `cat $FILES/pips/* | uniq`
+pip_install $(get_packages $FILES/pips | sort -u)
# compute service
git_clone $NOVA_REPO $NOVA_DIR $NOVA_BRANCH
@@ -886,6 +912,9 @@
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
apt_get install libvirt-bin
+ # Force IP forwarding on, just on case
+ sudo sysctl -w net.ipv4.ip_forward=1
+
# attempt to load modules: network block device - used to manage qcow images
sudo modprobe nbd || true
@@ -967,6 +996,9 @@
clean_iptables
rm -rf $NOVA_DIR/networks
mkdir -p $NOVA_DIR/networks
+
+ # Force IP forwarding on, just on case
+ sudo sysctl -w net.ipv4.ip_forward=1
fi
# Storage Service
@@ -1356,6 +1388,12 @@
# creating a new window in screen and then sends characters, so if
# bash isn't running by the time we send the command, nothing happens
sleep 1.5
+
+ if [[ -n ${SCREEN_LOGDIR} ]]; then
+ screen -S stack -p $1 -X logfile ${SCREEN_LOGDIR}/screen-${1}.${CURRENT_LOG_TIME}.log
+ screen -S stack -p $1 -X log on
+ ln -sf ${SCREEN_LOGDIR}/screen-${1}.${CURRENT_LOG_TIME}.log ${SCREEN_LOGDIR}/screen-${1}.log
+ fi
screen -S stack -p $1 -X stuff "$2$NL"
fi
}
@@ -1519,7 +1557,7 @@
# If we're using Quantum (i.e. q-svc is enabled), network creation has to
# happen after we've started the Quantum service.
-if is_service_enabled mysql; then
+if is_service_enabled mysql && is_service_enabled nova; then
# create a small network
$NOVA_DIR/bin/nova-manage network create private $FIXED_RANGE 1 $FIXED_NETWORK_SIZE
@@ -1540,24 +1578,17 @@
# ``libvirtd`` to our user in this script, when nova-compute is run it is
# within the context of our original shell (so our groups won't be updated).
# Use 'sg' to execute nova-compute as a member of the libvirtd group.
+# We don't check for is_service_enable as screen_it does it for us
screen_it n-cpu "cd $NOVA_DIR && sg libvirtd $NOVA_DIR/bin/nova-compute"
screen_it n-crt "cd $NOVA_DIR && $NOVA_DIR/bin/nova-cert"
screen_it n-obj "cd $NOVA_DIR && $NOVA_DIR/bin/nova-objectstore"
screen_it n-vol "cd $NOVA_DIR && $NOVA_DIR/bin/nova-volume"
screen_it n-net "cd $NOVA_DIR && $NOVA_DIR/bin/nova-network"
screen_it n-sch "cd $NOVA_DIR && $NOVA_DIR/bin/nova-scheduler"
-if is_service_enabled n-novnc; then
- screen_it n-novnc "cd $NOVNC_DIR && ./utils/nova-novncproxy --config-file $NOVA_CONF_DIR/$NOVA_CONF --web ."
-fi
-if is_service_enabled n-xvnc; then
- screen_it n-xvnc "cd $NOVA_DIR && ./bin/nova-xvpvncproxy --config-file $NOVA_CONF_DIR/$NOVA_CONF"
-fi
-if is_service_enabled n-cauth; then
- screen_it n-cauth "cd $NOVA_DIR && ./bin/nova-consoleauth"
-fi
-if is_service_enabled horizon; then
- screen_it horizon "cd $HORIZON_DIR && sudo tail -f /var/log/apache2/error.log"
-fi
+screen_it n-novnc "cd $NOVNC_DIR && ./utils/nova-novncproxy --config-file $NOVA_CONF_DIR/$NOVA_CONF --web ."
+screen_it n-xvnc "cd $NOVA_DIR && ./bin/nova-xvpvncproxy --config-file $NOVA_CONF_DIR/$NOVA_CONF"
+screen_it n-cauth "cd $NOVA_DIR && ./bin/nova-consoleauth"
+screen_it horizon "cd $HORIZON_DIR && sudo tail -f /var/log/apache2/error.log"
# Install Images
# ==============
diff --git a/stackrc b/stackrc
index a20426b..8df3b83 100644
--- a/stackrc
+++ b/stackrc
@@ -1,3 +1,6 @@
+# Find the other rc files
+RC_DIR=$(cd $(dirname "$BASH_SOURCE") && pwd)
+
# compute service
NOVA_REPO=https://github.com/openstack/nova.git
NOVA_BRANCH=master
@@ -76,12 +79,7 @@
IMAGE_URLS="http://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz";;
esac
-# use stored ec2 env variables
-if [ -f ./ec2rc ]; then
- source ./ec2rc
-fi
-
# allow local overrides of env variables
-if [ -f ./localrc ]; then
- source ./localrc
+if [ -f $RC_DIR/localrc ]; then
+ source $RC_DIR/localrc
fi