Use stage_dir instead of /opt/stack
Historically we have collected devstack logs under /opt/stack.
Stop doing that and collect them in the stage_dir instead, so that
once the base job logs pull service comes around we are ready for it.
This add the benefit of writing things into a folder which is
already owned by the ansible user (ansible_user_dir), so we don't
run into issue writing there.
A few logs (devstack log, log summary and dstat) use to show up on
logs.o.o. just because they happened to already be in /opt/stack/logs.
With this change they would be lost, so adding them to post.yaml.
Depends-on: I5ad4dfccbc1389da3afc53f3c866d3475e006db6
Change-Id: Ib4be2f5056c0dc2b776de4a0d18b47b12624be92
diff --git a/playbooks/post.yaml b/playbooks/post.yaml
index 95c3669..aaa5cdd 100644
--- a/playbooks/post.yaml
+++ b/playbooks/post.yaml
@@ -1,14 +1,24 @@
- hosts: all
become: True
vars:
+ devstack_log_dir: "{{ devstack_base_dir|default('/opt/stack') }}/logs/"
devstack_conf_dir: "{{ devstack_base_dir|default('/opt/stack') }}/devstack/"
- stage_dir: "{{ devstack_base_dir|default('/opt/stack') }}"
roles:
- export-devstack-journal
- role: stage-output
zuul_copy_output:
{ '{{ devstack_conf_dir }}/local.conf': 'logs',
- '{{ devstack_conf_dir }}/.stackenv': 'logs' }
+ '{{ devstack_conf_dir }}/.stackenv': 'logs' ,
+ '{{ devstack_log_dir }}/dstat-csv.log': 'logs',
+ '{{ devstack_log_dir }}/devstacklog.txt': 'logs',
+ '{{ devstack_log_dir }}/devstacklog.txt.summary': 'logs' }
extensions_to_txt:
- conf
- - fetch-devstack-log-dir
+ - log
+ - summary
+ # NOTE(andreaf) We need fetch-devstack-log-dir only as long as the base job
+ # starts pulling logs for us from {{ ansible_user_dir }}/logs.
+ # Meanwhile we already store things in ansible_user_dir and use
+ # fetch-devstack-log-dir setting devstack_base_dir
+ - role: fetch-devstack-log-dir
+ devstack_base_dir: "{{ ansible_user_dir }}"
diff --git a/roles/apache-logs-conf/README.rst b/roles/apache-logs-conf/README.rst
new file mode 100644
index 0000000..eccee40
--- /dev/null
+++ b/roles/apache-logs-conf/README.rst
@@ -0,0 +1,12 @@
+Prepare apache configs and logs for staging
+
+Make sure apache config files and log files are available in a linux flavor
+independent location. Note that this relies on hard links, to the staging
+directory must be in the same partition where the logs and configs are.
+
+**Role Variables**
+
+.. zuul:rolevar:: stage_dir
+ :default: {{ ansible_user_dir }}
+
+ The base stage directory.
diff --git a/roles/export-devstack-journal/README.rst b/roles/export-devstack-journal/README.rst
index 5f00592..a34e070 100644
--- a/roles/export-devstack-journal/README.rst
+++ b/roles/export-devstack-journal/README.rst
@@ -5,11 +5,17 @@
kernal and sudo messages.
Writes the output to the ``logs/`` subdirectory of
-``devstack_base_dir``.
+``stage_dir``.
**Role Variables**
.. zuul:rolevar:: devstack_base_dir
:default: /opt/stack
- The devstack base directory.
+ The devstack base directory. This is used to obtain the
+ ``log-start-timestamp.txt``, used to filter the systemd journal.
+
+.. zuul:rolevar:: stage_dir
+ :default: {{ ansible_user_dir }}
+
+ The base stage directory.
diff --git a/roles/export-devstack-journal/defaults/main.yaml b/roles/export-devstack-journal/defaults/main.yaml
index fea05c8..1fb04fe 100644
--- a/roles/export-devstack-journal/defaults/main.yaml
+++ b/roles/export-devstack-journal/defaults/main.yaml
@@ -1 +1,2 @@
devstack_base_dir: /opt/stack
+stage_dir: "{{ ansible_user_dir }}"
diff --git a/roles/export-devstack-journal/tasks/main.yaml b/roles/export-devstack-journal/tasks/main.yaml
index b9af02a..3efa575 100644
--- a/roles/export-devstack-journal/tasks/main.yaml
+++ b/roles/export-devstack-journal/tasks/main.yaml
@@ -1,3 +1,11 @@
+# NOTE(andreaf) This bypasses the stage-output role
+- name: Ensure {{ stage_dir }}/logs exists
+ become: true
+ file:
+ path: "{{ stage_dir }}/logs"
+ state: directory
+ owner: "{{ ansible_user }}"
+
# TODO: convert this to ansible
- name: Export journal files
become: true
@@ -7,7 +15,7 @@
name=""
for u in `systemctl list-unit-files | grep devstack | awk '{print $1}'`; do
name=$(echo $u | sed 's/devstack@/screen-/' | sed 's/\.service//')
- journalctl -o short-precise --unit $u | tee {{ devstack_base_dir }}/logs/$name.txt > /dev/null
+ journalctl -o short-precise --unit $u | tee {{ stage_dir }}/logs/$name.txt > /dev/null
done
# Export the journal in export format to make it downloadable
@@ -16,7 +24,7 @@
# debugging much easier. We don't do the native conversion here as
# some distros do not package that tooling.
journalctl -u 'devstack@*' -o export | \
- xz --threads=0 - > {{ devstack_base_dir }}/logs/devstack.journal.xz
+ xz --threads=0 - > {{ stage_dir }}/logs/devstack.journal.xz
# The journal contains everything running under systemd, we'll
# build an old school version of the syslog with just the
@@ -26,4 +34,4 @@
-t sudo \
--no-pager \
--since="$(cat {{ devstack_base_dir }}/log-start-timestamp.txt)" \
- | tee {{ devstack_base_dir }}/logs/syslog.txt > /dev/null
+ | tee {{ stage_dir }}/logs/syslog.txt > /dev/null