Cleanup the tempest readme
This commit makes several needed cleanups and improvements to the
tempest readme to make things clearer and elaborate on some sections.
Also as the readme is intended to be a tempest users guide, this moves
sections which are more developer focused to the hacking guide.
Change-Id: I4180ce18268443873fe7d8d5e7d0aaebc2cd1e08
diff --git a/HACKING.rst b/HACKING.rst
index 81a7c2c..04b5eb6 100644
--- a/HACKING.rst
+++ b/HACKING.rst
@@ -312,3 +312,57 @@
* Boot an additional instance from the new snapshot based volume
* Check written content in the instance booted from snapshot
"""
+
+Branchless Tempest Considerations
+---------------------------------
+
+Starting with the OpenStack Icehouse release Tempest no longer has any stable
+branches. This is to better ensure API consistency between releases because
+the API behavior should not change between releases. This means that the stable
+branches are also gated by the Tempest master branch, which also means that
+proposed commits to Tempest must work against both the master and all the
+currently supported stable branches of the projects. As such there are a few
+special considerations that have to be accounted for when pushing new changes
+to tempest.
+
+1. New Tests for new features
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When adding tests for new features that were not in previous releases of the
+projects the new test has to be properly skipped with a feature flag. Whether
+this is just as simple as using the @test.requires_ext() decorator to check
+if the required extension (or discoverable optional API) is enabled or adding
+a new config option to the appropriate section. If there isn't a method of
+selecting the new **feature** from the config file then there won't be a
+mechanism to disable the test with older stable releases and the new test won't
+be able to merge.
+
+2. Bug fix on core project needing Tempest changes
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When trying to land a bug fix which changes a tested API you'll have to use the
+following procedure::
+
+ - Propose change to the project, get a +2 on the change even with failing
+ - Propose skip on Tempest which will only be approved after the
+ corresponding change in the project has a +2 on change
+ - Land project change in master and all open stable branches (if required)
+ - Land changed test in Tempest
+
+Otherwise the bug fix won't be able to land in the project.
+
+3. New Tests for existing features
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If a test is being added for a feature that exists in all the current releases
+of the projects then the only concern is that the API behavior is the same
+across all the versions of the project being tested. If the behavior is not
+consistent the test will not be able to merge.
+
+API Stability
+-------------
+
+For new tests being added to Tempest the assumption is that the API being
+tested is considered stable and adheres to the OpenStack API stability
+guidelines. If an API is still considered experimental or in development then
+it should not be tested by Tempest until it is considered stable.