Send swift upload instructions to workers
Have zuul send signed credentials as part of the job for workers to
consume and upload assets to a defined location.
Zuul currently doesn't care about logs however this change will
suggest a LOG_PATH to builders as a unqiue destination prefix
allowing zuul to know preemptively the destination.
The workers are still required to send a URL of the final location.
Change-Id: I042cdd2dd2407f381cafcabc5c6b83d9b9a9eb00
diff --git a/doc/source/launchers.rst b/doc/source/launchers.rst
index f3e45db..accd80b 100644
--- a/doc/source/launchers.rst
+++ b/doc/source/launchers.rst
@@ -11,6 +11,8 @@
.. _`Turbo-Hipster Documentation`:
http://turbo-hipster.rtfd.org/
+.. _FormPost: http://docs.openstack.org/developer/swift/misc.html#module-swift.common.middleware.formpost
+
.. _launchers:
Launchers
@@ -117,6 +119,34 @@
Your jobs can check whether the parameters are ``000000`` to act
differently on each kind of event.
+Swift parameters
+~~~~~~~~~~~~~~~~
+
+If swift information has been configured for the job zuul will also
+provide signed credentials for the builder to upload results and
+assets into containers using the `FormPost`_ middleware.
+
+Each zuul container/instruction set will contain each of the following
+parameters where $NAME is the ``name`` defined in the layout.
+
+*SWIFT_$NAME_URL*
+ The swift destination URL. This will be the entire URL including
+ the AUTH, container and path prefix (folder).
+*SWIFT_$NAME_HMAC_BODY*
+ The information signed in the HMAC body. The body is as follows::
+
+ PATH TO OBJECT PREFIX (excluding domain)
+ BLANK LINE (zuul implements no form redirect)
+ MAX FILE SIZE
+ MAX FILE COUNT
+ SIGNATURE EXPIRY
+
+*SWIFT_$NAME_SIGNATURE*
+ The HMAC body signed with the configured key.
+*SWIFT_$NAME_LOGSERVER_PREFIX*
+ The URL to prepend to the object path when returning the results
+ from a build.
+
Gearman
-------
diff --git a/doc/source/zuul.rst b/doc/source/zuul.rst
index 7274342..ee37f2c 100644
--- a/doc/source/zuul.rst
+++ b/doc/source/zuul.rst
@@ -168,6 +168,59 @@
This can be overridden by individual pipelines.
``default_to=you@example.com``
+.. _swift:
+
+swift
+"""""
+
+To send (optional) swift upload instructions this section must be
+present. Multiple destinations can be defined in the :ref:`jobs`
+section of the layout.
+
+**authurl**
+ The (keystone) Auth URL for swift
+ ``For example, https://identity.api.rackspacecloud.com/v2.0/``
+
+Any of the `swiftclient connection parameters`_ can also be defined
+here by the same name. Including the os_options by their key name (
+``for example tenant_id``)
+
+.. _swiftclient connection parameters: http://docs.openstack.org/developer/python-swiftclient/swiftclient.html#module-swiftclient.client
+
+**X-Account-Meta-Temp-Url-Key** (optional)
+ This is the key used to sign the HMAC message. zuul will send the
+ key to swift for you so you only need to define it here. If you do
+ not set a key zuul will generate one automatically.
+
+**region_name** (optional)
+ The region name holding the swift container
+ ``For example, SYD``
+
+Each destination defined by the :ref:`jobs` will have the following
+default values that it may overwrite.
+
+**default_container** (optional)
+ Container name to place the log into
+ ``For example, logs``
+
+**default_expiry** (optional)
+ How long the signed destination should be available for
+ ``default: 7200 (2hrs)``
+
+**default_max_file_size** (optional)
+ The maximum size of an individual file
+ ``default: 104857600 (100MB)``
+
+**default_max_file_count** (optional)
+ The maximum number of separate files to allow
+ ``default: 10``
+
+**default_logserver_prefix**
+ Provide a URL to the CDN or logserver app so that a worker knows
+ what URL to return. The worker should return the logserver_prefix
+ url and the object path.
+ ``For example: http://logs.example.org/server.app?obj=``
+
layout.yaml
~~~~~~~~~~~
@@ -567,6 +620,9 @@
the ``ref-updated`` event which does include the commit sha1 (but lacks the
Gerrit change number).
+
+.. _jobs:
+
Jobs
""""
@@ -649,6 +705,39 @@
be used to specify on what node (or class of node) the job should be
run.
+**swift**
+ If :ref:`swift` is configured then each job can define a destination
+ container for the builder to place logs and/or assets into. Multiple
+ containers can be listed for each job by providing a unique ``name``.
+
+ *name*
+ Set an identifying name for the container. This is used in the
+ parameter key sent to the builder. For example if it ``logs`` then
+ one of the parameters sent will be ``SWIFT_logs_CONTAINER``
+ (case-sensitive).
+
+ Each of the defaults defined in :ref:`swift` can be overwritten as:
+
+ *container* (optional)
+ Container name to place the log into
+ ``For example, logs``
+
+ *expiry* (optional)
+ How long the signed destination should be available for
+
+ *max_file_size** (optional)
+ The maximum size of an individual file
+
+ *max_file_count* (optional)
+ The maximum number of separate files to allow
+
+ *logserver_prefix*
+ Provide a URL to the CDN or logserver app so that a worker knows
+ what URL to return.
+ ``For example: http://logs.example.org/server.app?obj=``
+ The worker should return the logserver_prefix url and the object
+ path as the URL in the results data packet.
+
Here is an example of setting the failure message for jobs that check
whether a change merges cleanly::