Add configuration documentation

This adds the start of v3 configuration documentation.

Change-Id: I3cc2f69c61795ef82d0e1c71d639e46fd1d5a7d1
Story: 2000790
Task: 3314
diff --git a/doc/source/admin-config.rst b/doc/source/admin-config.rst
new file mode 100644
index 0000000..03f742b
--- /dev/null
+++ b/doc/source/admin-config.rst
@@ -0,0 +1,24 @@
+:title: Administrative Configuration
+
+.. _admin-config:
+.. _drivers:
+
+Administrative Configuration
+============================
+
+TODO:
+
+* zuul.conf
+* connections
+
+* main.yaml
+* tenants
+* trusted/untrusted
+
+* drivers
+* gerrit
+  * trigger config
+  * reporter config
+* github
+  * trigger config
+  * reporter config
diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst
new file mode 100644
index 0000000..7ca0744
--- /dev/null
+++ b/doc/source/configuration.rst
@@ -0,0 +1,38 @@
+:title: Configuration
+
+.. _config:
+
+Configuration
+=============
+
+Zuul uses a very flexible configuration system.  Some initial
+configuration must be supplied by the system administrator.  From that
+beginning, Zuul builds its configuration by dynamically evaluating the
+contents of every git repository indicated by the configuration.
+
+Zuul configuration is focused primarily around the concept of a
+*Pipeline*.  Through a pipeline, a Zuul user expresses a connection
+between events, projects, jobs, and reports.  The general flow of
+operations is as follows:
+
+A *Connection* to a remote system generates an event; if it matches a
+*Trigger* on a *Pipeline*, then an *Item* is enqueued into the
+*Pipeline*.  An *Item* is some kind of git reference, such as a branch
+tip, a proposed change, or a pull request, and is associated with a
+*Project*.  Zuul launches the *Jobs* specified for that *Project* in
+that *Pipeline*, and when they complete, Zuul reports the results as
+specified by the pipeline's *Reporter*.
+
+TODO: flow diagram
+
+The following sections describe how to configure Zuul.
+:ref:`admin-config` covers areas of interest primarily to system
+administrators.  :ref:`project-config` covers the portion of Zuul
+configuration of interest to most users -- that which is included
+directly in the git repositories operated on by Zuul.
+
+.. toctree::
+   :maxdepth: 2
+
+   admin-config
+   project-config
diff --git a/doc/source/encryption.rst b/doc/source/encryption.rst
new file mode 100644
index 0000000..6698763
--- /dev/null
+++ b/doc/source/encryption.rst
@@ -0,0 +1,8 @@
+:title: Encryption
+
+.. _encryption:
+
+Encryption
+==========
+
+TODO
diff --git a/doc/source/index.rst b/doc/source/index.rst
index fb30b92..a3a4df5 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -15,6 +15,8 @@
 
    quick-start
    gating
+   configuration
+   encryption
    connections
    triggers
    reporters
diff --git a/doc/source/project-config.rst b/doc/source/project-config.rst
new file mode 100644
index 0000000..7d72b30
--- /dev/null
+++ b/doc/source/project-config.rst
@@ -0,0 +1,941 @@
+:title: Project Configuration
+
+.. _project-config:
+
+Project Configuration
+=====================
+
+The following sections describe the main part of Zuul's configuration.
+All of what follows is found within files inside of the repositories
+that Zuul manages.
+
+Security Contexts
+-----------------
+
+When a system administrator configures Zuul to operate on a project,
+they specify one of two security contexts for that project.  A
+*config-project* is one which is primarily tasked with holding
+configuration information and job content for Zuul.  Jobs which are
+defined in a *config-project* are run with elevated privileges, and
+all Zuul configuration items are available for use.  It is expected
+that changes to *config-projects* will undergo careful scrutiny before
+being merged.
+
+An *untrusted-project* is a project whose primary focus is not to
+operate Zuul, but rather it is one of the projects being tested or
+deployed.  The Zuul configuration language available to these projects
+is somewhat restricted (as detailed in individual section below), and
+jobs defined in these projects run in a restricted execution
+environment since they may be operating on changes which have not yet
+undergone review.
+
+Configuration Loading
+---------------------
+
+When Zuul starts, it examines all of the git repositories which are
+specified by the system administrator in :ref:`admin-config` and searches
+for files in the root of each repository.  In the case of a
+*config-project*, Zuul looks for a file named `zuul.yaml`.  In the
+case of an *untrusted-project*, Zuul looks first for `zuul.yaml` and
+if that is not found, `.zuul.yaml` (with a leading dot).  In the case
+of an *untrusted-project*, the configuration from every branch is
+included, however, in the case of a *config-project*, only the
+`master` branch is examined.
+
+When a change is proposed to one of these files in an
+*untrusted-project*, the configuration proposed in the change is
+merged into the running configuration so that any changes to Zuul's
+configuration are self-testing as part of that change.  If there is a
+configuration error, no jobs will be run and the error will be
+reported by any applicable pipelines.  In the case of a change to a
+*config-project*, the new configuration is parsed and examined for
+errors, but the new configuration is not used in testing the change.
+This is because configuration in *config-projects* is able to access
+elevated privileges and should always be reviewed before being merged.
+
+As soon as a change containing a Zuul configuration change merges to
+any Zuul-managed repository, the new configuration takes effect
+immediately.
+
+Configuration Items
+-------------------
+
+The `zuul.yaml` and `.zuul.yaml` configuration files are
+YAML-formatted and are structured as a series of items, each of which
+is described below.
+
+.. _pipeline:
+
+Pipeline
+~~~~~~~~
+
+A pipeline describes a workflow operation in Zuul.  It associates jobs
+for a given project with triggering and reporting events.
+
+Its flexible configuration allows for characterizing any number of
+workflows, and by specifying each as a named configuration, makes it
+easy to apply similar workflow operations to projects or groups of
+projects.
+
+By way of example, one of the primary uses of Zuul is to perform
+project gating.  To do so, one can create a *gate* pipeline which
+tells Zuul that when a certain event (such as approval by a code
+reviewer) occurs, the corresponding change or pull request should be
+enqueued into the pipeline.  When that happens, the jobs which have
+been configured to run for that project in the *gate* pipeline are
+run, and when they complete, the pipeline reports the results to the
+user.
+
+Pipeline configuration items may only appear in *config-projects*.
+
+Generally, a Zuul administrator would define a small number of
+pipelines which represent the workflow processes used in their
+environment.  Each project can then be added to the available
+pipelines as appropriate.
+
+Here is an example *check* pipeline, which runs whenever a new
+patchset is created in Gerrit.  If the associated jobs all report
+success, the pipeline reports back to Gerrit with a *Verified* vote of
++1, or if at least one of them fails, a -1::
+
+  - pipeline:
+    name: check
+      manager: independent
+      trigger:
+        my_gerrit:
+          - event: patchset-created
+      success:
+        my_gerrit:
+          verified: 1
+      failure:
+        my_gerrit
+          verified: -1
+
+See TODO for more annotated examples of common pipeline configurations.
+
+The attributes available on a pipeline are as follows (all are
+optional unless otherwise specified):
+
+**name** (required)
+  This is used later in the project definition to indicate what jobs
+  should be run for events in the pipeline.
+
+**manager** (required)
+  There are currently two schemes for managing pipelines:
+
+  *independent*
+    Every event in this pipeline should be treated as independent of
+    other events in the pipeline.  This is appropriate when the order of
+    events in the pipeline doesn't matter because the results of the
+    actions this pipeline performs can not affect other events in the
+    pipeline.  For example, when a change is first uploaded for review,
+    you may want to run tests on that change to provide early feedback
+    to reviewers.  At the end of the tests, the change is not going to
+    be merged, so it is safe to run these tests in parallel without
+    regard to any other changes in the pipeline.  They are independent.
+
+    Another type of pipeline that is independent is a post-merge
+    pipeline. In that case, the changes have already merged, so the
+    results can not affect any other events in the pipeline.
+
+  *dependent*
+    The dependent pipeline manager is designed for gating.  It ensures
+    that every change is tested exactly as it is going to be merged
+    into the repository.  An ideal gating system would test one change
+    at a time, applied to the tip of the repository, and only if that
+    change passed tests would it be merged.  Then the next change in
+    line would be tested the same way.  In order to achieve parallel
+    testing of changes, the dependent pipeline manager performs
+    speculative execution on changes.  It orders changes based on
+    their entry into the pipeline.  It begins testing all changes in
+    parallel, assuming that each change ahead in the pipeline will pass
+    its tests.  If they all succeed, all the changes can be tested and
+    merged in parallel.  If a change near the front of the pipeline
+    fails its tests, each change behind it ignores whatever tests have
+    been completed and are tested again without the change in front.
+    This way gate tests may run in parallel but still be tested
+    correctly, exactly as they will appear in the repository when
+    merged.
+
+    For more detail on the theory and operation of Zuul's dependent
+    pipeline manager, see: :doc:`gating`.
+
+**description**
+  This field may be used to provide a textual description of the
+  pipeline.  It may appear in the status page or in documentation.
+
+**success-message**
+  The introductory text in reports when all the voting jobs are
+  successful.  Defaults to "Build successful."
+
+**failure-message**
+  The introductory text in reports when at least one voting job fails.
+  Defaults to "Build failed."
+
+**merge-failure-message**
+  The introductory text in the message reported when a change fails to
+  merge with the current state of the repository.  Defaults to "Merge
+  failed."
+
+**footer-message**
+  Supplies additional information after test results.  Useful for
+  adding information about the CI system such as debugging and contact
+  details.
+
+**trigger**
+  At least one trigger source must be supplied for each pipeline.
+  Triggers are not exclusive -- matching events may be placed in
+  multiple pipelines, and they will behave independently in each of
+  the pipelines they match.
+
+  Triggers are loaded from their connection name. The driver type of
+  the connection will dictate which options are available.
+  See :doc:`triggers`.
+
+**require**
+  If this section is present, it established pre-requisites for any
+  kind of item entering the Pipeline.  Regardless of how the item is
+  to be enqueued (via any trigger or automatic dependency resolution),
+  the conditions specified here must be met or the item will not be
+  enqueued.
+
+.. TODO this section is in flux in v3 _pipeline-require-approval:
+
+  **approval**
+  This requires that a certain kind of approval be present for the
+  current patchset of the change (the approval could be added by the
+  event in question).  It takes several sub-parameters, all of which
+  are optional and are combined together so that there must be an
+  approval matching all specified requirements.
+
+    *username*
+    If present, an approval from this username is required.  It is
+    treated as a regular expression.
+
+    *email*
+    If present, an approval with this email address is required.  It
+    is treated as a regular expression.
+
+    *email-filter* (deprecated)
+    A deprecated alternate spelling of *email*.  Only one of *email* or
+    *email_filter* should be used.
+
+    *older-than*
+    If present, the approval must be older than this amount of time
+    to match.  Provide a time interval as a number with a suffix of
+    "w" (weeks), "d" (days), "h" (hours), "m" (minutes), "s"
+    (seconds).  Example ``48h`` or ``2d``.
+
+    *newer-than*
+    If present, the approval must be newer than this amount of time
+    to match.  Same format as "older-than".
+
+    Any other field is interpreted as a review category and value
+    pair.  For example ``verified: 1`` would require that the approval
+    be for a +1 vote in the "Verified" column.  The value may either
+    be a single value or a list: ``verified: [1, 2]`` would match
+    either a +1 or +2 vote.
+
+  **open**
+  A boolean value (``true`` or ``false``) that indicates whether the change
+  must be open or closed in order to be enqueued.
+
+  **current-patchset**
+  A boolean value (``true`` or ``false``) that indicates whether the change
+  must be the current patchset in order to be enqueued.
+
+  **status**
+  A string value that corresponds with the status of the change
+  reported by the trigger.
+
+**reject**
+  If this section is present, it establishes pre-requisites that can
+  block an item from being enqueued. It can be considered a negative
+  version of **require**.
+
+  **approval**
+  This takes a list of approvals. If an approval matches the provided
+  criteria the change can not be entered into the pipeline. It follows
+  the same syntax as the :ref:`"require approval" pipeline above
+  <pipeline-require-approval>`.
+
+  Example to reject a change with any negative vote::
+
+    reject:
+      approval:
+        - code-review: [-1, -2]
+
+**dequeue-on-new-patchset**
+  Normally, if a new patchset is uploaded to a change that is in a
+  pipeline, the existing entry in the pipeline will be removed (with
+  jobs canceled and any dependent changes that can no longer merge as
+  well.  To suppress this behavior (and allow jobs to continue
+  running), set this to ``false``.  Default: ``true``.
+
+**ignore-dependencies**
+  In any kind of pipeline (dependent or independent), Zuul will
+  attempt to enqueue all dependencies ahead of the current change so
+  that they are tested together (independent pipelines report the
+  results of each change regardless of the results of changes ahead).
+  To ignore dependencies completely in an independent pipeline, set
+  this to ``true``.  This option is ignored by dependent pipelines.
+  The default is: ``false``.
+
+The following options configure *reporters*.  Reporters are
+complementary to triggers; where a trigger is an event on a connection
+which causes Zuul to enqueue an item, a reporter is the action
+performed on a connection when an item is dequeued after its jobs
+complete.  The actual syntax for a reporter is defined by the driver
+which implements it.  See :ref:`drivers` for more information.
+
+**success**
+  Describes where Zuul should report to if all the jobs complete
+  successfully.  This section is optional; if it is omitted, Zuul will
+  run jobs and do nothing on success -- it will not report at all.  If
+  the section is present, the listed reporters will be asked to report
+  on the jobs.  The reporters are listed by their connection name. The
+  options available depend on the driver for the supplied connection.
+
+**failure**
+  These reporters describe what Zuul should do if at least one job
+  fails.
+
+**merge-failure**
+  These reporters describe what Zuul should do if it is unable to
+  merge in the patchset. If no merge-failure reporters are listed then
+  the ``failure`` reporters will be used to notify of unsuccessful
+  merges.
+
+**start**
+  These reporters describe what Zuul should do when a change is added
+  to the pipeline.  This can be used, for example, to reset a
+  previously reported result.
+
+**disabled**
+  These reporters describe what Zuul should do when a pipeline is
+  disabled.  See ``disable-after-consecutive-failures``.
+
+**disable-after-consecutive-failures**
+  If set, a pipeline can enter a ''disabled'' state if too many changes
+  in a row fail. When this value is exceeded the pipeline will stop
+  reporting to any of the ``success``, ``failure`` or ``merge-failure``
+  reporters and instead only report to the ``disabled`` reporters.
+  (No ``start`` reports are made when a pipeline is disabled).
+
+**precedence**
+  Indicates how the build scheduler should prioritize jobs for
+  different pipelines.  Each pipeline may have one precedence, jobs
+  for pipelines with a higher precedence will be run before ones with
+  lower.  The value should be one of ``high``, ``normal``, or ``low``.
+  Default: ``normal``.
+
+**window**
+  Dependent pipeline managers only. Zuul can rate limit dependent
+  pipelines in a manner similar to TCP flow control.  Jobs are only
+  started for items in the queue if they are within the actionable
+  window for the pipeline. The initial length of this window is
+  configurable with this value. The value given should be a positive
+  integer value. A value of ``0`` disables rate limiting on the
+  DependentPipelineManager.  Default: ``20``.
+
+**window-floor**
+  Dependent pipeline managers only. This is the minimum value for the
+  window described above. Should be a positive non zero integer value.
+  Default: ``3``.
+
+**window-increase-type**
+  Dependent pipeline managers only. This value describes how the window
+  should grow when changes are successfully merged by zuul. A value of
+  ``linear`` indicates that ``window-increase-factor`` should be added
+  to the previous window value. A value of ``exponential`` indicates
+  that ``window-increase-factor`` should be multiplied against the
+  previous window value and the result will become the window size.
+  Default: ``linear``.
+
+**window-increase-factor**
+  Dependent pipeline managers only. The value to be added or multiplied
+  against the previous window value to determine the new window after
+  successful change merges.
+  Default: ``1``.
+
+**window-decrease-type**
+  Dependent pipeline managers only. This value describes how the window
+  should shrink when changes are not able to be merged by Zuul. A value
+  of ``linear`` indicates that ``window-decrease-factor`` should be
+  subtracted from the previous window value. A value of ``exponential``
+  indicates that ``window-decrease-factor`` should be divided against
+  the previous window value and the result will become the window size.
+  Default: ``exponential``.
+
+**window-decrease-factor**
+  Dependent pipline managers only. The value to be subtracted or divided
+  against the previous window value to determine the new window after
+  unsuccessful change merges.
+  Default: ``2``.
+
+
+.. _job:
+
+Job
+~~~
+
+A job is a unit of work performed by Zuul on an item enqueued into a
+pipeline.  Items may run any number of jobs (which may depend on each
+other).  Each job is an invocation of an Ansible playbook with a
+specific inventory of hosts.  The actual tasks that are run by the job
+appear in the playbook for that job while the attributes that appear in the
+Zuul configuration specify information about when, where, and how the
+job should be run.
+
+Jobs in Zuul support inheritance.  Any job may specify a single parent
+job, and any attributes not set on the child job are collected from
+the parent job.  In this way, a configuration structure may be built
+starting with very basic jobs which describe characteristics that all
+jobs on the system should have, progressing through stages of
+specialization before arriving at a particular job.  A job may inherit
+from any other job in any project (however, if the other job is marked
+as `final`, some attributes may not be overidden).
+
+Jobs also support a concept called variance.  The first time a job
+definition appears is called the reference definition of the job.
+Subsequent job definitions with the same name are called variants.
+These may have different selection criteria which indicate to Zuul
+that, for instance, the job should behave differently on a different
+git branch.  Unlike inheritance, all job variants must be defined in
+the same project.
+
+When Zuul decides to run a job, it performs a process known as
+freezing the job.  Because any number of job variants may be
+applicable, Zuul collects all of the matching variants and applies
+them in the order they appeared in the configuration.  The resulting
+frozen job is built from attributes gathered from all of the
+matching variants.  In this way, exactly what is run is dependent on
+the pipeline, project, branch, and content of the item.
+
+In addition to the job's main playbook, each job may specify one or
+more pre- and post-playbooks.  These are run, in order, before and
+after (respectively) the main playbook.  They may be used to set up
+and tear down resources needed by the main playbook.  When combined
+with inheritance, they provide powerful tools for job construction.  A
+job only has a single main playbook, and when inheriting from a
+parent, the child's main playbook overrides (or replaces) the
+parent's.  However, the pre- and post-playbooks are appended and
+prepended in a nesting fashion.  So if a parent job and child job both
+specified pre and post playbooks, the sequence of playbooks run would
+be:
+
+* parent pre-run playbook
+* child pre-run playbook
+* child playbook
+* child post-run playbook
+* parent post-run playbook
+
+Further inheritance would nest even deeper.
+
+Here is an example of two job definitions::
+
+  - job:
+      name: base
+      pre-run: copy-git-repos
+      post-run: copy-logs
+
+  - job:
+      name: run-tests
+      parent: base
+      nodes:
+        - name: test-node
+	  image: fedora
+
+The following attributes are available on a job; all are optional
+unless otherwise specified:
+
+**name** (required)
+  The name of the job.  By default, Zuul looks for a playbook with
+  this name to use as the main playbook for the job.  This name is
+  also referenced later in a project pipeline configuration.
+
+**parent**
+  Specifies a job to inherit from.  The parent job can be defined in
+  this or any other project.  Any attributes not specified on a job
+  will be collected from its parent.
+
+**description**
+  A textual description of the job.  Not currently used directly by
+  Zuul, but it is used by the zuul-sphinx extension to Sphinx to
+  auto-document Zuul jobs (in which case it is interpreted as
+  ReStructuredText.
+
+**success-message**
+  Normally when a job succeeds, the string "SUCCESS" is reported as
+  the result for the job.  If set, this option may be used to supply a
+  different string.  Default: "SUCCESS".
+
+**failure-message**
+  Normally when a job fails, the string "FAILURE" is reported as
+  the result for the job.  If set, this option may be used to supply a
+  different string.  Default: "FAILURE".
+
+**success-url**
+  When a job succeeds, this URL is reported along with the result.
+  Default: none.
+
+**failure-url**
+  When a job fails, this URL is reported along with the result.
+  Default: none.
+
+**hold-following-changes**
+  In a dependent pipeline, this option may be used to indicate that no
+  jobs should start on any items which depend on the current item
+  until this job has completed successfully.  This may be used to
+  conserve build resources, at the expense of inhibiting the
+  parallelization which speeds the processing of items in a dependent
+  pipeline.  A boolean value, default: false.
+
+**voting**
+  Indicates whether the result of this job should be used in
+  determining the overall result of the item.  A boolean value,
+  default: true.
+
+**semaphore**
+  The name of a :ref:`semaphore` which should be acquired and released
+  when the job begins and ends.  If the semaphore is at maximum
+  capacity, then Zuul will wait until it can be acquired before
+  starting the job.  Default: none.
+
+**tags**
+  Metadata about this job.  Tags are units of information attached to
+  the job; they do not affect Zuul's behavior, but they can be used
+  within the job to characterize the job.  For example, a job which
+  tests a certain subsystem could be tagged with the name of that
+  subsystem, and if the job's results are reported into a database,
+  then the results of all jobs affecting that subsystem could be
+  queried.  This attribute is specified as a list of strings, and when
+  inheriting jobs or applying variants, tags accumulate in a set, so
+  the result is always a set of all the tags from all the jobs and
+  variants used in constructing the frozen job, with no duplication.
+  Default: none.
+
+** branches **
+  A regular expression (or list of regular expressions) which describe
+  on what branches a job should run (or in the case of variants: to
+  alter the behavior of a job for a certain branch).
+
+  If there is no job definition for a given job which matches the
+  branch of an item, then that job is not run for the item.
+  Otherwise, all of the job variants which match that branch (and any
+  other selection criteria) are used when freezing the job.
+
+  This example illustrates a job called *run-tests* which uses a
+  nodeset based on the current release of an operating system to
+  perform its tests, except when testing changes to the stable/2.0
+  branch, in which case it uses an older release::
+
+    - job:
+        name: run-tests
+        nodes: current-release
+
+    - job:
+        name: run-tests
+        branch: stable/2.0
+        nodes: old-release
+
+  In some cases, Zuul uses an implied value for the branch specifier
+  if none is supplied:
+
+  * For a job definition in a *config-project*, no implied branch
+    specifier is used.  If no branch specifier appears, the job
+    applies to all branches.
+
+  * In the case of an *untrusted-project*, no implied branch specifier
+    is applied to the reference definition of a job.  That is to say,
+    that if the first appearance of the job definition appears without
+    a branch specifier, then it will apply to all branches.  Note that
+    when collecting its configuration, Zuul reads the `master` branch
+    of a given project first, then other branches in alphabetical
+    order.
+
+  * Any further job variants other than the reference definition in an
+    *untrusted-project* will, if they do not have a branch specifier,
+    will have an implied branch specifier for the current branch
+    applied.
+
+  This allows for the very simple and expected workflow where if a
+  project defines a job on the master branch with no branch specifier,
+  and then creates a new branch based on master, any changes to that
+  job definition within the new branch only affect that branch.
+
+**files**
+  This attribute indicates that the job should only run on changes
+  where the specified files are modified.  This is a regular
+  expression or list of regular expressions.  Default: none.
+
+**irrelevant-files**
+  This is a negative complement of `files`.  It indicates that the job
+  should run unless *all* of the files changed match this list.  In
+  other words, if the regular expression `docs/.*` is supplied, then
+  this job will not run if the only files changed are in the docs
+  directory.  A regular expression or list of regular expressions.
+  Default: none.
+
+**auth**
+  Authentication information to be made available to the job.  This is
+  a dictionary with two potential keys:
+
+  **inherit**
+  A boolean indicating that the authentication information referenced
+  by this job should be able to be inherited by child jobs.  Normally
+  when a job inherits from another job, the auth section is not
+  included.  This permits jobs to inherit the same basic structure and
+  playbook, but ensures that secret information is unable to be
+  exposed by a child job which may alter the job's behavior.  If it is
+  safe for the contents of the authentication section to be used by
+  child jobs, set this to ``true``.  Default: ``false``.
+
+  **secrets**
+  A list of secrets which may be used by the job.  A :ref:`secret` is
+  a named collection of private information defined separately in the
+  configuration.  The secrets that appear here must be defined in the
+  same project as this job definition.
+
+  In the future, other types of authentication information may be
+  added.
+
+**nodes**
+  A list of nodes which should be supplied to the job.  This parameter
+  may be supplied either as a string, in which case it references a
+  :ref:`nodeset` definition which appears elsewhere in the
+  configuration, or a list, in which case it is interpreted in the
+  same way as a Nodeset definition (in essence, it is an anonymous
+  Node definition unique to this job).  See the :ref:`nodeset`
+  reference for the syntax to use in that case.
+
+  If a job has an empty or no node definition, it will still run and
+  may be able to perform actions on the Zuul executor.
+
+**override-branch**
+  When Zuul runs jobs for a proposed change, it normally checks out
+  the branch associated with that change on every project present in
+  the job.  If jobs are running on a ref (such as a branch tip or
+  tag), then that ref is normally checked out.  This attribute is used
+  to override that behavior and indicate that this job should,
+  regardless of the branch for the queue item, use the indicated
+  branch instead.  This can be used, for example, to run a previous
+  version of the software (from a stable maintenance branch) under
+  test even if the change being tested applies to a different branch
+  (this is only likely to be useful if there is some cross-branch
+  interaction with some component of the system being tested).  See
+  also the project-specific **override-branch** attribute under
+  **required-projects** to apply this behavior to a subset of a job's
+  projects.
+
+**timeout**
+  The time in minutes that the job should be allowed to run before it
+  is automatically aborted and failure is reported.  If no timeout is
+  supplied, the job may run indefinitely.  Supplying a timeout is
+  highly recommended.
+
+**attempts**
+  When Zuul encounters an error running a job's pre-run playbook, Zuul
+  will stop and restart the job.  Errors during the main or
+  post-run -playbook phase of a job are not affected by this parameter
+  (they are reported immediately).  This parameter controls the number
+  of attempts to make before an error is reported.  Default: 3.
+
+**pre-run**
+  The name of a playbook or list of playbooks to run before the main
+  body of a job.  The playbook is expected to reside in the
+  `playbooks/` directory of the project where the job is defined.
+
+  When a job inherits from a parent, the child's pre-run playbooks are
+  run after the parent's.  See :ref:`job` for more information.
+
+**post-run**
+  The name of a playbook or list of playbooks to run after the main
+  body of a job.  The playbook is expected to reside in the
+  `playbooks/` directory of the project where the job is defined.
+
+  When a job inherits from a parent, the child's post-run playbooks
+  are run before the parent's.  See :ref:`job` for more information.
+
+**run**
+  The name of the main playbook for this job.  This parameter is not
+  normally necessary, as it defaults to the name of the job.  However,
+  if a playbook with a different name is needed, it can be specified
+  here.  The playbook is expected to reside in the `playbooks/`
+  directory of the project where the job is defined.  When a child
+  inherits from a parent, a playbook with the name of the child job is
+  implicitly searched first, before falling back on the playbook used
+  by the parent job (unless the child job specifies a ``run``
+  attribute, in which case that value is used).  Default: the name of
+  the job.
+
+**roles**
+  A list of Ansible roles to prepare for the job.  Because a job runs
+  an Ansible playbook, any roles which are used by the job must be
+  prepared and installed by Zuul before the job begins.  This value is
+  a list of dictionaries, each of which indicates one of two types of
+  roles: a Galaxy role, which is simply a role that is installed from
+  Ansible Galaxy, or a Zuul role, which is a role provided by a
+  project managed by Zuul.  Zuul roles are able to benefit from
+  speculative merging and cross-project dependencies when used by jobs
+  in untrusted projects.
+
+  A project which supplies a role may be structured in one of two
+  configurations: a bare role (in which the role exists at the root of
+  the project), or a contained role (in which the role exists within
+  the `roles/` directory of the project, perhaps along with other
+  roles).  In the case of a contained role, the `roles/` directory of
+  the project is added to the role search path.  In the case of a bare
+  role, the project itself is added to the role search path.  In case
+  the name of the project is not the name under which the role should
+  be installed (and therefore referenced from Ansible), the `name`
+  attribute may be used to specify an alternate.
+
+  **galaxy**
+    The name of the role in Ansible Galaxy.  If this attribute is
+    supplied, Zuul will search Ansible Galaxy for a role by this name
+    and install it.  Mutually exclusive with ``zuul``; either
+    ``galaxy`` or ``zuul`` must be supplied.
+
+  **zuul**
+    The name of a Zuul project which supplies the role.  Mutually
+    exclusive with ``galaxy``; either ``galaxy`` or ``zuul`` must be
+    supplied.
+
+  **name**
+    The installation name of the role.  In the case of a bare role,
+    the role will be made available under this name.  Ignored in the
+    case of a contained role.
+
+**required-projects**
+  A list of other projects which are used by this job.  Any Zuul
+  projects specified here will also be checked out by Zuul into the
+  working directory for the job.  Speculative merging and cross-repo
+  dependencies will be honored.
+
+  The format for this attribute is either a list of strings or
+  dictionaries.  Strings are interpreted as project names,
+  dictionaries may have the following attributes:
+
+  **name**
+    The name of the required project.
+
+  **override-branch**
+    When Zuul runs jobs for a proposed change, it normally checks out
+    the branch associated with that change on every project present in
+    the job.  If jobs are running on a ref (such as a branch tip or
+    tag), then that ref is normally checked out.  This attribute is
+    used to override that behavior and indicate that this job should,
+    regardless of the branch for the queue item, use the indicated
+    branch instead, for only this project.  See also the
+    **override-branch** attribute of jobs to apply the same behavior
+    to all projects in a job.
+
+**vars**
+
+A dictionary of variables to supply to Ansible.  When inheriting from
+a job (or creating a variant of a job) vars are merged with previous
+definitions.  This means a variable definition with the same name will
+override a previously defined variable, but new variable names will be
+added to the set of defined variables.
+
+**dependencies**
+  A list of other jobs upon which this job depends.  Zuul will not
+  start executing this job until all of its dependencies have
+  completed successfully, and if one or more of them fail, this job
+  will not be run.
+
+**allowed-projects**
+  A list of Zuul projects which may use this job.  By default, a job
+  may be used by any other project known to Zuul, however, some jobs
+  use resources or perform actions which are not appropriate for other
+  projects.  In these cases, a list of projects which are allowed to
+  use this job may be supplied.  If this list is not empty, then it
+  must be an exhaustive list of all projects permitted to use the job.
+  The current project (where the job is defined) is not automatically
+  included, so if it should be able to run this job, then it must be
+  explicitly listed.  Default: the empty list (all projects may use
+  the job).
+
+
+.. _project:
+
+Project
+~~~~~~~
+
+A project corresponds to a source code repository with which Zuul is
+configured to interact.  The main responsibility of the `Project`
+configuration item is to specify which jobs should run in which
+pipelines for a given project.  Within each `Project` definition, a
+section for each `Pipeline` may appear.  This project-pipeline
+definition is what determines how a project participates in a
+pipeline.
+
+Consider the following `Project` definition::
+
+  - project:
+      name: yoyodyne
+      check:
+        jobs:
+          - check-syntax
+          - unit-tests
+      gate:
+        queue: integrated
+        jobs:
+          - unit-tests
+          - integration-tests
+
+The project has two project-pipeline stanzas, one for the `check`
+pipeline, and one for `gate`.  Each specifies which jobs shuld run
+when a change for that project enteres the respective pipeline -- when
+a change enters `check`, the `check-syntax` and `unit-test` jobs are
+run.
+
+Pipelines which use the dependent pipeline manager (e.g., the `gate`
+example shown earlier) maintain separate queues for groups of
+projects.  When Zuul serializes a set of changes which represent
+future potential project states, it must know about all of the
+projects within Zuul which may have an effect on the outcome of the
+jobs it runs.  If project *A* uses project *B* as a library, then Zuul
+must be told about that relationship so that it knows to serialize
+changes to A and B together, so that it does not merge a change to B
+while it is testing a change to A.
+
+Zuul could simply assume that all projects are related, or even infer
+relationships by which projects a job indicates it uses, however, in a
+large system that would become unwieldy very quickly, and
+unnecessarily delay changes to unrelated projects.  To allow for
+flexibility in the construction of groups of related projects, the
+change queues used by dependent pipeline managers are specified
+manually.  To group two or more related projects into a shared queue
+for a dependent pipeline, set the ``queue`` parameter to the same
+value for those projects.
+
+The `gate` project-pipeline definition above specifies that this
+project participates in the `integrated` shared queue for that
+pipeline.
+
+In addition to a project-pipeline definition for one or more
+`Pipelines`, the following attributes may appear in a Project:
+
+**name** (required)
+  The name of the project.  If Zuul is configured with two or more
+  unique projects with the same name, the canonical hostname for the
+  project should be included (e.g., `git.example.com/foo`).
+
+**templates**
+  A list of :ref:`project-template` references; the project-pipeline
+  definitions of each Project Template will be applied to this
+  project.  If more than one template includes jobs for a given
+  pipeline, they will be combined, as will any jobs specified in
+  project-pipeline definitions on the project itself.
+
+.. _project-template:
+
+Project Template
+~~~~~~~~~~~~~~~~
+
+A Project Template defines one or more project-pipeline definitions
+which can be re-used by multiple projects.
+
+A Project Template uses the same syntax as a :ref:`project`
+definition, however, in the case of a template, the ``name`` attribute
+does not refer to the name of a project, but rather names the template
+so that it can be referenced in a `Project` definition.
+
+.. _secret:
+
+Secret
+~~~~~~
+
+A Secret is a collection of private data for use by one or more jobs.
+In order to maintain the security of the data, the values are usually
+encrypted, however, data which are not sensitive may be provided
+unencrypted as well for convenience.
+
+A Secret may only be used by jobs defined within the same project.  To
+use a secret, a :ref:`job` must specify the secret within its `auth`
+section.  To protect against jobs in other repositories declaring a
+job with a secret as a parent and then exposing that secret, jobs
+which inherit from a job with secrets will not inherit the secrets
+themselves.  To alter that behavior, see the `inherit` job attribute.
+Further, jobs which do not permit children to inherit secrets (the
+default) are also automatically marked `final`, meaning that their
+execution related attributes may not be changed in a project-pipeline
+stanza.  This is to protect against a job with secrets defined in one
+project being used by another project in a way which might expose the
+secrets.  If a job with secrets is unsafe to be used by other
+projects, the `allowed-projects` job attribute can be used to restrict
+the projects which can invoke that job.  Finally, pipelines which are
+used to execute proposed but unreviewed changes can set the
+`allow-secrets` attribute to indicate that they should not supply
+secrets at all in order to protect against someone proposing a change
+which exposes a secret.
+
+The following attributes are required:
+
+**name** (required)
+  The name of the secret, used in a :ref:`Job` definition to request
+  the secret.
+
+**data** (required)
+  A dictionary which will be added to the Ansible variables available
+  to the job.  The values can either be plain text strings, or
+  encrypted values.  See :ref:`encryption` for more information.
+
+.. _nodeset:
+
+Nodeset
+~~~~~~~
+
+A Nodeset is a named collection of nodes for use by a job.  Jobs may
+specify what nodes they require individually, however, by defining
+groups of node types once and referring to them by name, job
+configuration may be simplified.
+
+A Nodeset requires two attributes:
+
+**name** (required)
+  The name of the Nodeset, to be referenced by a :ref:`job`.
+
+**nodes** (required)
+  A list of node definitions, each of which has the following format:
+
+  **name** (required)
+    The name of the node.  This will appear in the Ansible inventory
+    for the job.
+
+  **label** (required)
+    The Nodepool label for the node.  Zuul will request a node with
+    this label.
+
+.. _semaphore:
+
+Semaphore
+~~~~~~~~~
+
+Semaphores can be used to restrict the number of certain jobs which
+are running at the same time.  This may be useful for jobs which
+access shared or limited resources.  A semaphore has a value which
+represents the maximum number of jobs which use that semaphore at the
+same time.
+
+Semaphores are never subject to dynamic reconfiguration.  If the value
+of a semaphore is changed, it will take effect only when the change
+where it is updated is merged.  An example follows::
+
+  - semaphore:
+      name: semaphore-foo
+      max: 5
+  - semaphore:
+      name: semaphore-bar
+      max: 3
+
+The following attributes are available:
+
+**name** (required)
+  The name of the semaphore, referenced by jobs.
+
+**max**
+  The maximum number of running jobs which can use this semaphore.
+  Defaults to 1.