Reorganize docs into user/admin guide

Refresh the user and admin guide for v3 changes, and reorganize into
a narrative structure which makes more sense for v3.

Change-Id: I4ac3b18d5ed33b0fea4e2ef0318b19bfc3447ccc
diff --git a/doc/source/user/concepts.rst b/doc/source/user/concepts.rst
new file mode 100644
index 0000000..6197396
--- /dev/null
+++ b/doc/source/user/concepts.rst
@@ -0,0 +1,85 @@
+:title: Zuul Concepts
+
+Zuul Concepts
+=============
+
+Zuul's configuration is organized around the concept of a *pipeline*.
+In Zuul, a pipeline encompasses a workflow process which can be
+applied to one or more projects.  For instance, a "check" pipeline
+might describe the actions which should cause newly proposed changes
+to projects to be tested.  A "gate" pipeline might implement
+:ref:`project_gating` to automate merging changes to projects only if
+their tests pass.  A "post" pipeline might update published
+documentation for a project when changes land.
+
+The names "check", "gate", and "post" are arbitrary -- these are not
+concepts built into Zuul, but rather they are just a few common
+examples of workflow processes that can be described to Zuul and
+implemented as pipelines.
+
+Once a pipeline has been defined, any number of projects may be
+associated with it, each one specifying what jobs should be run for
+that project in a given pipeline.
+
+Pipelines have associated *triggers* which are descriptions of events
+which should cause something to be enqueued into a pipeline.  For
+example, when a patchset is uploaded to Gerrit, a Gerrit
+*patchset-created* event is emitted.  A pipeline configured to trigger
+on *patchset-created* events would then enqueue the associated change
+when Zuul receives that event.  If there are jobs associated with that
+project and pipeline, they will be run.  In addition to Gerrit, other
+triggers are available, including GitHub, timer, and zuul.  See
+:ref:`drivers` for a full list of available triggers.
+
+Once all of the jobs for an item in a pipeline have been run, the
+pipeline's *reporters* are responsible for reporting the results of
+all of the jobs.  Continuing the example from earlier, if a pipeline
+were configured with a Gerrit reporter, it would leave a review
+comment on the change and set any approval votes that are configured.
+There are several reporting phases available; each of which may be
+configured with any number of reporters.  See :ref:`drivers` for a
+full list of available reporters.
+
+The items enqueued into a pipeline are each associated with a git ref.
+That ref may point to a proposed change, or it may be the tip of a
+branch or a tag.  The triggering event determines the ref, and whether
+it represents a proposed or merged commit.  Zuul prepares the ref for
+an item before running jobs.  In the case of a proposed change, that
+means speculatively merging the change into its target branch.  This
+means that any jobs that operate on the change will run with the git
+repo in the state it will be in after the change merges (which may be
+substantially different than the git repo state of the change itself
+since the repo may have merged other changes since the change was
+originally authored).  Items in a pipeline may depend on other items,
+and if they do, all of their dependent changes will be included in the
+git repo state that Zuul prepares.  For more detail on this process,
+see :ref:`project_gating` and :ref:`dependencies`.
+
+The configuration for nearly everything described above is held in
+files inside of the git repos upon which Zuul operates.  Zuul's
+configuration is global, but distributed.  Jobs which are defined in
+one project might be used in another project while pipelines are
+available to all projects.  When Zuul starts, it reads its
+configuration from all of the projects it knows about, and when
+changes to its configuration are proposed, those changes may take
+effect temporarily as part of the proposed change, or immediately
+after the change merges, depending on the type of project in which the
+change appears.
+
+Jobs specify the type and quantity of nodes which they require.
+Before executing each job, Zuul will contact it's companion program,
+Nodepool, to supply them.  Nodepool may be configured to supply static
+nodes or contact cloud providers to create or delete nodes as
+necessary.  The types of nodes available to Zuul are determined by the
+Nodepool administrator.
+
+The executable contents of jobs themselves are Ansible playbooks.
+Ansible's support for orchestrating tasks on remote nodes is
+particularly suited to Zuul's support for multi-node testing.  Ansible
+is also easy to use for simple tasks (such as executing a shell
+script) or sophisticated deployment scenarios.  When Zuul runs
+Ansible, it attempts to do so in a manner most similar to the way that
+Ansible might be used to orchestrate remote systems.  Ansible itself
+is run on the executor and acts remotely upon the test nodes supplied
+to a job.  This facilitates continuous delivery by making it possible
+to use the same Ansible playbooks in testing and production.
diff --git a/doc/source/user/config.rst b/doc/source/user/config.rst
new file mode 100644
index 0000000..e7226e9
--- /dev/null
+++ b/doc/source/user/config.rst
@@ -0,0 +1,952 @@
+:title: Project Configuration
+
+.. _project-config:
+
+Project Configuration
+=====================
+
+The following sections describe the main part of Zuul's configuration.
+All of what follows is found within files inside of the repositories
+that Zuul manages.
+
+Security Contexts
+-----------------
+
+When a system administrator configures Zuul to operate on a project,
+they specify one of two security contexts for that project.  A
+*config-project* is one which is primarily tasked with holding
+configuration information and job content for Zuul.  Jobs which are
+defined in a *config-project* are run with elevated privileges, and
+all Zuul configuration items are available for use.  It is expected
+that changes to *config-projects* will undergo careful scrutiny before
+being merged.
+
+An *untrusted-project* is a project whose primary focus is not to
+operate Zuul, but rather it is one of the projects being tested or
+deployed.  The Zuul configuration language available to these projects
+is somewhat restricted (as detailed in individual section below), and
+jobs defined in these projects run in a restricted execution
+environment since they may be operating on changes which have not yet
+undergone review.
+
+Configuration Loading
+---------------------
+
+When Zuul starts, it examines all of the git repositories which are
+specified by the system administrator in :ref:`tenant-config` and searches
+for files in the root of each repository.  In the case of a
+*config-project*, Zuul looks for a file named `zuul.yaml`.  In the
+case of an *untrusted-project*, Zuul looks first for `zuul.yaml` and
+if that is not found, `.zuul.yaml` (with a leading dot).  In the case
+of an *untrusted-project*, the configuration from every branch is
+included, however, in the case of a *config-project*, only the
+`master` branch is examined.
+
+When a change is proposed to one of these files in an
+*untrusted-project*, the configuration proposed in the change is
+merged into the running configuration so that any changes to Zuul's
+configuration are self-testing as part of that change.  If there is a
+configuration error, no jobs will be run and the error will be
+reported by any applicable pipelines.  In the case of a change to a
+*config-project*, the new configuration is parsed and examined for
+errors, but the new configuration is not used in testing the change.
+This is because configuration in *config-projects* is able to access
+elevated privileges and should always be reviewed before being merged.
+
+As soon as a change containing a Zuul configuration change merges to
+any Zuul-managed repository, the new configuration takes effect
+immediately.
+
+Configuration Items
+-------------------
+
+The `zuul.yaml` and `.zuul.yaml` configuration files are
+YAML-formatted and are structured as a series of items, each of which
+is described below.
+
+.. _pipeline:
+
+Pipeline
+~~~~~~~~
+
+A pipeline describes a workflow operation in Zuul.  It associates jobs
+for a given project with triggering and reporting events.
+
+Its flexible configuration allows for characterizing any number of
+workflows, and by specifying each as a named configuration, makes it
+easy to apply similar workflow operations to projects or groups of
+projects.
+
+By way of example, one of the primary uses of Zuul is to perform
+project gating.  To do so, one can create a *gate* pipeline which
+tells Zuul that when a certain event (such as approval by a code
+reviewer) occurs, the corresponding change or pull request should be
+enqueued into the pipeline.  When that happens, the jobs which have
+been configured to run for that project in the *gate* pipeline are
+run, and when they complete, the pipeline reports the results to the
+user.
+
+Pipeline configuration items may only appear in *config-projects*.
+
+Generally, a Zuul administrator would define a small number of
+pipelines which represent the workflow processes used in their
+environment.  Each project can then be added to the available
+pipelines as appropriate.
+
+Here is an example *check* pipeline, which runs whenever a new
+patchset is created in Gerrit.  If the associated jobs all report
+success, the pipeline reports back to Gerrit with a *Verified* vote of
++1, or if at least one of them fails, a -1::
+
+  - pipeline:
+    name: check
+      manager: independent
+      trigger:
+        my_gerrit:
+          - event: patchset-created
+      success:
+        my_gerrit:
+          verified: 1
+      failure:
+        my_gerrit
+          verified: -1
+
+.. TODO: See TODO for more annotated examples of common pipeline configurations.
+
+The attributes available on a pipeline are as follows (all are
+optional unless otherwise specified):
+
+**name** (required)
+  This is used later in the project definition to indicate what jobs
+  should be run for events in the pipeline.
+
+**manager** (required)
+  There are currently two schemes for managing pipelines:
+
+  .. _independent_pipeline_manager:
+
+  *independent*
+    Every event in this pipeline should be treated as independent of
+    other events in the pipeline.  This is appropriate when the order of
+    events in the pipeline doesn't matter because the results of the
+    actions this pipeline performs can not affect other events in the
+    pipeline.  For example, when a change is first uploaded for review,
+    you may want to run tests on that change to provide early feedback
+    to reviewers.  At the end of the tests, the change is not going to
+    be merged, so it is safe to run these tests in parallel without
+    regard to any other changes in the pipeline.  They are independent.
+
+    Another type of pipeline that is independent is a post-merge
+    pipeline. In that case, the changes have already merged, so the
+    results can not affect any other events in the pipeline.
+
+  .. _dependent_pipeline_manager:
+
+  *dependent*
+    The dependent pipeline manager is designed for gating.  It ensures
+    that every change is tested exactly as it is going to be merged
+    into the repository.  An ideal gating system would test one change
+    at a time, applied to the tip of the repository, and only if that
+    change passed tests would it be merged.  Then the next change in
+    line would be tested the same way.  In order to achieve parallel
+    testing of changes, the dependent pipeline manager performs
+    speculative execution on changes.  It orders changes based on
+    their entry into the pipeline.  It begins testing all changes in
+    parallel, assuming that each change ahead in the pipeline will pass
+    its tests.  If they all succeed, all the changes can be tested and
+    merged in parallel.  If a change near the front of the pipeline
+    fails its tests, each change behind it ignores whatever tests have
+    been completed and are tested again without the change in front.
+    This way gate tests may run in parallel but still be tested
+    correctly, exactly as they will appear in the repository when
+    merged.
+
+    For more detail on the theory and operation of Zuul's dependent
+    pipeline manager, see: :doc:`gating`.
+
+**description**
+  This field may be used to provide a textual description of the
+  pipeline.  It may appear in the status page or in documentation.
+
+**success-message**
+  The introductory text in reports when all the voting jobs are
+  successful.  Defaults to "Build successful."
+
+**failure-message**
+  The introductory text in reports when at least one voting job fails.
+  Defaults to "Build failed."
+
+**merge-failure-message**
+  The introductory text in the message reported when a change fails to
+  merge with the current state of the repository.  Defaults to "Merge
+  failed."
+
+**footer-message**
+  Supplies additional information after test results.  Useful for
+  adding information about the CI system such as debugging and contact
+  details.
+
+**trigger**
+  At least one trigger source must be supplied for each pipeline.
+  Triggers are not exclusive -- matching events may be placed in
+  multiple pipelines, and they will behave independently in each of
+  the pipelines they match.
+
+  Triggers are loaded from their connection name. The driver type of
+  the connection will dictate which options are available.
+  See :ref:`drivers`.
+
+**require**
+  If this section is present, it established pre-requisites for any
+  kind of item entering the Pipeline.  Regardless of how the item is
+  to be enqueued (via any trigger or automatic dependency resolution),
+  the conditions specified here must be met or the item will not be
+  enqueued.
+
+.. _pipeline-require-approval:
+
+  **approval**
+  This requires that a certain kind of approval be present for the
+  current patchset of the change (the approval could be added by the
+  event in question).  It takes several sub-parameters, all of which
+  are optional and are combined together so that there must be an
+  approval matching all specified requirements.
+
+    *username*
+    If present, an approval from this username is required.  It is
+    treated as a regular expression.
+
+    *email*
+    If present, an approval with this email address is required.  It
+    is treated as a regular expression.
+
+    *email-filter* (deprecated)
+    A deprecated alternate spelling of *email*.  Only one of *email* or
+    *email_filter* should be used.
+
+    *older-than*
+    If present, the approval must be older than this amount of time
+    to match.  Provide a time interval as a number with a suffix of
+    "w" (weeks), "d" (days), "h" (hours), "m" (minutes), "s"
+    (seconds).  Example ``48h`` or ``2d``.
+
+    *newer-than*
+    If present, the approval must be newer than this amount of time
+    to match.  Same format as "older-than".
+
+    Any other field is interpreted as a review category and value
+    pair.  For example ``verified: 1`` would require that the approval
+    be for a +1 vote in the "Verified" column.  The value may either
+    be a single value or a list: ``verified: [1, 2]`` would match
+    either a +1 or +2 vote.
+
+  **open**
+  A boolean value (``true`` or ``false``) that indicates whether the change
+  must be open or closed in order to be enqueued.
+
+  **current-patchset**
+  A boolean value (``true`` or ``false``) that indicates whether the change
+  must be the current patchset in order to be enqueued.
+
+  **status**
+  A string value that corresponds with the status of the change
+  reported by the trigger.
+
+**reject**
+  If this section is present, it establishes pre-requisites that can
+  block an item from being enqueued. It can be considered a negative
+  version of **require**.
+
+  **approval**
+  This takes a list of approvals. If an approval matches the provided
+  criteria the change can not be entered into the pipeline. It follows
+  the same syntax as the :ref:`"require approval" pipeline above
+  <pipeline-require-approval>`.
+
+  Example to reject a change with any negative vote::
+
+    reject:
+      approval:
+        - code-review: [-1, -2]
+
+**dequeue-on-new-patchset**
+  Normally, if a new patchset is uploaded to a change that is in a
+  pipeline, the existing entry in the pipeline will be removed (with
+  jobs canceled and any dependent changes that can no longer merge as
+  well.  To suppress this behavior (and allow jobs to continue
+  running), set this to ``false``.  Default: ``true``.
+
+**ignore-dependencies**
+  In any kind of pipeline (dependent or independent), Zuul will
+  attempt to enqueue all dependencies ahead of the current change so
+  that they are tested together (independent pipelines report the
+  results of each change regardless of the results of changes ahead).
+  To ignore dependencies completely in an independent pipeline, set
+  this to ``true``.  This option is ignored by dependent pipelines.
+  The default is: ``false``.
+
+**precedence**
+  Indicates how the build scheduler should prioritize jobs for
+  different pipelines.  Each pipeline may have one precedence, jobs
+  for pipelines with a higher precedence will be run before ones with
+  lower.  The value should be one of ``high``, ``normal``, or ``low``.
+  Default: ``normal``.
+
+The following options configure *reporters*.  Reporters are
+complementary to triggers; where a trigger is an event on a connection
+which causes Zuul to enqueue an item, a reporter is the action
+performed on a connection when an item is dequeued after its jobs
+complete.  The actual syntax for a reporter is defined by the driver
+which implements it.  See :ref:`drivers` for more information.
+
+**success**
+  Describes where Zuul should report to if all the jobs complete
+  successfully.  This section is optional; if it is omitted, Zuul will
+  run jobs and do nothing on success -- it will not report at all.  If
+  the section is present, the listed reporters will be asked to report
+  on the jobs.  The reporters are listed by their connection name. The
+  options available depend on the driver for the supplied connection.
+
+**failure**
+  These reporters describe what Zuul should do if at least one job
+  fails.
+
+**merge-failure**
+  These reporters describe what Zuul should do if it is unable to
+  merge in the patchset. If no merge-failure reporters are listed then
+  the ``failure`` reporters will be used to notify of unsuccessful
+  merges.
+
+**start**
+  These reporters describe what Zuul should do when a change is added
+  to the pipeline.  This can be used, for example, to reset a
+  previously reported result.
+
+**disabled**
+  These reporters describe what Zuul should do when a pipeline is
+  disabled.  See ``disable-after-consecutive-failures``.
+
+The following options can be used to alter Zuul's behavior to mitigate
+situations in which jobs are failing frequently (perhaps due to a
+problem with an external dependency, or unusually high
+non-deterministic test failures).
+
+**disable-after-consecutive-failures**
+  If set, a pipeline can enter a ''disabled'' state if too many changes
+  in a row fail. When this value is exceeded the pipeline will stop
+  reporting to any of the ``success``, ``failure`` or ``merge-failure``
+  reporters and instead only report to the ``disabled`` reporters.
+  (No ``start`` reports are made when a pipeline is disabled).
+
+**window**
+  Dependent pipeline managers only. Zuul can rate limit dependent
+  pipelines in a manner similar to TCP flow control.  Jobs are only
+  started for items in the queue if they are within the actionable
+  window for the pipeline. The initial length of this window is
+  configurable with this value. The value given should be a positive
+  integer value. A value of ``0`` disables rate limiting on the
+  DependentPipelineManager.  Default: ``20``.
+
+**window-floor**
+  Dependent pipeline managers only. This is the minimum value for the
+  window described above. Should be a positive non zero integer value.
+  Default: ``3``.
+
+**window-increase-type**
+  Dependent pipeline managers only. This value describes how the window
+  should grow when changes are successfully merged by zuul. A value of
+  ``linear`` indicates that ``window-increase-factor`` should be added
+  to the previous window value. A value of ``exponential`` indicates
+  that ``window-increase-factor`` should be multiplied against the
+  previous window value and the result will become the window size.
+  Default: ``linear``.
+
+**window-increase-factor**
+  Dependent pipeline managers only. The value to be added or multiplied
+  against the previous window value to determine the new window after
+  successful change merges.
+  Default: ``1``.
+
+**window-decrease-type**
+  Dependent pipeline managers only. This value describes how the window
+  should shrink when changes are not able to be merged by Zuul. A value
+  of ``linear`` indicates that ``window-decrease-factor`` should be
+  subtracted from the previous window value. A value of ``exponential``
+  indicates that ``window-decrease-factor`` should be divided against
+  the previous window value and the result will become the window size.
+  Default: ``exponential``.
+
+**window-decrease-factor**
+  Dependent pipline managers only. The value to be subtracted or divided
+  against the previous window value to determine the new window after
+  unsuccessful change merges.
+  Default: ``2``.
+
+
+.. _job:
+
+Job
+~~~
+
+A job is a unit of work performed by Zuul on an item enqueued into a
+pipeline.  Items may run any number of jobs (which may depend on each
+other).  Each job is an invocation of an Ansible playbook with a
+specific inventory of hosts.  The actual tasks that are run by the job
+appear in the playbook for that job while the attributes that appear in the
+Zuul configuration specify information about when, where, and how the
+job should be run.
+
+Jobs in Zuul support inheritance.  Any job may specify a single parent
+job, and any attributes not set on the child job are collected from
+the parent job.  In this way, a configuration structure may be built
+starting with very basic jobs which describe characteristics that all
+jobs on the system should have, progressing through stages of
+specialization before arriving at a particular job.  A job may inherit
+from any other job in any project (however, if the other job is marked
+as `final`, some attributes may not be overidden).
+
+Jobs also support a concept called variance.  The first time a job
+definition appears is called the reference definition of the job.
+Subsequent job definitions with the same name are called variants.
+These may have different selection criteria which indicate to Zuul
+that, for instance, the job should behave differently on a different
+git branch.  Unlike inheritance, all job variants must be defined in
+the same project.
+
+When Zuul decides to run a job, it performs a process known as
+freezing the job.  Because any number of job variants may be
+applicable, Zuul collects all of the matching variants and applies
+them in the order they appeared in the configuration.  The resulting
+frozen job is built from attributes gathered from all of the
+matching variants.  In this way, exactly what is run is dependent on
+the pipeline, project, branch, and content of the item.
+
+In addition to the job's main playbook, each job may specify one or
+more pre- and post-playbooks.  These are run, in order, before and
+after (respectively) the main playbook.  They may be used to set up
+and tear down resources needed by the main playbook.  When combined
+with inheritance, they provide powerful tools for job construction.  A
+job only has a single main playbook, and when inheriting from a
+parent, the child's main playbook overrides (or replaces) the
+parent's.  However, the pre- and post-playbooks are appended and
+prepended in a nesting fashion.  So if a parent job and child job both
+specified pre and post playbooks, the sequence of playbooks run would
+be:
+
+* parent pre-run playbook
+* child pre-run playbook
+* child playbook
+* child post-run playbook
+* parent post-run playbook
+
+Further inheritance would nest even deeper.
+
+Here is an example of two job definitions::
+
+  - job:
+      name: base
+      pre-run: copy-git-repos
+      post-run: copy-logs
+
+  - job:
+      name: run-tests
+      parent: base
+      nodes:
+        - name: test-node
+	  image: fedora
+
+The following attributes are available on a job; all are optional
+unless otherwise specified:
+
+**name** (required)
+  The name of the job.  By default, Zuul looks for a playbook with
+  this name to use as the main playbook for the job.  This name is
+  also referenced later in a project pipeline configuration.
+
+**parent**
+  Specifies a job to inherit from.  The parent job can be defined in
+  this or any other project.  Any attributes not specified on a job
+  will be collected from its parent.
+
+**description**
+  A textual description of the job.  Not currently used directly by
+  Zuul, but it is used by the zuul-sphinx extension to Sphinx to
+  auto-document Zuul jobs (in which case it is interpreted as
+  ReStructuredText.
+
+**success-message**
+  Normally when a job succeeds, the string "SUCCESS" is reported as
+  the result for the job.  If set, this option may be used to supply a
+  different string.  Default: "SUCCESS".
+
+**failure-message**
+  Normally when a job fails, the string "FAILURE" is reported as
+  the result for the job.  If set, this option may be used to supply a
+  different string.  Default: "FAILURE".
+
+**success-url**
+  When a job succeeds, this URL is reported along with the result.
+  Default: none.
+
+**failure-url**
+  When a job fails, this URL is reported along with the result.
+  Default: none.
+
+**hold-following-changes**
+  In a dependent pipeline, this option may be used to indicate that no
+  jobs should start on any items which depend on the current item
+  until this job has completed successfully.  This may be used to
+  conserve build resources, at the expense of inhibiting the
+  parallelization which speeds the processing of items in a dependent
+  pipeline.  A boolean value, default: false.
+
+**voting**
+  Indicates whether the result of this job should be used in
+  determining the overall result of the item.  A boolean value,
+  default: true.
+
+**semaphore**
+  The name of a :ref:`semaphore` which should be acquired and released
+  when the job begins and ends.  If the semaphore is at maximum
+  capacity, then Zuul will wait until it can be acquired before
+  starting the job.  Default: none.
+
+**tags**
+  Metadata about this job.  Tags are units of information attached to
+  the job; they do not affect Zuul's behavior, but they can be used
+  within the job to characterize the job.  For example, a job which
+  tests a certain subsystem could be tagged with the name of that
+  subsystem, and if the job's results are reported into a database,
+  then the results of all jobs affecting that subsystem could be
+  queried.  This attribute is specified as a list of strings, and when
+  inheriting jobs or applying variants, tags accumulate in a set, so
+  the result is always a set of all the tags from all the jobs and
+  variants used in constructing the frozen job, with no duplication.
+  Default: none.
+
+**branches**
+  A regular expression (or list of regular expressions) which describe
+  on what branches a job should run (or in the case of variants: to
+  alter the behavior of a job for a certain branch).
+
+  If there is no job definition for a given job which matches the
+  branch of an item, then that job is not run for the item.
+  Otherwise, all of the job variants which match that branch (and any
+  other selection criteria) are used when freezing the job.
+
+  This example illustrates a job called *run-tests* which uses a
+  nodeset based on the current release of an operating system to
+  perform its tests, except when testing changes to the stable/2.0
+  branch, in which case it uses an older release::
+
+    - job:
+        name: run-tests
+        nodes: current-release
+
+    - job:
+        name: run-tests
+        branch: stable/2.0
+        nodes: old-release
+
+  In some cases, Zuul uses an implied value for the branch specifier
+  if none is supplied:
+
+  * For a job definition in a *config-project*, no implied branch
+    specifier is used.  If no branch specifier appears, the job
+    applies to all branches.
+
+  * In the case of an *untrusted-project*, no implied branch specifier
+    is applied to the reference definition of a job.  That is to say,
+    that if the first appearance of the job definition appears without
+    a branch specifier, then it will apply to all branches.  Note that
+    when collecting its configuration, Zuul reads the `master` branch
+    of a given project first, then other branches in alphabetical
+    order.
+
+  * Any further job variants other than the reference definition in an
+    *untrusted-project* will, if they do not have a branch specifier,
+    will have an implied branch specifier for the current branch
+    applied.
+
+  This allows for the very simple and expected workflow where if a
+  project defines a job on the master branch with no branch specifier,
+  and then creates a new branch based on master, any changes to that
+  job definition within the new branch only affect that branch.
+
+**files**
+  This attribute indicates that the job should only run on changes
+  where the specified files are modified.  This is a regular
+  expression or list of regular expressions.  Default: none.
+
+**irrelevant-files**
+  This is a negative complement of `files`.  It indicates that the job
+  should run unless *all* of the files changed match this list.  In
+  other words, if the regular expression `docs/.*` is supplied, then
+  this job will not run if the only files changed are in the docs
+  directory.  A regular expression or list of regular expressions.
+  Default: none.
+
+**auth**
+  Authentication information to be made available to the job.  This is
+  a dictionary with two potential keys:
+
+  **inherit**
+  A boolean indicating that the authentication information referenced
+  by this job should be able to be inherited by child jobs.  Normally
+  when a job inherits from another job, the auth section is not
+  included.  This permits jobs to inherit the same basic structure and
+  playbook, but ensures that secret information is unable to be
+  exposed by a child job which may alter the job's behavior.  If it is
+  safe for the contents of the authentication section to be used by
+  child jobs, set this to ``true``.  Default: ``false``.
+
+  **secrets**
+  A list of secrets which may be used by the job.  A :ref:`secret` is
+  a named collection of private information defined separately in the
+  configuration.  The secrets that appear here must be defined in the
+  same project as this job definition.
+
+  In the future, other types of authentication information may be
+  added.
+
+**nodes**
+  A list of nodes which should be supplied to the job.  This parameter
+  may be supplied either as a string, in which case it references a
+  :ref:`nodeset` definition which appears elsewhere in the
+  configuration, or a list, in which case it is interpreted in the
+  same way as a Nodeset definition (in essence, it is an anonymous
+  Node definition unique to this job).  See the :ref:`nodeset`
+  reference for the syntax to use in that case.
+
+  If a job has an empty or no node definition, it will still run and
+  may be able to perform actions on the Zuul executor.
+
+**override-branch**
+  When Zuul runs jobs for a proposed change, it normally checks out
+  the branch associated with that change on every project present in
+  the job.  If jobs are running on a ref (such as a branch tip or
+  tag), then that ref is normally checked out.  This attribute is used
+  to override that behavior and indicate that this job should,
+  regardless of the branch for the queue item, use the indicated
+  branch instead.  This can be used, for example, to run a previous
+  version of the software (from a stable maintenance branch) under
+  test even if the change being tested applies to a different branch
+  (this is only likely to be useful if there is some cross-branch
+  interaction with some component of the system being tested).  See
+  also the project-specific **override-branch** attribute under
+  **required-projects** to apply this behavior to a subset of a job's
+  projects.
+
+**timeout**
+  The time in minutes that the job should be allowed to run before it
+  is automatically aborted and failure is reported.  If no timeout is
+  supplied, the job may run indefinitely.  Supplying a timeout is
+  highly recommended.
+
+**attempts**
+  When Zuul encounters an error running a job's pre-run playbook, Zuul
+  will stop and restart the job.  Errors during the main or
+  post-run -playbook phase of a job are not affected by this parameter
+  (they are reported immediately).  This parameter controls the number
+  of attempts to make before an error is reported.  Default: 3.
+
+**pre-run**
+  The name of a playbook or list of playbooks to run before the main
+  body of a job.  The playbook is expected to reside in the
+  `playbooks/` directory of the project where the job is defined.
+
+  When a job inherits from a parent, the child's pre-run playbooks are
+  run after the parent's.  See :ref:`job` for more information.
+
+**post-run**
+  The name of a playbook or list of playbooks to run after the main
+  body of a job.  The playbook is expected to reside in the
+  `playbooks/` directory of the project where the job is defined.
+
+  When a job inherits from a parent, the child's post-run playbooks
+  are run before the parent's.  See :ref:`job` for more information.
+
+**run**
+  The name of the main playbook for this job.  This parameter is not
+  normally necessary, as it defaults to the name of the job.  However,
+  if a playbook with a different name is needed, it can be specified
+  here.  The playbook is expected to reside in the `playbooks/`
+  directory of the project where the job is defined.  When a child
+  inherits from a parent, a playbook with the name of the child job is
+  implicitly searched first, before falling back on the playbook used
+  by the parent job (unless the child job specifies a ``run``
+  attribute, in which case that value is used).  Default: the name of
+  the job.
+
+**roles**
+  A list of Ansible roles to prepare for the job.  Because a job runs
+  an Ansible playbook, any roles which are used by the job must be
+  prepared and installed by Zuul before the job begins.  This value is
+  a list of dictionaries, each of which indicates one of two types of
+  roles: a Galaxy role, which is simply a role that is installed from
+  Ansible Galaxy, or a Zuul role, which is a role provided by a
+  project managed by Zuul.  Zuul roles are able to benefit from
+  speculative merging and cross-project dependencies when used by jobs
+  in untrusted projects.
+
+  A project which supplies a role may be structured in one of two
+  configurations: a bare role (in which the role exists at the root of
+  the project), or a contained role (in which the role exists within
+  the `roles/` directory of the project, perhaps along with other
+  roles).  In the case of a contained role, the `roles/` directory of
+  the project is added to the role search path.  In the case of a bare
+  role, the project itself is added to the role search path.  In case
+  the name of the project is not the name under which the role should
+  be installed (and therefore referenced from Ansible), the `name`
+  attribute may be used to specify an alternate.
+
+  .. note:: galaxy roles are not yet implemented
+
+  **galaxy**
+    The name of the role in Ansible Galaxy.  If this attribute is
+    supplied, Zuul will search Ansible Galaxy for a role by this name
+    and install it.  Mutually exclusive with ``zuul``; either
+    ``galaxy`` or ``zuul`` must be supplied.
+
+  **zuul**
+    The name of a Zuul project which supplies the role.  Mutually
+    exclusive with ``galaxy``; either ``galaxy`` or ``zuul`` must be
+    supplied.
+
+  **name**
+    The installation name of the role.  In the case of a bare role,
+    the role will be made available under this name.  Ignored in the
+    case of a contained role.
+
+**required-projects**
+  A list of other projects which are used by this job.  Any Zuul
+  projects specified here will also be checked out by Zuul into the
+  working directory for the job.  Speculative merging and cross-repo
+  dependencies will be honored.
+
+  The format for this attribute is either a list of strings or
+  dictionaries.  Strings are interpreted as project names,
+  dictionaries may have the following attributes:
+
+  **name**
+    The name of the required project.
+
+  **override-branch**
+    When Zuul runs jobs for a proposed change, it normally checks out
+    the branch associated with that change on every project present in
+    the job.  If jobs are running on a ref (such as a branch tip or
+    tag), then that ref is normally checked out.  This attribute is
+    used to override that behavior and indicate that this job should,
+    regardless of the branch for the queue item, use the indicated
+    branch instead, for only this project.  See also the
+    **override-branch** attribute of jobs to apply the same behavior
+    to all projects in a job.
+
+**vars**
+
+A dictionary of variables to supply to Ansible.  When inheriting from
+a job (or creating a variant of a job) vars are merged with previous
+definitions.  This means a variable definition with the same name will
+override a previously defined variable, but new variable names will be
+added to the set of defined variables.
+
+**dependencies**
+  A list of other jobs upon which this job depends.  Zuul will not
+  start executing this job until all of its dependencies have
+  completed successfully, and if one or more of them fail, this job
+  will not be run.
+
+**allowed-projects**
+  A list of Zuul projects which may use this job.  By default, a job
+  may be used by any other project known to Zuul, however, some jobs
+  use resources or perform actions which are not appropriate for other
+  projects.  In these cases, a list of projects which are allowed to
+  use this job may be supplied.  If this list is not empty, then it
+  must be an exhaustive list of all projects permitted to use the job.
+  The current project (where the job is defined) is not automatically
+  included, so if it should be able to run this job, then it must be
+  explicitly listed.  Default: the empty list (all projects may use
+  the job).
+
+
+.. _project:
+
+Project
+~~~~~~~
+
+A project corresponds to a source code repository with which Zuul is
+configured to interact.  The main responsibility of the `Project`
+configuration item is to specify which jobs should run in which
+pipelines for a given project.  Within each `Project` definition, a
+section for each `Pipeline` may appear.  This project-pipeline
+definition is what determines how a project participates in a
+pipeline.
+
+Consider the following `Project` definition::
+
+  - project:
+      name: yoyodyne
+      check:
+        jobs:
+          - check-syntax
+          - unit-tests
+      gate:
+        queue: integrated
+        jobs:
+          - unit-tests
+          - integration-tests
+
+The project has two project-pipeline stanzas, one for the `check`
+pipeline, and one for `gate`.  Each specifies which jobs shuld run
+when a change for that project enteres the respective pipeline -- when
+a change enters `check`, the `check-syntax` and `unit-test` jobs are
+run.
+
+Pipelines which use the dependent pipeline manager (e.g., the `gate`
+example shown earlier) maintain separate queues for groups of
+projects.  When Zuul serializes a set of changes which represent
+future potential project states, it must know about all of the
+projects within Zuul which may have an effect on the outcome of the
+jobs it runs.  If project *A* uses project *B* as a library, then Zuul
+must be told about that relationship so that it knows to serialize
+changes to A and B together, so that it does not merge a change to B
+while it is testing a change to A.
+
+Zuul could simply assume that all projects are related, or even infer
+relationships by which projects a job indicates it uses, however, in a
+large system that would become unwieldy very quickly, and
+unnecessarily delay changes to unrelated projects.  To allow for
+flexibility in the construction of groups of related projects, the
+change queues used by dependent pipeline managers are specified
+manually.  To group two or more related projects into a shared queue
+for a dependent pipeline, set the ``queue`` parameter to the same
+value for those projects.
+
+The `gate` project-pipeline definition above specifies that this
+project participates in the `integrated` shared queue for that
+pipeline.
+
+In addition to a project-pipeline definition for one or more
+`Pipelines`, the following attributes may appear in a Project:
+
+**name** (required)
+  The name of the project.  If Zuul is configured with two or more
+  unique projects with the same name, the canonical hostname for the
+  project should be included (e.g., `git.example.com/foo`).
+
+**templates**
+  A list of :ref:`project-template` references; the project-pipeline
+  definitions of each Project Template will be applied to this
+  project.  If more than one template includes jobs for a given
+  pipeline, they will be combined, as will any jobs specified in
+  project-pipeline definitions on the project itself.
+
+.. _project-template:
+
+Project Template
+~~~~~~~~~~~~~~~~
+
+A Project Template defines one or more project-pipeline definitions
+which can be re-used by multiple projects.
+
+A Project Template uses the same syntax as a :ref:`project`
+definition, however, in the case of a template, the ``name`` attribute
+does not refer to the name of a project, but rather names the template
+so that it can be referenced in a `Project` definition.
+
+.. _secret:
+
+Secret
+~~~~~~
+
+A Secret is a collection of private data for use by one or more jobs.
+In order to maintain the security of the data, the values are usually
+encrypted, however, data which are not sensitive may be provided
+unencrypted as well for convenience.
+
+A Secret may only be used by jobs defined within the same project.  To
+use a secret, a :ref:`job` must specify the secret within its `auth`
+section.  To protect against jobs in other repositories declaring a
+job with a secret as a parent and then exposing that secret, jobs
+which inherit from a job with secrets will not inherit the secrets
+themselves.  To alter that behavior, see the `inherit` job attribute.
+Further, jobs which do not permit children to inherit secrets (the
+default) are also automatically marked `final`, meaning that their
+execution related attributes may not be changed in a project-pipeline
+stanza.  This is to protect against a job with secrets defined in one
+project being used by another project in a way which might expose the
+secrets.  If a job with secrets is unsafe to be used by other
+projects, the `allowed-projects` job attribute can be used to restrict
+the projects which can invoke that job.  Finally, pipelines which are
+used to execute proposed but unreviewed changes can set the
+`allow-secrets` attribute to indicate that they should not supply
+secrets at all in order to protect against someone proposing a change
+which exposes a secret.
+
+The following attributes are required:
+
+**name** (required)
+  The name of the secret, used in a :ref:`Job` definition to request
+  the secret.
+
+**data** (required)
+  A dictionary which will be added to the Ansible variables available
+  to the job.  The values can either be plain text strings, or
+  encrypted values.  See :ref:`encryption` for more information.
+
+.. _nodeset:
+
+Nodeset
+~~~~~~~
+
+A Nodeset is a named collection of nodes for use by a job.  Jobs may
+specify what nodes they require individually, however, by defining
+groups of node types once and referring to them by name, job
+configuration may be simplified.
+
+A Nodeset requires two attributes:
+
+**name** (required)
+  The name of the Nodeset, to be referenced by a :ref:`job`.
+
+**nodes** (required)
+  A list of node definitions, each of which has the following format:
+
+  **name** (required)
+    The name of the node.  This will appear in the Ansible inventory
+    for the job.
+
+  **label** (required)
+    The Nodepool label for the node.  Zuul will request a node with
+    this label.
+
+.. _semaphore:
+
+Semaphore
+~~~~~~~~~
+
+Semaphores can be used to restrict the number of certain jobs which
+are running at the same time.  This may be useful for jobs which
+access shared or limited resources.  A semaphore has a value which
+represents the maximum number of jobs which use that semaphore at the
+same time.
+
+Semaphores are never subject to dynamic reconfiguration.  If the value
+of a semaphore is changed, it will take effect only when the change
+where it is updated is merged.  An example follows::
+
+  - semaphore:
+      name: semaphore-foo
+      max: 5
+  - semaphore:
+      name: semaphore-bar
+      max: 3
+
+The following attributes are available:
+
+**name** (required)
+  The name of the semaphore, referenced by jobs.
+
+**max**
+  The maximum number of running jobs which can use this semaphore.
+  Defaults to 1.
diff --git a/doc/source/user/encryption.rst b/doc/source/user/encryption.rst
new file mode 100644
index 0000000..fdf2c5a
--- /dev/null
+++ b/doc/source/user/encryption.rst
@@ -0,0 +1,46 @@
+:title: Encryption
+
+.. _encryption:
+
+Encryption
+==========
+
+Zuul supports storing encrypted data directly in the git repositories
+of projects it operates on.  If you have a job which requires private
+information in order to run (e.g., credentials to interact with a
+third-party service) those credentials can be stored along with the
+job definition.
+
+Each project in Zuul has its own automatically generated RSA keypair
+which can be used by anyone to encrypt a secret and only Zuul is able
+to decrypt it.  Zuul serves each project's public key using its
+build-in webserver.  They can be fetched at the path
+``/keys/<source>/<project>.pub`` where ``<project>`` is the name of a
+project and ``<source>`` is the name of that project's connection in
+the main Zuul configuration file.
+
+Zuul currently supports one encryption scheme, PKCS#1 with OAEP, which
+can not store secrets longer than the key length, 4096 bits.  The
+padding used by this scheme ensures that someone examining the
+encrypted data can not determine the length of the plaintext version
+of the data, except to know that it is not longer than 4096 bits.
+
+In the config files themselves, Zuul uses an extensible method of
+specifying the encryption scheme used for a secret so that other
+schemes may be added later.  To specify a secret, use the
+``!encrypted/pkcs1-oaep`` YAML tag along with the base64 encoded
+value.  For example::
+
+  - secret:
+      name: test_secret
+      data:
+        password: !encrypted/pkcs1-oaep |
+          BFhtdnm8uXx7kn79RFL/zJywmzLkT1GY78P3bOtp4WghUFWobkifSu7ZpaV4NeO0s71YUsi1wGZZ
+          ...
+
+Zuul provides a standalone script to make encrypting values easy; it
+can be found at `tools/encrypt_secret.py` in the Zuul source
+directory.
+
+.. program-output:: python3 ../../tools/encrypt_secret.py --help
+
diff --git a/doc/source/user/gating.rst b/doc/source/user/gating.rst
new file mode 100644
index 0000000..3398892
--- /dev/null
+++ b/doc/source/user/gating.rst
@@ -0,0 +1,428 @@
+:title: Project Gating
+
+.. _project_gating:
+
+Project Gating
+==============
+
+Traditionally, many software development projects merge changes from
+developers into the repository, and then identify regressions
+resulting from those changes (perhaps by running a test suite with a
+continuous integration system), followed by more patches to fix those
+bugs.  When the mainline of development is broken, it can be very
+frustrating for developers and can cause lost productivity,
+particularly so when the number of contributors or contributions is
+large.
+
+The process of gating attempts to prevent changes that introduce
+regressions from being merged.  This keeps the mainline of development
+open and working for all developers, and only when a change is
+confirmed to work without disruption is it merged.
+
+Many projects practice an informal method of gating where developers
+with mainline commit access ensure that a test suite runs before
+merging a change.  With more developers, more changes, and more
+comprehensive test suites, that process does not scale very well, and
+is not the best use of a developer's time.  Zuul can help automate
+this process, with a particular emphasis on ensuring large numbers of
+changes are tested correctly.
+
+Testing in parallel
+-------------------
+
+A particular focus of Zuul is ensuring correctly ordered testing of
+changes in parallel.  A gating system should always test each change
+applied to the tip of the branch exactly as it is going to be merged.
+A simple way to do that would be to test one change at a time, and
+merge it only if it passes tests.  That works very well, but if
+changes take a long time to test, developers may have to wait a long
+time for their changes to make it into the repository.  With some
+projects, it may take hours to test changes, and it is easy for
+developers to create changes at a rate faster than they can be tested
+and merged.
+
+Zuul's :ref:`dependent pipeline manager<dependent_pipeline_manager>`
+allows for parallel execution of test jobs for gating while ensuring
+changes are tested correctly, exactly as if they had been tested one
+at a time.  It does this by performing speculative execution of test
+jobs; it assumes that all jobs will succeed and tests them in parallel
+accordingly.  If they do succeed, they can all be merged.  However, if
+one fails, then changes that were expecting it to succeed are
+re-tested without the failed change.  In the best case, as many
+changes as execution contexts are available may be tested in parallel
+and merged at once.  In the worst case, changes are tested one at a
+time (as each subsequent change fails, changes behind it start again).
+
+For example, if a core developer approves five changes in rapid
+succession::
+
+  A, B, C, D, E
+
+Zuul queues those changes in the order they were approved, and notes
+that each subsequent change depends on the one ahead of it merging:
+
+.. blockdiag::
+
+  blockdiag foo {
+    node_width = 40;
+    span_width = 40;
+    A <- B <- C <- D <- E;
+  }
+
+Zuul then starts immediately testing all of the changes in parallel.
+But in the case of changes that depend on others, it instructs the
+test system to include the changes ahead of it, with the assumption
+they pass.  That means jobs testing change *B* include change *A* as
+well::
+
+  Jobs for A: merge change A, then test
+  Jobs for B: merge changes A and B, then test
+  Jobs for C: merge changes A, B and C, then test
+  Jobs for D: merge changes A, B, C and D, then test
+  Jobs for E: merge changes A, B, C, D and E, then test
+
+Hence jobs triggered to tests A will only test A and ignore B, C, D:
+
+.. blockdiag::
+
+  blockdiag foo {
+    node_width = 40;
+    span_width = 40;
+    master -> A -> B -> C -> D -> E;
+    group jobs_for_A {
+        label = "Merged changes for A";
+        master -> A;
+    }
+    group ignored_to_test_A {
+        label = "Ignored changes";
+        color = "lightgray";
+        B -> C -> D -> E;
+    }
+  }
+
+The jobs for E would include the whole dependency chain: A, B, C, D, and E.
+E will be tested assuming A, B, C, and D passed:
+
+.. blockdiag::
+
+  blockdiag foo {
+    node_width = 40;
+    span_width = 40;
+    group jobs_for_E {
+        label = "Merged changes for E";
+        master -> A -> B -> C -> D -> E;
+    }
+  }
+
+If changes *A* and *B* pass tests (green), and *C*, *D*, and *E* fail (red):
+
+.. blockdiag::
+
+  blockdiag foo {
+    node_width = 40;
+    span_width = 40;
+
+    A [color = lightgreen];
+    B [color = lightgreen];
+    C [color = pink];
+    D [color = pink];
+    E [color = pink];
+
+    master <- A <- B <- C <- D <- E;
+  }
+
+Zuul will merge change *A* followed by change *B*, leaving this queue:
+
+.. blockdiag::
+
+  blockdiag foo {
+    node_width = 40;
+    span_width = 40;
+
+    C [color = pink];
+    D [color = pink];
+    E [color = pink];
+
+    C <- D <- E;
+  }
+
+Since *D* was dependent on *C*, it is not clear whether *D*'s failure is the
+result of a defect in *D* or *C*:
+
+.. blockdiag::
+
+  blockdiag foo {
+    node_width = 40;
+    span_width = 40;
+
+    C [color = pink];
+    D [label = "D\n?"];
+    E [label = "E\n?"];
+
+    C <- D <- E;
+  }
+
+Since *C* failed, Zuul will report its failure and drop *C* from the queue,
+keeping D and E:
+
+.. blockdiag::
+
+  blockdiag foo {
+    node_width = 40;
+    span_width = 40;
+
+    D [label = "D\n?"];
+    E [label = "E\n?"];
+
+    D <- E;
+  }
+
+This queue is the same as if two new changes had just arrived, so Zuul
+starts the process again testing *D* against the tip of the branch, and
+*E* against *D*:
+
+.. blockdiag::
+
+  blockdiag foo {
+    node_width = 40;
+    span_width = 40;
+    master -> D -> E;
+    group jobs_for_D {
+        label = "Merged changes for D";
+        master -> D;
+    }
+    group ignored_to_test_D {
+        label = "Skip";
+        color = "lightgray";
+        E;
+    }
+  }
+
+.. blockdiag::
+
+  blockdiag foo {
+    node_width = 40;
+    span_width = 40;
+    group jobs_for_E {
+        label = "Merged changes for E";
+        master -> D -> E;
+    }
+  }
+
+
+Cross Project Testing
+---------------------
+
+When your projects are closely coupled together, you want to make sure
+changes entering the gate are going to be tested with the version of
+other projects currently enqueued in the gate (since they will
+eventually be merged and might introduce breaking features).
+
+Such relationships can be defined in Zuul configuration by placing
+projects in a shared queue within a dependent pipeline.  Whenever
+changes for any project enter a pipeline with such a shared queue,
+they are tested together, such that the commits for the changes ahead
+in the queue are automatically present in the jobs for the changes
+behind them.  See :ref:`project` for more details.
+
+A given dependent pipeline may have as many shared change queues as
+necessary, so groups of related projects may share a change queue
+without interfering with unrelated projects.  Independent pipelines do
+not use shared change queues, however, they may still be used to test
+changes across projects using cross-project dependencies.
+
+.. _dependencies:
+
+Cross-Project Dependencies
+--------------------------
+
+Zuul permits users to specify dependencies across projects.  Using a
+special footer in Git commit messages, users may specify that a change
+depends on another change in any repository known to Zuul.
+
+Zuul's cross-project dependencies behave like a directed acyclic graph
+(DAG), like git itself, to indicate a one-way dependency relationship
+between changes in different git repositories.  Change A may depend on
+B, but B may not depend on A.
+
+.. TODO: update for v3 crd syntax
+
+To use them, include ``Depends-On: <gerrit-change-id>`` in the footer of
+a commit message.  Use the full Change-ID ('I' + 40 characters).
+
+
+Dependent Pipeline
+~~~~~~~~~~~~~~~~~~
+
+When Zuul sees changes with cross-project dependencies, it serializes
+them in the usual manner when enqueuing them into a pipeline.  This
+means that if change A depends on B, then when they are added to a
+dependent pipeline, B will appear first and A will follow:
+
+.. blockdiag::
+  :align: center
+
+  blockdiag crd {
+    orientation = portrait
+    span_width = 30
+    class greendot [
+        label = "",
+        shape = circle,
+        color = green,
+        width = 20, height = 20
+    ]
+
+    A_status [ class = greendot ]
+    B_status [ class = greendot ]
+    B_status -- A_status
+
+    'Change B\nChange-Id: Iabc' <- 'Change A\nDepends-On: Iabc'
+  }
+
+If tests for B fail, both B and A will be removed from the pipeline, and
+it will not be possible for A to merge until B does.
+
+
+.. note::
+
+   If changes with cross-project dependencies do not share a change
+   queue then Zuul is unable to enqueue them together, and the first
+   will be required to merge before the second is enqueued.
+
+Independent Pipeline
+~~~~~~~~~~~~~~~~~~~~
+
+When changes are enqueued into an independent pipeline, all of the
+related dependencies (both normal git-dependencies that come from
+parent commits as well as cross-project dependencies) appear in a
+dependency graph, as in a dependent pipeline. This means that even in
+an independent pipeline, your change will be tested with its
+dependencies.  Changes that were previously unable to be fully tested
+until a related change landed in a different repository may now be
+tested together from the start.
+
+All of the changes are still independent (you will note that the whole
+pipeline does not share a graph as in a dependent pipeline), but for
+each change tested, all of its dependencies are visually connected to
+it, and they are used to construct the git repositories that Zuul uses
+when testing.
+
+When looking at this graph on the status page, you will note that the
+dependencies show up as grey dots, while the actual change tested shows
+up as red or green (depending on the jobs results):
+
+.. blockdiag::
+  :align: center
+
+  blockdiag crdgrey {
+    orientation = portrait
+    span_width = 30
+    class dot [
+        label = "",
+        shape = circle,
+        width = 20, height = 20
+    ]
+
+    A_status [class = "dot", color = green]
+    B_status [class = "dot", color = grey]
+    B_status -- A_status
+
+    "Change B" <- "Change A\nDepends-On: B"
+  }
+
+This is to indicate that the grey changes are only there to establish
+dependencies.  Even if one of the dependencies is also being tested, it
+will show up as a grey dot when used as a dependency, but separately and
+additionally will appear as its own red or green dot for its test.
+
+
+.. TODO: relevant for v3?
+
+Multiple Changes
+~~~~~~~~~~~~~~~~
+
+A Gerrit change ID may refer to multiple changes (on multiple branches
+of the same project, or even multiple projects).  In these cases, Zuul
+will treat all of the changes with that change ID as dependencies.  So
+if you say that change in project A Depends-On a change ID that has
+changes in two branches of project B, then when testing the change to
+project A, both project B changes will be applied, and when deciding
+whether the project A change can merge, both changes must merge ahead
+of it.
+
+.. blockdiag::
+  :align: center
+
+  blockdiag crdmultirepos {
+    orientation = portrait
+    span_width = 30
+    class greendot [
+        label = "",
+        shape = circle,
+        color = green,
+        width = 20, height = 20
+    ]
+
+    B_stable_status [ class = "greendot" ]
+    B_master_status [ class = "greendot" ]
+    A_status [ class = "greendot" ]
+    B_stable_status -- B_master_status -- A_status
+
+    A [ label = "Repo A\nDepends-On: I123" ]
+    group {
+        orientation = portrait
+        label = "Dependencies"
+        color = "lightgray"
+
+        B_stable [ label = "Repo B\nChange-Id: I123\nBranch: stable" ]
+        B_master [ label = "Repo B\nChange-Id: I123\nBranch: master" ]
+    }
+    B_master <- A
+    B_stable <- A
+
+  }
+
+A change may depend on more than one Gerrit change ID as well.  So it
+is possible for a change in project A to depend on a change in project
+B and a change in project C.  Simply add more ``Depends-On:`` lines to
+the commit message footer.
+
+.. blockdiag::
+  :align: center
+
+  blockdiag crdmultichanges {
+    orientation = portrait
+    span_width = 30
+    class greendot [
+        label = "",
+        shape = circle,
+        color = green,
+        width = 20, height = 20
+    ]
+
+    C_status [ class = "greendot" ]
+    B_status [ class = "greendot" ]
+    A_status [ class = "greendot" ]
+    C_status -- B_status -- A_status
+
+    A [ label = "Repo A\nDepends-On: I123\nDepends-On: Iabc" ]
+    group {
+        orientation = portrait
+        label = "Dependencies"
+        color = "lightgray"
+
+        B [ label = "Repo B\nChange-Id: I123" ]
+        C [ label = "Repo C\nChange-Id: Iabc" ]
+    }
+    B, C <- A
+  }
+
+.. TODO: update for v3
+
+Cycles
+~~~~~~
+
+If a cycle is created by use of cross-project dependencies, Zuul will
+abort its work very early.  There will be no message in Gerrit and no
+changes that are part of the cycle will be enqueued into any pipeline.
+This is to protect Zuul from infinite loops.
diff --git a/doc/source/user/index.rst b/doc/source/user/index.rst
new file mode 100644
index 0000000..3eca04b
--- /dev/null
+++ b/doc/source/user/index.rst
@@ -0,0 +1,18 @@
+User's Guide
+============
+
+This guide is for all users of Zuul.  If you work on a project where
+Zuul is used to drive automation (whether that's testing proposed
+changes, building artifacts, or deploying builds), this guide will
+help you understand the concepts that underly Zuul, and how to
+configure it to meet your needs.
+
+
+.. toctree::
+   :maxdepth: 2
+
+   concepts
+   gating
+   config
+   jobs
+   encryption
diff --git a/doc/source/user/jobs.rst b/doc/source/user/jobs.rst
new file mode 100644
index 0000000..ad26327
--- /dev/null
+++ b/doc/source/user/jobs.rst
@@ -0,0 +1,90 @@
+:title: Job Content
+
+Job Content
+===========
+
+Zuul jobs are implemneted as Ansible playbooks.  Zuul prepares the
+repositories used for a job, installs any required Ansible roles, and
+then executes the job's playbooks.  Any setup or artifact collection
+required is the responsibility of the job itself.  While this flexible
+arrangement allows for almost any kind of job to be run by Zuul,
+batteries are included.  Zuul has a standard library of jobs upon
+which to build.
+
+Working Directory
+-----------------
+
+Before starting each job, the Zuul executor creates a directory to
+hold all of the content related to the job.  This includes some
+directories which are used by Zuul to configure and run Ansible and
+may not be accessible, as well as a directory tree, under ``work/``,
+that is readable and writable by the job.  The hierarchy is:
+
+**work/**
+  The working directory of the job.
+
+**work/src/**
+  Contains the prepared git repositories for the job.
+
+**work/logs/**
+  Where the Ansible log for the job is written; your job
+  may place other logs here as well.
+
+Git Repositories
+----------------
+
+The git repositories in ``work/src`` contain the repositories for all
+of the projects specified in the ``required-projects`` section of the
+job, plus the project associated with the queue item if it isn't
+already in that list.  In the case of a proposed change, that change
+and all of the changes ahead of it in the pipeline queue will already
+be merged into their respective repositories and target branches.  The
+change's project will have the change's branch checked out, as will
+all of the other projects, if that branch exists (otherwise, a
+fallback or default branch will be used).  If your job needs to
+operate on multiple branches, simply checkout the appropriate branches
+of these git repos to ensure that the job results reflect the proposed
+future state that Zuul is testing, and all dependencies are present.
+Do not use any git remotes; the local repositories are guaranteed to
+be up to date.
+
+Note that these git repositories are located on the executor; in order
+to be useful to most kinds of jobs, they will need to be present on
+the test nodes.  The ``base`` job in the standard library contains a
+pre-playbook which copies the repositories to all of the job's nodes.
+It is recommended to always inherit from this base job to ensure that
+behavior.
+
+.. TODO: link to base job documentation and/or document src (and logs?) directory
+
+Zuul Variables
+--------------
+
+Zuul supplies not only the variables specified by the job definition
+to Ansible, but also some variables from the executor itself.  They
+are:
+
+**zuul.executor.hostname**
+  The hostname of the executor.
+
+**zuul.executor.src_root**
+  The path to the source directory.
+
+**zuul.executor.log_root**
+  The path to the logs directory.
+
+SSH Keys
+--------
+
+Zuul starts each job with an SSH agent running and the key used to
+access the job's nodes added to that agent.  Generally you won't need
+to be aware of this since Ansible will use this when performing any
+tasks on remote nodes.  However, under some circumstances you may want
+to interact with the agent.  For example, you may wish to add a key
+provided as a secret to the job in order to access a specific host, or
+you may want to, in a pre-playbook, replace the key used to log into
+the assigned nodes in order to further protect it from being abused by
+untrusted job content.
+
+.. TODO: describe standard lib and link to published docs for it.
+