Split the merger into a separate process
Connect it to Zuul via Gearman. Any number of mergers may be
deployed.
Directly find the pipeline for a build when processing a result,
so that the procedure is roughly the same for build and merge
results.
The timer trigger currently requires the gerrit trigger also be
configured. Make that explicit inside of the timer trigger so
that the scheduler API interaction with triggers is cleaner.
Change-Id: I69498813764753c97c426e42d17596c2ef1d87cf
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 4b7b4b0..c5beda0 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -20,6 +20,7 @@
gating
triggers
+ merger
launchers
reporters
zuul
diff --git a/doc/source/merger.rst b/doc/source/merger.rst
new file mode 100644
index 0000000..4c445c6
--- /dev/null
+++ b/doc/source/merger.rst
@@ -0,0 +1,63 @@
+:title: Merger
+
+Merger
+======
+
+The Zuul Merger is a separate component which communicates with the
+main Zuul server via Gearman. Its purpose is to speculatively merge
+the changes for Zuul in preparation for testing. The resulting git
+commits also must be served to the test workers, and the server(s)
+running the Zuul Merger are expected to do this as well. Because both
+of these tasks are resource intensive, any number of Zuul Mergers can
+be run in parallel on distinct hosts.
+
+Configuration
+~~~~~~~~~~~~~
+
+The Zuul Merger can read the same zuul.conf file as the main Zuul
+server and requires the ``gearman``, ``gerrit``, ``merger``, and
+``zuul`` sections (indicated fields only). Be sure the zuul_url is
+set appropriately on each host that runs a zuul-merger.
+
+Zuul References
+~~~~~~~~~~~~~~~
+
+As the DependentPipelineManager may combine several changes together
+for testing when performing speculative execution, determining exactly
+how the workspace should be set up when running a Job can be complex.
+To alleviate this problem, Zuul performs merges itself, merging or
+cherry-picking changes as required and identifies the result with a
+Git reference of the form ``refs/zuul/<branch>/Z<random sha1>``.
+Preparing the workspace is then a simple matter of fetching that ref
+and checking it out. The parameters that provide this information are
+described in :ref:`launchers`.
+
+These references need to be made available via a Git repository that
+is available to Jenkins. This is accomplished by serving Zuul's Git
+repositories directly.
+
+Serving Zuul Git Repos
+~~~~~~~~~~~~~~~~~~~~~~
+
+Zuul maintains its own copies of any needed Git repositories in the
+directory specified by ``git_dir`` in the ``merger`` section of
+zuul.conf (by default, /var/lib/zuul/git). To directly serve Zuul's
+Git repositories in order to provide Zuul refs for Jenkins, you can
+configure Apache to do so using the following directives::
+
+ SetEnv GIT_PROJECT_ROOT /var/lib/zuul/git
+ SetEnv GIT_HTTP_EXPORT_ALL
+
+ AliasMatch ^/p/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$ /var/lib/zuul/git/$1
+ AliasMatch ^/p/(.*/objects/pack/pack-[0-9a-f]{40}.(pack|idx))$ /var/lib/zuul/git/$1
+ ScriptAlias /p/ /usr/lib/git-core/git-http-backend/
+
+And set ``push_change_refs`` to ``false`` (the default) in the
+``zuul`` section of zuul.conf.
+
+Note that Zuul's Git repositories are not bare, which means they have
+a working tree, and are not suitable for public consumption (for
+instance, a clone will produce a repository in an unpredictable state
+depending on what the state of Zuul's repository is when the clone
+happens). They are, however, suitable for automated systems that
+respond to Zuul triggers.
diff --git a/doc/source/triggers.rst b/doc/source/triggers.rst
index 287246c..c4485bf 100644
--- a/doc/source/triggers.rst
+++ b/doc/source/triggers.rst
@@ -35,49 +35,6 @@
be added to Gerrit. Zuul is very flexible and can take advantage of
those.
-Zuul References
-~~~~~~~~~~~~~~~
-
-As the DependentPipelineManager may combine several changes together
-for testing when performing speculative execution, determining exactly
-how the workspace should be set up when running a Job can be complex.
-To alleviate this problem, Zuul performs merges itself, merging or
-cherry-picking changes as required and identifies the result with a
-Git reference of the form ``refs/zuul/<branch>/Z<random sha1>``.
-Preparing the workspace is then a simple matter of fetching that ref
-and checking it out. The parameters that provide this information are
-described in :ref:`launchers`.
-
-These references need to be made available via a Git repository that
-is available to Jenkins. This is accomplished by serving Zuul's Git
-repositories directly.
-
-Serving Zuul Git Repos
-""""""""""""""""""""""
-
-Zuul maintains its own copies of any needed Git repositories in the
-directory specified by ``git_dir`` in the ``zuul`` section of
-zuul.conf (by default, /var/lib/zuul/git). To directly serve Zuul's
-Git repositories in order to provide Zuul refs for Jenkins, you can
-configure Apache to do so using the following directives::
-
- SetEnv GIT_PROJECT_ROOT /var/lib/zuul/git
- SetEnv GIT_HTTP_EXPORT_ALL
-
- AliasMatch ^/p/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$ /var/lib/zuul/git/$1
- AliasMatch ^/p/(.*/objects/pack/pack-[0-9a-f]{40}.(pack|idx))$ /var/lib/zuul/git/$1
- ScriptAlias /p/ /usr/lib/git-core/git-http-backend/
-
-And set ``push_change_refs`` to ``false`` (the default) in the
-``zuul`` section of zuul.conf.
-
-Note that Zuul's Git repositories are not bare, which means they have
-a working tree, and are not suitable for public consumption (for
-instance, a clone will produce a repository in an unpredictable state
-depending on what the state of Zuul's repository is when the clone
-happens). They are, however, suitable for automated systems that
-respond to Zuul triggers.
-
Timer
-----
diff --git a/doc/source/zuul.rst b/doc/source/zuul.rst
index d71912c..ee70523 100644
--- a/doc/source/zuul.rst
+++ b/doc/source/zuul.rst
@@ -83,21 +83,49 @@
""""
**layout_config**
- Path to layout config file.
+ Path to layout config file. Used by zuul-server only.
``layout_config=/etc/zuul/layout.yaml``
**log_config**
- Path to log config file.
+ Path to log config file. Used by all Zuul commands.
``log_config=/etc/zuul/logging.yaml``
**pidfile**
- Path to PID lock file.
+ Path to PID lock file. Used by all Zuul commands.
``pidfile=/var/run/zuul/zuul.pid``
**state_dir**
- Path to directory that Zuul should save state to.
+ Path to directory that Zuul should save state to. Used by all Zuul
+ commands.
``state_dir=/var/lib/zuul``
+**report_times**
+ Boolean value (``true`` or ``false``) that determines if Zuul should
+ include elapsed times for each job in the textual report. Used by
+ zuul-server only.
+ ``report_times=true``
+
+**status_url**
+ URL that will be posted in Zuul comments made to Gerrit changes when
+ starting jobs for a change. Used by zuul-server only.
+ ``status_url=https://zuul.example.com/status``
+
+**url_pattern**
+ If you are storing build logs external to the system that originally
+ ran jobs and wish to link to those logs when Zuul makes comments on
+ Gerrit changes for completed jobs this setting configures what the
+ URLs for those links should be. Used by zuul-server only.
+ ``http://logs.example.com/{change.number}/{change.patchset}/{pipeline.name}/{job.name}/{build.number}``
+
+**job_name_in_report**
+ Boolean value (``true`` or ``false``) that indicates whether the
+ job name should be included in the report (normally only the URL
+ is included). Defaults to ``false``. Used by zuul-server only.
+ ``job_name_in_report=true``
+
+merger
+""""""
+
**git_dir**
Directory that Zuul should clone local git repositories to.
``git_dir=/var/lib/zuul/git``
@@ -110,32 +138,10 @@
Optional: Value to pass to `git config user.name`.
``git_user_name=zuul``
-**report_times**
- Boolean value (``true`` or ``false``) that determines if Zuul should
- include elapsed times for each job in the textual report.
- ``report_times=true``
-
-**status_url**
- URL that will be posted in Zuul comments made to Gerrit changes when
- starting jobs for a change.
- ``status_url=https://zuul.example.com/status``
-
-**url_pattern**
- If you are storing build logs external to the system that originally
- ran jobs and wish to link to those logs when Zuul makes comments on
- Gerrit changes for completed jobs this setting configures what the
- URLs for those links should be.
- ``http://logs.example.com/{change.number}/{change.patchset}/{pipeline.name}/{job.name}/{build.number}``
-
-**job_name_in_report**
- Boolean value (``true`` or ``false``) that indicates whether the
- job name should be included in the report (normally only the URL
- is included). Defaults to ``false``.
- ``job_name_in_report=true``
-
**zuul_url**
- URL of Zuul's git repos, accessible to test workers.
- Usually "http://zuul.example.com/p".
+ URL of this merger's git repos, accessible to test workers. Usually
+ "http://zuul.example.com/p" or "http://zuul-merger01.example.com/p"
+ depending on whether the merger is co-located with the Zuul server.
smtp
""""