blob: d8de7028de3483daaaa01b2b3d1f400f22d65edd [file] [log] [blame]
James E. Blair4076e2b2014-01-28 12:42:20 -08001:title: Merger
2
3Merger
4======
5
6The Zuul Merger is a separate component which communicates with the
7main Zuul server via Gearman. Its purpose is to speculatively merge
8the changes for Zuul in preparation for testing. The resulting git
9commits also must be served to the test workers, and the server(s)
10running the Zuul Merger are expected to do this as well. Because both
11of these tasks are resource intensive, any number of Zuul Mergers can
12be run in parallel on distinct hosts.
13
14Configuration
15~~~~~~~~~~~~~
16
17The Zuul Merger can read the same zuul.conf file as the main Zuul
18server and requires the ``gearman``, ``gerrit``, ``merger``, and
19``zuul`` sections (indicated fields only). Be sure the zuul_url is
20set appropriately on each host that runs a zuul-merger.
21
22Zuul References
23~~~~~~~~~~~~~~~
24
25As the DependentPipelineManager may combine several changes together
26for testing when performing speculative execution, determining exactly
27how the workspace should be set up when running a Job can be complex.
28To alleviate this problem, Zuul performs merges itself, merging or
29cherry-picking changes as required and identifies the result with a
30Git reference of the form ``refs/zuul/<branch>/Z<random sha1>``.
31Preparing the workspace is then a simple matter of fetching that ref
32and checking it out. The parameters that provide this information are
Paul Belanger174a8272017-03-14 13:20:10 -040033described in :ref:`executors`.
James E. Blair4076e2b2014-01-28 12:42:20 -080034
35These references need to be made available via a Git repository that
Joshua Heskethe8987162014-03-13 13:05:33 +110036is available to workers (such as Jenkins). This is accomplished by
37serving Zuul's Git repositories directly.
James E. Blair4076e2b2014-01-28 12:42:20 -080038
39Serving Zuul Git Repos
40~~~~~~~~~~~~~~~~~~~~~~
41
42Zuul maintains its own copies of any needed Git repositories in the
43directory specified by ``git_dir`` in the ``merger`` section of
44zuul.conf (by default, /var/lib/zuul/git). To directly serve Zuul's
Joshua Heskethe8987162014-03-13 13:05:33 +110045Git repositories in order to provide Zuul refs for workers, you can
James E. Blair4076e2b2014-01-28 12:42:20 -080046configure Apache to do so using the following directives::
47
48 SetEnv GIT_PROJECT_ROOT /var/lib/zuul/git
49 SetEnv GIT_HTTP_EXPORT_ALL
50
51 AliasMatch ^/p/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$ /var/lib/zuul/git/$1
52 AliasMatch ^/p/(.*/objects/pack/pack-[0-9a-f]{40}.(pack|idx))$ /var/lib/zuul/git/$1
53 ScriptAlias /p/ /usr/lib/git-core/git-http-backend/
54
James E. Blair4076e2b2014-01-28 12:42:20 -080055Note that Zuul's Git repositories are not bare, which means they have
56a working tree, and are not suitable for public consumption (for
57instance, a clone will produce a repository in an unpredictable state
58depending on what the state of Zuul's repository is when the clone
59happens). They are, however, suitable for automated systems that
60respond to Zuul triggers.
Antoine Mussod9ac4c12014-07-24 14:44:02 +020061
62Clearing old references
63~~~~~~~~~~~~~~~~~~~~~~~
64
65The references created under refs/zuul are not garbage collected. Since
66git fetch send them all to Gerrit to sync the repositories, the time
67spent on merge will slightly grow overtime and start being noticeable.
68
69To clean them you can use the ``tools/zuul-clear-refs.py`` script on
70each repositories. It will delete Zuul references that point to commits
71for which the commit date is older than a given amount of days (default
72360)::
73
74 ./tools/zuul-clear-refs.py /path/to/zuul/git/repo