blob: f5e2226ba3c4e8ec6c1bb8d46cdeb6c5f44e7328 [file] [log] [blame]
James E. Blaircdd00072012-06-08 19:17:28 -07001:title: Zuul
2
3Zuul
4====
5
6Configuration
7-------------
8
9Zuul has three configuration files:
10
11**zuul.conf**
James E. Blair1f4c2bb2013-04-26 08:40:46 -070012 Connection information for Gerrit and Gearman, locations of the
13 other config files
James E. Blaircdd00072012-06-08 19:17:28 -070014**layout.yaml**
Clark Boylan00635dc2012-09-19 14:03:08 -070015 Project and pipeline configuration -- what Zuul does
James E. Blaircdd00072012-06-08 19:17:28 -070016**logging.conf**
17 Python logging config
18
19Examples of each of the three files can be found in the etc/ directory
20of the source distribution.
21
James E. Blair6aea36d2012-12-17 13:03:24 -080022.. _zuulconf:
23
James E. Blaircdd00072012-06-08 19:17:28 -070024zuul.conf
25~~~~~~~~~
26
27Zuul will look for ``/etc/zuul/zuul.conf`` or ``~/zuul.conf`` to
28bootstrap its configuration. Alternately, you may specify ``-c
29/path/to/zuul.conf`` on the command line.
30
James E. Blair1f4c2bb2013-04-26 08:40:46 -070031Gerrit and Gearman connection information are each described in a
32section of zuul.conf. The location of the other two configuration
33files (as well as the location of the PID file when running Zuul as a
34server) are specified in a third section.
James E. Blaircdd00072012-06-08 19:17:28 -070035
Clark Boylan9b670902012-09-28 13:47:56 -070036The three sections of this config and their options are documented below.
37You can also find an example zuul.conf file in the git
38`repository
K Jonathan Harker97e457e2013-02-26 13:29:38 -080039<https://github.com/openstack-infra/zuul/blob/master/etc/zuul.conf-sample>`_
Clark Boylan9b670902012-09-28 13:47:56 -070040
James E. Blair1f4c2bb2013-04-26 08:40:46 -070041gearman
Clark Boylan9b670902012-09-28 13:47:56 -070042"""""""
43
44**server**
James E. Blair1f4c2bb2013-04-26 08:40:46 -070045 Hostname or IP address of the Gearman server.
46 ``server=gearman.example.com``
Clark Boylan9b670902012-09-28 13:47:56 -070047
James E. Blair1f4c2bb2013-04-26 08:40:46 -070048**port**
49 Port on which the Gearman server is listening
50 ``port=4730``
Clark Boylan9b670902012-09-28 13:47:56 -070051
James E. Blair77cc7b82013-07-15 13:22:37 -070052gearman_server
53""""""""""""""
54
55**start**
56 Whether to start the internal Gearman server (default: False).
57 ``start=true``
58
59**log_config**
60 Path to log config file for internal Gearman server.
61 ``log_config=/etc/zuul/gearman-logging.yaml``
62
Clark Boylan9b670902012-09-28 13:47:56 -070063gerrit
64""""""
65
66**server**
67 FQDN of Gerrit server.
68 ``server=review.example.com``
69
Antoine Musso27475012012-11-26 09:53:01 +010070**baseurl**
71 Optional: path to Gerrit web interface. Defaults to ``https://<value
72 of server>/``. ``baseurl=https://review.example.com/review_site/``
73
Clark Boylan9b670902012-09-28 13:47:56 -070074**user**
75 User name to use when logging into above server via ssh.
James E. Blair1f4c2bb2013-04-26 08:40:46 -070076 ``user=zuul``
Clark Boylan9b670902012-09-28 13:47:56 -070077
78**sshkey**
79 Path to SSH key to use when logging into above server.
James E. Blair1f4c2bb2013-04-26 08:40:46 -070080 ``sshkey=/home/zuul/.ssh/id_rsa``
Clark Boylan9b670902012-09-28 13:47:56 -070081
82zuul
83""""
84
85**layout_config**
86 Path to layout config file.
87 ``layout_config=/etc/zuul/layout.yaml``
88
89**log_config**
90 Path to log config file.
91 ``log_config=/etc/zuul/logging.yaml``
92
93**pidfile**
94 Path to PID lock file.
95 ``pidfile=/var/run/zuul/zuul.pid``
96
97**state_dir**
98 Path to directory that Zuul should save state to.
99 ``state_dir=/var/lib/zuul``
100
101**git_dir**
102 Directory that Zuul should clone local git repositories to.
103 ``git_dir=/var/lib/zuul/git``
104
Paul Belangerb67aba12013-05-13 19:22:14 -0400105**git_user_email**
106 Optional: Value to pass to `git config user.email`.
107 ``git_user_email=zuul@example.com``
108
109**git_user_name**
110 Optional: Value to pass to `git config user.name`.
111 ``git_user_name=zuul``
112
Clark Boylan9b670902012-09-28 13:47:56 -0700113**push_change_refs**
114 Boolean value (``true`` or ``false``) that determines if Zuul should
115 push change refs to the git origin server for the git repositories in
116 git_dir.
117 ``push_change_refs=true``
118
James E. Blair0ac6c012013-04-26 09:04:23 -0700119**report_times**
120 Boolean value (``true`` or ``false``) that determines if Zuul should
121 include elapsed times for each job in the textual report.
122 ``report_times=true``
123
Clark Boylan9b670902012-09-28 13:47:56 -0700124**status_url**
125 URL that will be posted in Zuul comments made to Gerrit changes when
James E. Blair1f4c2bb2013-04-26 08:40:46 -0700126 starting jobs for a change.
127 ``status_url=https://zuul.example.com/status``
Clark Boylan9b670902012-09-28 13:47:56 -0700128
129**url_pattern**
James E. Blair1f4c2bb2013-04-26 08:40:46 -0700130 If you are storing build logs external to the system that originally
131 ran jobs and wish to link to those logs when Zuul makes comments on
132 Gerrit changes for completed jobs this setting configures what the
133 URLs for those links should be.
Clark Boylan9b670902012-09-28 13:47:56 -0700134 ``http://logs.example.com/{change.number}/{change.patchset}/{pipeline.name}/{job.name}/{build.number}``
135
James E. Blair8fef52b2013-08-17 17:32:50 -0700136**job_name_in_report**
137 Boolean value (``true`` or ``false``) that indicates whether the
138 job name should be included in the report (normally only the URL
139 is included). Defaults to ``false``.
140 ``job_name_in_report=true``
141
James E. Blaircdd00072012-06-08 19:17:28 -0700142layout.yaml
143~~~~~~~~~~~
144
Clark Boylan00635dc2012-09-19 14:03:08 -0700145This is the main configuration file for Zuul, where all of the pipelines
James E. Blaircdd00072012-06-08 19:17:28 -0700146and projects are defined, what tests should be run, and what actions
Clark Boylan00635dc2012-09-19 14:03:08 -0700147Zuul should perform. There are three sections: pipelines, jobs, and
James E. Blaircdd00072012-06-08 19:17:28 -0700148projects.
149
James E. Blaire5a847f2012-07-10 15:29:14 -0700150.. _includes:
151
152Includes
153""""""""
154
155Custom functions to be used in Zuul's configuration may be provided
156using the ``includes`` directive. It accepts a list of files to
157include, and currently supports one type of inclusion, a python file::
158
159 includes:
160 - python-file: local_functions.py
161
162**python-file**
163 The path to a python file. The file will be loaded and objects that
164 it defines will be placed in a special environment which can be
165 referenced in the Zuul configuration. Currently only the
166 parameter-function attribute of a Job uses this feature.
167
Clark Boylan00635dc2012-09-19 14:03:08 -0700168Pipelines
169"""""""""
James E. Blaircdd00072012-06-08 19:17:28 -0700170
Clark Boylan00635dc2012-09-19 14:03:08 -0700171Zuul can have any number of independent pipelines. Whenever a matching
172Gerrit event is found for a pipeline, that event is added to the
173pipeline, and the jobs specified for that pipeline are run. When all
174jobs specified for the pipeline that were triggered by an event are
175completed, Zuul reports back to Gerrit the results.
James E. Blaircdd00072012-06-08 19:17:28 -0700176
Clark Boylan00635dc2012-09-19 14:03:08 -0700177There are no pre-defined pipelines in Zuul, rather you can define
178whatever pipelines you need in the layout file. This is a very flexible
179system that can accommodate many kinds of workflows.
James E. Blaircdd00072012-06-08 19:17:28 -0700180
Clark Boylan00635dc2012-09-19 14:03:08 -0700181Here is a quick example of a pipeline definition followed by an
James E. Blaircdd00072012-06-08 19:17:28 -0700182explanation of each of the parameters::
183
184 - name: check
Clark Boylan00635dc2012-09-19 14:03:08 -0700185 manager: IndependentPipelineManager
James E. Blaircdd00072012-06-08 19:17:28 -0700186 trigger:
James E. Blair6c358e72013-07-29 17:06:47 -0700187 gerrit:
188 - event: patchset-created
James E. Blaircdd00072012-06-08 19:17:28 -0700189 success:
190 verified: 1
191 failure:
192 verified: -1
193
194**name**
195 This is used later in the project definition to indicate what jobs
Clark Boylan00635dc2012-09-19 14:03:08 -0700196 should be run for events in the pipeline.
James E. Blaircdd00072012-06-08 19:17:28 -0700197
James E. Blair8dbd56a2012-12-22 10:55:10 -0800198**description**
199 This is an optional field that may be used to provide a textual
200 description of the pipeline.
201
James E. Blair56370192013-01-14 15:47:28 -0800202**success-message**
203 An optional field that supplies the introductory text in message
204 reported back to Gerrit when all the voting builds are successful.
205 Defaults to "Build successful."
206
207**failure-message**
208 An optional field that supplies the introductory text in message
209 reported back to Gerrit when at least one voting build fails.
210 Defaults to "Build failed."
211
James E. Blaircdd00072012-06-08 19:17:28 -0700212**manager**
Clark Boylan00635dc2012-09-19 14:03:08 -0700213 There are currently two schemes for managing pipelines:
James E. Blaircdd00072012-06-08 19:17:28 -0700214
Clark Boylan00635dc2012-09-19 14:03:08 -0700215 *IndependentPipelineManager*
216 Every event in this pipeline should be treated as independent of
217 other events in the pipeline. This is appropriate when the order of
218 events in the pipeline doesn't matter because the results of the
219 actions this pipeline performs can not affect other events in the
220 pipeline. For example, when a change is first uploaded for review,
James E. Blaircdd00072012-06-08 19:17:28 -0700221 you may want to run tests on that change to provide early feedback
222 to reviewers. At the end of the tests, the change is not going to
223 be merged, so it is safe to run these tests in parallel without
Clark Boylan00635dc2012-09-19 14:03:08 -0700224 regard to any other changes in the pipeline. They are independent.
James E. Blaircdd00072012-06-08 19:17:28 -0700225
Clark Boylan00635dc2012-09-19 14:03:08 -0700226 Another type of pipeline that is independent is a post-merge
227 pipeline. In that case, the changes have already merged, so the
228 results can not affect any other events in the pipeline.
James E. Blaircdd00072012-06-08 19:17:28 -0700229
Clark Boylan00635dc2012-09-19 14:03:08 -0700230 *DependentPipelineManager*
231 The dependent pipeline manager is designed for gating. It ensures
James E. Blaircdd00072012-06-08 19:17:28 -0700232 that every change is tested exactly as it is going to be merged
233 into the repository. An ideal gating system would test one change
234 at a time, applied to the tip of the repository, and only if that
235 change passed tests would it be merged. Then the next change in
236 line would be tested the same way. In order to achieve parallel
Clark Boylan00635dc2012-09-19 14:03:08 -0700237 testing of changes, the dependent pipeline manager performs
James E. Blaircdd00072012-06-08 19:17:28 -0700238 speculative execution on changes. It orders changes based on
Clark Boylan00635dc2012-09-19 14:03:08 -0700239 their entry into the pipeline. It begins testing all changes in
240 parallel, assuming that each change ahead in the pipeline will pass
James E. Blaircdd00072012-06-08 19:17:28 -0700241 its tests. If they all succeed, all the changes can be tested and
Clark Boylan00635dc2012-09-19 14:03:08 -0700242 merged in parallel. If a change near the front of the pipeline
243 fails its tests, each change behind it ignores whatever tests have
244 been completed and are tested again without the change in front.
245 This way gate tests may run in parallel but still be tested
246 correctly, exactly as they will appear in the repository when
247 merged.
James E. Blaircdd00072012-06-08 19:17:28 -0700248
Clark Boylan00635dc2012-09-19 14:03:08 -0700249 One important characteristic of the DependentPipelineManager is that
James E. Blaircdd00072012-06-08 19:17:28 -0700250 it analyzes the jobs that are triggered by different projects, and
251 if those projects have jobs in common, it treats those projects as
252 related, and they share a single virtual queue of changes. Thus,
253 if there is a job that performs integration testing on two
254 projects, those two projects will automatically share a virtual
255 change queue. If a third project does not invoke that job, it
Clark Boylan00635dc2012-09-19 14:03:08 -0700256 will be part of a separate virtual change queue, and changes to
257 it will not depend on changes to the first two jobs.
James E. Blaircdd00072012-06-08 19:17:28 -0700258
259 For more detail on the theory and operation of Zuul's
Clark Boylan00635dc2012-09-19 14:03:08 -0700260 DependentPipelineManager, see: :doc:`gating`.
James E. Blaircdd00072012-06-08 19:17:28 -0700261
262**trigger**
James E. Blair6c358e72013-07-29 17:06:47 -0700263 Exactly one trigger source must be supplied for each pipeline.
James E. Blaircdd00072012-06-08 19:17:28 -0700264 Triggers are not exclusive -- matching events may be placed in
James E. Blair6c358e72013-07-29 17:06:47 -0700265 multiple pipelines, and they will behave independently in each of
266 the pipelines they match. You may select from the following:
James E. Blaircdd00072012-06-08 19:17:28 -0700267
James E. Blair6c358e72013-07-29 17:06:47 -0700268 **gerrit**
269 This describes what Gerrit events should be placed in the
270 pipeline. Multiple gerrit triggers may be listed. Further
271 parameters describe the kind of events that match:
James E. Blaircdd00072012-06-08 19:17:28 -0700272
James E. Blair6c358e72013-07-29 17:06:47 -0700273 *event*
274 The event name from gerrit. Examples: ``patchset-created``,
275 ``comment-added``, ``ref-updated``. This field is treated as a
276 regular expression.
James E. Blaircdd00072012-06-08 19:17:28 -0700277
James E. Blair6c358e72013-07-29 17:06:47 -0700278 *branch*
279 The branch associated with the event. Example: ``master``. This
280 field is treated as a regular expression, and multiple branches may
281 be listed.
James E. Blaircdd00072012-06-08 19:17:28 -0700282
James E. Blair6c358e72013-07-29 17:06:47 -0700283 *ref*
284 On ref-updated events, the branch parameter is not used, instead the
285 ref is provided. Currently Gerrit has the somewhat idiosyncratic
286 behavior of specifying bare refs for branch names (e.g., ``master``),
287 but full ref names for other kinds of refs (e.g., ``refs/tags/foo``).
288 Zuul matches what you put here exactly against what Gerrit
289 provides. This field is treated as a regular expression, and
290 multiple refs may be listed.
James E. Blaircdd00072012-06-08 19:17:28 -0700291
James E. Blair6c358e72013-07-29 17:06:47 -0700292 *approval*
293 This is only used for ``comment-added`` events. It only matches if
294 the event has a matching approval associated with it. Example:
295 ``code-review: 2`` matches a ``+2`` vote on the code review category.
296 Multiple approvals may be listed.
Antoine Mussob4e809e2012-12-06 16:58:06 +0100297
James E. Blair6c358e72013-07-29 17:06:47 -0700298 *email_filter*
299 This is used for any event. It takes a regex applied on the performer
300 email, i.e Gerrit account email address. If you want to specify
301 several email filters, you must use a YAML list. Make sure to use non
302 greedy matchers and to escapes dots!
303 Example: ``email_filter: ^.*?@example\.org$``.
304
305 *comment_filter*
306 This is only used for ``comment-added`` events. It accepts a list of
307 regexes that are searched for in the comment string. If any of these
308 regexes matches a portion of the comment string the trigger is
309 matched. ``comment_filter: retrigger`` will match when comments
310 containing 'retrigger' somewhere in the comment text are added to a
311 change.
Clark Boylanb9bcb402012-06-29 17:44:05 -0700312
James E. Blair63bb0ef2013-07-29 17:14:51 -0700313 **timer**
314 This trigger will run based on a cron-style time specification.
315 It will enqueue an event into its pipeline for every project
316 defined in the configuration. Any job associated with the
317 pipeline will run in response to that event.
318
319 *time*
320 The time specification in cron syntax. Only the 5 part syntax is
321 supported, not the symbolic names. Example: ``0 0 * * *`` runs
322 at midnight.
323
324
James E. Blair2fa50962013-01-30 21:50:41 -0800325**dequeue-on-new-patchset**
326 Normally, if a new patchset is uploaded to a change that is in a
327 pipeline, the existing entry in the pipeline will be removed (with
328 jobs canceled and any dependent changes that can no longer merge as
329 well. To suppress this behavior (and allow jobs to continue
330 running), set this to ``false``. Default: ``true``.
331
James E. Blair6736beb2013-07-11 15:18:15 -0700332**dequeue-on-conflict**
333 Normally, if there is a merge conflict or similar error with a
334 change, Zuul will immediately remove it from the queue, even if the
335 error is only due to a change that happened to be enqueued ahead of
336 it. If you would like to keep the change in the queue until it is
337 at the head to be certain that the merge conflict is intrinsic to
338 the change, set this to ``false``. Default: ``true``.
339
James E. Blaircdd00072012-06-08 19:17:28 -0700340**success**
341 Describes what Zuul should do if all the jobs complete successfully.
342 This section is optional; if it is omitted, Zuul will run jobs and
343 do nothing on success; it will not even report a message to Gerrit.
344 If the section is present, it will leave a message on the Gerrit
345 review. Each additional argument is assumed to be an argument to
346 ``gerrit review``, with the boolean value of ``true`` simply
347 indicating that the argument should be present without following it
348 with a value. For example, ``verified: 1`` becomes ``gerrit
349 review --verified 1`` and ``submit: true`` becomes ``gerrit review
350 --submit``.
351
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400352**failure**
James E. Blaircdd00072012-06-08 19:17:28 -0700353 Uses the same syntax as **success**, but describes what Zuul should
354 do if at least one job fails.
James E. Blairdc253862012-06-13 17:12:42 -0700355
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400356**start**
James E. Blairdc253862012-06-13 17:12:42 -0700357 Uses the same syntax as **success**, but describes what Zuul should
Clark Boylan00635dc2012-09-19 14:03:08 -0700358 do when a change is added to the pipeline manager. This can be used,
James E. Blairdc253862012-06-13 17:12:42 -0700359 for example, to reset the value of the Verified review category.
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400360
James E. Blair64ed6f22013-07-10 14:07:23 -0700361**precedence**
362 Indicates how the build scheduler should prioritize jobs for
363 different pipelines. Each pipeline may have one precedence, jobs
364 for pipelines with a higher precedence will be run before ones with
365 lower. The value should be one of ``high``, ``normal``, or ``low``.
366 Default: ``normal``.
367
Clark Boylan00635dc2012-09-19 14:03:08 -0700368Some example pipeline configurations are included in the sample layout
369file. The first is called a *check* pipeline::
James E. Blaircdd00072012-06-08 19:17:28 -0700370
371 - name: check
Clark Boylan00635dc2012-09-19 14:03:08 -0700372 manager: IndependentPipelineManager
James E. Blaircdd00072012-06-08 19:17:28 -0700373 trigger:
374 - event: patchset-created
375 success:
376 verified: 1
377 failure:
378 verified: -1
379
380This will trigger jobs each time a new patchset (or change) is
381uploaded to Gerrit, and report +/-1 values to Gerrit in the
382``verified`` review category. ::
383
384 - name: gate
Clark Boylan00635dc2012-09-19 14:03:08 -0700385 manager: DependentPipelineManager
James E. Blaircdd00072012-06-08 19:17:28 -0700386 trigger:
387 - event: comment-added
388 approval:
389 - approved: 1
390 success:
391 verified: 2
392 submit: true
393 failure:
394 verified: -2
395
396This will trigger jobs whenever a reviewer leaves a vote of ``1`` in the
397``approved`` review category in Gerrit (a non-standard category).
398Changes will be tested in such a way as to guarantee that they will be
399merged exactly as tested, though that will happen in parallel by
400creating a virtual queue of dependent changes and performing
401speculative execution of jobs. ::
402
403 - name: post
Clark Boylan00635dc2012-09-19 14:03:08 -0700404 manager: IndependentPipelineManager
James E. Blaircdd00072012-06-08 19:17:28 -0700405 trigger:
406 - event: ref-updated
407 ref: ^(?!refs/).*$
408
409This will trigger jobs whenever a change is merged to a named branch
410(e.g., ``master``). No output will be reported to Gerrit. This is
411useful for side effects such as creating per-commit tarballs. ::
412
413 - name: silent
Clark Boylan00635dc2012-09-19 14:03:08 -0700414 manager: IndependentPipelineManager
James E. Blaircdd00072012-06-08 19:17:28 -0700415 trigger:
416 - event: patchset-created
417
418This also triggers jobs when changes are uploaded to Gerrit, but no
419results are reported to Gerrit. This is useful for jobs that are in
Antoine Mussoce333842012-10-16 14:42:35 +0200420development and not yet ready to be presented to developers. ::
421
422 pipelines:
423 - name: post-merge
424 manager: IndependentPipelineManager
425 trigger:
426 - event: change-merged
427 success:
428 force-message: True
429 failure:
430 force-message: True
431
432The ``change-merged`` events happen when a change has been merged in the git
433repository. The change is thus closed and Gerrit will not accept modifications
434to the review scoring such as ``code-review`` or ``verified``. By using the
435``force-message: True`` parameter, Zuul will pass ``--force-message`` to the
436``gerrit review`` command, thus making sure the message is actually
437sent back to Gerrit regardless of approval scores.
438That kind of pipeline is nice to run regression or performance tests.
439
440.. note::
441 The ``change-merged`` event does not include the commit sha1 which can be
442 hazardous, it would let you report back to Gerrit though. If you were to
443 build a tarball for a specific commit, you should consider insteading using
444 the ``ref-updated`` event which does include the commit sha1 (but lack the
445 Gerrit change number).
James E. Blaircdd00072012-06-08 19:17:28 -0700446
447Jobs
448""""
449
450The jobs section is optional, and can be used to set attributes of
451jobs that are independent of their association with a project. For
452example, if a job should return a customized message on failure, that
453may be specified here. Otherwise, Zuul does not need to be told about
454each job as it builds a list from the project specification.
455
456**name**
457 The name of the job. This field is treated as a regular expression
458 and will be applied to each job that matches.
459
James E. Blaire5a847f2012-07-10 15:29:14 -0700460**failure-message (optional)**
461 The message that should be reported to Gerrit if the job fails.
James E. Blaircdd00072012-06-08 19:17:28 -0700462
James E. Blaire5a847f2012-07-10 15:29:14 -0700463**success-message (optional)**
464 The message that should be reported to Gerrit if the job fails.
James E. Blaircdd00072012-06-08 19:17:28 -0700465
James E. Blair6aea36d2012-12-17 13:03:24 -0800466**failure-pattern (optional)**
467 The URL that should be reported to Gerrit if the job fails.
James E. Blair1f4c2bb2013-04-26 08:40:46 -0700468 Defaults to the build URL or the url_pattern configured in
James E. Blair6aea36d2012-12-17 13:03:24 -0800469 zuul.conf. May be supplied as a string pattern with substitutions
470 as described in url_pattern in :ref:`zuulconf`.
471
472**success-pattern (optional)**
473 The URL that should be reported to Gerrit if the job succeeds.
James E. Blair1f4c2bb2013-04-26 08:40:46 -0700474 Defaults to the build URL or the url_pattern configured in
James E. Blair6aea36d2012-12-17 13:03:24 -0800475 zuul.conf. May be supplied as a string pattern with substitutions
476 as described in url_pattern in :ref:`zuulconf`.
477
James E. Blair222d4982012-07-16 09:31:19 -0700478**hold-following-changes (optional)**
479 This is a boolean that indicates that changes that follow this
Clark Boylan00635dc2012-09-19 14:03:08 -0700480 change in a dependent change pipeline should wait until this job
James E. Blair222d4982012-07-16 09:31:19 -0700481 succeeds before launching. If this is applied to a very short job
482 that can predict whether longer jobs will fail early, this can be
483 used to reduce the number of jobs that Zuul will launch and
484 ultimately have to cancel. In that case, a small amount of
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400485 parallelization of jobs is traded for more efficient use of testing
James E. Blair222d4982012-07-16 09:31:19 -0700486 resources. On the other hand, to apply this to a long running job
487 would largely defeat the parallelization of dependent change testing
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400488 that is the main feature of Zuul. Default: ``false``.
James E. Blair222d4982012-07-16 09:31:19 -0700489
James E. Blaire5a847f2012-07-10 15:29:14 -0700490**branch (optional)**
James E. Blaircdd00072012-06-08 19:17:28 -0700491 This job should only be run on matching branches. This field is
492 treated as a regular expression and multiple branches may be
493 listed.
494
James E. Blair70c71582013-03-06 08:50:50 -0800495**files (optional)**
496 This job should only be run if at least one of the files involved in
497 the change (added, deleted, or modified) matches at least one of the
498 file patterns listed here. This field is treated as a regular
499 expression and multiple expressions may be listed.
500
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400501**voting (optional)**
502 Boolean value (``true`` or ``false``) that indicates whatever
503 a job is voting or not. Default: ``true``.
504
James E. Blaire5a847f2012-07-10 15:29:14 -0700505**parameter-function (optional)**
506 Specifies a function that should be applied to the parameters before
507 the job is launched. The function should be defined in a python file
508 included with the :ref:`includes` directive. The function
509 should have the following signature:
510
James E. Blaird0470972013-07-29 10:05:43 -0700511 .. function:: parameters(item, job, parameters)
James E. Blaire5a847f2012-07-10 15:29:14 -0700512
513 Manipulate the parameters passed to a job before a build is
514 launched. The ``parameters`` dictionary will already contain the
515 standard Zuul job parameters, and is expected to be modified
516 in-place.
517
James E. Blaird78576a2013-07-09 10:39:17 -0700518 :param item: the current queue item
519 :type item: zuul.model.QueueItem
James E. Blaird0470972013-07-29 10:05:43 -0700520 :param job: the job about to be run
521 :type job: zuul.model.Job
James E. Blaire5a847f2012-07-10 15:29:14 -0700522 :param parameters: parameters to be passed to the job
523 :type parameters: dict
524
James E. Blair1f4c2bb2013-04-26 08:40:46 -0700525 If the parameter **ZUUL_NODE** is set by this function, then it will
526 be used to specify on what node (or class of node) the job should be
527 run.
528
James E. Blaircdd00072012-06-08 19:17:28 -0700529Here is an example of setting the failure message for jobs that check
530whether a change merges cleanly::
531
532 - name: ^.*-merge$
533 failure-message: This change was unable to be automatically merged
534 with the current state of the repository. Please rebase your
535 change and upload a new patchset.
536
537Projects
538""""""""
539
Clark Boylan00635dc2012-09-19 14:03:08 -0700540The projects section indicates what jobs should be run in each pipeline
James E. Blaircdd00072012-06-08 19:17:28 -0700541for events associated with each project. It contains a list of
542projects. Here is an example::
543
544 - name: example/project
545 check:
546 - project-merge:
547 - project-unittest
Clark Boylan00635dc2012-09-19 14:03:08 -0700548 - project-pep8
549 - project-pyflakes
James E. Blaircdd00072012-06-08 19:17:28 -0700550 gate:
551 - project-merge:
552 - project-unittest
Clark Boylan00635dc2012-09-19 14:03:08 -0700553 - project-pep8
554 - project-pyflakes
James E. Blaircdd00072012-06-08 19:17:28 -0700555 post:
556 - project-publish
557
558**name**
559 The name of the project (as known by Gerrit).
560
James E. Blair19deff22013-08-25 13:17:35 -0700561**merge-mode (optional)**
562 An optional value that indicates what strategy should be used to
563 merge changes to this project. Supported values are:
564
565 ** merge-resolve **
566 Equivalent to 'git merge -s resolve'. This corresponds closely to
567 what Gerrit performs (using JGit) for a project if the "Merge if
568 necessary" merge mode is selected and "Automatically resolve
569 conflicts" is checked. This is the default.
570
571 ** merge **
572 Equivalent to 'git merge'.
573
574 ** cherry-pick **
575 Equivalent to 'git cherry-pick'.
576
Clark Boylan00635dc2012-09-19 14:03:08 -0700577This is followed by a section for each of the pipelines defined above.
578Pipelines may be omitted if no jobs should run for this project in a
579given pipeline. Within the pipeline section, the jobs that should be
James E. Blaircdd00072012-06-08 19:17:28 -0700580executed are listed. If a job is entered as a dictionary key, then
581jobs contained within that key are only executed if the key job
582succeeds. In the above example, project-unittest, project-pep8, and
583project-pyflakes are only executed if project-merge succeeds. This
584can help avoid running unnecessary jobs.
585
Paul Belangerffef9e42013-02-11 22:15:18 -0500586.. seealso:: The OpenStack Zuul configuration for a comprehensive example: https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/zuul/layout.yaml
James E. Blaircdd00072012-06-08 19:17:28 -0700587
Antoine Musso80edd5a2013-02-13 15:37:53 +0100588Project Templates
589"""""""""""""""""
590
591Whenever you have lot of similiar projects (such as plugins for a project) you
592will most probably want to use the same pipeline configurations. The
593project templates let you define pipelines and job name templates to trigger.
594One can then just apply the template on its project which make it easier to
595update several similiar projects. As an example::
596
597 project-templates:
598 # Name of the template
599 - name: plugin-triggering
600 # Definition of pipelines just like for a `project`
601 check:
602 - '{jobprefix}-merge':
603 - '{jobprefix}-pep8'
604 - '{jobprefix}-pyflakes'
605 gate:
606 - '{jobprefix}-merge':
607 - '{jobprefix}-unittest'
608 - '{jobprefix}-pep8'
609 - '{jobprefix}-pyflakes'
610
611In your projects definition, you will then apply the template using the template
612key::
613
614 projects:
615 - name: plugin/foobar
616 template:
617 - name: plugin-triggering
618 jobprefix: plugin-foobar
619
620You can pass several parameters to a template. A ``parameter`` value will be
621used for expansion of ``{parameter}`` in the template strings.
James E. Blaircdd00072012-06-08 19:17:28 -0700622
623logging.conf
624~~~~~~~~~~~~
625This file is optional. If provided, it should be a standard
626:mod:`logging.config` module configuration file. If not present, Zuul will
627output all log messages of DEBUG level or higher to the console.
628
629Starting Zuul
630-------------
631
632To start Zuul, run **zuul-server**::
633
Antoine Mussob3aa8282013-04-19 15:16:59 +0200634 usage: zuul-server [-h] [-c CONFIG] [-l LAYOUT] [-d] [-t] [--version]
James E. Blaircdd00072012-06-08 19:17:28 -0700635
636 Project gating system.
637
638 optional arguments:
639 -h, --help show this help message and exit
640 -c CONFIG specify the config file
Antoine Mussob3aa8282013-04-19 15:16:59 +0200641 -l LAYOUT specify the layout file
James E. Blaircdd00072012-06-08 19:17:28 -0700642 -d do not run as a daemon
Antoine Mussob3aa8282013-04-19 15:16:59 +0200643 -t validate layout file syntax
644 --version show zuul version
James E. Blaircdd00072012-06-08 19:17:28 -0700645
646You may want to use the ``-d`` argument while you are initially setting
647up Zuul so you can detect any configuration errors quickly. Under
648normal operation, omit ``-d`` and let Zuul run as a daemon.
649
650If you send signal 1 (SIGHUP) to the zuul-server process, Zuul will
651stop executing new jobs, wait until all executing jobs are finished,
652reload its configuration, and resume. Any values in any of the
653configuration files may be changed, except the location of Zuul's PID
654file (a change to that will be ignored until Zuul is restarted).
Clark Boylanf231fa22013-02-08 12:28:53 -0800655
656If you send a SIGUSR1 to the zuul-server process, Zuul will stop
657executing new jobs, wait until all executing jobs are finished,
658then exit. While waiting to exit Zuul will queue Gerrit events and
659save these events prior to exiting. When Zuul starts again it will
660read these saved events and act on them.
Jeremy Stanley93e05f42013-03-08 17:29:17 +0000661
662If you need to abort zuul and intend to manually requeue changes for
663jobs which were running in its pipelines, prior to terminating you can
664use the zuul-changes.py tool script to simplify the process. For
665example, this would give you a list of Gerrit commands to reverify or
666recheck changes for the gate and check pipelines respectively::
667
668 ./tools/zuul-changes.py --review-host=review.openstack.org \
669 http://zuul.openstack.org/ gate 'reverify no bug'
670 ./tools/zuul-changes.py --review-host=review.openstack.org \
671 http://zuul.openstack.org/ check 'recheck no bug'
Clark Boylanfba9b242013-08-20 10:11:17 -0700672
673If you send a SIGUSR2 to the zuul-server process, Zuul will dump a stack
674trace for each running thread into its debug log. This is useful for
675tracking down deadlock or otherwise slow threads.