blob: c711394b315aff28797ad95dc299c2eba7da43b0 [file] [log] [blame]
James E. Blaircdd00072012-06-08 19:17:28 -07001:title: Zuul
2
3Zuul
4====
5
6Configuration
7-------------
8
9Zuul has three configuration files:
10
11**zuul.conf**
James E. Blair1f4c2bb2013-04-26 08:40:46 -070012 Connection information for Gerrit and Gearman, locations of the
Łukasz Jernaś048acb42014-03-02 18:49:41 +010013 other config files.
James E. Blaircdd00072012-06-08 19:17:28 -070014**layout.yaml**
Łukasz Jernaś048acb42014-03-02 18:49:41 +010015 Project and pipeline configuration -- what Zuul does.
James E. Blaircdd00072012-06-08 19:17:28 -070016**logging.conf**
Łukasz Jernaś048acb42014-03-02 18:49:41 +010017 Python logging config.
James E. Blaircdd00072012-06-08 19:17:28 -070018
19Examples of each of the three files can be found in the etc/ directory
20of the source distribution.
21
James E. Blair6aea36d2012-12-17 13:03:24 -080022.. _zuulconf:
23
James E. Blaircdd00072012-06-08 19:17:28 -070024zuul.conf
25~~~~~~~~~
26
27Zuul will look for ``/etc/zuul/zuul.conf`` or ``~/zuul.conf`` to
28bootstrap its configuration. Alternately, you may specify ``-c
29/path/to/zuul.conf`` on the command line.
30
James E. Blair1f4c2bb2013-04-26 08:40:46 -070031Gerrit and Gearman connection information are each described in a
32section of zuul.conf. The location of the other two configuration
33files (as well as the location of the PID file when running Zuul as a
34server) are specified in a third section.
James E. Blaircdd00072012-06-08 19:17:28 -070035
Clark Boylan9b670902012-09-28 13:47:56 -070036The three sections of this config and their options are documented below.
37You can also find an example zuul.conf file in the git
38`repository
K Jonathan Harker97e457e2013-02-26 13:29:38 -080039<https://github.com/openstack-infra/zuul/blob/master/etc/zuul.conf-sample>`_
Clark Boylan9b670902012-09-28 13:47:56 -070040
James E. Blair1f4c2bb2013-04-26 08:40:46 -070041gearman
Clark Boylan9b670902012-09-28 13:47:56 -070042"""""""
43
44**server**
James E. Blair1f4c2bb2013-04-26 08:40:46 -070045 Hostname or IP address of the Gearman server.
46 ``server=gearman.example.com``
Clark Boylan9b670902012-09-28 13:47:56 -070047
James E. Blair1f4c2bb2013-04-26 08:40:46 -070048**port**
Łukasz Jernaś048acb42014-03-02 18:49:41 +010049 Port on which the Gearman server is listening.
James E. Blair1f4c2bb2013-04-26 08:40:46 -070050 ``port=4730``
Clark Boylan9b670902012-09-28 13:47:56 -070051
James E. Blair77cc7b82013-07-15 13:22:37 -070052gearman_server
53""""""""""""""
54
55**start**
56 Whether to start the internal Gearman server (default: False).
57 ``start=true``
58
59**log_config**
60 Path to log config file for internal Gearman server.
61 ``log_config=/etc/zuul/gearman-logging.yaml``
62
Clark Boylan9b670902012-09-28 13:47:56 -070063gerrit
64""""""
65
66**server**
67 FQDN of Gerrit server.
68 ``server=review.example.com``
69
Antoine Musso27475012012-11-26 09:53:01 +010070**baseurl**
71 Optional: path to Gerrit web interface. Defaults to ``https://<value
72 of server>/``. ``baseurl=https://review.example.com/review_site/``
73
Clark Boylan9b670902012-09-28 13:47:56 -070074**user**
75 User name to use when logging into above server via ssh.
James E. Blair1f4c2bb2013-04-26 08:40:46 -070076 ``user=zuul``
Clark Boylan9b670902012-09-28 13:47:56 -070077
78**sshkey**
79 Path to SSH key to use when logging into above server.
James E. Blair1f4c2bb2013-04-26 08:40:46 -070080 ``sshkey=/home/zuul/.ssh/id_rsa``
Clark Boylan9b670902012-09-28 13:47:56 -070081
82zuul
83""""
84
85**layout_config**
James E. Blair4076e2b2014-01-28 12:42:20 -080086 Path to layout config file. Used by zuul-server only.
Clark Boylan9b670902012-09-28 13:47:56 -070087 ``layout_config=/etc/zuul/layout.yaml``
88
89**log_config**
James E. Blaira4430132014-02-17 08:32:07 -080090 Path to log config file. Used by zuul-server only.
Clark Boylan9b670902012-09-28 13:47:56 -070091 ``log_config=/etc/zuul/logging.yaml``
92
93**pidfile**
James E. Blaira4430132014-02-17 08:32:07 -080094 Path to PID lock file. Used by zuul-server only.
Clark Boylan9b670902012-09-28 13:47:56 -070095 ``pidfile=/var/run/zuul/zuul.pid``
96
97**state_dir**
James E. Blair4076e2b2014-01-28 12:42:20 -080098 Path to directory that Zuul should save state to. Used by all Zuul
99 commands.
Clark Boylan9b670902012-09-28 13:47:56 -0700100 ``state_dir=/var/lib/zuul``
101
James E. Blair4076e2b2014-01-28 12:42:20 -0800102**report_times**
103 Boolean value (``true`` or ``false``) that determines if Zuul should
104 include elapsed times for each job in the textual report. Used by
105 zuul-server only.
106 ``report_times=true``
107
108**status_url**
109 URL that will be posted in Zuul comments made to Gerrit changes when
110 starting jobs for a change. Used by zuul-server only.
111 ``status_url=https://zuul.example.com/status``
112
113**url_pattern**
114 If you are storing build logs external to the system that originally
115 ran jobs and wish to link to those logs when Zuul makes comments on
116 Gerrit changes for completed jobs this setting configures what the
117 URLs for those links should be. Used by zuul-server only.
118 ``http://logs.example.com/{change.number}/{change.patchset}/{pipeline.name}/{job.name}/{build.number}``
119
120**job_name_in_report**
121 Boolean value (``true`` or ``false``) that indicates whether the
122 job name should be included in the report (normally only the URL
123 is included). Defaults to ``false``. Used by zuul-server only.
124 ``job_name_in_report=true``
125
126merger
127""""""
128
Clark Boylan9b670902012-09-28 13:47:56 -0700129**git_dir**
130 Directory that Zuul should clone local git repositories to.
131 ``git_dir=/var/lib/zuul/git``
132
Paul Belangerb67aba12013-05-13 19:22:14 -0400133**git_user_email**
134 Optional: Value to pass to `git config user.email`.
135 ``git_user_email=zuul@example.com``
136
137**git_user_name**
138 Optional: Value to pass to `git config user.name`.
139 ``git_user_name=zuul``
140
Arx Cruzb1b010d2013-10-28 19:49:59 -0200141**zuul_url**
James E. Blair4076e2b2014-01-28 12:42:20 -0800142 URL of this merger's git repos, accessible to test workers. Usually
143 "http://zuul.example.com/p" or "http://zuul-merger01.example.com/p"
144 depending on whether the merger is co-located with the Zuul server.
Arx Cruzb1b010d2013-10-28 19:49:59 -0200145
James E. Blaira4430132014-02-17 08:32:07 -0800146**log_config**
147 Path to log config file for the merger process.
148 ``log_config=/etc/zuul/logging.yaml``
149
150**pidfile**
151 Path to PID lock file for the merger process.
152 ``pidfile=/var/run/zuul-merger/merger.pid``
153
Joshua Hesketh5fea8672013-08-19 17:32:01 +1000154smtp
155""""
156
157**server**
158 SMTP server hostname or address to use.
159 ``server=localhost``
160
161**default_from**
162 Who the email should appear to be sent from when emailing the report.
163 This can be overridden by individual pipelines.
164 ``default_from=zuul@example.com``
165
166**default_to**
167 Who the report should be emailed to by default.
168 This can be overridden by individual pipelines.
169 ``default_to=you@example.com``
170
James E. Blaircdd00072012-06-08 19:17:28 -0700171layout.yaml
172~~~~~~~~~~~
173
Clark Boylan00635dc2012-09-19 14:03:08 -0700174This is the main configuration file for Zuul, where all of the pipelines
James E. Blaircdd00072012-06-08 19:17:28 -0700175and projects are defined, what tests should be run, and what actions
Clark Boylan00635dc2012-09-19 14:03:08 -0700176Zuul should perform. There are three sections: pipelines, jobs, and
James E. Blaircdd00072012-06-08 19:17:28 -0700177projects.
178
James E. Blaire5a847f2012-07-10 15:29:14 -0700179.. _includes:
180
181Includes
182""""""""
183
184Custom functions to be used in Zuul's configuration may be provided
185using the ``includes`` directive. It accepts a list of files to
186include, and currently supports one type of inclusion, a python file::
187
188 includes:
189 - python-file: local_functions.py
190
191**python-file**
192 The path to a python file. The file will be loaded and objects that
193 it defines will be placed in a special environment which can be
194 referenced in the Zuul configuration. Currently only the
195 parameter-function attribute of a Job uses this feature.
196
Clark Boylan00635dc2012-09-19 14:03:08 -0700197Pipelines
198"""""""""
James E. Blaircdd00072012-06-08 19:17:28 -0700199
Clark Boylan00635dc2012-09-19 14:03:08 -0700200Zuul can have any number of independent pipelines. Whenever a matching
201Gerrit event is found for a pipeline, that event is added to the
202pipeline, and the jobs specified for that pipeline are run. When all
203jobs specified for the pipeline that were triggered by an event are
204completed, Zuul reports back to Gerrit the results.
James E. Blaircdd00072012-06-08 19:17:28 -0700205
Clark Boylan00635dc2012-09-19 14:03:08 -0700206There are no pre-defined pipelines in Zuul, rather you can define
207whatever pipelines you need in the layout file. This is a very flexible
208system that can accommodate many kinds of workflows.
James E. Blaircdd00072012-06-08 19:17:28 -0700209
Clark Boylan00635dc2012-09-19 14:03:08 -0700210Here is a quick example of a pipeline definition followed by an
James E. Blaircdd00072012-06-08 19:17:28 -0700211explanation of each of the parameters::
212
213 - name: check
Clark Boylan00635dc2012-09-19 14:03:08 -0700214 manager: IndependentPipelineManager
James E. Blaircdd00072012-06-08 19:17:28 -0700215 trigger:
James E. Blair6c358e72013-07-29 17:06:47 -0700216 gerrit:
217 - event: patchset-created
James E. Blaircdd00072012-06-08 19:17:28 -0700218 success:
219 verified: 1
220 failure:
221 verified: -1
222
223**name**
224 This is used later in the project definition to indicate what jobs
Clark Boylan00635dc2012-09-19 14:03:08 -0700225 should be run for events in the pipeline.
James E. Blaircdd00072012-06-08 19:17:28 -0700226
James E. Blair8dbd56a2012-12-22 10:55:10 -0800227**description**
228 This is an optional field that may be used to provide a textual
229 description of the pipeline.
230
James E. Blair56370192013-01-14 15:47:28 -0800231**success-message**
232 An optional field that supplies the introductory text in message
233 reported back to Gerrit when all the voting builds are successful.
234 Defaults to "Build successful."
235
236**failure-message**
237 An optional field that supplies the introductory text in message
238 reported back to Gerrit when at least one voting build fails.
239 Defaults to "Build failed."
240
Joshua Hesketh3979e3e2014-03-04 11:21:10 +1100241**footer-message**
242 An optional field to supply additional information after test results.
243 Useful for adding information about the CI system such as debugging
244 and contact details.
245
James E. Blaircdd00072012-06-08 19:17:28 -0700246**manager**
Clark Boylan00635dc2012-09-19 14:03:08 -0700247 There are currently two schemes for managing pipelines:
James E. Blaircdd00072012-06-08 19:17:28 -0700248
Clark Boylan00635dc2012-09-19 14:03:08 -0700249 *IndependentPipelineManager*
250 Every event in this pipeline should be treated as independent of
251 other events in the pipeline. This is appropriate when the order of
252 events in the pipeline doesn't matter because the results of the
253 actions this pipeline performs can not affect other events in the
254 pipeline. For example, when a change is first uploaded for review,
James E. Blaircdd00072012-06-08 19:17:28 -0700255 you may want to run tests on that change to provide early feedback
256 to reviewers. At the end of the tests, the change is not going to
257 be merged, so it is safe to run these tests in parallel without
Clark Boylan00635dc2012-09-19 14:03:08 -0700258 regard to any other changes in the pipeline. They are independent.
James E. Blaircdd00072012-06-08 19:17:28 -0700259
Clark Boylan00635dc2012-09-19 14:03:08 -0700260 Another type of pipeline that is independent is a post-merge
261 pipeline. In that case, the changes have already merged, so the
262 results can not affect any other events in the pipeline.
James E. Blaircdd00072012-06-08 19:17:28 -0700263
Clark Boylan00635dc2012-09-19 14:03:08 -0700264 *DependentPipelineManager*
265 The dependent pipeline manager is designed for gating. It ensures
James E. Blaircdd00072012-06-08 19:17:28 -0700266 that every change is tested exactly as it is going to be merged
267 into the repository. An ideal gating system would test one change
268 at a time, applied to the tip of the repository, and only if that
269 change passed tests would it be merged. Then the next change in
270 line would be tested the same way. In order to achieve parallel
Clark Boylan00635dc2012-09-19 14:03:08 -0700271 testing of changes, the dependent pipeline manager performs
James E. Blaircdd00072012-06-08 19:17:28 -0700272 speculative execution on changes. It orders changes based on
Clark Boylan00635dc2012-09-19 14:03:08 -0700273 their entry into the pipeline. It begins testing all changes in
274 parallel, assuming that each change ahead in the pipeline will pass
James E. Blaircdd00072012-06-08 19:17:28 -0700275 its tests. If they all succeed, all the changes can be tested and
Clark Boylan00635dc2012-09-19 14:03:08 -0700276 merged in parallel. If a change near the front of the pipeline
277 fails its tests, each change behind it ignores whatever tests have
278 been completed and are tested again without the change in front.
279 This way gate tests may run in parallel but still be tested
280 correctly, exactly as they will appear in the repository when
281 merged.
James E. Blaircdd00072012-06-08 19:17:28 -0700282
Clark Boylan00635dc2012-09-19 14:03:08 -0700283 One important characteristic of the DependentPipelineManager is that
James E. Blaircdd00072012-06-08 19:17:28 -0700284 it analyzes the jobs that are triggered by different projects, and
285 if those projects have jobs in common, it treats those projects as
286 related, and they share a single virtual queue of changes. Thus,
287 if there is a job that performs integration testing on two
288 projects, those two projects will automatically share a virtual
289 change queue. If a third project does not invoke that job, it
Clark Boylan00635dc2012-09-19 14:03:08 -0700290 will be part of a separate virtual change queue, and changes to
291 it will not depend on changes to the first two jobs.
James E. Blaircdd00072012-06-08 19:17:28 -0700292
293 For more detail on the theory and operation of Zuul's
Clark Boylan00635dc2012-09-19 14:03:08 -0700294 DependentPipelineManager, see: :doc:`gating`.
James E. Blaircdd00072012-06-08 19:17:28 -0700295
296**trigger**
James E. Blair6c358e72013-07-29 17:06:47 -0700297 Exactly one trigger source must be supplied for each pipeline.
James E. Blaircdd00072012-06-08 19:17:28 -0700298 Triggers are not exclusive -- matching events may be placed in
James E. Blair6c358e72013-07-29 17:06:47 -0700299 multiple pipelines, and they will behave independently in each of
300 the pipelines they match. You may select from the following:
James E. Blaircdd00072012-06-08 19:17:28 -0700301
James E. Blair6c358e72013-07-29 17:06:47 -0700302 **gerrit**
303 This describes what Gerrit events should be placed in the
304 pipeline. Multiple gerrit triggers may be listed. Further
305 parameters describe the kind of events that match:
James E. Blaircdd00072012-06-08 19:17:28 -0700306
James E. Blair6c358e72013-07-29 17:06:47 -0700307 *event*
308 The event name from gerrit. Examples: ``patchset-created``,
309 ``comment-added``, ``ref-updated``. This field is treated as a
310 regular expression.
James E. Blaircdd00072012-06-08 19:17:28 -0700311
James E. Blair6c358e72013-07-29 17:06:47 -0700312 *branch*
313 The branch associated with the event. Example: ``master``. This
314 field is treated as a regular expression, and multiple branches may
315 be listed.
James E. Blaircdd00072012-06-08 19:17:28 -0700316
James E. Blair6c358e72013-07-29 17:06:47 -0700317 *ref*
318 On ref-updated events, the branch parameter is not used, instead the
319 ref is provided. Currently Gerrit has the somewhat idiosyncratic
320 behavior of specifying bare refs for branch names (e.g., ``master``),
321 but full ref names for other kinds of refs (e.g., ``refs/tags/foo``).
322 Zuul matches what you put here exactly against what Gerrit
323 provides. This field is treated as a regular expression, and
324 multiple refs may be listed.
James E. Blaircdd00072012-06-08 19:17:28 -0700325
James E. Blair6c358e72013-07-29 17:06:47 -0700326 *approval*
327 This is only used for ``comment-added`` events. It only matches if
328 the event has a matching approval associated with it. Example:
329 ``code-review: 2`` matches a ``+2`` vote on the code review category.
330 Multiple approvals may be listed.
Antoine Mussob4e809e2012-12-06 16:58:06 +0100331
James E. Blair6c358e72013-07-29 17:06:47 -0700332 *email_filter*
333 This is used for any event. It takes a regex applied on the performer
Michael Prokop526926a2013-10-24 16:16:57 +0200334 email, i.e. Gerrit account email address. If you want to specify
James E. Blair6c358e72013-07-29 17:06:47 -0700335 several email filters, you must use a YAML list. Make sure to use non
336 greedy matchers and to escapes dots!
337 Example: ``email_filter: ^.*?@example\.org$``.
338
Joshua Heskethb8a817e2013-12-27 11:21:38 +1100339 *username_filter*
340 This is used for any event. It takes a regex applied on the performer
341 username, i.e. Gerrit account name. If you want to specify several
342 username filters, you must use a YAML list. Make sure to use non greedy
343 matchers and to escapes dots!
344 Example: ``username_filter: ^jenkins$``.
345
James E. Blair6c358e72013-07-29 17:06:47 -0700346 *comment_filter*
347 This is only used for ``comment-added`` events. It accepts a list of
348 regexes that are searched for in the comment string. If any of these
349 regexes matches a portion of the comment string the trigger is
350 matched. ``comment_filter: retrigger`` will match when comments
351 containing 'retrigger' somewhere in the comment text are added to a
352 change.
Clark Boylanb9bcb402012-06-29 17:44:05 -0700353
James E. Blairc053d022014-01-22 14:57:33 -0800354 *require-approval*
355 This may be used for any event. It requires that a certain kind
356 of approval be present for the current patchset of the change (the
357 approval could be added by the event in question). It takes
358 several sub-parameters, all of which are optional and are combined
359 together so that there must be an approval matching all specified
360 requirements.
361
362 *username*
363 If present, an approval from this username is required.
364
365 *email-filter*
366 If present, an approval with this email address is required. It
367 is treated as a regular expression as above.
368
369 *older-than*
370 If present, the approval must be older than this amount of time
371 to match. Provide a time interval as a number with a suffix of
372 "w" (weeks), "d" (days), "h" (hours), "m" (minutes), "s"
Łukasz Jernaś048acb42014-03-02 18:49:41 +0100373 (seconds). Example ``48h`` or ``2d``.
James E. Blairc053d022014-01-22 14:57:33 -0800374
375 *newer-than*
376 If present, the approval must be newer than this amount of time
377 to match. Same format as "older-than".
378
379 Any other field is interpreted as a review category and value
Łukasz Jernaś048acb42014-03-02 18:49:41 +0100380 pair. For example ``verified: 1`` would require that the approval
James E. Blairc053d022014-01-22 14:57:33 -0800381 be for a +1 vote in the "Verified" column.
382
James E. Blair63bb0ef2013-07-29 17:14:51 -0700383 **timer**
384 This trigger will run based on a cron-style time specification.
385 It will enqueue an event into its pipeline for every project
386 defined in the configuration. Any job associated with the
387 pipeline will run in response to that event.
388
389 *time*
390 The time specification in cron syntax. Only the 5 part syntax is
391 supported, not the symbolic names. Example: ``0 0 * * *`` runs
392 at midnight.
393
394
James E. Blair2fa50962013-01-30 21:50:41 -0800395**dequeue-on-new-patchset**
396 Normally, if a new patchset is uploaded to a change that is in a
397 pipeline, the existing entry in the pipeline will be removed (with
398 jobs canceled and any dependent changes that can no longer merge as
399 well. To suppress this behavior (and allow jobs to continue
400 running), set this to ``false``. Default: ``true``.
401
James E. Blaircdd00072012-06-08 19:17:28 -0700402**success**
Joshua Hesketh1879cf72013-08-19 14:13:15 +1000403 Describes where Zuul should report to if all the jobs complete
404 successfully.
James E. Blaircdd00072012-06-08 19:17:28 -0700405 This section is optional; if it is omitted, Zuul will run jobs and
406 do nothing on success; it will not even report a message to Gerrit.
Joshua Hesketh1879cf72013-08-19 14:13:15 +1000407 If the section is present, the listed reporter plugins will be
408 asked to report on the jobs.
409 Each reporter's value dictionary is handled by the reporter. See
410 reporters for more details.
James E. Blaircdd00072012-06-08 19:17:28 -0700411
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400412**failure**
James E. Blaircdd00072012-06-08 19:17:28 -0700413 Uses the same syntax as **success**, but describes what Zuul should
414 do if at least one job fails.
James E. Blairdc253862012-06-13 17:12:42 -0700415
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400416**start**
James E. Blairdc253862012-06-13 17:12:42 -0700417 Uses the same syntax as **success**, but describes what Zuul should
Clark Boylan00635dc2012-09-19 14:03:08 -0700418 do when a change is added to the pipeline manager. This can be used,
James E. Blairdc253862012-06-13 17:12:42 -0700419 for example, to reset the value of the Verified review category.
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400420
James E. Blair64ed6f22013-07-10 14:07:23 -0700421**precedence**
422 Indicates how the build scheduler should prioritize jobs for
423 different pipelines. Each pipeline may have one precedence, jobs
424 for pipelines with a higher precedence will be run before ones with
425 lower. The value should be one of ``high``, ``normal``, or ``low``.
426 Default: ``normal``.
427
Clark Boylanc2d19e42014-01-23 14:08:58 -0800428**window**
429 DependentPipelineManagers only. Zuul can rate limit
430 DependentPipelineManagers in a manner similar to TCP flow control.
431 Jobs are only started for changes in the queue if they sit in the
432 actionable window for the pipeline. The initial length of this window
433 is configurable with this value. The value given should be a positive
434 integer value. A value of ``0`` disables rate limiting on the
435 DependentPipelineManager.
436 Default: ``20``.
437
438**window-floor**
439 DependentPipelineManagers only. This is the minimum value for the
440 window described above. Should be a positive non zero integer value.
441 Default: ``3``.
442
443**window-increase-type**
444 DependentPipelineManagers only. This value describes how the window
445 should grow when changes are successfully merged by zuul. A value of
446 ``linear`` indicates that ``window-increase-factor`` should be added
447 to the previous window value. A value of ``exponential`` indicates
448 that ``window-increase-factor`` should be multiplied against the
449 previous window value and the result will become the window size.
450 Default: ``linear``.
451
452**window-increase-factor**
453 DependentPipelineManagers only. The value to be added or mulitplied
454 against the previous window value to determine the new window after
455 successful change merges.
456 Default: ``1``.
457
458**window-decrease-type**
459 DependentPipelineManagers only. This value describes how the window
460 should shrink when changes are not able to be merged by Zuul. A value
461 of ``linear`` indicates that ``window-decrease-factor`` should be
462 subtracted from the previous window value. A value of ``exponential``
463 indicates that ``window-decrease-factor`` should be divided against
464 the previous window value and the result will become the window size.
465 Default: ``exponential``.
466
467**window-decrease-factor**
468 DependentPipelineManagers only. The value to be subtracted or divided
469 against the previous window value to determine the new window after
470 unsuccessful change merges.
471 Default: ``2``.
472
Clark Boylan00635dc2012-09-19 14:03:08 -0700473Some example pipeline configurations are included in the sample layout
474file. The first is called a *check* pipeline::
James E. Blaircdd00072012-06-08 19:17:28 -0700475
476 - name: check
Clark Boylan00635dc2012-09-19 14:03:08 -0700477 manager: IndependentPipelineManager
James E. Blaircdd00072012-06-08 19:17:28 -0700478 trigger:
479 - event: patchset-created
480 success:
Joshua Hesketh1879cf72013-08-19 14:13:15 +1000481 gerrit:
482 verified: 1
James E. Blaircdd00072012-06-08 19:17:28 -0700483 failure:
Joshua Hesketh1879cf72013-08-19 14:13:15 +1000484 gerrit:
485 verified: -1
James E. Blaircdd00072012-06-08 19:17:28 -0700486
487This will trigger jobs each time a new patchset (or change) is
488uploaded to Gerrit, and report +/-1 values to Gerrit in the
489``verified`` review category. ::
490
491 - name: gate
Clark Boylan00635dc2012-09-19 14:03:08 -0700492 manager: DependentPipelineManager
James E. Blaircdd00072012-06-08 19:17:28 -0700493 trigger:
494 - event: comment-added
495 approval:
496 - approved: 1
497 success:
Joshua Hesketh1879cf72013-08-19 14:13:15 +1000498 gerrit:
499 verified: 2
500 submit: true
James E. Blaircdd00072012-06-08 19:17:28 -0700501 failure:
Joshua Hesketh1879cf72013-08-19 14:13:15 +1000502 gerrit:
503 verified: -2
James E. Blaircdd00072012-06-08 19:17:28 -0700504
505This will trigger jobs whenever a reviewer leaves a vote of ``1`` in the
506``approved`` review category in Gerrit (a non-standard category).
507Changes will be tested in such a way as to guarantee that they will be
508merged exactly as tested, though that will happen in parallel by
509creating a virtual queue of dependent changes and performing
510speculative execution of jobs. ::
511
512 - name: post
Clark Boylan00635dc2012-09-19 14:03:08 -0700513 manager: IndependentPipelineManager
James E. Blaircdd00072012-06-08 19:17:28 -0700514 trigger:
515 - event: ref-updated
516 ref: ^(?!refs/).*$
517
518This will trigger jobs whenever a change is merged to a named branch
519(e.g., ``master``). No output will be reported to Gerrit. This is
520useful for side effects such as creating per-commit tarballs. ::
521
522 - name: silent
Clark Boylan00635dc2012-09-19 14:03:08 -0700523 manager: IndependentPipelineManager
James E. Blaircdd00072012-06-08 19:17:28 -0700524 trigger:
525 - event: patchset-created
526
527This also triggers jobs when changes are uploaded to Gerrit, but no
528results are reported to Gerrit. This is useful for jobs that are in
Antoine Mussoce333842012-10-16 14:42:35 +0200529development and not yet ready to be presented to developers. ::
530
531 pipelines:
532 - name: post-merge
533 manager: IndependentPipelineManager
534 trigger:
535 - event: change-merged
536 success:
Joshua Hesketh1879cf72013-08-19 14:13:15 +1000537 gerrit:
538 force-message: True
Antoine Mussoce333842012-10-16 14:42:35 +0200539 failure:
Joshua Hesketh1879cf72013-08-19 14:13:15 +1000540 gerrit:
541 force-message: True
Antoine Mussoce333842012-10-16 14:42:35 +0200542
543The ``change-merged`` events happen when a change has been merged in the git
544repository. The change is thus closed and Gerrit will not accept modifications
545to the review scoring such as ``code-review`` or ``verified``. By using the
546``force-message: True`` parameter, Zuul will pass ``--force-message`` to the
547``gerrit review`` command, thus making sure the message is actually
548sent back to Gerrit regardless of approval scores.
549That kind of pipeline is nice to run regression or performance tests.
550
551.. note::
552 The ``change-merged`` event does not include the commit sha1 which can be
553 hazardous, it would let you report back to Gerrit though. If you were to
Michael Prokop526926a2013-10-24 16:16:57 +0200554 build a tarball for a specific commit, you should consider instead using
Łukasz Jernaś048acb42014-03-02 18:49:41 +0100555 the ``ref-updated`` event which does include the commit sha1 (but lacks the
Antoine Mussoce333842012-10-16 14:42:35 +0200556 Gerrit change number).
James E. Blaircdd00072012-06-08 19:17:28 -0700557
558Jobs
559""""
560
561The jobs section is optional, and can be used to set attributes of
562jobs that are independent of their association with a project. For
563example, if a job should return a customized message on failure, that
564may be specified here. Otherwise, Zuul does not need to be told about
565each job as it builds a list from the project specification.
566
567**name**
568 The name of the job. This field is treated as a regular expression
569 and will be applied to each job that matches.
570
James E. Blaire5a847f2012-07-10 15:29:14 -0700571**failure-message (optional)**
572 The message that should be reported to Gerrit if the job fails.
James E. Blaircdd00072012-06-08 19:17:28 -0700573
James E. Blaire5a847f2012-07-10 15:29:14 -0700574**success-message (optional)**
575 The message that should be reported to Gerrit if the job fails.
James E. Blaircdd00072012-06-08 19:17:28 -0700576
James E. Blair6aea36d2012-12-17 13:03:24 -0800577**failure-pattern (optional)**
578 The URL that should be reported to Gerrit if the job fails.
James E. Blair1f4c2bb2013-04-26 08:40:46 -0700579 Defaults to the build URL or the url_pattern configured in
James E. Blair6aea36d2012-12-17 13:03:24 -0800580 zuul.conf. May be supplied as a string pattern with substitutions
581 as described in url_pattern in :ref:`zuulconf`.
582
583**success-pattern (optional)**
584 The URL that should be reported to Gerrit if the job succeeds.
James E. Blair1f4c2bb2013-04-26 08:40:46 -0700585 Defaults to the build URL or the url_pattern configured in
James E. Blair6aea36d2012-12-17 13:03:24 -0800586 zuul.conf. May be supplied as a string pattern with substitutions
587 as described in url_pattern in :ref:`zuulconf`.
588
James E. Blair222d4982012-07-16 09:31:19 -0700589**hold-following-changes (optional)**
590 This is a boolean that indicates that changes that follow this
Clark Boylan00635dc2012-09-19 14:03:08 -0700591 change in a dependent change pipeline should wait until this job
James E. Blair222d4982012-07-16 09:31:19 -0700592 succeeds before launching. If this is applied to a very short job
593 that can predict whether longer jobs will fail early, this can be
594 used to reduce the number of jobs that Zuul will launch and
595 ultimately have to cancel. In that case, a small amount of
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400596 parallelization of jobs is traded for more efficient use of testing
James E. Blair222d4982012-07-16 09:31:19 -0700597 resources. On the other hand, to apply this to a long running job
598 would largely defeat the parallelization of dependent change testing
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400599 that is the main feature of Zuul. Default: ``false``.
James E. Blair222d4982012-07-16 09:31:19 -0700600
James E. Blaire5a847f2012-07-10 15:29:14 -0700601**branch (optional)**
James E. Blaircdd00072012-06-08 19:17:28 -0700602 This job should only be run on matching branches. This field is
603 treated as a regular expression and multiple branches may be
604 listed.
605
James E. Blair70c71582013-03-06 08:50:50 -0800606**files (optional)**
607 This job should only be run if at least one of the files involved in
608 the change (added, deleted, or modified) matches at least one of the
609 file patterns listed here. This field is treated as a regular
610 expression and multiple expressions may be listed.
611
Mathieu Gagnéd6d2a642013-06-11 20:59:58 -0400612**voting (optional)**
613 Boolean value (``true`` or ``false``) that indicates whatever
614 a job is voting or not. Default: ``true``.
615
James E. Blaire5a847f2012-07-10 15:29:14 -0700616**parameter-function (optional)**
617 Specifies a function that should be applied to the parameters before
618 the job is launched. The function should be defined in a python file
619 included with the :ref:`includes` directive. The function
620 should have the following signature:
621
James E. Blaird0470972013-07-29 10:05:43 -0700622 .. function:: parameters(item, job, parameters)
James E. Blaire5a847f2012-07-10 15:29:14 -0700623
624 Manipulate the parameters passed to a job before a build is
625 launched. The ``parameters`` dictionary will already contain the
626 standard Zuul job parameters, and is expected to be modified
627 in-place.
628
James E. Blaird78576a2013-07-09 10:39:17 -0700629 :param item: the current queue item
630 :type item: zuul.model.QueueItem
James E. Blaird0470972013-07-29 10:05:43 -0700631 :param job: the job about to be run
632 :type job: zuul.model.Job
James E. Blaire5a847f2012-07-10 15:29:14 -0700633 :param parameters: parameters to be passed to the job
634 :type parameters: dict
635
James E. Blair1f4c2bb2013-04-26 08:40:46 -0700636 If the parameter **ZUUL_NODE** is set by this function, then it will
637 be used to specify on what node (or class of node) the job should be
638 run.
639
James E. Blaircdd00072012-06-08 19:17:28 -0700640Here is an example of setting the failure message for jobs that check
641whether a change merges cleanly::
642
643 - name: ^.*-merge$
644 failure-message: This change was unable to be automatically merged
645 with the current state of the repository. Please rebase your
646 change and upload a new patchset.
647
648Projects
649""""""""
650
Clark Boylan00635dc2012-09-19 14:03:08 -0700651The projects section indicates what jobs should be run in each pipeline
James E. Blaircdd00072012-06-08 19:17:28 -0700652for events associated with each project. It contains a list of
653projects. Here is an example::
654
655 - name: example/project
656 check:
657 - project-merge:
658 - project-unittest
Clark Boylan00635dc2012-09-19 14:03:08 -0700659 - project-pep8
660 - project-pyflakes
James E. Blaircdd00072012-06-08 19:17:28 -0700661 gate:
662 - project-merge:
663 - project-unittest
Clark Boylan00635dc2012-09-19 14:03:08 -0700664 - project-pep8
665 - project-pyflakes
James E. Blaircdd00072012-06-08 19:17:28 -0700666 post:
667 - project-publish
668
669**name**
670 The name of the project (as known by Gerrit).
671
James E. Blair19deff22013-08-25 13:17:35 -0700672**merge-mode (optional)**
673 An optional value that indicates what strategy should be used to
674 merge changes to this project. Supported values are:
675
676 ** merge-resolve **
677 Equivalent to 'git merge -s resolve'. This corresponds closely to
678 what Gerrit performs (using JGit) for a project if the "Merge if
679 necessary" merge mode is selected and "Automatically resolve
680 conflicts" is checked. This is the default.
681
682 ** merge **
683 Equivalent to 'git merge'.
684
685 ** cherry-pick **
686 Equivalent to 'git cherry-pick'.
687
Clark Boylan00635dc2012-09-19 14:03:08 -0700688This is followed by a section for each of the pipelines defined above.
689Pipelines may be omitted if no jobs should run for this project in a
690given pipeline. Within the pipeline section, the jobs that should be
James E. Blaircdd00072012-06-08 19:17:28 -0700691executed are listed. If a job is entered as a dictionary key, then
692jobs contained within that key are only executed if the key job
693succeeds. In the above example, project-unittest, project-pep8, and
694project-pyflakes are only executed if project-merge succeeds. This
695can help avoid running unnecessary jobs.
696
James E. Blair18c64442014-03-18 10:14:45 -0700697The special job named ``noop`` is internal to Zuul and will always
698return ``SUCCESS`` immediately. This can be useful if you require
699that all changes be processed by a pipeline but a project has no jobs
700that can be run on it.
701
Paul Belangerffef9e42013-02-11 22:15:18 -0500702.. seealso:: The OpenStack Zuul configuration for a comprehensive example: https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/zuul/layout.yaml
James E. Blaircdd00072012-06-08 19:17:28 -0700703
Antoine Musso80edd5a2013-02-13 15:37:53 +0100704Project Templates
705"""""""""""""""""
706
Michael Prokop526926a2013-10-24 16:16:57 +0200707Whenever you have lot of similar projects (such as plugins for a project) you
Antoine Musso80edd5a2013-02-13 15:37:53 +0100708will most probably want to use the same pipeline configurations. The
709project templates let you define pipelines and job name templates to trigger.
710One can then just apply the template on its project which make it easier to
Michael Prokop526926a2013-10-24 16:16:57 +0200711update several similar projects. As an example::
Antoine Musso80edd5a2013-02-13 15:37:53 +0100712
713 project-templates:
714 # Name of the template
715 - name: plugin-triggering
716 # Definition of pipelines just like for a `project`
717 check:
718 - '{jobprefix}-merge':
719 - '{jobprefix}-pep8'
720 - '{jobprefix}-pyflakes'
721 gate:
722 - '{jobprefix}-merge':
723 - '{jobprefix}-unittest'
724 - '{jobprefix}-pep8'
725 - '{jobprefix}-pyflakes'
726
727In your projects definition, you will then apply the template using the template
728key::
729
730 projects:
731 - name: plugin/foobar
732 template:
733 - name: plugin-triggering
734 jobprefix: plugin-foobar
735
James E. Blairaea6cf62013-12-16 15:38:12 -0800736You can pass several parameters to a template. A ``parameter`` value
737will be used for expansion of ``{parameter}`` in the template
738strings. The parameter ``name`` will be automatically provided and
739will contain the short name of the project, that is the portion of the
740project name after the last ``/`` character.
James E. Blaircdd00072012-06-08 19:17:28 -0700741
James E. Blair3e98c022013-12-16 15:25:38 -0800742Multiple templates can be combined in a project, and the jobs from all
743of those templates will be added to the project. Individual jobs may
744also be added::
745
746 projects:
747 - name: plugin/foobar
748 template:
749 - name: plugin-triggering
750 jobprefix: plugin-foobar
751 - name: plugin-extras
752 jobprefix: plugin-foobar
753 check:
754 - foobar-extra-special-job
755
756The order of the jobs listed in the project (which only affects the
757order of jobs listed on the report) will be the jobs from each
758template in the order listed, followed by any jobs individually listed
759for the project.
760
761Note that if multiple templates are used for a project and one
762template specifies a job that is also specified in another template,
763or specified in the project itself, those jobs will be duplicated in
764the resulting project configuration.
765
James E. Blaircdd00072012-06-08 19:17:28 -0700766logging.conf
767~~~~~~~~~~~~
768This file is optional. If provided, it should be a standard
769:mod:`logging.config` module configuration file. If not present, Zuul will
770output all log messages of DEBUG level or higher to the console.
771
772Starting Zuul
773-------------
774
775To start Zuul, run **zuul-server**::
776
Antoine Mussob3aa8282013-04-19 15:16:59 +0200777 usage: zuul-server [-h] [-c CONFIG] [-l LAYOUT] [-d] [-t] [--version]
James E. Blaircdd00072012-06-08 19:17:28 -0700778
779 Project gating system.
780
781 optional arguments:
782 -h, --help show this help message and exit
783 -c CONFIG specify the config file
Antoine Mussob3aa8282013-04-19 15:16:59 +0200784 -l LAYOUT specify the layout file
James E. Blaircdd00072012-06-08 19:17:28 -0700785 -d do not run as a daemon
Antoine Mussob3aa8282013-04-19 15:16:59 +0200786 -t validate layout file syntax
787 --version show zuul version
James E. Blaircdd00072012-06-08 19:17:28 -0700788
789You may want to use the ``-d`` argument while you are initially setting
790up Zuul so you can detect any configuration errors quickly. Under
791normal operation, omit ``-d`` and let Zuul run as a daemon.
792
793If you send signal 1 (SIGHUP) to the zuul-server process, Zuul will
794stop executing new jobs, wait until all executing jobs are finished,
795reload its configuration, and resume. Any values in any of the
796configuration files may be changed, except the location of Zuul's PID
797file (a change to that will be ignored until Zuul is restarted).
Clark Boylanf231fa22013-02-08 12:28:53 -0800798
799If you send a SIGUSR1 to the zuul-server process, Zuul will stop
800executing new jobs, wait until all executing jobs are finished,
801then exit. While waiting to exit Zuul will queue Gerrit events and
802save these events prior to exiting. When Zuul starts again it will
803read these saved events and act on them.
Jeremy Stanley93e05f42013-03-08 17:29:17 +0000804
Michael Prokop526926a2013-10-24 16:16:57 +0200805If you need to abort Zuul and intend to manually requeue changes for
Jeremy Stanley93e05f42013-03-08 17:29:17 +0000806jobs which were running in its pipelines, prior to terminating you can
807use the zuul-changes.py tool script to simplify the process. For
808example, this would give you a list of Gerrit commands to reverify or
809recheck changes for the gate and check pipelines respectively::
810
811 ./tools/zuul-changes.py --review-host=review.openstack.org \
812 http://zuul.openstack.org/ gate 'reverify no bug'
813 ./tools/zuul-changes.py --review-host=review.openstack.org \
814 http://zuul.openstack.org/ check 'recheck no bug'
Clark Boylanfba9b242013-08-20 10:11:17 -0700815
816If you send a SIGUSR2 to the zuul-server process, Zuul will dump a stack
817trace for each running thread into its debug log. This is useful for
818tracking down deadlock or otherwise slow threads.