blob: 70edf76a6fd7ee3cff0e0f9bdc7ab5fa5673f3a7 [file] [log] [blame]
James E. Blair1de8d402017-05-07 17:08:04 -07001:title: Project Configuration
2
3.. _project-config:
4
5Project Configuration
6=====================
7
8The following sections describe the main part of Zuul's configuration.
9All of what follows is found within files inside of the repositories
10that Zuul manages.
11
12Security Contexts
13-----------------
14
15When a system administrator configures Zuul to operate on a project,
16they specify one of two security contexts for that project. A
17*config-project* is one which is primarily tasked with holding
18configuration information and job content for Zuul. Jobs which are
19defined in a *config-project* are run with elevated privileges, and
20all Zuul configuration items are available for use. It is expected
21that changes to *config-projects* will undergo careful scrutiny before
22being merged.
23
24An *untrusted-project* is a project whose primary focus is not to
25operate Zuul, but rather it is one of the projects being tested or
26deployed. The Zuul configuration language available to these projects
27is somewhat restricted (as detailed in individual section below), and
28jobs defined in these projects run in a restricted execution
29environment since they may be operating on changes which have not yet
30undergone review.
31
32Configuration Loading
33---------------------
34
35When Zuul starts, it examines all of the git repositories which are
James E. Blaireff5a9d2017-06-20 00:00:37 -070036specified by the system administrator in :ref:`tenant-config` and searches
James E. Blair1de8d402017-05-07 17:08:04 -070037for files in the root of each repository. In the case of a
38*config-project*, Zuul looks for a file named `zuul.yaml`. In the
39case of an *untrusted-project*, Zuul looks first for `zuul.yaml` and
40if that is not found, `.zuul.yaml` (with a leading dot). In the case
41of an *untrusted-project*, the configuration from every branch is
42included, however, in the case of a *config-project*, only the
43`master` branch is examined.
44
45When a change is proposed to one of these files in an
46*untrusted-project*, the configuration proposed in the change is
47merged into the running configuration so that any changes to Zuul's
48configuration are self-testing as part of that change. If there is a
49configuration error, no jobs will be run and the error will be
50reported by any applicable pipelines. In the case of a change to a
51*config-project*, the new configuration is parsed and examined for
52errors, but the new configuration is not used in testing the change.
53This is because configuration in *config-projects* is able to access
54elevated privileges and should always be reviewed before being merged.
55
56As soon as a change containing a Zuul configuration change merges to
57any Zuul-managed repository, the new configuration takes effect
58immediately.
59
60Configuration Items
61-------------------
62
63The `zuul.yaml` and `.zuul.yaml` configuration files are
64YAML-formatted and are structured as a series of items, each of which
65is described below.
66
67.. _pipeline:
68
69Pipeline
70~~~~~~~~
71
72A pipeline describes a workflow operation in Zuul. It associates jobs
73for a given project with triggering and reporting events.
74
75Its flexible configuration allows for characterizing any number of
76workflows, and by specifying each as a named configuration, makes it
77easy to apply similar workflow operations to projects or groups of
78projects.
79
80By way of example, one of the primary uses of Zuul is to perform
81project gating. To do so, one can create a *gate* pipeline which
82tells Zuul that when a certain event (such as approval by a code
83reviewer) occurs, the corresponding change or pull request should be
84enqueued into the pipeline. When that happens, the jobs which have
85been configured to run for that project in the *gate* pipeline are
86run, and when they complete, the pipeline reports the results to the
87user.
88
89Pipeline configuration items may only appear in *config-projects*.
90
91Generally, a Zuul administrator would define a small number of
92pipelines which represent the workflow processes used in their
93environment. Each project can then be added to the available
94pipelines as appropriate.
95
96Here is an example *check* pipeline, which runs whenever a new
97patchset is created in Gerrit. If the associated jobs all report
98success, the pipeline reports back to Gerrit with a *Verified* vote of
99+1, or if at least one of them fails, a -1::
100
101 - pipeline:
James E. Blaira8387ec2017-07-05 14:00:12 -0700102 name: check
James E. Blair1de8d402017-05-07 17:08:04 -0700103 manager: independent
104 trigger:
105 my_gerrit:
106 - event: patchset-created
107 success:
108 my_gerrit:
109 verified: 1
110 failure:
111 my_gerrit
112 verified: -1
113
James E. Blaireff5a9d2017-06-20 00:00:37 -0700114.. TODO: See TODO for more annotated examples of common pipeline configurations.
James E. Blair1de8d402017-05-07 17:08:04 -0700115
116The attributes available on a pipeline are as follows (all are
117optional unless otherwise specified):
118
119**name** (required)
120 This is used later in the project definition to indicate what jobs
121 should be run for events in the pipeline.
122
123**manager** (required)
124 There are currently two schemes for managing pipelines:
125
James E. Blaireff5a9d2017-06-20 00:00:37 -0700126 .. _independent_pipeline_manager:
127
James E. Blair1de8d402017-05-07 17:08:04 -0700128 *independent*
129 Every event in this pipeline should be treated as independent of
130 other events in the pipeline. This is appropriate when the order of
131 events in the pipeline doesn't matter because the results of the
132 actions this pipeline performs can not affect other events in the
133 pipeline. For example, when a change is first uploaded for review,
134 you may want to run tests on that change to provide early feedback
135 to reviewers. At the end of the tests, the change is not going to
136 be merged, so it is safe to run these tests in parallel without
137 regard to any other changes in the pipeline. They are independent.
138
139 Another type of pipeline that is independent is a post-merge
140 pipeline. In that case, the changes have already merged, so the
141 results can not affect any other events in the pipeline.
142
James E. Blaireff5a9d2017-06-20 00:00:37 -0700143 .. _dependent_pipeline_manager:
144
James E. Blair1de8d402017-05-07 17:08:04 -0700145 *dependent*
146 The dependent pipeline manager is designed for gating. It ensures
147 that every change is tested exactly as it is going to be merged
148 into the repository. An ideal gating system would test one change
149 at a time, applied to the tip of the repository, and only if that
150 change passed tests would it be merged. Then the next change in
151 line would be tested the same way. In order to achieve parallel
152 testing of changes, the dependent pipeline manager performs
153 speculative execution on changes. It orders changes based on
154 their entry into the pipeline. It begins testing all changes in
155 parallel, assuming that each change ahead in the pipeline will pass
156 its tests. If they all succeed, all the changes can be tested and
157 merged in parallel. If a change near the front of the pipeline
158 fails its tests, each change behind it ignores whatever tests have
159 been completed and are tested again without the change in front.
160 This way gate tests may run in parallel but still be tested
161 correctly, exactly as they will appear in the repository when
162 merged.
163
164 For more detail on the theory and operation of Zuul's dependent
165 pipeline manager, see: :doc:`gating`.
166
James E. Blairf17aa9c2017-07-05 13:21:23 -0700167**allow-secrets**
168 This is a boolean which can be used to prevent jobs which require
169 secrets from running in this pipeline. Some pipelines run on
170 proposed changes and therefore execute code which has not yet been
171 reviewed. In such a case, allowing a job to use a secret could
172 result in that secret being exposed. The default is False, meaning
173 that in order to run jobs with secrets, this must be explicitly
174 enabled on each Pipeline where that is safe.
175
176 For more information, see :ref:`secret`.
177
James E. Blair1de8d402017-05-07 17:08:04 -0700178**description**
179 This field may be used to provide a textual description of the
180 pipeline. It may appear in the status page or in documentation.
181
182**success-message**
183 The introductory text in reports when all the voting jobs are
184 successful. Defaults to "Build successful."
185
186**failure-message**
187 The introductory text in reports when at least one voting job fails.
188 Defaults to "Build failed."
189
190**merge-failure-message**
191 The introductory text in the message reported when a change fails to
192 merge with the current state of the repository. Defaults to "Merge
193 failed."
194
195**footer-message**
196 Supplies additional information after test results. Useful for
197 adding information about the CI system such as debugging and contact
198 details.
199
200**trigger**
201 At least one trigger source must be supplied for each pipeline.
202 Triggers are not exclusive -- matching events may be placed in
203 multiple pipelines, and they will behave independently in each of
204 the pipelines they match.
205
206 Triggers are loaded from their connection name. The driver type of
207 the connection will dictate which options are available.
James E. Blaireff5a9d2017-06-20 00:00:37 -0700208 See :ref:`drivers`.
James E. Blair1de8d402017-05-07 17:08:04 -0700209
210**require**
211 If this section is present, it established pre-requisites for any
212 kind of item entering the Pipeline. Regardless of how the item is
213 to be enqueued (via any trigger or automatic dependency resolution),
214 the conditions specified here must be met or the item will not be
215 enqueued.
216
James E. Blaireff5a9d2017-06-20 00:00:37 -0700217.. _pipeline-require-approval:
James E. Blair1de8d402017-05-07 17:08:04 -0700218
219 **approval**
220 This requires that a certain kind of approval be present for the
221 current patchset of the change (the approval could be added by the
222 event in question). It takes several sub-parameters, all of which
223 are optional and are combined together so that there must be an
224 approval matching all specified requirements.
225
226 *username*
227 If present, an approval from this username is required. It is
228 treated as a regular expression.
229
230 *email*
231 If present, an approval with this email address is required. It
232 is treated as a regular expression.
233
234 *email-filter* (deprecated)
235 A deprecated alternate spelling of *email*. Only one of *email* or
236 *email_filter* should be used.
237
238 *older-than*
239 If present, the approval must be older than this amount of time
240 to match. Provide a time interval as a number with a suffix of
241 "w" (weeks), "d" (days), "h" (hours), "m" (minutes), "s"
242 (seconds). Example ``48h`` or ``2d``.
243
244 *newer-than*
245 If present, the approval must be newer than this amount of time
246 to match. Same format as "older-than".
247
248 Any other field is interpreted as a review category and value
249 pair. For example ``verified: 1`` would require that the approval
250 be for a +1 vote in the "Verified" column. The value may either
251 be a single value or a list: ``verified: [1, 2]`` would match
252 either a +1 or +2 vote.
253
254 **open**
255 A boolean value (``true`` or ``false``) that indicates whether the change
256 must be open or closed in order to be enqueued.
257
258 **current-patchset**
259 A boolean value (``true`` or ``false``) that indicates whether the change
260 must be the current patchset in order to be enqueued.
261
262 **status**
263 A string value that corresponds with the status of the change
264 reported by the trigger.
265
266**reject**
267 If this section is present, it establishes pre-requisites that can
268 block an item from being enqueued. It can be considered a negative
269 version of **require**.
270
271 **approval**
272 This takes a list of approvals. If an approval matches the provided
273 criteria the change can not be entered into the pipeline. It follows
274 the same syntax as the :ref:`"require approval" pipeline above
275 <pipeline-require-approval>`.
276
277 Example to reject a change with any negative vote::
278
279 reject:
280 approval:
281 - code-review: [-1, -2]
282
283**dequeue-on-new-patchset**
284 Normally, if a new patchset is uploaded to a change that is in a
285 pipeline, the existing entry in the pipeline will be removed (with
286 jobs canceled and any dependent changes that can no longer merge as
287 well. To suppress this behavior (and allow jobs to continue
288 running), set this to ``false``. Default: ``true``.
289
290**ignore-dependencies**
291 In any kind of pipeline (dependent or independent), Zuul will
292 attempt to enqueue all dependencies ahead of the current change so
293 that they are tested together (independent pipelines report the
294 results of each change regardless of the results of changes ahead).
295 To ignore dependencies completely in an independent pipeline, set
296 this to ``true``. This option is ignored by dependent pipelines.
297 The default is: ``false``.
298
James E. Blaireff5a9d2017-06-20 00:00:37 -0700299**precedence**
300 Indicates how the build scheduler should prioritize jobs for
301 different pipelines. Each pipeline may have one precedence, jobs
302 for pipelines with a higher precedence will be run before ones with
303 lower. The value should be one of ``high``, ``normal``, or ``low``.
304 Default: ``normal``.
305
James E. Blair1de8d402017-05-07 17:08:04 -0700306The following options configure *reporters*. Reporters are
307complementary to triggers; where a trigger is an event on a connection
308which causes Zuul to enqueue an item, a reporter is the action
309performed on a connection when an item is dequeued after its jobs
310complete. The actual syntax for a reporter is defined by the driver
311which implements it. See :ref:`drivers` for more information.
312
313**success**
314 Describes where Zuul should report to if all the jobs complete
315 successfully. This section is optional; if it is omitted, Zuul will
316 run jobs and do nothing on success -- it will not report at all. If
317 the section is present, the listed reporters will be asked to report
318 on the jobs. The reporters are listed by their connection name. The
319 options available depend on the driver for the supplied connection.
320
321**failure**
322 These reporters describe what Zuul should do if at least one job
323 fails.
324
325**merge-failure**
326 These reporters describe what Zuul should do if it is unable to
327 merge in the patchset. If no merge-failure reporters are listed then
328 the ``failure`` reporters will be used to notify of unsuccessful
329 merges.
330
331**start**
332 These reporters describe what Zuul should do when a change is added
333 to the pipeline. This can be used, for example, to reset a
334 previously reported result.
335
336**disabled**
337 These reporters describe what Zuul should do when a pipeline is
338 disabled. See ``disable-after-consecutive-failures``.
339
James E. Blaireff5a9d2017-06-20 00:00:37 -0700340The following options can be used to alter Zuul's behavior to mitigate
341situations in which jobs are failing frequently (perhaps due to a
342problem with an external dependency, or unusually high
343non-deterministic test failures).
344
James E. Blair1de8d402017-05-07 17:08:04 -0700345**disable-after-consecutive-failures**
346 If set, a pipeline can enter a ''disabled'' state if too many changes
347 in a row fail. When this value is exceeded the pipeline will stop
348 reporting to any of the ``success``, ``failure`` or ``merge-failure``
349 reporters and instead only report to the ``disabled`` reporters.
350 (No ``start`` reports are made when a pipeline is disabled).
351
James E. Blair1de8d402017-05-07 17:08:04 -0700352**window**
353 Dependent pipeline managers only. Zuul can rate limit dependent
354 pipelines in a manner similar to TCP flow control. Jobs are only
355 started for items in the queue if they are within the actionable
356 window for the pipeline. The initial length of this window is
357 configurable with this value. The value given should be a positive
358 integer value. A value of ``0`` disables rate limiting on the
359 DependentPipelineManager. Default: ``20``.
360
361**window-floor**
362 Dependent pipeline managers only. This is the minimum value for the
363 window described above. Should be a positive non zero integer value.
364 Default: ``3``.
365
366**window-increase-type**
367 Dependent pipeline managers only. This value describes how the window
368 should grow when changes are successfully merged by zuul. A value of
369 ``linear`` indicates that ``window-increase-factor`` should be added
370 to the previous window value. A value of ``exponential`` indicates
371 that ``window-increase-factor`` should be multiplied against the
372 previous window value and the result will become the window size.
373 Default: ``linear``.
374
375**window-increase-factor**
376 Dependent pipeline managers only. The value to be added or multiplied
377 against the previous window value to determine the new window after
378 successful change merges.
379 Default: ``1``.
380
381**window-decrease-type**
382 Dependent pipeline managers only. This value describes how the window
383 should shrink when changes are not able to be merged by Zuul. A value
384 of ``linear`` indicates that ``window-decrease-factor`` should be
385 subtracted from the previous window value. A value of ``exponential``
386 indicates that ``window-decrease-factor`` should be divided against
387 the previous window value and the result will become the window size.
388 Default: ``exponential``.
389
390**window-decrease-factor**
391 Dependent pipline managers only. The value to be subtracted or divided
392 against the previous window value to determine the new window after
393 unsuccessful change merges.
394 Default: ``2``.
395
396
397.. _job:
398
399Job
400~~~
401
402A job is a unit of work performed by Zuul on an item enqueued into a
403pipeline. Items may run any number of jobs (which may depend on each
404other). Each job is an invocation of an Ansible playbook with a
405specific inventory of hosts. The actual tasks that are run by the job
406appear in the playbook for that job while the attributes that appear in the
407Zuul configuration specify information about when, where, and how the
408job should be run.
409
410Jobs in Zuul support inheritance. Any job may specify a single parent
411job, and any attributes not set on the child job are collected from
412the parent job. In this way, a configuration structure may be built
413starting with very basic jobs which describe characteristics that all
414jobs on the system should have, progressing through stages of
415specialization before arriving at a particular job. A job may inherit
416from any other job in any project (however, if the other job is marked
417as `final`, some attributes may not be overidden).
418
419Jobs also support a concept called variance. The first time a job
420definition appears is called the reference definition of the job.
421Subsequent job definitions with the same name are called variants.
422These may have different selection criteria which indicate to Zuul
423that, for instance, the job should behave differently on a different
424git branch. Unlike inheritance, all job variants must be defined in
425the same project.
426
427When Zuul decides to run a job, it performs a process known as
428freezing the job. Because any number of job variants may be
429applicable, Zuul collects all of the matching variants and applies
430them in the order they appeared in the configuration. The resulting
431frozen job is built from attributes gathered from all of the
432matching variants. In this way, exactly what is run is dependent on
433the pipeline, project, branch, and content of the item.
434
435In addition to the job's main playbook, each job may specify one or
436more pre- and post-playbooks. These are run, in order, before and
437after (respectively) the main playbook. They may be used to set up
438and tear down resources needed by the main playbook. When combined
439with inheritance, they provide powerful tools for job construction. A
440job only has a single main playbook, and when inheriting from a
441parent, the child's main playbook overrides (or replaces) the
442parent's. However, the pre- and post-playbooks are appended and
443prepended in a nesting fashion. So if a parent job and child job both
444specified pre and post playbooks, the sequence of playbooks run would
445be:
446
447* parent pre-run playbook
448* child pre-run playbook
449* child playbook
450* child post-run playbook
451* parent post-run playbook
452
453Further inheritance would nest even deeper.
454
455Here is an example of two job definitions::
456
457 - job:
458 name: base
459 pre-run: copy-git-repos
460 post-run: copy-logs
461
462 - job:
463 name: run-tests
464 parent: base
465 nodes:
466 - name: test-node
467 image: fedora
468
469The following attributes are available on a job; all are optional
470unless otherwise specified:
471
472**name** (required)
473 The name of the job. By default, Zuul looks for a playbook with
474 this name to use as the main playbook for the job. This name is
475 also referenced later in a project pipeline configuration.
476
477**parent**
478 Specifies a job to inherit from. The parent job can be defined in
479 this or any other project. Any attributes not specified on a job
480 will be collected from its parent.
481
482**description**
483 A textual description of the job. Not currently used directly by
484 Zuul, but it is used by the zuul-sphinx extension to Sphinx to
485 auto-document Zuul jobs (in which case it is interpreted as
486 ReStructuredText.
487
488**success-message**
489 Normally when a job succeeds, the string "SUCCESS" is reported as
490 the result for the job. If set, this option may be used to supply a
491 different string. Default: "SUCCESS".
492
493**failure-message**
494 Normally when a job fails, the string "FAILURE" is reported as
495 the result for the job. If set, this option may be used to supply a
496 different string. Default: "FAILURE".
497
498**success-url**
James E. Blair88e79c02017-07-07 13:36:54 -0700499 When a job succeeds, this URL is reported along with the result. If
500 this value is not supplied, Zuul uses the content of the job
501 :ref:`return value <return_values>` **zuul.log_url**. This is
502 recommended as it allows the code which stores the URL to the job
503 artifacts to report exactly where they were stored. To override
504 this value, or if it is not set, supply an absolute URL in this
505 field. If a relative URL is supplied in this field, and
506 **zuul.log_url** is set, then the two will be combined to produce
507 the URL used for the report. This can be used to specify that
508 certain jobs should "deep link" into the stored job artifacts.
James E. Blair1de8d402017-05-07 17:08:04 -0700509 Default: none.
510
511**failure-url**
512 When a job fails, this URL is reported along with the result.
James E. Blair88e79c02017-07-07 13:36:54 -0700513 Otherwise behaves the same as **success-url**.
James E. Blair1de8d402017-05-07 17:08:04 -0700514
515**hold-following-changes**
516 In a dependent pipeline, this option may be used to indicate that no
517 jobs should start on any items which depend on the current item
518 until this job has completed successfully. This may be used to
519 conserve build resources, at the expense of inhibiting the
520 parallelization which speeds the processing of items in a dependent
521 pipeline. A boolean value, default: false.
522
523**voting**
524 Indicates whether the result of this job should be used in
525 determining the overall result of the item. A boolean value,
526 default: true.
527
528**semaphore**
529 The name of a :ref:`semaphore` which should be acquired and released
530 when the job begins and ends. If the semaphore is at maximum
531 capacity, then Zuul will wait until it can be acquired before
532 starting the job. Default: none.
533
534**tags**
535 Metadata about this job. Tags are units of information attached to
536 the job; they do not affect Zuul's behavior, but they can be used
537 within the job to characterize the job. For example, a job which
538 tests a certain subsystem could be tagged with the name of that
539 subsystem, and if the job's results are reported into a database,
540 then the results of all jobs affecting that subsystem could be
541 queried. This attribute is specified as a list of strings, and when
542 inheriting jobs or applying variants, tags accumulate in a set, so
543 the result is always a set of all the tags from all the jobs and
544 variants used in constructing the frozen job, with no duplication.
545 Default: none.
546
James E. Blaireff5a9d2017-06-20 00:00:37 -0700547**branches**
James E. Blair1de8d402017-05-07 17:08:04 -0700548 A regular expression (or list of regular expressions) which describe
549 on what branches a job should run (or in the case of variants: to
550 alter the behavior of a job for a certain branch).
551
552 If there is no job definition for a given job which matches the
553 branch of an item, then that job is not run for the item.
554 Otherwise, all of the job variants which match that branch (and any
555 other selection criteria) are used when freezing the job.
556
557 This example illustrates a job called *run-tests* which uses a
558 nodeset based on the current release of an operating system to
559 perform its tests, except when testing changes to the stable/2.0
560 branch, in which case it uses an older release::
561
562 - job:
563 name: run-tests
564 nodes: current-release
565
566 - job:
567 name: run-tests
568 branch: stable/2.0
569 nodes: old-release
570
571 In some cases, Zuul uses an implied value for the branch specifier
572 if none is supplied:
573
574 * For a job definition in a *config-project*, no implied branch
575 specifier is used. If no branch specifier appears, the job
576 applies to all branches.
577
578 * In the case of an *untrusted-project*, no implied branch specifier
579 is applied to the reference definition of a job. That is to say,
580 that if the first appearance of the job definition appears without
581 a branch specifier, then it will apply to all branches. Note that
582 when collecting its configuration, Zuul reads the `master` branch
583 of a given project first, then other branches in alphabetical
584 order.
585
586 * Any further job variants other than the reference definition in an
587 *untrusted-project* will, if they do not have a branch specifier,
588 will have an implied branch specifier for the current branch
589 applied.
590
591 This allows for the very simple and expected workflow where if a
592 project defines a job on the master branch with no branch specifier,
593 and then creates a new branch based on master, any changes to that
594 job definition within the new branch only affect that branch.
595
596**files**
597 This attribute indicates that the job should only run on changes
598 where the specified files are modified. This is a regular
599 expression or list of regular expressions. Default: none.
600
601**irrelevant-files**
602 This is a negative complement of `files`. It indicates that the job
603 should run unless *all* of the files changed match this list. In
604 other words, if the regular expression `docs/.*` is supplied, then
605 this job will not run if the only files changed are in the docs
606 directory. A regular expression or list of regular expressions.
607 Default: none.
608
609**auth**
610 Authentication information to be made available to the job. This is
611 a dictionary with two potential keys:
612
613 **inherit**
614 A boolean indicating that the authentication information referenced
615 by this job should be able to be inherited by child jobs. Normally
616 when a job inherits from another job, the auth section is not
617 included. This permits jobs to inherit the same basic structure and
618 playbook, but ensures that secret information is unable to be
619 exposed by a child job which may alter the job's behavior. If it is
620 safe for the contents of the authentication section to be used by
621 child jobs, set this to ``true``. Default: ``false``.
622
623 **secrets**
624 A list of secrets which may be used by the job. A :ref:`secret` is
625 a named collection of private information defined separately in the
626 configuration. The secrets that appear here must be defined in the
627 same project as this job definition.
628
629 In the future, other types of authentication information may be
630 added.
631
632**nodes**
633 A list of nodes which should be supplied to the job. This parameter
634 may be supplied either as a string, in which case it references a
635 :ref:`nodeset` definition which appears elsewhere in the
636 configuration, or a list, in which case it is interpreted in the
637 same way as a Nodeset definition (in essence, it is an anonymous
638 Node definition unique to this job). See the :ref:`nodeset`
639 reference for the syntax to use in that case.
640
641 If a job has an empty or no node definition, it will still run and
642 may be able to perform actions on the Zuul executor.
643
644**override-branch**
645 When Zuul runs jobs for a proposed change, it normally checks out
646 the branch associated with that change on every project present in
647 the job. If jobs are running on a ref (such as a branch tip or
648 tag), then that ref is normally checked out. This attribute is used
649 to override that behavior and indicate that this job should,
650 regardless of the branch for the queue item, use the indicated
651 branch instead. This can be used, for example, to run a previous
652 version of the software (from a stable maintenance branch) under
653 test even if the change being tested applies to a different branch
654 (this is only likely to be useful if there is some cross-branch
655 interaction with some component of the system being tested). See
656 also the project-specific **override-branch** attribute under
657 **required-projects** to apply this behavior to a subset of a job's
658 projects.
659
660**timeout**
661 The time in minutes that the job should be allowed to run before it
662 is automatically aborted and failure is reported. If no timeout is
663 supplied, the job may run indefinitely. Supplying a timeout is
664 highly recommended.
665
666**attempts**
667 When Zuul encounters an error running a job's pre-run playbook, Zuul
668 will stop and restart the job. Errors during the main or
669 post-run -playbook phase of a job are not affected by this parameter
670 (they are reported immediately). This parameter controls the number
671 of attempts to make before an error is reported. Default: 3.
672
673**pre-run**
674 The name of a playbook or list of playbooks to run before the main
675 body of a job. The playbook is expected to reside in the
676 `playbooks/` directory of the project where the job is defined.
677
678 When a job inherits from a parent, the child's pre-run playbooks are
679 run after the parent's. See :ref:`job` for more information.
680
681**post-run**
682 The name of a playbook or list of playbooks to run after the main
683 body of a job. The playbook is expected to reside in the
684 `playbooks/` directory of the project where the job is defined.
685
686 When a job inherits from a parent, the child's post-run playbooks
687 are run before the parent's. See :ref:`job` for more information.
688
689**run**
690 The name of the main playbook for this job. This parameter is not
691 normally necessary, as it defaults to the name of the job. However,
692 if a playbook with a different name is needed, it can be specified
693 here. The playbook is expected to reside in the `playbooks/`
694 directory of the project where the job is defined. When a child
695 inherits from a parent, a playbook with the name of the child job is
696 implicitly searched first, before falling back on the playbook used
697 by the parent job (unless the child job specifies a ``run``
698 attribute, in which case that value is used). Default: the name of
699 the job.
700
701**roles**
702 A list of Ansible roles to prepare for the job. Because a job runs
703 an Ansible playbook, any roles which are used by the job must be
704 prepared and installed by Zuul before the job begins. This value is
705 a list of dictionaries, each of which indicates one of two types of
706 roles: a Galaxy role, which is simply a role that is installed from
707 Ansible Galaxy, or a Zuul role, which is a role provided by a
708 project managed by Zuul. Zuul roles are able to benefit from
709 speculative merging and cross-project dependencies when used by jobs
710 in untrusted projects.
711
712 A project which supplies a role may be structured in one of two
713 configurations: a bare role (in which the role exists at the root of
714 the project), or a contained role (in which the role exists within
715 the `roles/` directory of the project, perhaps along with other
716 roles). In the case of a contained role, the `roles/` directory of
717 the project is added to the role search path. In the case of a bare
718 role, the project itself is added to the role search path. In case
719 the name of the project is not the name under which the role should
720 be installed (and therefore referenced from Ansible), the `name`
721 attribute may be used to specify an alternate.
722
James E. Blaireff5a9d2017-06-20 00:00:37 -0700723 .. note:: galaxy roles are not yet implemented
724
James E. Blair1de8d402017-05-07 17:08:04 -0700725 **galaxy**
726 The name of the role in Ansible Galaxy. If this attribute is
727 supplied, Zuul will search Ansible Galaxy for a role by this name
728 and install it. Mutually exclusive with ``zuul``; either
729 ``galaxy`` or ``zuul`` must be supplied.
730
731 **zuul**
732 The name of a Zuul project which supplies the role. Mutually
733 exclusive with ``galaxy``; either ``galaxy`` or ``zuul`` must be
734 supplied.
735
736 **name**
737 The installation name of the role. In the case of a bare role,
738 the role will be made available under this name. Ignored in the
739 case of a contained role.
740
741**required-projects**
742 A list of other projects which are used by this job. Any Zuul
743 projects specified here will also be checked out by Zuul into the
744 working directory for the job. Speculative merging and cross-repo
745 dependencies will be honored.
746
747 The format for this attribute is either a list of strings or
748 dictionaries. Strings are interpreted as project names,
749 dictionaries may have the following attributes:
750
751 **name**
752 The name of the required project.
753
754 **override-branch**
755 When Zuul runs jobs for a proposed change, it normally checks out
756 the branch associated with that change on every project present in
757 the job. If jobs are running on a ref (such as a branch tip or
758 tag), then that ref is normally checked out. This attribute is
759 used to override that behavior and indicate that this job should,
760 regardless of the branch for the queue item, use the indicated
761 branch instead, for only this project. See also the
762 **override-branch** attribute of jobs to apply the same behavior
763 to all projects in a job.
764
765**vars**
766
767A dictionary of variables to supply to Ansible. When inheriting from
768a job (or creating a variant of a job) vars are merged with previous
769definitions. This means a variable definition with the same name will
770override a previously defined variable, but new variable names will be
771added to the set of defined variables.
772
773**dependencies**
774 A list of other jobs upon which this job depends. Zuul will not
775 start executing this job until all of its dependencies have
776 completed successfully, and if one or more of them fail, this job
777 will not be run.
778
779**allowed-projects**
780 A list of Zuul projects which may use this job. By default, a job
781 may be used by any other project known to Zuul, however, some jobs
782 use resources or perform actions which are not appropriate for other
783 projects. In these cases, a list of projects which are allowed to
784 use this job may be supplied. If this list is not empty, then it
785 must be an exhaustive list of all projects permitted to use the job.
786 The current project (where the job is defined) is not automatically
787 included, so if it should be able to run this job, then it must be
788 explicitly listed. Default: the empty list (all projects may use
789 the job).
790
791
792.. _project:
793
794Project
795~~~~~~~
796
797A project corresponds to a source code repository with which Zuul is
798configured to interact. The main responsibility of the `Project`
799configuration item is to specify which jobs should run in which
800pipelines for a given project. Within each `Project` definition, a
801section for each `Pipeline` may appear. This project-pipeline
802definition is what determines how a project participates in a
803pipeline.
804
805Consider the following `Project` definition::
806
807 - project:
808 name: yoyodyne
809 check:
810 jobs:
811 - check-syntax
812 - unit-tests
813 gate:
814 queue: integrated
815 jobs:
816 - unit-tests
817 - integration-tests
818
819The project has two project-pipeline stanzas, one for the `check`
820pipeline, and one for `gate`. Each specifies which jobs shuld run
821when a change for that project enteres the respective pipeline -- when
822a change enters `check`, the `check-syntax` and `unit-test` jobs are
823run.
824
825Pipelines which use the dependent pipeline manager (e.g., the `gate`
826example shown earlier) maintain separate queues for groups of
827projects. When Zuul serializes a set of changes which represent
828future potential project states, it must know about all of the
829projects within Zuul which may have an effect on the outcome of the
830jobs it runs. If project *A* uses project *B* as a library, then Zuul
831must be told about that relationship so that it knows to serialize
832changes to A and B together, so that it does not merge a change to B
833while it is testing a change to A.
834
835Zuul could simply assume that all projects are related, or even infer
836relationships by which projects a job indicates it uses, however, in a
837large system that would become unwieldy very quickly, and
838unnecessarily delay changes to unrelated projects. To allow for
839flexibility in the construction of groups of related projects, the
840change queues used by dependent pipeline managers are specified
841manually. To group two or more related projects into a shared queue
842for a dependent pipeline, set the ``queue`` parameter to the same
843value for those projects.
844
845The `gate` project-pipeline definition above specifies that this
846project participates in the `integrated` shared queue for that
847pipeline.
848
849In addition to a project-pipeline definition for one or more
850`Pipelines`, the following attributes may appear in a Project:
851
852**name** (required)
853 The name of the project. If Zuul is configured with two or more
854 unique projects with the same name, the canonical hostname for the
855 project should be included (e.g., `git.example.com/foo`).
856
857**templates**
858 A list of :ref:`project-template` references; the project-pipeline
859 definitions of each Project Template will be applied to this
860 project. If more than one template includes jobs for a given
861 pipeline, they will be combined, as will any jobs specified in
862 project-pipeline definitions on the project itself.
863
864.. _project-template:
865
866Project Template
867~~~~~~~~~~~~~~~~
868
869A Project Template defines one or more project-pipeline definitions
870which can be re-used by multiple projects.
871
872A Project Template uses the same syntax as a :ref:`project`
873definition, however, in the case of a template, the ``name`` attribute
874does not refer to the name of a project, but rather names the template
875so that it can be referenced in a `Project` definition.
876
877.. _secret:
878
879Secret
880~~~~~~
881
882A Secret is a collection of private data for use by one or more jobs.
883In order to maintain the security of the data, the values are usually
884encrypted, however, data which are not sensitive may be provided
885unencrypted as well for convenience.
886
887A Secret may only be used by jobs defined within the same project. To
888use a secret, a :ref:`job` must specify the secret within its `auth`
889section. To protect against jobs in other repositories declaring a
890job with a secret as a parent and then exposing that secret, jobs
891which inherit from a job with secrets will not inherit the secrets
892themselves. To alter that behavior, see the `inherit` job attribute.
893Further, jobs which do not permit children to inherit secrets (the
894default) are also automatically marked `final`, meaning that their
895execution related attributes may not be changed in a project-pipeline
896stanza. This is to protect against a job with secrets defined in one
897project being used by another project in a way which might expose the
898secrets. If a job with secrets is unsafe to be used by other
899projects, the `allowed-projects` job attribute can be used to restrict
900the projects which can invoke that job. Finally, pipelines which are
901used to execute proposed but unreviewed changes can set the
902`allow-secrets` attribute to indicate that they should not supply
903secrets at all in order to protect against someone proposing a change
904which exposes a secret.
905
906The following attributes are required:
907
908**name** (required)
909 The name of the secret, used in a :ref:`Job` definition to request
910 the secret.
911
912**data** (required)
913 A dictionary which will be added to the Ansible variables available
914 to the job. The values can either be plain text strings, or
915 encrypted values. See :ref:`encryption` for more information.
916
917.. _nodeset:
918
919Nodeset
920~~~~~~~
921
922A Nodeset is a named collection of nodes for use by a job. Jobs may
923specify what nodes they require individually, however, by defining
924groups of node types once and referring to them by name, job
925configuration may be simplified.
926
927A Nodeset requires two attributes:
928
929**name** (required)
930 The name of the Nodeset, to be referenced by a :ref:`job`.
931
932**nodes** (required)
933 A list of node definitions, each of which has the following format:
934
935 **name** (required)
936 The name of the node. This will appear in the Ansible inventory
937 for the job.
938
939 **label** (required)
940 The Nodepool label for the node. Zuul will request a node with
941 this label.
942
943.. _semaphore:
944
945Semaphore
946~~~~~~~~~
947
948Semaphores can be used to restrict the number of certain jobs which
949are running at the same time. This may be useful for jobs which
950access shared or limited resources. A semaphore has a value which
951represents the maximum number of jobs which use that semaphore at the
952same time.
953
954Semaphores are never subject to dynamic reconfiguration. If the value
955of a semaphore is changed, it will take effect only when the change
956where it is updated is merged. An example follows::
957
958 - semaphore:
959 name: semaphore-foo
960 max: 5
961 - semaphore:
962 name: semaphore-bar
963 max: 3
964
965The following attributes are available:
966
967**name** (required)
968 The name of the semaphore, referenced by jobs.
969
970**max**
971 The maximum number of running jobs which can use this semaphore.
972 Defaults to 1.