blob: c1379183e0a3546bef2bc6ff37f9a32a51d0600f [file] [log] [blame]
James E. Blair1de8d402017-05-07 17:08:04 -07001:title: Project Configuration
2
3.. _project-config:
4
5Project Configuration
6=====================
7
8The following sections describe the main part of Zuul's configuration.
9All of what follows is found within files inside of the repositories
10that Zuul manages.
11
12Security Contexts
13-----------------
14
15When a system administrator configures Zuul to operate on a project,
16they specify one of two security contexts for that project. A
17*config-project* is one which is primarily tasked with holding
18configuration information and job content for Zuul. Jobs which are
19defined in a *config-project* are run with elevated privileges, and
20all Zuul configuration items are available for use. It is expected
21that changes to *config-projects* will undergo careful scrutiny before
22being merged.
23
24An *untrusted-project* is a project whose primary focus is not to
25operate Zuul, but rather it is one of the projects being tested or
26deployed. The Zuul configuration language available to these projects
27is somewhat restricted (as detailed in individual section below), and
28jobs defined in these projects run in a restricted execution
29environment since they may be operating on changes which have not yet
30undergone review.
31
32Configuration Loading
33---------------------
34
35When Zuul starts, it examines all of the git repositories which are
James E. Blaireff5a9d2017-06-20 00:00:37 -070036specified by the system administrator in :ref:`tenant-config` and searches
Tristan Cacqueray4a015832017-07-11 05:18:14 +000037for files in the root of each repository. Zuul looks first for a file named
38`zuul.yaml` or a directory named `zuul.d`, and if they are not found,
39`.zuul.yaml` or `.zuul.d` (with a leading dot). In the case of an
40*untrusted-project*, the configuration from every branch is included,
41however, in the case of a *config-project*, only the `master` branch is
42examined.
James E. Blair1de8d402017-05-07 17:08:04 -070043
44When a change is proposed to one of these files in an
45*untrusted-project*, the configuration proposed in the change is
46merged into the running configuration so that any changes to Zuul's
47configuration are self-testing as part of that change. If there is a
48configuration error, no jobs will be run and the error will be
49reported by any applicable pipelines. In the case of a change to a
50*config-project*, the new configuration is parsed and examined for
51errors, but the new configuration is not used in testing the change.
52This is because configuration in *config-projects* is able to access
53elevated privileges and should always be reviewed before being merged.
54
55As soon as a change containing a Zuul configuration change merges to
56any Zuul-managed repository, the new configuration takes effect
57immediately.
58
59Configuration Items
60-------------------
61
62The `zuul.yaml` and `.zuul.yaml` configuration files are
63YAML-formatted and are structured as a series of items, each of which
64is described below.
65
Tristan Cacqueray4a015832017-07-11 05:18:14 +000066In the case of a `zuul.d` directory, Zuul recurses the directory and extends
67the configuration using all the .yaml files in the sorted path order.
68For example, to keep job's variants in a separate file, it needs to be loaded
69after the main entries, for example using number prefixes in file's names::
70
71* zuul.d/pipelines.yaml
72* zuul.d/projects.yaml
73* zuul.d/01_jobs.yaml
74* zuul.d/02_jobs-variants.yaml
75
James E. Blair1de8d402017-05-07 17:08:04 -070076.. _pipeline:
77
78Pipeline
79~~~~~~~~
80
81A pipeline describes a workflow operation in Zuul. It associates jobs
82for a given project with triggering and reporting events.
83
84Its flexible configuration allows for characterizing any number of
85workflows, and by specifying each as a named configuration, makes it
86easy to apply similar workflow operations to projects or groups of
87projects.
88
89By way of example, one of the primary uses of Zuul is to perform
90project gating. To do so, one can create a *gate* pipeline which
91tells Zuul that when a certain event (such as approval by a code
92reviewer) occurs, the corresponding change or pull request should be
93enqueued into the pipeline. When that happens, the jobs which have
94been configured to run for that project in the *gate* pipeline are
95run, and when they complete, the pipeline reports the results to the
96user.
97
98Pipeline configuration items may only appear in *config-projects*.
99
100Generally, a Zuul administrator would define a small number of
101pipelines which represent the workflow processes used in their
102environment. Each project can then be added to the available
103pipelines as appropriate.
104
105Here is an example *check* pipeline, which runs whenever a new
106patchset is created in Gerrit. If the associated jobs all report
107success, the pipeline reports back to Gerrit with a *Verified* vote of
108+1, or if at least one of them fails, a -1::
109
110 - pipeline:
James E. Blaira8387ec2017-07-05 14:00:12 -0700111 name: check
James E. Blair1de8d402017-05-07 17:08:04 -0700112 manager: independent
113 trigger:
114 my_gerrit:
115 - event: patchset-created
116 success:
117 my_gerrit:
118 verified: 1
119 failure:
120 my_gerrit
121 verified: -1
122
James E. Blaireff5a9d2017-06-20 00:00:37 -0700123.. TODO: See TODO for more annotated examples of common pipeline configurations.
James E. Blair1de8d402017-05-07 17:08:04 -0700124
125The attributes available on a pipeline are as follows (all are
126optional unless otherwise specified):
127
128**name** (required)
129 This is used later in the project definition to indicate what jobs
130 should be run for events in the pipeline.
131
132**manager** (required)
133 There are currently two schemes for managing pipelines:
134
James E. Blaireff5a9d2017-06-20 00:00:37 -0700135 .. _independent_pipeline_manager:
136
James E. Blair1de8d402017-05-07 17:08:04 -0700137 *independent*
138 Every event in this pipeline should be treated as independent of
139 other events in the pipeline. This is appropriate when the order of
140 events in the pipeline doesn't matter because the results of the
141 actions this pipeline performs can not affect other events in the
142 pipeline. For example, when a change is first uploaded for review,
143 you may want to run tests on that change to provide early feedback
144 to reviewers. At the end of the tests, the change is not going to
145 be merged, so it is safe to run these tests in parallel without
146 regard to any other changes in the pipeline. They are independent.
147
148 Another type of pipeline that is independent is a post-merge
149 pipeline. In that case, the changes have already merged, so the
150 results can not affect any other events in the pipeline.
151
James E. Blaireff5a9d2017-06-20 00:00:37 -0700152 .. _dependent_pipeline_manager:
153
James E. Blair1de8d402017-05-07 17:08:04 -0700154 *dependent*
155 The dependent pipeline manager is designed for gating. It ensures
156 that every change is tested exactly as it is going to be merged
157 into the repository. An ideal gating system would test one change
158 at a time, applied to the tip of the repository, and only if that
159 change passed tests would it be merged. Then the next change in
160 line would be tested the same way. In order to achieve parallel
161 testing of changes, the dependent pipeline manager performs
162 speculative execution on changes. It orders changes based on
163 their entry into the pipeline. It begins testing all changes in
164 parallel, assuming that each change ahead in the pipeline will pass
165 its tests. If they all succeed, all the changes can be tested and
166 merged in parallel. If a change near the front of the pipeline
167 fails its tests, each change behind it ignores whatever tests have
168 been completed and are tested again without the change in front.
169 This way gate tests may run in parallel but still be tested
170 correctly, exactly as they will appear in the repository when
171 merged.
172
173 For more detail on the theory and operation of Zuul's dependent
174 pipeline manager, see: :doc:`gating`.
175
James E. Blairf17aa9c2017-07-05 13:21:23 -0700176**allow-secrets**
177 This is a boolean which can be used to prevent jobs which require
178 secrets from running in this pipeline. Some pipelines run on
179 proposed changes and therefore execute code which has not yet been
180 reviewed. In such a case, allowing a job to use a secret could
181 result in that secret being exposed. The default is False, meaning
182 that in order to run jobs with secrets, this must be explicitly
183 enabled on each Pipeline where that is safe.
184
185 For more information, see :ref:`secret`.
186
James E. Blair1de8d402017-05-07 17:08:04 -0700187**description**
188 This field may be used to provide a textual description of the
189 pipeline. It may appear in the status page or in documentation.
190
191**success-message**
192 The introductory text in reports when all the voting jobs are
193 successful. Defaults to "Build successful."
194
195**failure-message**
196 The introductory text in reports when at least one voting job fails.
197 Defaults to "Build failed."
198
199**merge-failure-message**
200 The introductory text in the message reported when a change fails to
201 merge with the current state of the repository. Defaults to "Merge
202 failed."
203
204**footer-message**
205 Supplies additional information after test results. Useful for
206 adding information about the CI system such as debugging and contact
207 details.
208
209**trigger**
210 At least one trigger source must be supplied for each pipeline.
211 Triggers are not exclusive -- matching events may be placed in
212 multiple pipelines, and they will behave independently in each of
213 the pipelines they match.
214
215 Triggers are loaded from their connection name. The driver type of
216 the connection will dictate which options are available.
James E. Blaireff5a9d2017-06-20 00:00:37 -0700217 See :ref:`drivers`.
James E. Blair1de8d402017-05-07 17:08:04 -0700218
219**require**
220 If this section is present, it established pre-requisites for any
221 kind of item entering the Pipeline. Regardless of how the item is
222 to be enqueued (via any trigger or automatic dependency resolution),
223 the conditions specified here must be met or the item will not be
224 enqueued.
225
James E. Blaireff5a9d2017-06-20 00:00:37 -0700226.. _pipeline-require-approval:
James E. Blair1de8d402017-05-07 17:08:04 -0700227
228 **approval**
229 This requires that a certain kind of approval be present for the
230 current patchset of the change (the approval could be added by the
231 event in question). It takes several sub-parameters, all of which
232 are optional and are combined together so that there must be an
233 approval matching all specified requirements.
234
235 *username*
236 If present, an approval from this username is required. It is
237 treated as a regular expression.
238
239 *email*
240 If present, an approval with this email address is required. It
241 is treated as a regular expression.
242
243 *email-filter* (deprecated)
244 A deprecated alternate spelling of *email*. Only one of *email* or
245 *email_filter* should be used.
246
247 *older-than*
248 If present, the approval must be older than this amount of time
249 to match. Provide a time interval as a number with a suffix of
250 "w" (weeks), "d" (days), "h" (hours), "m" (minutes), "s"
251 (seconds). Example ``48h`` or ``2d``.
252
253 *newer-than*
254 If present, the approval must be newer than this amount of time
255 to match. Same format as "older-than".
256
257 Any other field is interpreted as a review category and value
258 pair. For example ``verified: 1`` would require that the approval
259 be for a +1 vote in the "Verified" column. The value may either
260 be a single value or a list: ``verified: [1, 2]`` would match
261 either a +1 or +2 vote.
262
263 **open**
264 A boolean value (``true`` or ``false``) that indicates whether the change
265 must be open or closed in order to be enqueued.
266
267 **current-patchset**
268 A boolean value (``true`` or ``false``) that indicates whether the change
269 must be the current patchset in order to be enqueued.
270
271 **status**
272 A string value that corresponds with the status of the change
273 reported by the trigger.
274
275**reject**
276 If this section is present, it establishes pre-requisites that can
277 block an item from being enqueued. It can be considered a negative
278 version of **require**.
279
280 **approval**
281 This takes a list of approvals. If an approval matches the provided
282 criteria the change can not be entered into the pipeline. It follows
283 the same syntax as the :ref:`"require approval" pipeline above
284 <pipeline-require-approval>`.
285
286 Example to reject a change with any negative vote::
287
288 reject:
289 approval:
290 - code-review: [-1, -2]
291
292**dequeue-on-new-patchset**
293 Normally, if a new patchset is uploaded to a change that is in a
294 pipeline, the existing entry in the pipeline will be removed (with
295 jobs canceled and any dependent changes that can no longer merge as
296 well. To suppress this behavior (and allow jobs to continue
297 running), set this to ``false``. Default: ``true``.
298
299**ignore-dependencies**
300 In any kind of pipeline (dependent or independent), Zuul will
301 attempt to enqueue all dependencies ahead of the current change so
302 that they are tested together (independent pipelines report the
303 results of each change regardless of the results of changes ahead).
304 To ignore dependencies completely in an independent pipeline, set
305 this to ``true``. This option is ignored by dependent pipelines.
306 The default is: ``false``.
307
James E. Blaireff5a9d2017-06-20 00:00:37 -0700308**precedence**
309 Indicates how the build scheduler should prioritize jobs for
310 different pipelines. Each pipeline may have one precedence, jobs
311 for pipelines with a higher precedence will be run before ones with
312 lower. The value should be one of ``high``, ``normal``, or ``low``.
313 Default: ``normal``.
314
James E. Blair1de8d402017-05-07 17:08:04 -0700315The following options configure *reporters*. Reporters are
316complementary to triggers; where a trigger is an event on a connection
317which causes Zuul to enqueue an item, a reporter is the action
318performed on a connection when an item is dequeued after its jobs
319complete. The actual syntax for a reporter is defined by the driver
320which implements it. See :ref:`drivers` for more information.
321
322**success**
323 Describes where Zuul should report to if all the jobs complete
324 successfully. This section is optional; if it is omitted, Zuul will
325 run jobs and do nothing on success -- it will not report at all. If
326 the section is present, the listed reporters will be asked to report
327 on the jobs. The reporters are listed by their connection name. The
328 options available depend on the driver for the supplied connection.
329
330**failure**
331 These reporters describe what Zuul should do if at least one job
332 fails.
333
334**merge-failure**
335 These reporters describe what Zuul should do if it is unable to
336 merge in the patchset. If no merge-failure reporters are listed then
337 the ``failure`` reporters will be used to notify of unsuccessful
338 merges.
339
340**start**
341 These reporters describe what Zuul should do when a change is added
342 to the pipeline. This can be used, for example, to reset a
343 previously reported result.
344
345**disabled**
346 These reporters describe what Zuul should do when a pipeline is
347 disabled. See ``disable-after-consecutive-failures``.
348
James E. Blaireff5a9d2017-06-20 00:00:37 -0700349The following options can be used to alter Zuul's behavior to mitigate
350situations in which jobs are failing frequently (perhaps due to a
351problem with an external dependency, or unusually high
352non-deterministic test failures).
353
James E. Blair1de8d402017-05-07 17:08:04 -0700354**disable-after-consecutive-failures**
355 If set, a pipeline can enter a ''disabled'' state if too many changes
356 in a row fail. When this value is exceeded the pipeline will stop
357 reporting to any of the ``success``, ``failure`` or ``merge-failure``
358 reporters and instead only report to the ``disabled`` reporters.
359 (No ``start`` reports are made when a pipeline is disabled).
360
James E. Blair1de8d402017-05-07 17:08:04 -0700361**window**
362 Dependent pipeline managers only. Zuul can rate limit dependent
363 pipelines in a manner similar to TCP flow control. Jobs are only
364 started for items in the queue if they are within the actionable
365 window for the pipeline. The initial length of this window is
366 configurable with this value. The value given should be a positive
367 integer value. A value of ``0`` disables rate limiting on the
368 DependentPipelineManager. Default: ``20``.
369
370**window-floor**
371 Dependent pipeline managers only. This is the minimum value for the
372 window described above. Should be a positive non zero integer value.
373 Default: ``3``.
374
375**window-increase-type**
376 Dependent pipeline managers only. This value describes how the window
377 should grow when changes are successfully merged by zuul. A value of
378 ``linear`` indicates that ``window-increase-factor`` should be added
379 to the previous window value. A value of ``exponential`` indicates
380 that ``window-increase-factor`` should be multiplied against the
381 previous window value and the result will become the window size.
382 Default: ``linear``.
383
384**window-increase-factor**
385 Dependent pipeline managers only. The value to be added or multiplied
386 against the previous window value to determine the new window after
387 successful change merges.
388 Default: ``1``.
389
390**window-decrease-type**
391 Dependent pipeline managers only. This value describes how the window
392 should shrink when changes are not able to be merged by Zuul. A value
393 of ``linear`` indicates that ``window-decrease-factor`` should be
394 subtracted from the previous window value. A value of ``exponential``
395 indicates that ``window-decrease-factor`` should be divided against
396 the previous window value and the result will become the window size.
397 Default: ``exponential``.
398
399**window-decrease-factor**
400 Dependent pipline managers only. The value to be subtracted or divided
401 against the previous window value to determine the new window after
402 unsuccessful change merges.
403 Default: ``2``.
404
405
406.. _job:
407
408Job
409~~~
410
411A job is a unit of work performed by Zuul on an item enqueued into a
412pipeline. Items may run any number of jobs (which may depend on each
413other). Each job is an invocation of an Ansible playbook with a
414specific inventory of hosts. The actual tasks that are run by the job
415appear in the playbook for that job while the attributes that appear in the
416Zuul configuration specify information about when, where, and how the
417job should be run.
418
419Jobs in Zuul support inheritance. Any job may specify a single parent
420job, and any attributes not set on the child job are collected from
421the parent job. In this way, a configuration structure may be built
422starting with very basic jobs which describe characteristics that all
423jobs on the system should have, progressing through stages of
424specialization before arriving at a particular job. A job may inherit
425from any other job in any project (however, if the other job is marked
426as `final`, some attributes may not be overidden).
427
428Jobs also support a concept called variance. The first time a job
429definition appears is called the reference definition of the job.
430Subsequent job definitions with the same name are called variants.
431These may have different selection criteria which indicate to Zuul
432that, for instance, the job should behave differently on a different
433git branch. Unlike inheritance, all job variants must be defined in
434the same project.
435
436When Zuul decides to run a job, it performs a process known as
437freezing the job. Because any number of job variants may be
438applicable, Zuul collects all of the matching variants and applies
439them in the order they appeared in the configuration. The resulting
440frozen job is built from attributes gathered from all of the
441matching variants. In this way, exactly what is run is dependent on
442the pipeline, project, branch, and content of the item.
443
444In addition to the job's main playbook, each job may specify one or
445more pre- and post-playbooks. These are run, in order, before and
446after (respectively) the main playbook. They may be used to set up
447and tear down resources needed by the main playbook. When combined
448with inheritance, they provide powerful tools for job construction. A
449job only has a single main playbook, and when inheriting from a
450parent, the child's main playbook overrides (or replaces) the
451parent's. However, the pre- and post-playbooks are appended and
452prepended in a nesting fashion. So if a parent job and child job both
453specified pre and post playbooks, the sequence of playbooks run would
454be:
455
456* parent pre-run playbook
457* child pre-run playbook
458* child playbook
459* child post-run playbook
460* parent post-run playbook
461
462Further inheritance would nest even deeper.
463
464Here is an example of two job definitions::
465
466 - job:
467 name: base
468 pre-run: copy-git-repos
469 post-run: copy-logs
470
471 - job:
472 name: run-tests
473 parent: base
474 nodes:
475 - name: test-node
476 image: fedora
477
478The following attributes are available on a job; all are optional
479unless otherwise specified:
480
481**name** (required)
482 The name of the job. By default, Zuul looks for a playbook with
483 this name to use as the main playbook for the job. This name is
484 also referenced later in a project pipeline configuration.
485
486**parent**
487 Specifies a job to inherit from. The parent job can be defined in
488 this or any other project. Any attributes not specified on a job
489 will be collected from its parent.
490
491**description**
492 A textual description of the job. Not currently used directly by
493 Zuul, but it is used by the zuul-sphinx extension to Sphinx to
494 auto-document Zuul jobs (in which case it is interpreted as
495 ReStructuredText.
496
497**success-message**
498 Normally when a job succeeds, the string "SUCCESS" is reported as
499 the result for the job. If set, this option may be used to supply a
500 different string. Default: "SUCCESS".
501
502**failure-message**
503 Normally when a job fails, the string "FAILURE" is reported as
504 the result for the job. If set, this option may be used to supply a
505 different string. Default: "FAILURE".
506
507**success-url**
James E. Blair88e79c02017-07-07 13:36:54 -0700508 When a job succeeds, this URL is reported along with the result. If
509 this value is not supplied, Zuul uses the content of the job
510 :ref:`return value <return_values>` **zuul.log_url**. This is
511 recommended as it allows the code which stores the URL to the job
512 artifacts to report exactly where they were stored. To override
513 this value, or if it is not set, supply an absolute URL in this
514 field. If a relative URL is supplied in this field, and
515 **zuul.log_url** is set, then the two will be combined to produce
516 the URL used for the report. This can be used to specify that
517 certain jobs should "deep link" into the stored job artifacts.
James E. Blair1de8d402017-05-07 17:08:04 -0700518 Default: none.
519
520**failure-url**
521 When a job fails, this URL is reported along with the result.
James E. Blair88e79c02017-07-07 13:36:54 -0700522 Otherwise behaves the same as **success-url**.
James E. Blair1de8d402017-05-07 17:08:04 -0700523
524**hold-following-changes**
525 In a dependent pipeline, this option may be used to indicate that no
526 jobs should start on any items which depend on the current item
527 until this job has completed successfully. This may be used to
528 conserve build resources, at the expense of inhibiting the
529 parallelization which speeds the processing of items in a dependent
530 pipeline. A boolean value, default: false.
531
532**voting**
533 Indicates whether the result of this job should be used in
534 determining the overall result of the item. A boolean value,
535 default: true.
536
537**semaphore**
538 The name of a :ref:`semaphore` which should be acquired and released
539 when the job begins and ends. If the semaphore is at maximum
540 capacity, then Zuul will wait until it can be acquired before
541 starting the job. Default: none.
542
543**tags**
544 Metadata about this job. Tags are units of information attached to
545 the job; they do not affect Zuul's behavior, but they can be used
546 within the job to characterize the job. For example, a job which
547 tests a certain subsystem could be tagged with the name of that
548 subsystem, and if the job's results are reported into a database,
549 then the results of all jobs affecting that subsystem could be
550 queried. This attribute is specified as a list of strings, and when
551 inheriting jobs or applying variants, tags accumulate in a set, so
552 the result is always a set of all the tags from all the jobs and
553 variants used in constructing the frozen job, with no duplication.
554 Default: none.
555
James E. Blaireff5a9d2017-06-20 00:00:37 -0700556**branches**
James E. Blair1de8d402017-05-07 17:08:04 -0700557 A regular expression (or list of regular expressions) which describe
558 on what branches a job should run (or in the case of variants: to
559 alter the behavior of a job for a certain branch).
560
561 If there is no job definition for a given job which matches the
562 branch of an item, then that job is not run for the item.
563 Otherwise, all of the job variants which match that branch (and any
564 other selection criteria) are used when freezing the job.
565
566 This example illustrates a job called *run-tests* which uses a
567 nodeset based on the current release of an operating system to
568 perform its tests, except when testing changes to the stable/2.0
569 branch, in which case it uses an older release::
570
571 - job:
572 name: run-tests
573 nodes: current-release
574
575 - job:
576 name: run-tests
577 branch: stable/2.0
578 nodes: old-release
579
580 In some cases, Zuul uses an implied value for the branch specifier
581 if none is supplied:
582
583 * For a job definition in a *config-project*, no implied branch
584 specifier is used. If no branch specifier appears, the job
585 applies to all branches.
586
587 * In the case of an *untrusted-project*, no implied branch specifier
588 is applied to the reference definition of a job. That is to say,
589 that if the first appearance of the job definition appears without
590 a branch specifier, then it will apply to all branches. Note that
591 when collecting its configuration, Zuul reads the `master` branch
592 of a given project first, then other branches in alphabetical
593 order.
594
595 * Any further job variants other than the reference definition in an
596 *untrusted-project* will, if they do not have a branch specifier,
597 will have an implied branch specifier for the current branch
598 applied.
599
600 This allows for the very simple and expected workflow where if a
601 project defines a job on the master branch with no branch specifier,
602 and then creates a new branch based on master, any changes to that
603 job definition within the new branch only affect that branch.
604
605**files**
606 This attribute indicates that the job should only run on changes
607 where the specified files are modified. This is a regular
608 expression or list of regular expressions. Default: none.
609
610**irrelevant-files**
611 This is a negative complement of `files`. It indicates that the job
612 should run unless *all* of the files changed match this list. In
613 other words, if the regular expression `docs/.*` is supplied, then
614 this job will not run if the only files changed are in the docs
615 directory. A regular expression or list of regular expressions.
616 Default: none.
617
618**auth**
619 Authentication information to be made available to the job. This is
620 a dictionary with two potential keys:
621
622 **inherit**
623 A boolean indicating that the authentication information referenced
624 by this job should be able to be inherited by child jobs. Normally
625 when a job inherits from another job, the auth section is not
626 included. This permits jobs to inherit the same basic structure and
627 playbook, but ensures that secret information is unable to be
628 exposed by a child job which may alter the job's behavior. If it is
629 safe for the contents of the authentication section to be used by
630 child jobs, set this to ``true``. Default: ``false``.
631
632 **secrets**
633 A list of secrets which may be used by the job. A :ref:`secret` is
634 a named collection of private information defined separately in the
635 configuration. The secrets that appear here must be defined in the
636 same project as this job definition.
637
638 In the future, other types of authentication information may be
639 added.
640
641**nodes**
642 A list of nodes which should be supplied to the job. This parameter
643 may be supplied either as a string, in which case it references a
644 :ref:`nodeset` definition which appears elsewhere in the
645 configuration, or a list, in which case it is interpreted in the
646 same way as a Nodeset definition (in essence, it is an anonymous
647 Node definition unique to this job). See the :ref:`nodeset`
648 reference for the syntax to use in that case.
649
650 If a job has an empty or no node definition, it will still run and
651 may be able to perform actions on the Zuul executor.
652
653**override-branch**
654 When Zuul runs jobs for a proposed change, it normally checks out
655 the branch associated with that change on every project present in
656 the job. If jobs are running on a ref (such as a branch tip or
657 tag), then that ref is normally checked out. This attribute is used
658 to override that behavior and indicate that this job should,
659 regardless of the branch for the queue item, use the indicated
660 branch instead. This can be used, for example, to run a previous
661 version of the software (from a stable maintenance branch) under
662 test even if the change being tested applies to a different branch
663 (this is only likely to be useful if there is some cross-branch
664 interaction with some component of the system being tested). See
665 also the project-specific **override-branch** attribute under
666 **required-projects** to apply this behavior to a subset of a job's
667 projects.
668
669**timeout**
670 The time in minutes that the job should be allowed to run before it
671 is automatically aborted and failure is reported. If no timeout is
672 supplied, the job may run indefinitely. Supplying a timeout is
673 highly recommended.
674
675**attempts**
676 When Zuul encounters an error running a job's pre-run playbook, Zuul
677 will stop and restart the job. Errors during the main or
678 post-run -playbook phase of a job are not affected by this parameter
679 (they are reported immediately). This parameter controls the number
680 of attempts to make before an error is reported. Default: 3.
681
682**pre-run**
Tobias Henkel2aade262017-07-12 16:09:06 +0200683 The name of a playbook or list of playbooks without file extension
684 to run before the main body of a job. The full path to the playbook
685 in the repo where the job is defined is expected.
James E. Blair1de8d402017-05-07 17:08:04 -0700686
687 When a job inherits from a parent, the child's pre-run playbooks are
688 run after the parent's. See :ref:`job` for more information.
689
690**post-run**
Tobias Henkel2aade262017-07-12 16:09:06 +0200691 The name of a playbook or list of playbooks without file extension
692 to run after the main body of a job. The full path to the playbook
693 in the repo where the job is defined is expected.
James E. Blair1de8d402017-05-07 17:08:04 -0700694
695 When a job inherits from a parent, the child's post-run playbooks
696 are run before the parent's. See :ref:`job` for more information.
697
698**run**
Tobias Henkel2aade262017-07-12 16:09:06 +0200699 The name of the main playbook for this job. This parameter is
700 not normally necessary, as it defaults to a playbook with the
701 same name as the job inside of the `playbooks/` directory (e.g.,
702 the `foo` job would default to `playbooks/foo`. However, if a
703 playbook with a different name is needed, it can be specified
704 here. The file extension is not required, but the full path
705 within the repo is. When a child inherits from a parent, a
706 playbook with the name of the child job is implicitly searched
707 first, before falling back on the playbook used by the parent
708 job (unless the child job specifies a ``run`` attribute, in which
709 case that value is used). Example::
710
711 run: playbooks/<name of the job>
James E. Blair1de8d402017-05-07 17:08:04 -0700712
713**roles**
714 A list of Ansible roles to prepare for the job. Because a job runs
715 an Ansible playbook, any roles which are used by the job must be
716 prepared and installed by Zuul before the job begins. This value is
717 a list of dictionaries, each of which indicates one of two types of
718 roles: a Galaxy role, which is simply a role that is installed from
719 Ansible Galaxy, or a Zuul role, which is a role provided by a
720 project managed by Zuul. Zuul roles are able to benefit from
James E. Blair74a82cf2017-07-12 17:23:08 -0700721 speculative merging and cross-project dependencies when used by
James E. Blair4eec8282017-07-12 17:33:26 -0700722 playbooks in untrusted projects. Roles are added to the Ansible
723 role path in the order they appear on the job -- roles earlier in
724 the list will take precedence over those which follow.
James E. Blair74a82cf2017-07-12 17:23:08 -0700725
726 In the case of job inheritance or variance, the roles used for each
727 of the playbooks run by the job will be only those which were
728 defined along with that playbook. If a child job inherits from a
729 parent which defines a pre and post playbook, then the pre and post
730 playbooks it inherits from the parent job will run only with the
731 roles that were defined on the parent. If the child adds its own
732 pre and post playbooks, then any roles added by the child will be
733 available to the child's playbooks. This is so that a job which
734 inherits from a parent does not inadvertantly alter the behavior of
James E. Blair4eec8282017-07-12 17:33:26 -0700735 the parent's playbooks by the addition of conflicting roles. Roles
736 added by a child will appear before those it inherits from its
737 parent.
James E. Blair1de8d402017-05-07 17:08:04 -0700738
739 A project which supplies a role may be structured in one of two
740 configurations: a bare role (in which the role exists at the root of
741 the project), or a contained role (in which the role exists within
742 the `roles/` directory of the project, perhaps along with other
743 roles). In the case of a contained role, the `roles/` directory of
744 the project is added to the role search path. In the case of a bare
745 role, the project itself is added to the role search path. In case
746 the name of the project is not the name under which the role should
747 be installed (and therefore referenced from Ansible), the `name`
748 attribute may be used to specify an alternate.
749
James E. Blairbb94dfa2017-07-11 07:45:19 -0700750 A job automatically has the project in which it is defined added to
751 the roles path if that project appears to contain a role or `roles/`
752 directory. By default, the project is added to the path under its
753 own name, however, that may be changed by explicitly listing the
754 project in the roles list in the usual way.
755
James E. Blaireff5a9d2017-06-20 00:00:37 -0700756 .. note:: galaxy roles are not yet implemented
757
James E. Blair1de8d402017-05-07 17:08:04 -0700758 **galaxy**
759 The name of the role in Ansible Galaxy. If this attribute is
760 supplied, Zuul will search Ansible Galaxy for a role by this name
761 and install it. Mutually exclusive with ``zuul``; either
762 ``galaxy`` or ``zuul`` must be supplied.
763
764 **zuul**
765 The name of a Zuul project which supplies the role. Mutually
766 exclusive with ``galaxy``; either ``galaxy`` or ``zuul`` must be
767 supplied.
768
769 **name**
770 The installation name of the role. In the case of a bare role,
771 the role will be made available under this name. Ignored in the
772 case of a contained role.
773
774**required-projects**
775 A list of other projects which are used by this job. Any Zuul
776 projects specified here will also be checked out by Zuul into the
777 working directory for the job. Speculative merging and cross-repo
778 dependencies will be honored.
779
780 The format for this attribute is either a list of strings or
781 dictionaries. Strings are interpreted as project names,
782 dictionaries may have the following attributes:
783
784 **name**
785 The name of the required project.
786
787 **override-branch**
788 When Zuul runs jobs for a proposed change, it normally checks out
789 the branch associated with that change on every project present in
790 the job. If jobs are running on a ref (such as a branch tip or
791 tag), then that ref is normally checked out. This attribute is
792 used to override that behavior and indicate that this job should,
793 regardless of the branch for the queue item, use the indicated
794 branch instead, for only this project. See also the
795 **override-branch** attribute of jobs to apply the same behavior
796 to all projects in a job.
797
798**vars**
799
800A dictionary of variables to supply to Ansible. When inheriting from
801a job (or creating a variant of a job) vars are merged with previous
802definitions. This means a variable definition with the same name will
803override a previously defined variable, but new variable names will be
804added to the set of defined variables.
805
806**dependencies**
807 A list of other jobs upon which this job depends. Zuul will not
808 start executing this job until all of its dependencies have
809 completed successfully, and if one or more of them fail, this job
810 will not be run.
811
812**allowed-projects**
813 A list of Zuul projects which may use this job. By default, a job
814 may be used by any other project known to Zuul, however, some jobs
815 use resources or perform actions which are not appropriate for other
816 projects. In these cases, a list of projects which are allowed to
817 use this job may be supplied. If this list is not empty, then it
818 must be an exhaustive list of all projects permitted to use the job.
819 The current project (where the job is defined) is not automatically
820 included, so if it should be able to run this job, then it must be
821 explicitly listed. Default: the empty list (all projects may use
822 the job).
823
824
825.. _project:
826
827Project
828~~~~~~~
829
830A project corresponds to a source code repository with which Zuul is
831configured to interact. The main responsibility of the `Project`
832configuration item is to specify which jobs should run in which
833pipelines for a given project. Within each `Project` definition, a
834section for each `Pipeline` may appear. This project-pipeline
835definition is what determines how a project participates in a
836pipeline.
837
838Consider the following `Project` definition::
839
840 - project:
841 name: yoyodyne
842 check:
843 jobs:
844 - check-syntax
845 - unit-tests
846 gate:
847 queue: integrated
848 jobs:
849 - unit-tests
850 - integration-tests
851
852The project has two project-pipeline stanzas, one for the `check`
853pipeline, and one for `gate`. Each specifies which jobs shuld run
854when a change for that project enteres the respective pipeline -- when
855a change enters `check`, the `check-syntax` and `unit-test` jobs are
856run.
857
858Pipelines which use the dependent pipeline manager (e.g., the `gate`
859example shown earlier) maintain separate queues for groups of
860projects. When Zuul serializes a set of changes which represent
861future potential project states, it must know about all of the
862projects within Zuul which may have an effect on the outcome of the
863jobs it runs. If project *A* uses project *B* as a library, then Zuul
864must be told about that relationship so that it knows to serialize
865changes to A and B together, so that it does not merge a change to B
866while it is testing a change to A.
867
868Zuul could simply assume that all projects are related, or even infer
869relationships by which projects a job indicates it uses, however, in a
870large system that would become unwieldy very quickly, and
871unnecessarily delay changes to unrelated projects. To allow for
872flexibility in the construction of groups of related projects, the
873change queues used by dependent pipeline managers are specified
874manually. To group two or more related projects into a shared queue
875for a dependent pipeline, set the ``queue`` parameter to the same
876value for those projects.
877
878The `gate` project-pipeline definition above specifies that this
879project participates in the `integrated` shared queue for that
880pipeline.
881
882In addition to a project-pipeline definition for one or more
883`Pipelines`, the following attributes may appear in a Project:
884
885**name** (required)
886 The name of the project. If Zuul is configured with two or more
887 unique projects with the same name, the canonical hostname for the
888 project should be included (e.g., `git.example.com/foo`).
889
890**templates**
891 A list of :ref:`project-template` references; the project-pipeline
892 definitions of each Project Template will be applied to this
893 project. If more than one template includes jobs for a given
894 pipeline, they will be combined, as will any jobs specified in
895 project-pipeline definitions on the project itself.
896
897.. _project-template:
898
899Project Template
900~~~~~~~~~~~~~~~~
901
902A Project Template defines one or more project-pipeline definitions
903which can be re-used by multiple projects.
904
905A Project Template uses the same syntax as a :ref:`project`
906definition, however, in the case of a template, the ``name`` attribute
907does not refer to the name of a project, but rather names the template
908so that it can be referenced in a `Project` definition.
909
910.. _secret:
911
912Secret
913~~~~~~
914
915A Secret is a collection of private data for use by one or more jobs.
916In order to maintain the security of the data, the values are usually
917encrypted, however, data which are not sensitive may be provided
918unencrypted as well for convenience.
919
920A Secret may only be used by jobs defined within the same project. To
921use a secret, a :ref:`job` must specify the secret within its `auth`
922section. To protect against jobs in other repositories declaring a
923job with a secret as a parent and then exposing that secret, jobs
924which inherit from a job with secrets will not inherit the secrets
925themselves. To alter that behavior, see the `inherit` job attribute.
926Further, jobs which do not permit children to inherit secrets (the
927default) are also automatically marked `final`, meaning that their
928execution related attributes may not be changed in a project-pipeline
929stanza. This is to protect against a job with secrets defined in one
930project being used by another project in a way which might expose the
931secrets. If a job with secrets is unsafe to be used by other
932projects, the `allowed-projects` job attribute can be used to restrict
933the projects which can invoke that job. Finally, pipelines which are
934used to execute proposed but unreviewed changes can set the
935`allow-secrets` attribute to indicate that they should not supply
936secrets at all in order to protect against someone proposing a change
937which exposes a secret.
938
939The following attributes are required:
940
941**name** (required)
942 The name of the secret, used in a :ref:`Job` definition to request
943 the secret.
944
945**data** (required)
946 A dictionary which will be added to the Ansible variables available
947 to the job. The values can either be plain text strings, or
948 encrypted values. See :ref:`encryption` for more information.
949
950.. _nodeset:
951
952Nodeset
953~~~~~~~
954
955A Nodeset is a named collection of nodes for use by a job. Jobs may
956specify what nodes they require individually, however, by defining
957groups of node types once and referring to them by name, job
958configuration may be simplified.
959
960A Nodeset requires two attributes:
961
962**name** (required)
963 The name of the Nodeset, to be referenced by a :ref:`job`.
964
965**nodes** (required)
966 A list of node definitions, each of which has the following format:
967
968 **name** (required)
969 The name of the node. This will appear in the Ansible inventory
970 for the job.
971
972 **label** (required)
973 The Nodepool label for the node. Zuul will request a node with
974 this label.
975
976.. _semaphore:
977
978Semaphore
979~~~~~~~~~
980
981Semaphores can be used to restrict the number of certain jobs which
982are running at the same time. This may be useful for jobs which
983access shared or limited resources. A semaphore has a value which
984represents the maximum number of jobs which use that semaphore at the
985same time.
986
987Semaphores are never subject to dynamic reconfiguration. If the value
988of a semaphore is changed, it will take effect only when the change
989where it is updated is merged. An example follows::
990
991 - semaphore:
992 name: semaphore-foo
993 max: 5
994 - semaphore:
995 name: semaphore-bar
996 max: 3
997
998The following attributes are available:
999
1000**name** (required)
1001 The name of the semaphore, referenced by jobs.
1002
1003**max**
1004 The maximum number of running jobs which can use this semaphore.
1005 Defaults to 1.