project templating system
On setup where Zuul ends up triggering hundreds of projects, you end up
having projects using roughly the same pipeline/jobs. Whenever one want
to add a job in all the similiar project, he has to edit each project
one by one.
To save some precious time, this patch introduces the concept of project
templates. It lets you define a set of pipeline and attached jobs
though the job names can be passed parameters defined on a per project
basis. Thus, updating similiar projects is all about editing a single
template.
A basic example is provided in the documentation.
The voluptuous schema has been updated. It does check whether all
parameters are properly passed to a template but does NOT check whether
the resulting job name exist.
The parameter expansion in templates is borrowed from Jenkins Job
Builder (deep_format function). It has been tweaked to also expand
dictionary keys.
Layout test plan:
$ nosetests -m layout --nocapture
Test layout file validation ...
<...>
bad_template1.yaml
required key not provided @
data['projects'][0]['template']['project']
bad_template2.yaml
extra keys not allowed @
data['projects'][0]['template']['extraparam']
good_template1.yaml
ok
<...>
$
A basic test hasbeen added to verify whether a project-template properly
triggers its tests:
$ nosetests --nocapture \
tests/test_scheduler.py:testScheduler.test_job_from_templates_launched
Test whether a job generated via a template can be launched ... ok
----------------------------------------------------------------------
Ran 1 test in 0.863s
OK
$
Change-Id: Ib82e4719331c204de87fbb4b20c198842b7e32f4
Reviewed-on: https://review.openstack.org/21881
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
diff --git a/tests/fixtures/layout.yaml b/tests/fixtures/layout.yaml
index 5276d83..ecdd2da 100644
--- a/tests/fixtures/layout.yaml
+++ b/tests/fixtures/layout.yaml
@@ -47,6 +47,12 @@
files:
- '.*-requires'
+project-templates:
+ - name: test-one-and-two
+ check:
+ - '{projectname}-test1'
+ - '{projectname}-test2'
+
projects:
- name: org/project
merge-mode: cherry-pick
@@ -124,3 +130,8 @@
- nonvoting-project-test2
post:
- nonvoting-project-post
+
+ - name: org/templated-project
+ template:
+ - name: test-one-and-two
+ projectname: project
diff --git a/tests/fixtures/layouts/bad_template1.yaml b/tests/fixtures/layouts/bad_template1.yaml
new file mode 100644
index 0000000..43da793
--- /dev/null
+++ b/tests/fixtures/layouts/bad_template1.yaml
@@ -0,0 +1,19 @@
+# Template is going to be called but missing a parameter
+
+pipelines:
+ - name: 'check'
+ manager: IndependentPipelineManager
+ trigger:
+ - event: patchset-created
+
+project-templates:
+ - name: template-generic
+ check:
+ # Template uses the 'project' parameter' which must
+ - '{project}-merge'
+
+projects:
+ - name: organization/project
+ template:
+ - name: template-generic
+ # Here we 'forgot' to pass 'project'
diff --git a/tests/fixtures/layouts/bad_template2.yaml b/tests/fixtures/layouts/bad_template2.yaml
new file mode 100644
index 0000000..0e40d2d
--- /dev/null
+++ b/tests/fixtures/layouts/bad_template2.yaml
@@ -0,0 +1,22 @@
+# Template is going to be called with an extra parameter
+
+pipelines:
+ - name: 'check'
+ manager: IndependentPipelineManager
+ trigger:
+ - event: patchset-created
+
+project-templates:
+ - name: template-generic
+ check:
+ # Template only uses the 'project' parameter'
+ - '{project}-merge'
+
+projects:
+ - name: organization/project
+ template:
+ - name: template-generic
+ project: 'MyProjectName'
+ # Feed an extra parameters which is not going to be used
+ # by the template. That is an error.
+ extraparam: 'IShouldNotBeSet'
diff --git a/tests/fixtures/layouts/bad_template3.yaml b/tests/fixtures/layouts/bad_template3.yaml
new file mode 100644
index 0000000..70412b8
--- /dev/null
+++ b/tests/fixtures/layouts/bad_template3.yaml
@@ -0,0 +1,13 @@
+# Template refers to an unexisting pipeline
+
+pipelines:
+ # We have no pipelines at all
+
+project-templates:
+ - name: template-generic
+ unexisting-pipeline: # pipeline does not exist
+
+projects:
+ - name: organization/project
+ template:
+ - name: template-generic
diff --git a/tests/fixtures/layouts/good_template1.yaml b/tests/fixtures/layouts/good_template1.yaml
new file mode 100644
index 0000000..1d179f7
--- /dev/null
+++ b/tests/fixtures/layouts/good_template1.yaml
@@ -0,0 +1,16 @@
+pipelines:
+ - name: 'check'
+ manager: IndependentPipelineManager
+ trigger:
+ - event: patchset-created
+
+project-templates:
+ - name: template-generic
+ check:
+ - '{project}-merge'
+
+projects:
+ - name: organization/project
+ template:
+ - name: template-generic
+ project: 'myproject'