blob: 16e73850d915b74f5ed01bbc8550b8fe15eac1e1 [file] [log] [blame]
Paul Belanger82595102013-04-08 11:30:51 -04001Zuul
2====
James E. Blaird65e22d2012-06-01 13:50:21 -07003
James E. Blair4c3e0a32016-10-12 14:15:20 -07004Zuul is a project gating system developed for the OpenStack Project.
James E. Blaird65e22d2012-06-01 13:50:21 -07005
James E. Blair75260742016-10-12 15:12:06 -07006We are currently engaged in a significant development effort in
7preparation for the third major version of Zuul. We call this effort
8`Zuul v3`_ and it is described in more detail below.
9
Paul Belanger82595102013-04-08 11:30:51 -040010Contributing
11------------
James E. Blaird65e22d2012-06-01 13:50:21 -070012
James E. Blair1a426402016-10-12 15:17:07 -070013We are currently engaged in a significant development effort in
14preparation for the third major version of Zuul. We call this effort
15`Zuul v3`_ and it is described in this file in the `feature/zuulv3`
16branch of this repo.
17
Anita Kuno84ed8cd2013-12-17 10:14:45 -050018To browse the latest code, see: https://git.openstack.org/cgit/openstack-infra/zuul/tree/
19To clone the latest code, use `git clone git://git.openstack.org/openstack-infra/zuul`
James E. Blaird65e22d2012-06-01 13:50:21 -070020
Michael Krotscheck8c81dc32014-11-11 15:59:06 -080021Bugs are handled at: https://storyboard.openstack.org/#!/project/679
James E. Blaird65e22d2012-06-01 13:50:21 -070022
James E. Blair4c3e0a32016-10-12 14:15:20 -070023Code reviews are, as you might expect, handled by gerrit at
24https://review.openstack.org
James E. Blaird65e22d2012-06-01 13:50:21 -070025
James E. Blair4c3e0a32016-10-12 14:15:20 -070026Use `git review` to submit patches (after creating a Gerrit account
27that links to your launchpad account). Example::
James E. Blaird65e22d2012-06-01 13:50:21 -070028
29 # Do your commits
Paul Belanger82595102013-04-08 11:30:51 -040030 $ git review
Ori Livneh7191ee82013-05-02 19:13:53 -070031 # Enter your username if prompted
James E. Blair75260742016-10-12 15:12:06 -070032
33Zuul v3
34-------
35
36The Zuul v3 effort involves significant changes to Zuul, and its
37companion program, Nodepool. The intent is for Zuul to become more
38generally useful outside of the OpenStack community. This is the best
39way to get started with this effort:
40
411) Read the Zuul v3 spec: http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html
42
43 We use specification documents like this to describe large efforts
44 where we want to make sure that all the participants are in
45 agreement about what will happen and generally how before starting
46 development. These specs should contain enough information for
47 people to evaluate the proposal generally, and sometimes include
48 specific details that need to be agreed upon in advance. They are
49 living documents which can change as work gets underway. However,
50 every change or detail does not need to be reflected in the spec --
51 most work is simply done with patches (and revised if necessary in
52 code review).
53
542) Read the Nodepool build-workers spec: http://specs.openstack.org/openstack-infra/infra-specs/specs/nodepool-zookeeper-workers.html
55
563) Review any proposed updates to these specs: https://review.openstack.org/#/q/status:open+project:openstack-infra/infra-specs+topic:zuulv3
57
58 Some of the information in the specs may be effectively superceded
59 by changes here, which are still undergoing review.
60
David Shrewsbury49bee7b2017-03-28 14:28:02 -0400614) Read developer documentation on the internal data model and testing: http://docs.openstack.org/infra/zuul/feature/zuulv3/developer.html
James E. Blair75260742016-10-12 15:12:06 -070062
63 The general philosophy for Zuul tests is to perform functional
64 testing of either the individual component or the entire end-to-end
65 system with external systems (such as Gerrit) replaced with fakes.
66 Before adding additional unit tests with a narrower focus, consider
67 whether they add value to this system or are merely duplicative of
68 functional tests.
69
705) Review open changes: https://review.openstack.org/#/q/status:open+branch:feature/zuulv3
71
72 We find that the most valuable code reviews are ones that spot
73 problems with the proposed change, or raise questions about how
74 that might affect other systems or subsequent work. It is also a
75 great way to stay involved as a team in work performed by others
76 (for instance, by observing and asking questions about development
77 while it is in progress). We try not to sweat the small things and
78 don't worry too much about style suggestions or other nitpicky
79 things (unless they are relevant -- for instance, a -1 vote on a
80 change that introduces a yaml change out of character with existing
81 conventions is useful because it makes the system more
82 user-friendly; a -1 vote on a change which uses a sub-optimal line
83 breaking strategy is probably not the best use of anyone's time).
84
856) Join #zuul on Freenode. Let others (especially jeblair who is
86 trying to coordinate and prioritize work) know what you would like
87 to work on.
88
James E. Blaird5dcaa12016-12-05 13:39:30 -0800897) Check storyboard for status of current work items: https://storyboard.openstack.org/#!/board/41
James E. Blair75260742016-10-12 15:12:06 -070090
James E. Blair903a7462017-03-07 09:28:40 -080091 Work items tagged with ``low-hanging-fruit`` are tasks that have
92 been identified as not requiring an expansive knowledge of the
93 system. They may still require either some knowledge or
94 investigation into a specific area, but should be suitable for a
95 developer who is becoming acquainted with the system. Those items
96 can be found at:
97 https://storyboard.openstack.org/#!/story/list?tags=low-hanging-fruit&tags=zuulv3
98
James E. Blair75260742016-10-12 15:12:06 -070099Once you are up to speed on those items, it will be helpful to know
100the following:
101
102* Zuul v3 includes some substantial changes to Zuul, and in order to
103 implement them quickly and simultaneously, we temporarily disabled
104 most of the test suite. That test suite still has relevance, but
105 tests are likely to need updating individually, with reasons ranging
106 from something simple such as a test-framework method changing its
107 name, to more substantial issues, such as a feature being removed as
108 part of the v3 work. Each test will need to be evaluated
James E. Blaird5dcaa12016-12-05 13:39:30 -0800109 individually. Feel free to, at any time, claim a test name in this
110 story and work on re-enabling it:
111 https://storyboard.openstack.org/#!/story/2000773
James E. Blair75260742016-10-12 15:12:06 -0700112
113* Because of the importance of external systems, as well as the number
114 of internal Zuul components, actually running Zuul in a development
115 mode quickly becomes unweildy (imagine uploading changes to Gerrit
116 repeatedly while altering Zuul source code). Instead, the best way
117 to develop with Zuul is in fact to write a functional test.
118 Construct a test to fully simulate the series of events you want to
119 see, then run it in the foreground. For example::
120
Paul Belanger6a1825d2017-03-18 12:40:43 -0400121 .tox/py27/bin/python -m testtools.run tests.unit.test_scheduler.TestScheduler.test_jobs_executed
James E. Blair75260742016-10-12 15:12:06 -0700122
123 See TESTING.rst for more information.
124
125* There are many occasions, when working on sweeping changes to Zuul
126 v3, we left notes for future work items in the code marked with
127 "TODOv3". These represent potentially serious missing functionality
128 or other issues which must be resolved before an initial v3 release
129 (unlike a more conventional TODO note, these really can not be left
130 indefinitely). These present an opportunity to identify work items
131 not otherwise tracked. The names associated with TODO or TODOv3
132 items do not mean that only that person can address them -- they
133 simply reflect who to ask to explain the item in more detail if it
134 is too cryptic. In your own work, feel free to leave TODOv3 notes
135 if a change would otherwise become too large or unweildy.
James E. Blaira3c03ed2016-12-05 13:39:39 -0800136
Monty Taylor9c817e92017-06-16 12:31:23 -0500137Python Version Support
138----------------------
139
140Zuul v3 requires Python 3. It does not support Python 2.
141
142As Ansible is used for the execution of jobs, it's important to note that
143while Ansible does support Python 3, not all of Ansible's modules do. Zuul
144currently sets ``ansible_python_interpreter`` to python2 so that remote
145content will be executed with Python2.
146
James E. Blaira3c03ed2016-12-05 13:39:39 -0800147Roadmap
148-------
149
James E. Blair903a7462017-03-07 09:28:40 -0800150* Begin using Zuul v3 to run jobs for Zuul itself
James E. Blaira3c03ed2016-12-05 13:39:39 -0800151* Implement a shim to translate Zuul v2 demand into Nodepool Zookeeper
152 launcher requests
153* Begin using Zookeeper based Nodepool launchers with Zuul v2.5 in
154 OpenStack Infra
James E. Blaira3c03ed2016-12-05 13:39:39 -0800155* Move OpenStack Infra to use Zuul v3
156* Implement Github support
157* Begin using Zuul v3 to run tests on Ansible repos
158* Implement support in Nodepool for non-OpenStack clouds
159* Add native container support to Zuul / Nodepool