Merge "Add more logging to zuul merger process"
diff --git a/doc/source/launchers.rst b/doc/source/launchers.rst
index db49933..d6e8b64 100644
--- a/doc/source/launchers.rst
+++ b/doc/source/launchers.rst
@@ -73,51 +73,93 @@
Zuul will pass some parameters with every job it launches. The
Gearman Plugin will ensure these are supplied as Jenkins build
parameters, so they will be available for use in the job configuration
-as well as to the running job as environment variables. They are as
-follows:
+as well as to the running job as environment variables. Builds can
+be triggered either by an action on a change or by a reference update.
+Both events share a common set of parameters and more specific
+parameters as follows:
+
+Common parameters
+~~~~~~~~~~~~~~~~~
**ZUUL_UUID**
- Zuul provided key to link builds with Gerrit events
+ Zuul provided key to link builds with Gerrit events.
**ZUUL_REF**
- Zuul provided ref that includes commit(s) to build
+ Zuul provided ref that includes commit(s) to build.
**ZUUL_COMMIT**
- The commit SHA1 at the head of ZUUL_REF
+ The commit SHA1 at the head of ZUUL_REF.
**ZUUL_PROJECT**
- The project that triggered this build
+ The project that triggered this build.
**ZUUL_PIPELINE**
- The Zuul pipeline that is building this job
+ The Zuul pipeline that is building this job.
**ZUUL_URL**
- The url for the zuul server as configured in zuul.conf.
+ The URL for the zuul server as configured in zuul.conf.
A test runner may use this URL as the basis for fetching
git commits.
+Change related parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
The following additional parameters will only be provided for builds
associated with changes (i.e., in response to patchset-created or
comment-added events):
**ZUUL_BRANCH**
- The target branch for the change that triggered this build
+ The target branch for the change that triggered this build.
**ZUUL_CHANGE**
- The Gerrit change ID for the change that triggered this build
+ The Gerrit change ID for the change that triggered this build.
**ZUUL_CHANGE_IDS**
All of the Gerrit change IDs that are included in this build (useful
- when the DependentPipelineManager combines changes for testing)
+ when the DependentPipelineManager combines changes for testing).
**ZUUL_PATCHSET**
- The Gerrit patchset number for the change that triggered this build
+ The Gerrit patchset number for the change that triggered this build.
+
+Reference updated parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The following additional parameters will only be provided for
post-merge (ref-updated) builds:
**ZUUL_OLDREV**
The SHA1 of the old revision at this ref (recall the ref name is
- in ZUUL_REF)
+ in ZUUL_REF).
**ZUUL_NEWREV**
The SHA1 of the new revision at this ref (recall the ref name is
- in ZUUL_REF)
+ in ZUUL_REF).
**ZUUL_SHORT_OLDREV**
- The shortened (7 character) SHA1 of the old revision
+ The shortened (7 character) SHA1 of the old revision.
**ZUUL_SHORT_NEWREV**
- The shortened (7 character) SHA1 of the new revision
+ The shortened (7 character) SHA1 of the new revision.
+
+Unset revisions default to 00000000000000000000000000000000.
+
+Examples:
+
+When a reference is created::
+
+ ZUUL_OLDREV=00000000000000000000000000000000
+ ZUUL_NEWREV=123456789abcdef123456789abcdef12
+ ZUUL_SHORT_OLDREV=0000000
+ ZUUL_SHORT_NEWREV=1234567
+
+When a reference is deleted::
+
+ ZUUL_OLDREV=123456789abcdef123456789abcdef12
+ ZUUL_NEWREV=00000000000000000000000000000000
+ ZUUL_SHORT_OLDREV=1234567
+ ZUUL_SHORT_NEWREV=0000000
+
+And finally a reference being altered::
+
+ ZUUL_OLDREV=123456789abcdef123456789abcdef12
+ ZUUL_NEWREV=abcdef123456789abcdef123456789ab
+ ZUUL_SHORT_OLDREV=1234567
+ ZUUL_SHORT_NEWREV=abcdef1
+
+Your jobs can check whether the parameters are ``000000`` to act
+differently on each kind of event.
+
+Jenkins git plugin configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In order to test the correct build, configure the Jenkins Git SCM
plugin as follows::
diff --git a/doc/source/reporters.rst b/doc/source/reporters.rst
index 7c0214d..f94a439 100644
--- a/doc/source/reporters.rst
+++ b/doc/source/reporters.rst
@@ -39,10 +39,10 @@
SMTP Configuration
~~~~~~~~~~~~~~~~~~
-zuul.conf contains the SMTP server and default to/from as describe
+zuul.conf contains the SMTP server and default to/from as described
in :ref:`zuulconf`.
-Each pipeline can overwrite the subject or the to or from address by
+Each pipeline can overwrite the ``subject`` or the ``to`` or ``from`` address by
providing alternatives as arguments to the reporter. For example, ::
pipelines:
diff --git a/doc/source/zuul.rst b/doc/source/zuul.rst
index 1a6a23d..5086d4d 100644
--- a/doc/source/zuul.rst
+++ b/doc/source/zuul.rst
@@ -10,11 +10,11 @@
**zuul.conf**
Connection information for Gerrit and Gearman, locations of the
- other config files
+ other config files.
**layout.yaml**
- Project and pipeline configuration -- what Zuul does
+ Project and pipeline configuration -- what Zuul does.
**logging.conf**
- Python logging config
+ Python logging config.
Examples of each of the three files can be found in the etc/ directory
of the source distribution.
@@ -46,7 +46,7 @@
``server=gearman.example.com``
**port**
- Port on which the Gearman server is listening
+ Port on which the Gearman server is listening.
``port=4730``
gearman_server
@@ -238,6 +238,11 @@
reported back to Gerrit when at least one voting build fails.
Defaults to "Build failed."
+**footer-message**
+ An optional field to supply additional information after test results.
+ Useful for adding information about the CI system such as debugging
+ and contact details.
+
**manager**
There are currently two schemes for managing pipelines:
@@ -365,14 +370,14 @@
If present, the approval must be older than this amount of time
to match. Provide a time interval as a number with a suffix of
"w" (weeks), "d" (days), "h" (hours), "m" (minutes), "s"
- (seconds). Example "48h" or "2d".
+ (seconds). Example ``48h`` or ``2d``.
*newer-than*
If present, the approval must be newer than this amount of time
to match. Same format as "older-than".
Any other field is interpreted as a review category and value
- pair. For example "verified: 1" would require that the approval
+ pair. For example ``verified: 1`` would require that the approval
be for a +1 vote in the "Verified" column.
**timer**
@@ -547,7 +552,7 @@
The ``change-merged`` event does not include the commit sha1 which can be
hazardous, it would let you report back to Gerrit though. If you were to
build a tarball for a specific commit, you should consider instead using
- the ``ref-updated`` event which does include the commit sha1 (but lack the
+ the ``ref-updated`` event which does include the commit sha1 (but lacks the
Gerrit change number).
Jobs
diff --git a/tests/fixtures/layout-footer-message.yaml b/tests/fixtures/layout-footer-message.yaml
new file mode 100644
index 0000000..7977c19
--- /dev/null
+++ b/tests/fixtures/layout-footer-message.yaml
@@ -0,0 +1,34 @@
+includes:
+ - python-file: custom_functions.py
+
+pipelines:
+ - name: gate
+ manager: DependentPipelineManager
+ failure-message: Build failed. For information on how to proceed, see http://wiki.example.org/Test_Failures
+ footer-message: For CI problems and help debugging, contact ci@example.org
+ trigger:
+ gerrit:
+ - event: comment-added
+ approval:
+ - approved: 1
+ success:
+ gerrit:
+ verified: 2
+ submit: true
+ smtp:
+ to: success@example.org
+ failure:
+ gerrit:
+ verified: -2
+ smtp:
+ to: failure@example.org
+ start:
+ gerrit:
+ verified: 0
+ precedence: high
+
+projects:
+ - name: org/project
+ gate:
+ - test1
+ - test2
diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py
index 4a4d6de..1d7be4a 100755
--- a/tests/test_scheduler.py
+++ b/tests/test_scheduler.py
@@ -896,7 +896,6 @@
self.merge_server.join()
self.merge_client.stop()
self.worker.shutdown()
- self.gearman_server.shutdown()
self.gerrit.stop()
self.timer.stop()
self.sched.stop()
@@ -907,6 +906,7 @@
self.webapp.join()
self.rpc.stop()
self.rpc.join()
+ self.gearman_server.shutdown()
threads = threading.enumerate()
if len(threads) > 1:
self.log.error("More than one thread is running: %s" % threads)
@@ -1112,6 +1112,7 @@
self.sched.result_event_queue.empty() and
self.fake_gerrit.event_queue.empty() and
not self.merge_client.build_sets and
+ self.haveAllBuildsReported() and
self.areAllBuildsWaiting()):
self.sched.run_handler_lock.release()
self.worker.lock.release()
@@ -1945,6 +1946,15 @@
self.fake_gerrit.addEvent(B.addApproval('APRV', 1))
self.fake_gerrit.addEvent(C.addApproval('APRV', 1))
self.waitUntilSettled()
+ queue = self.gearman_server.getQueue()
+ job_A = None
+ for job in queue:
+ if 'project-merge' in job.name:
+ job_A = job
+ ref_A = self.getParameter(job_A, 'ZUUL_REF')
+ commit_A = self.getParameter(job_A, 'ZUUL_COMMIT')
+ self.log.debug("Got Zuul ref for change A: %s" % ref_A)
+ self.log.debug("Got Zuul commit for change A: %s" % commit_A)
self.gearman_server.release('.*-merge')
self.waitUntilSettled()
@@ -1954,7 +1964,10 @@
if 'project-merge' in job.name:
job_B = job
ref_B = self.getParameter(job_B, 'ZUUL_REF')
+ commit_B = self.getParameter(job_B, 'ZUUL_COMMIT')
self.log.debug("Got Zuul ref for change B: %s" % ref_B)
+ self.log.debug("Got Zuul commit for change B: %s" % commit_B)
+
self.gearman_server.release('.*-merge')
self.waitUntilSettled()
queue = self.gearman_server.getQueue()
@@ -1962,7 +1975,9 @@
if 'project-merge' in job.name:
job_C = job
ref_C = self.getParameter(job_C, 'ZUUL_REF')
+ commit_C = self.getParameter(job_C, 'ZUUL_COMMIT')
self.log.debug("Got Zuul ref for change C: %s" % ref_C)
+ self.log.debug("Got Zuul commit for change C: %s" % commit_C)
self.gearman_server.hold_jobs_in_queue = False
self.gearman_server.release()
self.waitUntilSettled()
@@ -1972,15 +1987,32 @@
repo_messages = [c.message.strip()
for c in repo.iter_commits(ref_C)]
+ repo_shas = [c.hexsha for c in repo.iter_commits(ref_C)]
repo_messages.reverse()
correct_messages = ['initial commit', 'A-1', 'C-1']
+ # Ensure the right commits are in the history for this ref
self.assertEqual(repo_messages, correct_messages)
+ # Ensure ZUUL_REF -> ZUUL_COMMIT
+ self.assertEqual(repo_shas[0], commit_C)
repo_messages = [c.message.strip()
for c in repo.iter_commits(ref_B)]
+ repo_shas = [c.hexsha for c in repo.iter_commits(ref_B)]
repo_messages.reverse()
correct_messages = ['initial commit', 'mp commit', 'B-1']
self.assertEqual(repo_messages, correct_messages)
+ self.assertEqual(repo_shas[0], commit_B)
+
+ repo_messages = [c.message.strip()
+ for c in repo.iter_commits(ref_A)]
+ repo_shas = [c.hexsha for c in repo.iter_commits(ref_A)]
+ repo_messages.reverse()
+ correct_messages = ['initial commit', 'A-1']
+ self.assertEqual(repo_messages, correct_messages)
+ self.assertEqual(repo_shas[0], commit_A)
+
+ self.assertNotEqual(ref_A, ref_B, ref_C)
+ self.assertNotEqual(commit_A, commit_B, commit_C)
def test_one_job_project(self):
"Test that queueing works with one job"
@@ -3640,3 +3672,43 @@
self.worker.hold_jobs_in_build = False
self.worker.release()
self.waitUntilSettled()
+
+ def test_footer_message(self):
+ "Test a pipeline's footer message is correctly added to the report."
+ self.config.set('zuul', 'layout_config',
+ 'tests/fixtures/layout-footer-message.yaml')
+ self.sched.reconfigure(self.config)
+ self.registerJobs()
+
+ A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A')
+ A.addApproval('CRVW', 2)
+ self.worker.addFailTest('test1', A)
+ self.fake_gerrit.addEvent(A.addApproval('APRV', 1))
+ self.waitUntilSettled()
+
+ B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B')
+ B.addApproval('CRVW', 2)
+ self.fake_gerrit.addEvent(B.addApproval('APRV', 1))
+ self.waitUntilSettled()
+
+ self.assertEqual(2, len(self.smtp_messages))
+
+ failure_body = """\
+Build failed. For information on how to proceed, see \
+http://wiki.example.org/Test_Failures
+
+- test1 http://logs.example.com/1/1/gate/test1/0 : FAILURE in 0s
+- test2 http://logs.example.com/1/1/gate/test2/1 : SUCCESS in 0s
+
+For CI problems and help debugging, contact ci@example.org"""
+
+ success_body = """\
+Build succeeded.
+
+- test1 http://logs.example.com/2/1/gate/test1/2 : SUCCESS in 0s
+- test2 http://logs.example.com/2/1/gate/test2/3 : SUCCESS in 0s
+
+For CI problems and help debugging, contact ci@example.org"""
+
+ self.assertEqual(failure_body, self.smtp_messages[0]['body'])
+ self.assertEqual(success_body, self.smtp_messages[1]['body'])
diff --git a/zuul/cmd/merger.py b/zuul/cmd/merger.py
index f046235..edf8da9 100644
--- a/zuul/cmd/merger.py
+++ b/zuul/cmd/merger.py
@@ -138,7 +138,7 @@
if server.config.has_option('merger', 'pidfile'):
pid_fn = os.path.expanduser(server.config.get('merger', 'pidfile'))
else:
- pid_fn = '/var/run/zuul-merger/merger.pid'
+ pid_fn = '/var/run/zuul-merger/zuul-merger.pid'
pid = pid_file_module.TimeoutPIDLockFile(pid_fn, 10)
if server.args.nodaemon:
diff --git a/zuul/launcher/gearman.py b/zuul/launcher/gearman.py
index 0e248af..3638add 100644
--- a/zuul/launcher/gearman.py
+++ b/zuul/launcher/gearman.py
@@ -352,11 +352,11 @@
self.log.debug("Still unable to find build %s to cancel" % build)
if build.number:
self.log.debug("Build %s has just started" % build)
- self.cancelRunningBuild(build)
- self.log.debug("Canceled just running build %s" % build)
else:
- self.log.error("Build %s has not started but "
- "was not found in queue" % build)
+ self.log.error("Build %s has not started but was not"
+ "found in queue; canceling anyway" % build)
+ self.cancelRunningBuild(build)
+ self.log.debug("Canceled possibly running build %s" % build)
def onBuildCompleted(self, job, result=None):
if job.unique in self.meta_jobs:
diff --git a/zuul/layoutvalidator.py b/zuul/layoutvalidator.py
index 48aab03..de58c25 100644
--- a/zuul/layoutvalidator.py
+++ b/zuul/layoutvalidator.py
@@ -79,6 +79,7 @@
'description': str,
'success-message': str,
'failure-message': str,
+ 'footer-message': str,
'dequeue-on-new-patchset': bool,
'trigger': trigger,
'success': report_actions,
diff --git a/zuul/merger/merger.py b/zuul/merger/merger.py
index 28e88ad..f150771 100644
--- a/zuul/merger/merger.py
+++ b/zuul/merger/merger.py
@@ -266,12 +266,12 @@
recent[key] = commit
# Set the Zuul ref for this item to point to the most recent
# commits of each project-branch
- for key, commit in recent.items():
+ for key, mrc in recent.items():
project, branch = key
try:
repo = self.getRepo(project, None)
zuul_ref = branch + '/' + item['ref']
- repo.createZuulRef(zuul_ref, commit)
+ repo.createZuulRef(zuul_ref, mrc)
except Exception:
self.log.exception("Unable to set zuul ref %s for "
"item %s" % (zuul_ref, item))
diff --git a/zuul/model.py b/zuul/model.py
index a90c6f7..b20c08e 100644
--- a/zuul/model.py
+++ b/zuul/model.py
@@ -64,6 +64,7 @@
self.description = None
self.failure_message = None
self.success_message = None
+ self.footer_message = None
self.dequeue_on_new_patchset = True
self.job_trees = {} # project -> JobTree
self.manager = None
diff --git a/zuul/scheduler.py b/zuul/scheduler.py
index d3ffdc8..c941a98 100644
--- a/zuul/scheduler.py
+++ b/zuul/scheduler.py
@@ -228,6 +228,7 @@
"Build failed.")
pipeline.success_message = conf_pipeline.get('success-message',
"Build succeeded.")
+ pipeline.footer_message = conf_pipeline.get('footer-message', "")
pipeline.dequeue_on_new_patchset = conf_pipeline.get(
'dequeue-on-new-patchset', True)
@@ -1452,6 +1453,8 @@
name = job.name + ' '
ret += '- %s%s : %s%s%s\n' % (name, url, result, elapsed,
voting)
+ ret += '\n'
+ ret += self.pipeline.footer_message
return ret
def formatDescription(self, build):
diff --git a/zuul/webapp.py b/zuul/webapp.py
index 193d1b6..6b04384 100644
--- a/zuul/webapp.py
+++ b/zuul/webapp.py
@@ -26,10 +26,11 @@
threading.Thread.__init__(self)
self.scheduler = scheduler
self.port = port
-
- def run(self):
+ self.daemon = True
self.server = httpserver.serve(self.app, host='0.0.0.0',
port=self.port, start_loop=False)
+
+ def run(self):
self.server.serve_forever()
def stop(self):