Merge "Don't ignore inexistent jobs in config" into feature/zuulv3
diff --git a/doc/source/user/config.rst b/doc/source/user/config.rst
index 25d192c..34d23f9 100644
--- a/doc/source/user/config.rst
+++ b/doc/source/user/config.rst
@@ -952,20 +952,76 @@
project participates in the ``integrated`` shared queue for that
pipeline.
-In addition to a project-pipeline definition for one or more
-pipelines, the following attributes may appear in a project:
+.. attr:: project
-**name** (required)
- The name of the project. If Zuul is configured with two or more
- unique projects with the same name, the canonical hostname for the
- project should be included (e.g., `git.example.com/foo`).
+ In addition to a project-pipeline definition for one or more
+ pipelines, the following attributes may appear in a project:
-**templates**
- A list of :ref:`project-template` references; the project-pipeline
- definitions of each Project Template will be applied to this
- project. If more than one template includes jobs for a given
- pipeline, they will be combined, as will any jobs specified in
- project-pipeline definitions on the project itself.
+ .. attr:: name
+ :required:
+
+ The name of the project. If Zuul is configured with two or more
+ unique projects with the same name, the canonical hostname for
+ the project should be included (e.g., `git.example.com/foo`).
+
+ .. attr:: templates
+
+ A list of :ref:`project-template` references; the
+ project-pipeline definitions of each Project Template will be
+ applied to this project. If more than one template includes
+ jobs for a given pipeline, they will be combined, as will any
+ jobs specified in project-pipeline definitions on the project
+ itself.
+
+ .. attr:: merge-mode
+ :default: merge-resolve
+
+ The merge mode which is used by Git for this project. Be sure
+ this matches what the remote system which performs merges (i.e.,
+ Gerrit or GitHub). Must be one of the following values:
+
+ .. value:: merge
+
+ Uses the default git merge strategy (recursive).
+
+ .. value:: merge-resolve
+
+ Uses the resolve git merge strategy. This is a very
+ conservative merge strategy which most closely matches the
+ behavior of Gerrit.
+
+ .. value:: cherry-pick
+
+ Cherry-picks each change onto the branch rather than
+ performing any merges.
+
+ .. attr:: <pipeline>
+
+ Each pipeline that the project participates in should have an
+ entry in the project. The value for this key should be a
+ dictionary with the following format:
+
+ .. attr:: jobs
+ :required:
+
+ A list of jobs that should be run when items for this project
+ are enqueued into the pipeline. Each item of this list may
+ be a string, in which case it is treated as a job name, or it
+ may be a dictionary, in which case it is treated as a job
+ variant local to this project and pipeline. In that case,
+ the format of the dictionary is the same as the top level
+ :attr:`job` definition. Any attributes set on the job here
+ will override previous versions of the job.
+
+ .. attr:: queue
+
+ If this pipeline is a :value:`dependent
+ <pipeline.manager.dependent>` pipeline, this specifies the
+ name of the shared queue this project is in. Any projects
+ which interact with each other in tests should be part of the
+ same shared queue in order to ensure that they don't merge
+ changes which break the others. This is a free-form string;
+ just set the same value for each group of projects.
.. _project-template:
@@ -976,9 +1032,10 @@
which can be re-used by multiple projects.
A Project Template uses the same syntax as a :ref:`project`
-definition, however, in the case of a template, the ``name`` attribute
-does not refer to the name of a project, but rather names the template
-so that it can be referenced in a `Project` definition.
+definition, however, in the case of a template, the
+:attr:`project.name` attribute does not refer to the name of a
+project, but rather names the template so that it can be referenced in
+a `Project` definition.
.. _secret:
@@ -1009,16 +1066,23 @@
secrets at all in order to protect against someone proposing a change
which exposes a secret.
-The following attributes are required:
+.. attr:: secret
-**name** (required)
- The name of the secret, used in a :ref:`Job` definition to request
- the secret.
+ The following attributes must appear on a secret:
-**data** (required)
- A dictionary which will be added to the Ansible variables available
- to the job. The values can either be plain text strings, or
- encrypted values. See :ref:`encryption` for more information.
+ .. attr:: name
+ :required:
+
+ The name of the secret, used in a :ref:`Job` definition to
+ request the secret.
+
+ .. attr:: data
+ :required:
+
+ A dictionary which will be added to the Ansible variables
+ available to the job. The values can either be plain text
+ strings, or encrypted values. See :ref:`encryption` for more
+ information.
.. _nodeset:
diff --git a/doc/source/user/jobs.rst b/doc/source/user/jobs.rst
index b6a8564..577d147 100644
--- a/doc/source/user/jobs.rst
+++ b/doc/source/user/jobs.rst
@@ -91,23 +91,25 @@
Job Variables
~~~~~~~~~~~~~
-Any variables specified in the job definition are available as Ansible
-host variables. They are added to the `vars` section of the inventory
-file under the `all` hosts group, so they are available to all hosts.
-Simply refer to them by the name specified in the job's `vars`
-section.
+Any variables specified in the job definition (using the
+:attr:`job.vars` attribute) are available as Ansible host variables.
+They are added to the ``vars`` section of the inventory file under the
+``all`` hosts group, so they are available to all hosts. Simply refer
+to them by the name specified in the job's ``vars`` section.
Secrets
~~~~~~~
-Secrets also appear as variables available to Ansible. Unlike job
-variables, these are not added to the inventory file (so that the
-inventory file may be kept for debugging purposes without revealing
-secrets). But they are still available to Ansible as normal
+:ref:`Secrets <secret>` also appear as variables available to Ansible.
+Unlike job variables, these are not added to the inventory file (so
+that the inventory file may be kept for debugging purposes without
+revealing secrets). But they are still available to Ansible as normal
variables. Because secrets are groups of variables, they will appear
as a dictionary structure in templates, with the dictionary itself
being the name of the secret, and its members the individual items in
-the secret. For example, a secret defined as::
+the secret. For example, a secret defined as:
+
+.. code-block:: yaml
- secret:
name: credentials
@@ -119,13 +121,12 @@
{{ credentials.username }} {{ credentials.password }}
-.. TODO: xref job vars
Zuul Variables
~~~~~~~~~~~~~~
Zuul supplies not only the variables specified by the job definition
-to Ansible, but also some variables from the Zuul itself.
+to Ansible, but also some variables from Zuul itself.
When a pipeline is triggered by an action, it enqueues items which may
vary based on the pipeline's configuration. For example, when a new
@@ -137,93 +138,123 @@
attributes in common. But other attributes may vary based on the type
of item.
-All items provide the following information as Ansible variables:
+.. var:: zuul
-**zuul.build**
- The UUID of the build. A build is a single execution of a job.
- When an item is enqueued into a pipeline, this usually results in
- one build of each job configured for that item's project. However,
- items may be re-enqueued in which case another build may run. In
- dependent pipelines, the same job may run multiple times for the
- same item as circumstances change ahead in the queue. Each time a
- job is run, for whatever reason, it is acompanied with a new
- unique id.
+ All items provide the following information as Ansible variables
+ under the ``zuul`` key:
-**zuul.buildset**
- The build set UUID. When Zuul runs jobs for an item, the collection
- of those jobs is known as a buildset. If the configuration of items
- ahead in a dependent pipeline changes, Zuul creates a new buildset
- and restarts all of the jobs.
+ .. var:: build
-**zuul.ref**
- The git ref of the item. This will be the full path (e.g.,
- 'refs/heads/master' or 'refs/changes/...').
+ The UUID of the build. A build is a single execution of a job.
+ When an item is enqueued into a pipeline, this usually results
+ in one build of each job configured for that item's project.
+ However, items may be re-enqueued in which case another build
+ may run. In dependent pipelines, the same job may run multiple
+ times for the same item as circumstances change ahead in the
+ queue. Each time a job is run, for whatever reason, it is
+ acompanied with a new unique id.
-**zuul.pipeline**
- The name of the pipeline in which the job is being run.
+ .. var:: buildset
-**zuul.job**
- The name of the job being run.
+ The build set UUID. When Zuul runs jobs for an item, the
+ collection of those jobs is known as a buildset. If the
+ configuration of items ahead in a dependent pipeline changes,
+ Zuul creates a new buildset and restarts all of the jobs.
-**zuul.voting**
- A boolean indicating whether the job is voting.
+ .. var:: ref
-**zuul.project**
- The item's project. This is a data structure with the following
- fields:
+ The git ref of the item. This will be the full path (e.g.,
+ `refs/heads/master` or `refs/changes/...`).
-**zuul.project.name**
- The name of the project, excluding hostname. E.g., `org/project`.
+ .. var:: pipeline
-**zuul.project.short_name**
- The name of the project, excluding directories or organizations.
- E.g., `project`.
+ The name of the pipeline in which the job is being run.
-**zuul.project.canonical_hostname**
- The canonical hostname where the project lives. E.g.,
- `git.example.com`.
+ .. var:: job
-**zuul.project.canonical_name**
- The full canonical name of the project including hostname. E.g.,
- `git.example.com/org/project`.
+ The name of the job being run.
-**zuul.tenant**
- The name of the current Zuul tenant.
+ .. var:: voting
-**zuul.jobtags**
- A list of tags associated with the job. Not to be confused with git
- tags, these are simply free-form text fields that can be used by the
- job for reporting or classification purposes.
+ A boolean indicating whether the job is voting.
-**zuul.items**
+ .. var:: project
- A list of dictionaries, each representing an item being tested with
- this change with the format:
+ The item's project. This is a data structure with the following
+ fields:
- **project.name**
- The name of the project, excluding hostname. E.g., `org/project`.
+ .. var:: name
- **project.short_name**
- The name of the project, excluding directories or organizations.
- E.g., `project`.
+ The name of the project, excluding hostname. E.g., `org/project`.
- **project.canonical_hostname**
- The canonical hostname where the project lives. E.g.,
- `git.example.com`.
+ .. var:: short_name
- **project.canonical_name**
- The full canonical name of the project including hostname. E.g.,
- `git.example.com/org/project`.
+ The name of the project, excluding directories or
+ organizations. E.g., `project`.
- **branch**
- The target branch of the change (without the `refs/heads/` prefix).
+ .. var:: canonical_hostname
- **change**
- The identifier for the change.
+ The canonical hostname where the project lives. E.g.,
+ `git.example.com`.
- **patchset**
- The patchset identifier for the change. If a change is revised,
- this will have a different value.
+ .. var:: canonical_name
+
+ The full canonical name of the project including hostname.
+ E.g., `git.example.com/org/project`.
+
+ .. var:: tenant
+
+ The name of the current Zuul tenant.
+
+ .. var:: jobtags
+
+ A list of tags associated with the job. Not to be confused with
+ git tags, these are simply free-form text fields that can be
+ used by the job for reporting or classification purposes.
+
+ .. var:: items
+ :type: list
+
+ A list of dictionaries, each representing an item being tested
+ with this change with the format:
+
+ .. var:: project
+
+ The item's project. This is a data structure with the
+ following fields:
+
+ .. var:: name
+
+ The name of the project, excluding hostname. E.g.,
+ `org/project`.
+
+ .. var:: short_name
+
+ The name of the project, excluding directories or
+ organizations. E.g., `project`.
+
+ .. var:: canonical_hostname
+
+ The canonical hostname where the project lives. E.g.,
+ `git.example.com`.
+
+ .. var:: canonical_name
+
+ The full canonical name of the project including hostname.
+ E.g., `git.example.com/org/project`.
+
+ .. var:: branch
+
+ The target branch of the change (without the `refs/heads/` prefix).
+
+ .. var:: change
+
+ The identifier for the change.
+
+ .. var:: patchset
+
+ The patchset identifier for the change. If a change is
+ revised, this will have a different value.
Change Items
++++++++++++
@@ -233,15 +264,21 @@
change or a GitHub pull request). The following additional variables
are available:
-**zuul.branch**
- The target branch of the change (without the `refs/heads/` prefix).
+.. var:: zuul
+ :hidden:
-**zuul.change**
- The identifier for the change.
+ .. var:: branch
-**zuul.patchset**
- The patchset identifier for the change. If a change is revised,
- this will have a different value.
+ The target branch of the change (without the `refs/heads/` prefix).
+
+ .. var:: change
+
+ The identifier for the change.
+
+ .. var:: patchset
+
+ The patchset identifier for the change. If a change is revised,
+ this will have a different value.
Branch Items
++++++++++++
@@ -252,18 +289,25 @@
of verifying the current condition of the branch. The following
additional variables are available:
-**zuul.branch**
- The name of the item's branch (without the `refs/heads/` prefix).
+.. var:: zuul
+ :hidden:
-**zuul.oldrev**
- If the item was enqueued as the result of a change merging or being
- pushed to the branch, the git sha of the old revision will be
- included here. Otherwise, this variable will be undefined.
+ .. var:: branch
-**zuul.newrev**
- If the item was enqueued as the result of a change merging or being
- pushed to the branch, the git sha of the new revision will be
- included here. Otherwise, this variable will be undefined.
+ The name of the item's branch (without the `refs/heads/`
+ prefix).
+
+ .. var:: oldrev
+
+ If the item was enqueued as the result of a change merging or
+ being pushed to the branch, the git sha of the old revision will
+ be included here. Otherwise, this variable will be undefined.
+
+ .. var:: newrev
+
+ If the item was enqueued as the result of a change merging or
+ being pushed to the branch, the git sha of the new revision will
+ be included here. Otherwise, this variable will be undefined.
Tag Items
+++++++++
@@ -272,20 +316,24 @@
tag was created or deleted. The following additional variables are
available:
-**zuul.tag**
- The name of the item's tag (without the `refs/tags/` prefix).
+.. var:: zuul
+ :hidden:
-**zuul.oldrev**
- If the item was enqueued as the result of a tag being deleted, the
- previous git sha of the tag will be included here. If the tag was
- created, this will be set to the value
- 0000000000000000000000000000000000000000.
+ .. var:: tag
-**zuul.newrev**
- If the item was enqueued as the result of a tag being created, the
- new git sha of the tag will be included here. If the tag was
- deleted, this will be set to the value
- 0000000000000000000000000000000000000000.
+ The name of the item's tag (without the `refs/tags/` prefix).
+
+ .. var:: oldrev
+
+ If the item was enqueued as the result of a tag being deleted,
+ the previous git sha of the tag will be included here. If the
+ tag was created, this variable will be undefined.
+
+ .. var:: newrev
+
+ If the item was enqueued as the result of a tag being created,
+ the new git sha of the tag will be included here. If the tag
+ was deleted, this variable will be undefined.
Ref Items
+++++++++
@@ -295,17 +343,20 @@
to identify the ref. The following additional variables are
available:
-**zuul.oldrev**
- If the item was enqueued as the result of a ref being deleted, the
- previous git sha of the ref will be included here. If the ref was
- created, this will be set to the value
- 0000000000000000000000000000000000000000.
+.. var:: zuul
+ :hidden:
-**zuul.newrev**
- If the item was enqueued as the result of a ref being created, the
- new git sha of the ref will be included here. If the ref was
- deleted, this will be set to the value
- 0000000000000000000000000000000000000000.
+ .. var:: oldrev
+
+ If the item was enqueued as the result of a ref being deleted,
+ the previous git sha of the ref will be included here. If the
+ ref was created, this variable will be undefined.
+
+ .. var:: newrev
+
+ If the item was enqueued as the result of a ref being created,
+ the new git sha of the ref will be included here. If the ref
+ was deleted, this variable will be undefined.
Working Directory
+++++++++++++++++
@@ -313,15 +364,29 @@
Additionally, some information about the working directory and the
executor running the job is available:
-**zuul.executor.hostname**
- The hostname of the executor.
+.. var:: zuul
+ :hidden:
-**zuul.executor.src_root**
- The path to the source directory.
+ .. var:: executor
-**zuul.executor.log_root**
- The path to the logs directory.
+ A number of values related to the executor running the job are
+ available:
+ .. var:: hostname
+
+ The hostname of the executor.
+
+ .. var:: src_root
+
+ The path to the source directory.
+
+ .. var:: log_root
+
+ The path to the logs directory.
+
+ .. var:: work_root
+
+ The path to the working directory.
.. _user_sitewide_variables:
@@ -357,7 +422,9 @@
The job may return some values to Zuul to affect its behavior. To
return a value, use the *zuul_return* Ansible module in a job
-playbook. For example::
+playbook. For example:
+
+.. code-block:: yaml
tasks:
- zuul_return:
@@ -370,7 +437,9 @@
Several uses of these values are planned, but the only currently
implemented use is to set the log URL for a build. To do so, set the
-**zuul.log_url** value. For example::
+**zuul.log_url** value. For example:
+
+.. code-block:: yaml
tasks:
- zuul_return:
diff --git a/tests/base.py b/tests/base.py
index 35c8324..7093e13 100755
--- a/tests/base.py
+++ b/tests/base.py
@@ -551,6 +551,18 @@
_points_to_commits_only = True
+class FakeGithub(object):
+
+ class FakeUser(object):
+ def __init__(self, login):
+ self.login = login
+ self.name = "Github User"
+ self.email = "github.user@example.com"
+
+ def user(self, login):
+ return self.FakeUser(login)
+
+
class FakeGithubPullRequest(object):
def __init__(self, github, number, project, branch,
@@ -879,6 +891,13 @@
self.merge_failure = False
self.merge_not_allowed_count = 0
self.reports = []
+ self.github_client = FakeGithub()
+
+ def getGithubClient(self,
+ project=None,
+ user_id=None,
+ use_app=True):
+ return self.github_client
def openFakePullRequest(self, project, branch, subject, files=[],
body=None):
@@ -965,14 +984,6 @@
pr = self.pull_requests[number - 1]
return pr.reviews
- def getUser(self, login):
- data = {
- 'username': login,
- 'name': 'Github User',
- 'email': 'github.user@example.com'
- }
- return data
-
def getRepoPermission(self, project, login):
owner, proj = project.split('/')
for pr in self.pull_requests:
diff --git a/tests/fixtures/config/ansible/git/common-config/playbooks/check-vars.yaml b/tests/fixtures/config/ansible/git/common-config/playbooks/check-vars.yaml
index 3f62c4c..cd343d0 100644
--- a/tests/fixtures/config/ansible/git/common-config/playbooks/check-vars.yaml
+++ b/tests/fixtures/config/ansible/git/common-config/playbooks/check-vars.yaml
@@ -13,6 +13,7 @@
- zuul.executor.hostname is defined
- zuul.executor.src_root is defined
- zuul.executor.log_root is defined
+ - zuul.executor.work_root is defined
- name: Assert zuul.project variables are valid.
assert:
@@ -29,4 +30,4 @@
that:
- vartest_job == 'vartest_job'
- vartest_secret.value == 'vartest_secret'
- - vartest_site == 'vartest_site'
\ No newline at end of file
+ - vartest_site == 'vartest_site'
diff --git a/tests/fixtures/layouts/reporting-github.yaml b/tests/fixtures/layouts/reporting-github.yaml
index 0fdec85..ddb0588 100644
--- a/tests/fixtures/layouts/reporting-github.yaml
+++ b/tests/fixtures/layouts/reporting-github.yaml
@@ -35,6 +35,27 @@
comment: false
- pipeline:
+ name: this_is_a_really_stupid_long_name_for_a_pipeline_that_should_never_be_used_in_production_becuase_it_will_be_too_long_for_the_API_to_make_use_of_without_crashing
+ description: Uncommon reporting
+ manager: independent
+ trigger:
+ github:
+ - event: pull_request
+ action: comment
+ comment: 'long pipeline'
+ start:
+ github:
+ status: 'pending'
+ success:
+ github:
+ comment: false
+ status: 'success'
+ status-url: http://logs.example.com/{tenant.name}/{pipeline.name}/{change.project}/{change.number}/{buildset.uuid}/
+ failure:
+ github:
+ comment: false
+
+- pipeline:
name: push-reporting
description: Uncommon reporting
manager: independent
@@ -68,6 +89,9 @@
reporting:
jobs:
- project-test1
+ this_is_a_really_stupid_long_name_for_a_pipeline_that_should_never_be_used_in_production_becuase_it_will_be_too_long_for_the_API_to_make_use_of_without_crashing:
+ jobs:
+ - project-test1
- project:
name: org/project2
diff --git a/tests/unit/test_executor.py b/tests/unit/test_executor.py
index 46f3b26..444d783 100755
--- a/tests/unit/test_executor.py
+++ b/tests/unit/test_executor.py
@@ -277,6 +277,11 @@
'layouts/repo-checkout-no-timer-override.yaml')
self.sched.reconfigure(self.config)
self.waitUntilSettled()
+ # If APScheduler is in mid-event when we remove the job, we
+ # can end up with one more event firing, so give it an extra
+ # second to settle.
+ time.sleep(1)
+ self.waitUntilSettled()
self.assertEquals(1, len(self.builds), "One build is running")
@@ -315,6 +320,11 @@
'layouts/repo-checkout-no-timer.yaml')
self.sched.reconfigure(self.config)
self.waitUntilSettled()
+ # If APScheduler is in mid-event when we remove the job, we
+ # can end up with one more event firing, so give it an extra
+ # second to settle.
+ time.sleep(1)
+ self.waitUntilSettled()
self.assertEquals(2, len(self.builds), "Two builds are running")
diff --git a/tests/unit/test_github_driver.py b/tests/unit/test_github_driver.py
index 1ae36aa..0e199df 100644
--- a/tests/unit/test_github_driver.py
+++ b/tests/unit/test_github_driver.py
@@ -272,7 +272,8 @@
check_url = ('http://zuul.example.com/status/#%s,%s' %
(A.number, A.head_sha))
self.assertEqual('tenant-one/check', check_status['context'])
- self.assertEqual('Standard check', check_status['description'])
+ self.assertEqual('check status: pending',
+ check_status['description'])
self.assertEqual('pending', check_status['state'])
self.assertEqual(check_url, check_status['url'])
self.assertEqual(0, len(A.comments))
@@ -287,6 +288,8 @@
check_url = ('http://zuul.example.com/status/#%s,%s' %
(A.number, A.head_sha))
self.assertEqual('tenant-one/check', check_status['context'])
+ self.assertEqual('check status: success',
+ check_status['description'])
self.assertEqual('success', check_status['state'])
self.assertEqual(check_url, check_status['url'])
self.assertEqual(1, len(A.comments))
@@ -312,6 +315,8 @@
self.assertEqual(3, len(statuses))
report_status = statuses[0]
self.assertEqual('tenant-one/reporting', report_status['context'])
+ self.assertEqual('reporting status: success',
+ report_status['description'])
self.assertEqual('success', report_status['state'])
self.assertEqual(2, len(A.comments))
@@ -330,6 +335,33 @@
MatchesRegex('^[a-fA-F0-9]{32}\/$'))
@simple_layout('layouts/reporting-github.yaml', driver='github')
+ def test_truncated_status_description(self):
+ project = 'org/project'
+ # pipeline reports pull status both on start and success
+ self.executor_server.hold_jobs_in_build = True
+ A = self.fake_github.openFakePullRequest(project, 'master', 'A')
+ self.fake_github.emitEvent(
+ A.getCommentAddedEvent('long pipeline'))
+ self.waitUntilSettled()
+ statuses = self.fake_github.statuses[project][A.head_sha]
+ self.assertEqual(1, len(statuses))
+ check_status = statuses[0]
+ # Status is truncated due to long pipeline name
+ self.assertEqual('status: pending',
+ check_status['description'])
+
+ self.executor_server.hold_jobs_in_build = False
+ self.executor_server.release()
+ self.waitUntilSettled()
+ # We should only have two statuses for the head sha
+ statuses = self.fake_github.statuses[project][A.head_sha]
+ self.assertEqual(2, len(statuses))
+ check_status = statuses[0]
+ # Status is truncated due to long pipeline name
+ self.assertEqual('status: success',
+ check_status['description'])
+
+ @simple_layout('layouts/reporting-github.yaml', driver='github')
def test_push_reporting(self):
project = 'org/project2'
# pipeline reports pull status both on start and success
@@ -427,7 +459,7 @@
check_url = ('http://zuul.example.com/status/#%s,%s' %
(A.number, A.head_sha))
self.assertEqual('tenant-one/check', check_status['context'])
- self.assertEqual('Standard check', check_status['description'])
+ self.assertEqual('check status: pending', check_status['description'])
self.assertEqual('pending', check_status['state'])
self.assertEqual(check_url, check_status['url'])
self.assertEqual(0, len(A.comments))
@@ -443,6 +475,7 @@
(A.number, A.head_sha))
self.assertEqual('tenant-one/check', check_status['context'])
self.assertEqual('success', check_status['state'])
+ self.assertEqual('check status: success', check_status['description'])
self.assertEqual(check_url, check_status['url'])
self.assertEqual(1, len(A.comments))
self.assertThat(A.comments[0],
diff --git a/tests/unit/test_scheduler.py b/tests/unit/test_scheduler.py
index c8ebeea..e7cc93d 100755
--- a/tests/unit/test_scheduler.py
+++ b/tests/unit/test_scheduler.py
@@ -1470,7 +1470,7 @@
'review.example.com/org/project',
'project-test2'])
)
- self.assertEqual(held_node['hold_reason'], "reason text")
+ self.assertEqual(held_node['comment'], "reason text")
# Another failed change should not hold any more nodes
B = self.fake_gerrit.addFakeChange('org/project', 'master', 'B')
@@ -1892,6 +1892,11 @@
self.commitConfigUpdate('common-config', 'layouts/no-timer.yaml')
self.sched.reconfigure(self.config)
self.waitUntilSettled()
+ # If APScheduler is in mid-event when we remove the job, we
+ # can end up with one more event firing, so give it an extra
+ # second to settle.
+ time.sleep(1)
+ self.waitUntilSettled()
self.assertEqual(len(self.builds), 1, "One timer job")
@@ -2855,6 +2860,12 @@
# below don't race against more jobs being queued.
self.commitConfigUpdate('common-config', 'layouts/no-timer.yaml')
self.sched.reconfigure(self.config)
+ self.waitUntilSettled()
+ # If APScheduler is in mid-event when we remove the job, we
+ # can end up with one more event firing, so give it an extra
+ # second to settle.
+ time.sleep(1)
+ self.waitUntilSettled()
self.executor_server.release()
self.waitUntilSettled()
@@ -2902,6 +2913,11 @@
'layouts/no-timer.yaml')
self.sched.reconfigure(self.config)
self.waitUntilSettled()
+ # If APScheduler is in mid-event when we remove the job,
+ # we can end up with one more event firing, so give it an
+ # extra second to settle.
+ time.sleep(1)
+ self.waitUntilSettled()
self.assertEqual(len(self.builds), 1,
'Timer builds iteration #%d' % x)
self.executor_server.release('.*')
@@ -2980,6 +2996,11 @@
self.commitConfigUpdate('common-config', 'layouts/no-timer.yaml')
self.sched.reconfigure(self.config)
self.waitUntilSettled()
+ # If APScheduler is in mid-event when we remove the job, we
+ # can end up with one more event firing, so give it an extra
+ # second to settle.
+ time.sleep(1)
+ self.waitUntilSettled()
self.executor_server.release('.*')
self.waitUntilSettled()
@@ -3024,6 +3045,11 @@
self.sched.reconfigure(self.config)
self.registerJobs()
self.waitUntilSettled()
+ # If APScheduler is in mid-event when we remove the job, we
+ # can end up with one more event firing, so give it an extra
+ # second to settle.
+ time.sleep(1)
+ self.waitUntilSettled()
self.worker.release('.*')
self.waitUntilSettled()
diff --git a/zuul/driver/github/githubconnection.py b/zuul/driver/github/githubconnection.py
index 48603a0..80ac573 100644
--- a/zuul/driver/github/githubconnection.py
+++ b/zuul/driver/github/githubconnection.py
@@ -326,25 +326,26 @@
self._data = None
def __getitem__(self, key):
- if self._data is None:
- self._data = self._init_data()
+ self._init_data()
return self._data[key]
def __iter__(self):
+ self._init_data()
return iter(self._data)
def __len__(self):
+ self._init_data()
return len(self._data)
def _init_data(self):
- user = self._github.user(self._username)
- log_rate_limit(self.log, self._github)
- data = {
- 'username': user.login,
- 'name': user.name,
- 'email': user.email
- }
- return data
+ if self._data is None:
+ user = self._github.user(self._username)
+ log_rate_limit(self.log, self._github)
+ self._data = {
+ 'username': user.login,
+ 'name': user.name,
+ 'email': user.email
+ }
class GithubConnection(BaseConnection):
diff --git a/zuul/driver/github/githubreporter.py b/zuul/driver/github/githubreporter.py
index b0791d9..3b8f518 100644
--- a/zuul/driver/github/githubreporter.py
+++ b/zuul/driver/github/githubreporter.py
@@ -101,9 +101,15 @@
url_pattern = sched_config.get('webapp', 'status_url')
url = item.formatUrlPattern(url_pattern) if url_pattern else ''
- description = ''
- if item.pipeline.description:
- description = item.pipeline.description
+ description = '%s status: %s' % (item.pipeline.name,
+ self._commit_status)
+
+ if len(description) >= 140:
+ # This pipeline is named with a long name and thus this
+ # desciption would overflow the GitHub limit of 1024 bytes.
+ # Truncate the description. In practice, anything over 140
+ # characters seems to trip the limit.
+ description = 'status: %s' % self._commit_status
self.log.debug(
'Reporting change %s, params %s, status:\n'
diff --git a/zuul/executor/client.py b/zuul/executor/client.py
index cf70520..85ae68c 100644
--- a/zuul/executor/client.py
+++ b/zuul/executor/client.py
@@ -173,9 +173,11 @@
zuul_params['change'] = str(item.change.number)
if hasattr(item.change, 'patchset'):
zuul_params['patchset'] = str(item.change.patchset)
- if hasattr(item.change, 'oldrev') and item.change.oldrev:
+ if (hasattr(item.change, 'oldrev') and item.change.oldrev
+ and item.change.oldrev != '0' * 40):
zuul_params['oldrev'] = item.change.oldrev
- if hasattr(item.change, 'newrev') and item.change.newrev:
+ if (hasattr(item.change, 'newrev') and item.change.newrev
+ and item.change.newrev != '0' * 40):
zuul_params['newrev'] = item.change.newrev
zuul_params['items'] = []
for i in all_items:
diff --git a/zuul/executor/server.py b/zuul/executor/server.py
index 22ca59f..b166111 100644
--- a/zuul/executor/server.py
+++ b/zuul/executor/server.py
@@ -1350,6 +1350,7 @@
hostname=self.executor_server.hostname,
src_root=self.jobdir.src_root,
log_root=self.jobdir.log_root,
+ work_root=self.jobdir.work_root,
result_data_file=self.jobdir.result_data_file)
nodes = self.getHostList(args)
diff --git a/zuul/model.py b/zuul/model.py
index 27ed243..9be8745 100644
--- a/zuul/model.py
+++ b/zuul/model.py
@@ -357,7 +357,7 @@
self.id = None
self.lock = None
self.hold_job = None
- self.hold_reason = None
+ self.comment = None
# Attributes from Nodepool
self._state = 'unknown'
self.state_time = time.time()
@@ -399,7 +399,7 @@
d = {}
d['state'] = self.state
d['hold_job'] = self.hold_job
- d['hold_reason'] = self.hold_reason
+ d['comment'] = self.comment
for k in self._keys:
d[k] = getattr(self, k)
return d
diff --git a/zuul/nodepool.py b/zuul/nodepool.py
index 6b3632b..0696c60 100644
--- a/zuul/nodepool.py
+++ b/zuul/nodepool.py
@@ -61,7 +61,7 @@
for node in nodes:
node.state = model.STATE_HOLD
node.hold_job = " ".join(autohold_key)
- node.hold_reason = reason
+ node.comment = reason
self.sched.zk.storeNode(node)
# We remove the autohold when the number of nodes in hold
diff --git a/zuul/sphinx/zuul.py b/zuul/sphinx/zuul.py
index b4133d7..7946074 100644
--- a/zuul/sphinx/zuul.py
+++ b/zuul/sphinx/zuul.py
@@ -25,18 +25,18 @@
class ZuulConfigObject(ObjectDescription):
object_names = {
'attr': 'attribute',
+ 'var': 'variable',
}
def get_path(self):
- attr_path = self.env.ref_context.get('zuul:attr_path', [])
- path = []
- if attr_path:
- path.extend(attr_path)
- return path
+ return self.env.ref_context.get('zuul:attr_path', [])
+
+ def get_display_path(self):
+ return self.env.ref_context.get('zuul:display_attr_path', [])
@property
def parent_pathname(self):
- return '.'.join(self.get_path())
+ return '.'.join(self.get_display_path())
@property
def full_pathname(self):
@@ -81,14 +81,19 @@
def before_content(self):
path = self.env.ref_context.setdefault('zuul:attr_path', [])
path.append(self.names[-1])
+ path = self.env.ref_context.setdefault('zuul:display_attr_path', [])
+ path.append(self.names[-1])
def after_content(self):
path = self.env.ref_context.get('zuul:attr_path')
if path:
path.pop()
+ path = self.env.ref_context.get('zuul:display_attr_path')
+ if path:
+ path.pop()
def handle_signature(self, sig, signode):
- path = self.get_path()
+ path = self.get_display_path()
signode['is_multiline'] = True
line = addnodes.desc_signature_line()
line['add_permalink'] = True
@@ -115,6 +120,50 @@
return sig
+class ZuulVarDirective(ZuulConfigObject):
+ has_content = True
+
+ option_spec = {
+ 'type': lambda x: x,
+ 'hidden': lambda x: x,
+ }
+
+ type_map = {
+ 'list': '[]',
+ 'dict': '{}',
+ }
+
+ def get_type_str(self):
+ if 'type' in self.options:
+ return self.type_map[self.options['type']]
+ return ''
+
+ def before_content(self):
+ path = self.env.ref_context.setdefault('zuul:attr_path', [])
+ element = self.names[-1]
+ path.append(element)
+ path = self.env.ref_context.setdefault('zuul:display_attr_path', [])
+ element = self.names[-1] + self.get_type_str()
+ path.append(element)
+
+ def after_content(self):
+ path = self.env.ref_context.get('zuul:attr_path')
+ if path:
+ path.pop()
+ path = self.env.ref_context.get('zuul:display_attr_path')
+ if path:
+ path.pop()
+
+ def handle_signature(self, sig, signode):
+ if 'hidden' in self.options:
+ return sig
+ path = self.get_display_path()
+ for x in path:
+ signode += addnodes.desc_addname(x + '.', x + '.')
+ signode += addnodes.desc_name(sig, sig)
+ return sig
+
+
class ZuulDomain(Domain):
name = 'zuul'
label = 'Zuul'
@@ -122,6 +171,7 @@
directives = {
'attr': ZuulAttrDirective,
'value': ZuulValueDirective,
+ 'var': ZuulVarDirective,
}
roles = {
@@ -129,6 +179,8 @@
warn_dangling=True),
'value': XRefRole(innernodeclass=nodes.inline, # type: ignore
warn_dangling=True),
+ 'var': XRefRole(innernodeclass=nodes.inline, # type: ignore
+ warn_dangling=True),
}
initial_data = {