Merge "tox: remove validate-layout" into feature/zuulv3
diff --git a/doc/source/admin/components.rst b/doc/source/admin/components.rst
index b3c2e44..86b01ef 100644
--- a/doc/source/admin/components.rst
+++ b/doc/source/admin/components.rst
@@ -224,6 +224,11 @@
.. attr:: scheduler
+ .. attr:: command_socket
+ :default: /var/lib/zuul/scheduler.socket
+
+ Path to command socket file for the scheduler process.
+
.. attr:: tenant_config
:required:
@@ -282,6 +287,11 @@
.. attr:: merger
+ ,, attr:: command_socket
+ :default: /var/lib/zuul/merger.socket
+
+ Path to command socket file for the merger process.
+
.. attr:: git_dir
Directory in which Zuul should clone git repositories.
@@ -392,6 +402,11 @@
.. attr:: executor
+ .. attr:: command_socket
+ :default: /var/lib/zuul/executor.socket
+
+ Path to command socket file for the executor process.
+
.. attr:: finger_port
:default: 79
diff --git a/doc/source/admin/drivers/github.rst b/doc/source/admin/drivers/github.rst
index 7eebbdc..8dd7764 100644
--- a/doc/source/admin/drivers/github.rst
+++ b/doc/source/admin/drivers/github.rst
@@ -7,18 +7,95 @@
interact with the public GitHub service as well as site-local
installations of GitHub enterprise.
-.. TODO: make this section more user friendly
+Configure GitHub
+----------------
-Configure GitHub `webhook events
-<https://developer.github.com/webhooks/creating/>`_.
+There are two options currently available. GitHub's project owner can either
+manually setup web-hook or install a GitHub Application. In the first case,
+the project's owner needs to know the zuul endpoint and the webhook secrets.
-Set *Payload URL* to
-``http://<zuul-hostname>/connection/<connection-name>/payload``.
-Set *Content Type* to ``application/json``.
+Web-Hook
+........
+
+To configure a project's `webhook events <https://developer.github.com/webhooks/creating/>`_:
+
+* Set *Payload URL* to ``http://<zuul-hostname>/connection/<connection-name>/payload``.
+
+* Set *Content Type* to ``application/json``.
Select *Events* you are interested in. See below for the supported events.
+You will also need to have a GitHub user created for your zuul:
+
+* Zuul public key needs to be added to the GitHub account
+
+* A api_token needs to be created too, see this `article <See https://help.github.com/articles/creating-an-access-token-for-command-line-use/>`_
+
+Then in the zuul.conf, set webhook_token and api_token.
+
+Application
+...........
+
+To create a `GitHub application <https://developer.github.com/apps/building-integrations/setting-up-and-registering-github-apps/registering-github-apps/>`_:
+
+* Go to your organization settings page to create the application, e.g.: https://github.com/organizations/my-org/settings/apps/new
+
+* Set GitHub App name to "my-org-zuul"
+
+* Set Setup URL to your setup documentation, when user install the application they are redirected to this url
+
+* Set Webhook URL to ``http://<zuul-hostname>/connection/<connection-name>/payload``.
+
+* Create a Webhook secret
+
+* Set permissions:
+
+ * Commit statuses: Read & Write
+
+ * Issues: Read & Write
+
+ * Pull requests: Read & Write
+
+ * Repository contents: Read & Write (write to let zuul merge change)
+
+* Set events subscription:
+
+ * Label
+
+ * Status
+
+ * Issue comment
+
+ * Issues
+
+ * Pull request
+
+ * Pull request review
+
+ * Pull request review comment
+
+ * Commit comment
+
+ * Create
+
+ * Push
+
+ * Release
+
+* Set Where can this GitHub App be installed to "Any account"
+
+* Create the App
+
+* Generate a Private key in the app settings page
+
+Then in the zuul.conf, set webhook_token, app_id and app_key.
+After restarting zuul-scheduler, verify in the 'Advanced' tab that the
+Ping payload works (green tick and 200 response)
+
+Users can now install the application using its public page, e.g.: https://github.com/apps/my-org-zuul
+
+
Connection Configuration
------------------------
diff --git a/doc/source/user/config.rst b/doc/source/user/config.rst
index 3ea20ab..4151eda 100644
--- a/doc/source/user/config.rst
+++ b/doc/source/user/config.rst
@@ -798,13 +798,6 @@
are run after the parent's. See :ref:`job` for more
information.
- .. warning::
-
- If the path as specified does not exist, Zuul will try
- appending the extensions ``.yaml`` and ``.yml``. This
- behavior is deprecated and will be removed in the future all
- playbook paths should include the file extension.
-
.. attr:: post-run
The name of a playbook or list of playbooks to run after the
@@ -815,13 +808,6 @@
playbooks are run before the parent's. See :ref:`job` for more
information.
- .. warning::
-
- If the path as specified does not exist, Zuul will try
- appending the extensions ``.yaml`` and ``.yml``. This
- behavior is deprecated and will be removed in the future all
- playbook paths should include the file extension.
-
.. attr:: run
The name of the main playbook for this job. If it is not
@@ -833,13 +819,6 @@
run: playbooks/job-playbook.yaml
- .. warning::
-
- If the path as specified does not exist, Zuul will try
- appending the extensions ``.yaml`` and ``.yml``. This
- behavior is deprecated and will be removed in the future all
- playbook paths should include the file extension.
-
.. attr:: roles
A list of Ansible roles to prepare for the job. Because a job
@@ -1210,7 +1189,9 @@
label: controller-label
- name: compute1
label: compute-label
- - name: compute2
+ - name:
+ - compute2
+ - web
label: compute-label
groups:
- name: ceph-osd
@@ -1221,6 +1202,9 @@
- controller
- compute1
- compute2
+ - name: ceph-web
+ nodes:
+ - web
.. attr:: nodeset
@@ -1242,6 +1226,9 @@
The name of the node. This will appear in the Ansible inventory
for the job.
+ This can also be as a list of strings. If so, then the list of hosts in
+ the Ansible inventory will share a common ansible_host address.
+
.. attr:: label
:required:
diff --git a/doc/source/user/jobs.rst b/doc/source/user/jobs.rst
index 989338a..278c4f4 100644
--- a/doc/source/user/jobs.rst
+++ b/doc/source/user/jobs.rst
@@ -220,14 +220,15 @@
`src/git.example.com/org/project`.
.. var:: projects
- :type: list
+ :type: dict
- A list of all projects prepared by Zuul for the item. It
+ A dictionary of all projects prepared by Zuul for the item. It
includes, at least, the item's own project. It also includes
the projects of any items this item depends on, as well as the
projects that appear in :attr:`job.required-projects`.
- This is a list of dictionaries, with each element consisting of:
+ This is a dictionary of dictionaries. Each value has a key of
+ the `canonical_name`, then each entry consists of:
.. var:: name
@@ -264,6 +265,20 @@
This may be influenced by the branch or tag associated with
the item as well as the job configuration.
+ For example, to access the source directory of a single known
+ project, you might use::
+
+ {{ zuul.projects['git.example.com/org/project'].src_dir }}
+
+ To iterate over the project list, you might write a task
+ something like::
+
+ - name: Sample project iteration
+ debug:
+ msg: "Project {{ item.name }} is at {{ item.src_dir }}
+ with_items: {{ zuul.projects.values() | list }}
+
+
.. var:: _projects
:type: dict
diff --git a/etc/status/public_html/zuul.app.js b/etc/status/public_html/zuul.app.js
index 7ceb2dd..bf90a4d 100644
--- a/etc/status/public_html/zuul.app.js
+++ b/etc/status/public_html/zuul.app.js
@@ -28,8 +28,6 @@
function zuul_build_dom($, container) {
// Build a default-looking DOM
var default_layout = '<div class="container">'
- + '<h1>Zuul Status</h1>'
- + '<p>Real-time status monitor of Zuul, the pipeline manager between Gerrit and Workers.</p>'
+ '<div class="zuul-container" id="zuul-container">'
+ '<div style="display: none;" class="alert" id="zuul_msg"></div>'
+ '<button class="btn pull-right zuul-spinner">updating <span class="glyphicon glyphicon-refresh"></span></button>'
diff --git a/etc/zuul.conf-sample b/etc/zuul.conf-sample
index f0e1765..17092af 100644
--- a/etc/zuul.conf-sample
+++ b/etc/zuul.conf-sample
@@ -38,6 +38,7 @@
listen_address=127.0.0.1
port=9000
static_cache_expiry=0
+;sql_connection_name=mydatabase
[webapp]
listen_address=0.0.0.0
diff --git a/playbooks/zuul-stream/templates/ansible.cfg.j2 b/playbooks/zuul-stream/templates/ansible.cfg.j2
index 24f459e..41ffc0c 100644
--- a/playbooks/zuul-stream/templates/ansible.cfg.j2
+++ b/playbooks/zuul-stream/templates/ansible.cfg.j2
@@ -1,5 +1,5 @@
[defaults]
-hostfile = {{ ansible_user_dir }}/inventory.yaml
+inventory = {{ ansible_user_dir }}/inventory.yaml
gathering = smart
gather_subset = !all
lookup_plugins = {{ ansible_user_dir }}/src/git.openstack.org/openstack-infra/zuul/zuul/ansible/lookup
diff --git a/tests/base.py b/tests/base.py
index 036515d..ea01d20 100755
--- a/tests/base.py
+++ b/tests/base.py
@@ -486,6 +486,29 @@
self.changes[self.change_number] = c
return c
+ def addFakeTag(self, project, branch, tag):
+ path = os.path.join(self.upstream_root, project)
+ repo = git.Repo(path)
+ commit = repo.heads[branch].commit
+ newrev = commit.hexsha
+ ref = 'refs/tags/' + tag
+
+ git.Tag.create(repo, tag, commit)
+
+ event = {
+ "type": "ref-updated",
+ "submitter": {
+ "name": "User Name",
+ },
+ "refUpdate": {
+ "oldRev": 40 * '0',
+ "newRev": newrev,
+ "refName": ref,
+ "project": project,
+ }
+ }
+ return event
+
def getFakeBranchCreatedEvent(self, project, branch):
path = os.path.join(self.upstream_root, project)
repo = git.Repo(path)
@@ -1412,7 +1435,7 @@
host['host_vars']['ansible_connection'] = 'local'
hosts.append(dict(
- name='localhost',
+ name=['localhost'],
host_vars=dict(ansible_connection='local'),
host_keys=[]))
return hosts
@@ -2043,10 +2066,16 @@
FIXTURE_DIR,
self.config.get('scheduler', 'tenant_config')))
self.config.set('scheduler', 'state_dir', self.state_root)
+ self.config.set(
+ 'scheduler', 'command_socket',
+ os.path.join(self.test_root, 'scheduler.socket'))
self.config.set('merger', 'git_dir', self.merger_src_root)
self.config.set('executor', 'git_dir', self.executor_src_root)
self.config.set('executor', 'private_key_file', self.private_key_file)
self.config.set('executor', 'state_dir', self.executor_state_root)
+ self.config.set(
+ 'executor', 'command_socket',
+ os.path.join(self.test_root, 'executor.socket'))
self.statsd = FakeStatsd()
if self.config.has_section('statsd'):
@@ -2233,13 +2262,13 @@
branch='master', tag='init')
if 'job' in item:
if 'run' in item['job']:
- files['%s.yaml' % item['job']['run']] = ''
+ files['%s' % item['job']['run']] = ''
for fn in zuul.configloader.as_list(
item['job'].get('pre-run', [])):
- files['%s.yaml' % fn] = ''
+ files['%s' % fn] = ''
for fn in zuul.configloader.as_list(
item['job'].get('post-run', [])):
- files['%s.yaml' % fn] = ''
+ files['%s' % fn] = ''
root = os.path.join(self.test_root, "config")
if not os.path.exists(root):
diff --git a/tests/fixtures/config/ansible/git/common-config/zuul.yaml b/tests/fixtures/config/ansible/git/common-config/zuul.yaml
index 28bfce1..d0a8f7b 100644
--- a/tests/fixtures/config/ansible/git/common-config/zuul.yaml
+++ b/tests/fixtures/config/ansible/git/common-config/zuul.yaml
@@ -129,10 +129,10 @@
parent: base-urls
name: hello
run: playbooks/hello-post.yaml
- post-run: playbooks/hello-post
+ post-run: playbooks/hello-post.yaml
- job:
parent: python27
name: failpost
run: playbooks/post-broken.yaml
- post-run: playbooks/post-broken
+ post-run: playbooks/post-broken.yaml
diff --git a/tests/fixtures/config/branch-negative/git/org_project/.zuul.yaml b/tests/fixtures/config/branch-negative/git/org_project/.zuul.yaml
new file mode 100644
index 0000000..f02f449
--- /dev/null
+++ b/tests/fixtures/config/branch-negative/git/org_project/.zuul.yaml
@@ -0,0 +1,10 @@
+- job:
+ name: test-job
+ run: playbooks/test-job.yaml
+
+- project:
+ name: org/project
+ check:
+ jobs:
+ - test-job:
+ branches: ^(?!stable)
diff --git a/tests/fixtures/config/branch-negative/git/org_project/README b/tests/fixtures/config/branch-negative/git/org_project/README
new file mode 100644
index 0000000..9daeafb
--- /dev/null
+++ b/tests/fixtures/config/branch-negative/git/org_project/README
@@ -0,0 +1 @@
+test
diff --git a/tests/fixtures/config/branch-negative/git/org_project/playbooks/test-job.yaml b/tests/fixtures/config/branch-negative/git/org_project/playbooks/test-job.yaml
new file mode 100644
index 0000000..f679dce
--- /dev/null
+++ b/tests/fixtures/config/branch-negative/git/org_project/playbooks/test-job.yaml
@@ -0,0 +1,2 @@
+- hosts: all
+ tasks: []
diff --git a/tests/fixtures/config/branch-negative/git/project-config/zuul.yaml b/tests/fixtures/config/branch-negative/git/project-config/zuul.yaml
new file mode 100644
index 0000000..dc4a182
--- /dev/null
+++ b/tests/fixtures/config/branch-negative/git/project-config/zuul.yaml
@@ -0,0 +1,26 @@
+- pipeline:
+ name: check
+ manager: independent
+ trigger:
+ gerrit:
+ - event: patchset-created
+ success:
+ gerrit:
+ Verified: 1
+ failure:
+ gerrit:
+ Verified: -1
+
+- job:
+ name: base
+ parent: null
+
+- project:
+ name: project-config
+ check:
+ jobs: []
+
+- project:
+ name: org/project
+ check:
+ jobs: []
diff --git a/tests/fixtures/config/branch-negative/main.yaml b/tests/fixtures/config/branch-negative/main.yaml
new file mode 100644
index 0000000..0ac232f
--- /dev/null
+++ b/tests/fixtures/config/branch-negative/main.yaml
@@ -0,0 +1,8 @@
+- tenant:
+ name: tenant-one
+ source:
+ gerrit:
+ config-projects:
+ - project-config
+ untrusted-projects:
+ - org/project
diff --git a/tests/fixtures/config/branch-tag/git/org_project/.zuul.yaml b/tests/fixtures/config/branch-tag/git/org_project/.zuul.yaml
new file mode 100644
index 0000000..acbba6c
--- /dev/null
+++ b/tests/fixtures/config/branch-tag/git/org_project/.zuul.yaml
@@ -0,0 +1,9 @@
+- job:
+ name: test-job
+ run: playbooks/test-job.yaml
+
+- project:
+ name: org/project
+ tag:
+ jobs:
+ - test-job
diff --git a/tests/fixtures/config/branch-tag/git/org_project/README b/tests/fixtures/config/branch-tag/git/org_project/README
new file mode 100644
index 0000000..9daeafb
--- /dev/null
+++ b/tests/fixtures/config/branch-tag/git/org_project/README
@@ -0,0 +1 @@
+test
diff --git a/tests/fixtures/config/branch-tag/git/org_project/playbooks/test-job.yaml b/tests/fixtures/config/branch-tag/git/org_project/playbooks/test-job.yaml
new file mode 100644
index 0000000..f679dce
--- /dev/null
+++ b/tests/fixtures/config/branch-tag/git/org_project/playbooks/test-job.yaml
@@ -0,0 +1,2 @@
+- hosts: all
+ tasks: []
diff --git a/tests/fixtures/config/branch-tag/git/project-config/zuul.yaml b/tests/fixtures/config/branch-tag/git/project-config/zuul.yaml
new file mode 100644
index 0000000..0ae6396
--- /dev/null
+++ b/tests/fixtures/config/branch-tag/git/project-config/zuul.yaml
@@ -0,0 +1,21 @@
+- pipeline:
+ name: tag
+ manager: independent
+ trigger:
+ gerrit:
+ - event: ref-updated
+ ref: ^refs/tags/.*$
+
+- job:
+ name: base
+ parent: null
+
+- project:
+ name: project-config
+ tag:
+ jobs: []
+
+- project:
+ name: org/project
+ tag:
+ jobs: []
diff --git a/tests/fixtures/config/branch-tag/main.yaml b/tests/fixtures/config/branch-tag/main.yaml
new file mode 100644
index 0000000..0ac232f
--- /dev/null
+++ b/tests/fixtures/config/branch-tag/main.yaml
@@ -0,0 +1,8 @@
+- tenant:
+ name: tenant-one
+ source:
+ gerrit:
+ config-projects:
+ - project-config
+ untrusted-projects:
+ - org/project
diff --git a/tests/fixtures/config/branch-variants/git/project-config/zuul.yaml b/tests/fixtures/config/branch-variants/git/project-config/zuul.yaml
index 161e5a1..48da2d4 100644
--- a/tests/fixtures/config/branch-variants/git/project-config/zuul.yaml
+++ b/tests/fixtures/config/branch-variants/git/project-config/zuul.yaml
@@ -34,10 +34,10 @@
- job:
name: base
parent: null
- pre-run: playbooks/base/pre
+ pre-run: playbooks/base/pre.yaml
post-run:
- - playbooks/base/post-ssh
- - playbooks/base/post-logs
+ - playbooks/base/post-ssh.yaml
+ - playbooks/base/post-logs.yaml
- project:
name: project-config
diff --git a/tests/fixtures/config/branch-variants/git/puppet-integration/.zuul.yaml b/tests/fixtures/config/branch-variants/git/puppet-integration/.zuul.yaml
index 322927f..7e9cbc3 100644
--- a/tests/fixtures/config/branch-variants/git/puppet-integration/.zuul.yaml
+++ b/tests/fixtures/config/branch-variants/git/puppet-integration/.zuul.yaml
@@ -1,16 +1,16 @@
- job:
name: puppet-base
- pre-run: playbooks/prepare-node-common
+ pre-run: playbooks/prepare-node-common.yaml
- job:
name: puppet-module-base
parent: puppet-base
- pre-run: playbooks/prepare-node-unit
+ pre-run: playbooks/prepare-node-unit.yaml
- job:
name: puppet-lint
parent: puppet-module-base
- run: playbooks/run-lint
+ run: playbooks/run-lint.yaml
tags:
- master
diff --git a/tests/fixtures/config/branch-variants/git/puppet-integration/stable.zuul.yaml b/tests/fixtures/config/branch-variants/git/puppet-integration/stable.zuul.yaml
index 4701b80..74704a0 100644
--- a/tests/fixtures/config/branch-variants/git/puppet-integration/stable.zuul.yaml
+++ b/tests/fixtures/config/branch-variants/git/puppet-integration/stable.zuul.yaml
@@ -1,16 +1,16 @@
- job:
name: puppet-base
- pre-run: playbooks/prepare-node-common
+ pre-run: playbooks/prepare-node-common.yaml
- job:
name: puppet-module-base
parent: puppet-base
- pre-run: playbooks/prepare-node-unit
+ pre-run: playbooks/prepare-node-unit.yaml
- job:
name: puppet-lint
parent: puppet-module-base
- run: playbooks/run-lint
+ run: playbooks/run-lint.yaml
tags:
- stable
diff --git a/tests/fixtures/config/inventory/git/common-config/zuul.yaml b/tests/fixtures/config/inventory/git/common-config/zuul.yaml
index 74ddf2d..ad530a7 100644
--- a/tests/fixtures/config/inventory/git/common-config/zuul.yaml
+++ b/tests/fixtures/config/inventory/git/common-config/zuul.yaml
@@ -52,6 +52,16 @@
run: playbooks/single-inventory.yaml
- job:
+ name: single-inventory-list
+ nodeset:
+ nodes:
+ - name:
+ - compute
+ - controller
+ label: ubuntu-xenial
+ run: playbooks/single-inventory.yaml
+
+- job:
name: group-inventory
nodeset: nodeset1
run: playbooks/group-inventory.yaml
diff --git a/tests/fixtures/config/inventory/git/org_project/.zuul.yaml b/tests/fixtures/config/inventory/git/org_project/.zuul.yaml
index 1a8bf5d..6a29049 100644
--- a/tests/fixtures/config/inventory/git/org_project/.zuul.yaml
+++ b/tests/fixtures/config/inventory/git/org_project/.zuul.yaml
@@ -3,5 +3,6 @@
check:
jobs:
- single-inventory
+ - single-inventory-list
- group-inventory
- hostvars-inventory
diff --git a/tests/fixtures/config/job-output/git/common-config/zuul.yaml b/tests/fixtures/config/job-output/git/common-config/zuul.yaml
index 4df0020..9373038 100644
--- a/tests/fixtures/config/job-output/git/common-config/zuul.yaml
+++ b/tests/fixtures/config/job-output/git/common-config/zuul.yaml
@@ -23,8 +23,8 @@
- job:
name: job-output-failure
- run: playbooks/job-output
- post-run: playbooks/job-output-failure-post
+ run: playbooks/job-output.yaml
+ post-run: playbooks/job-output-failure-post.yaml
- project:
name: org/project
diff --git a/tests/fixtures/config/post-playbook/git/common-config/zuul.yaml b/tests/fixtures/config/post-playbook/git/common-config/zuul.yaml
index 16d7dee..b00d4c2 100644
--- a/tests/fixtures/config/post-playbook/git/common-config/zuul.yaml
+++ b/tests/fixtures/config/post-playbook/git/common-config/zuul.yaml
@@ -18,8 +18,8 @@
- job:
name: python27
- pre-run: playbooks/pre
- post-run: playbooks/post
+ pre-run: playbooks/pre.yaml
+ post-run: playbooks/post.yaml
vars:
waitpath: '{{zuul._test.test_root}}/{{zuul.build}}/test_wait'
run: playbooks/python27.yaml
diff --git a/tests/fixtures/config/pre-playbook/git/common-config/zuul.yaml b/tests/fixtures/config/pre-playbook/git/common-config/zuul.yaml
index 7817745..16f48b1 100644
--- a/tests/fixtures/config/pre-playbook/git/common-config/zuul.yaml
+++ b/tests/fixtures/config/pre-playbook/git/common-config/zuul.yaml
@@ -18,6 +18,6 @@
- job:
name: python27
- pre-run: playbooks/pre
- post-run: playbooks/post
+ pre-run: playbooks/pre.yaml
+ post-run: playbooks/post.yaml
run: playbooks/python27.yaml
diff --git a/tests/fixtures/config/tenant-parser/git/common-config/zuul.yaml b/tests/fixtures/config/tenant-parser/git/common-config/zuul.yaml
index e21f967..a28ef54 100644
--- a/tests/fixtures/config/tenant-parser/git/common-config/zuul.yaml
+++ b/tests/fixtures/config/tenant-parser/git/common-config/zuul.yaml
@@ -18,8 +18,10 @@
- job:
name: common-config-job
+# Use the canonical name here. This should be merged with the org/project1 in
+# the other repo.
- project:
- name: org/project1
+ name: review.example.com/org/project1
check:
jobs:
- common-config-job
diff --git a/tests/unit/test_executor.py b/tests/unit/test_executor.py
index 5d27663..474859d 100755
--- a/tests/unit/test_executor.py
+++ b/tests/unit/test_executor.py
@@ -416,15 +416,15 @@
job)
def test_getHostList_host_keys(self):
- # Test without ssh_port set
+ # Test without connection_port set
node = {'name': 'fake-host',
'host_keys': ['fake-host-key'],
'interface_ip': 'localhost'}
keys = self.test_job.getHostList({'nodes': [node]})[0]['host_keys']
self.assertEqual(keys[0], 'localhost fake-host-key')
- # Test with custom ssh_port set
- node['ssh_port'] = 22022
+ # Test with custom connection_port set
+ node['connection_port'] = 22022
keys = self.test_job.getHostList({'nodes': [node]})[0]['host_keys']
self.assertEqual(keys[0], '[localhost]:22022 fake-host-key')
diff --git a/tests/unit/test_inventory.py b/tests/unit/test_inventory.py
index 04dcb05..1c41f5f 100644
--- a/tests/unit/test_inventory.py
+++ b/tests/unit/test_inventory.py
@@ -57,6 +57,26 @@
self.executor_server.release()
self.waitUntilSettled()
+ def test_single_inventory_list(self):
+
+ inventory = self._get_build_inventory('single-inventory-list')
+
+ all_nodes = ('compute', 'controller')
+ self.assertIn('all', inventory)
+ self.assertIn('hosts', inventory['all'])
+ self.assertIn('vars', inventory['all'])
+ for node_name in all_nodes:
+ self.assertIn(node_name, inventory['all']['hosts'])
+ self.assertIn('zuul', inventory['all']['vars'])
+ z_vars = inventory['all']['vars']['zuul']
+ self.assertIn('executor', z_vars)
+ self.assertIn('src_root', z_vars['executor'])
+ self.assertIn('job', z_vars)
+ self.assertEqual(z_vars['job'], 'single-inventory-list')
+
+ self.executor_server.release()
+ self.waitUntilSettled()
+
def test_group_inventory(self):
inventory = self._get_build_inventory('group-inventory')
diff --git a/tests/unit/test_log_streamer.py b/tests/unit/test_log_streamer.py
index c808540..27368e3 100644
--- a/tests/unit/test_log_streamer.py
+++ b/tests/unit/test_log_streamer.py
@@ -158,7 +158,7 @@
def runWSClient(self, build_uuid, event):
async def client(loop, build_uuid, event):
- uri = 'http://[::1]:9000/console-stream'
+ uri = 'http://[::1]:9000/tenant-one/console-stream'
try:
session = aiohttp.ClientSession(loop=loop)
async with session.ws_connect(uri) as ws:
diff --git a/tests/unit/test_nodepool.py b/tests/unit/test_nodepool.py
index d3f9ddb..aa0f082 100644
--- a/tests/unit/test_nodepool.py
+++ b/tests/unit/test_nodepool.py
@@ -67,8 +67,8 @@
# Test a simple node request
nodeset = model.NodeSet()
- nodeset.addNode(model.Node('controller', 'ubuntu-xenial'))
- nodeset.addNode(model.Node('compute', 'ubuntu-xenial'))
+ nodeset.addNode(model.Node(['controller', 'foo'], 'ubuntu-xenial'))
+ nodeset.addNode(model.Node(['compute'], 'ubuntu-xenial'))
job = model.Job('testjob')
job.nodeset = nodeset
request = self.nodepool.requestNodes(None, job)
@@ -99,8 +99,8 @@
# Test that node requests are re-submitted after disconnect
nodeset = model.NodeSet()
- nodeset.addNode(model.Node('controller', 'ubuntu-xenial'))
- nodeset.addNode(model.Node('compute', 'ubuntu-xenial'))
+ nodeset.addNode(model.Node(['controller'], 'ubuntu-xenial'))
+ nodeset.addNode(model.Node(['compute'], 'ubuntu-xenial'))
job = model.Job('testjob')
job.nodeset = nodeset
self.fake_nodepool.paused = True
@@ -116,8 +116,8 @@
# Test that node requests can be canceled
nodeset = model.NodeSet()
- nodeset.addNode(model.Node('controller', 'ubuntu-xenial'))
- nodeset.addNode(model.Node('compute', 'ubuntu-xenial'))
+ nodeset.addNode(model.Node(['controller'], 'ubuntu-xenial'))
+ nodeset.addNode(model.Node(['compute'], 'ubuntu-xenial'))
job = model.Job('testjob')
job.nodeset = nodeset
self.fake_nodepool.paused = True
@@ -131,8 +131,8 @@
# Test that a resubmitted request would not lock nodes
nodeset = model.NodeSet()
- nodeset.addNode(model.Node('controller', 'ubuntu-xenial'))
- nodeset.addNode(model.Node('compute', 'ubuntu-xenial'))
+ nodeset.addNode(model.Node(['controller'], 'ubuntu-xenial'))
+ nodeset.addNode(model.Node(['compute'], 'ubuntu-xenial'))
job = model.Job('testjob')
job.nodeset = nodeset
request = self.nodepool.requestNodes(None, job)
@@ -152,8 +152,8 @@
# Test that a lost request would not lock nodes
nodeset = model.NodeSet()
- nodeset.addNode(model.Node('controller', 'ubuntu-xenial'))
- nodeset.addNode(model.Node('compute', 'ubuntu-xenial'))
+ nodeset.addNode(model.Node(['controller'], 'ubuntu-xenial'))
+ nodeset.addNode(model.Node(['compute'], 'ubuntu-xenial'))
job = model.Job('testjob')
job.nodeset = nodeset
request = self.nodepool.requestNodes(None, job)
diff --git a/tests/unit/test_scheduler.py b/tests/unit/test_scheduler.py
index cad557e..aacc81e 100755
--- a/tests/unit/test_scheduler.py
+++ b/tests/unit/test_scheduler.py
@@ -2581,7 +2581,7 @@
self.assertEqual('project-merge', status_jobs[0]['name'])
# TODO(mordred) pull uuids from self.builds
self.assertEqual(
- 'static/stream.html?uuid={uuid}&logfile=console.log'.format(
+ 'stream.html?uuid={uuid}&logfile=console.log'.format(
uuid=status_jobs[0]['uuid']),
status_jobs[0]['url'])
self.assertEqual(
@@ -2597,7 +2597,7 @@
status_jobs[0]['report_url'])
self.assertEqual('project-test1', status_jobs[1]['name'])
self.assertEqual(
- 'static/stream.html?uuid={uuid}&logfile=console.log'.format(
+ 'stream.html?uuid={uuid}&logfile=console.log'.format(
uuid=status_jobs[1]['uuid']),
status_jobs[1]['url'])
self.assertEqual(
@@ -2613,7 +2613,7 @@
self.assertEqual('project-test2', status_jobs[2]['name'])
self.assertEqual(
- 'static/stream.html?uuid={uuid}&logfile=console.log'.format(
+ 'stream.html?uuid={uuid}&logfile=console.log'.format(
uuid=status_jobs[2]['uuid']),
status_jobs[2]['url'])
self.assertEqual(
@@ -4210,7 +4210,7 @@
self.assertEqual('gate', job['pipeline'])
self.assertEqual(False, job['retry'])
self.assertEqual(
- 'static/stream.html?uuid={uuid}&logfile=console.log'
+ 'stream.html?uuid={uuid}&logfile=console.log'
.format(uuid=job['uuid']), job['url'])
self.assertEqual(
'finger://{hostname}/{uuid}'.format(
diff --git a/tests/unit/test_v3.py b/tests/unit/test_v3.py
index e2da808..b9c9b32 100755
--- a/tests/unit/test_v3.py
+++ b/tests/unit/test_v3.py
@@ -157,6 +157,39 @@
self.assertIn('Unable to modify final job', A.messages[0])
+class TestBranchTag(ZuulTestCase):
+ tenant_config_file = 'config/branch-tag/main.yaml'
+
+ def test_negative_branch_match(self):
+ # Test that a negative branch matcher works with implied branches.
+ event = self.fake_gerrit.addFakeTag('org/project', 'master', 'foo')
+ self.fake_gerrit.addEvent(event)
+ self.waitUntilSettled()
+ self.assertHistory([
+ dict(name='test-job', result='SUCCESS', ref='refs/tags/foo')])
+
+
+class TestBranchNegative(ZuulTestCase):
+ tenant_config_file = 'config/branch-negative/main.yaml'
+
+ def test_negative_branch_match(self):
+ # Test that a negative branch matcher works with implied branches.
+ self.create_branch('org/project', 'stable/pike')
+ self.fake_gerrit.addEvent(
+ self.fake_gerrit.getFakeBranchCreatedEvent(
+ 'org/project', 'stable/pike'))
+ self.waitUntilSettled()
+
+ A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A')
+ self.fake_gerrit.addEvent(A.getPatchsetCreatedEvent(1))
+ self.waitUntilSettled()
+ B = self.fake_gerrit.addFakeChange('org/project', 'stable/pike', 'A')
+ self.fake_gerrit.addEvent(B.getPatchsetCreatedEvent(1))
+ self.waitUntilSettled()
+ self.assertHistory([
+ dict(name='test-job', result='SUCCESS', changes='1,1')])
+
+
class TestBranchTemplates(ZuulTestCase):
tenant_config_file = 'config/branch-templates/main.yaml'
@@ -1902,8 +1935,8 @@
name: parent
roles:
- zuul: bare-role
- pre-run: playbooks/parent-pre
- post-run: playbooks/parent-post
+ pre-run: playbooks/parent-pre.yaml
+ post-run: playbooks/parent-post.yaml
- job:
name: project-test
diff --git a/tools/test-logs.sh b/tools/test-logs.sh
index bf2147d..a514dd8 100644
--- a/tools/test-logs.sh
+++ b/tools/test-logs.sh
@@ -42,7 +42,7 @@
cat >$WORK_DIR/ansible.cfg <<EOF
[defaults]
-hostfile = $INVENTORY
+inventory = $INVENTORY
gathering = smart
gather_subset = !all
fact_caching = jsonfile
diff --git a/zuul/ansible/callback/zuul_stream.py b/zuul/ansible/callback/zuul_stream.py
index 8845e9b..df28a57 100644
--- a/zuul/ansible/callback/zuul_stream.py
+++ b/zuul/ansible/callback/zuul_stream.py
@@ -150,7 +150,7 @@
buff += more
if buff:
self._log_streamline(
- host, line.decode("utf-8", "backslashreplace"))
+ host, buff.decode("utf-8", "backslashreplace"))
def _log_streamline(self, host, line):
if "[Zuul] Task exit code" in line:
diff --git a/zuul/ansible/library/zuul_return.py b/zuul/ansible/library/zuul_return.py
index 9f3332b..4935226 100644
--- a/zuul/ansible/library/zuul_return.py
+++ b/zuul/ansible/library/zuul_return.py
@@ -63,7 +63,7 @@
path = os.path.join(os.environ['ZUUL_JOBDIR'], 'work',
'results.json')
set_value(path, p['data'], p['file'])
- module.exit_json(changed=True, e=os.environ)
+ module.exit_json(changed=True, e=os.environ.copy())
from ansible.module_utils.basic import * # noqa
from ansible.module_utils.basic import AnsibleModule
diff --git a/zuul/change_matcher.py b/zuul/change_matcher.py
index 7f6673d..eb12f9b 100644
--- a/zuul/change_matcher.py
+++ b/zuul/change_matcher.py
@@ -69,6 +69,20 @@
return False
+class ImpliedBranchMatcher(AbstractChangeMatcher):
+ """
+ A branch matcher that only considers branch refs, and always
+ succeeds on other types (e.g., tags).
+ """
+
+ def matches(self, change):
+ if hasattr(change, 'branch'):
+ if self.regex.match(change.branch):
+ return True
+ return False
+ return True
+
+
class FileMatcher(AbstractChangeMatcher):
def matches(self, change):
diff --git a/zuul/cmd/__init__.py b/zuul/cmd/__init__.py
index e150f9c..236fd9f 100755
--- a/zuul/cmd/__init__.py
+++ b/zuul/cmd/__init__.py
@@ -23,6 +23,7 @@
import logging.config
import os
import signal
+import socket
import sys
import traceback
import threading
@@ -184,3 +185,12 @@
pass
with daemon.DaemonContext(pidfile=pid):
self.run()
+
+ def send_command(self, cmd):
+ command_socket = get_default(
+ self.config, self.app_name, 'command_socket',
+ '/var/lib/zuul/%s.socket' % self.app_name)
+ s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+ s.connect(command_socket)
+ cmd = '%s\n' % cmd
+ s.sendall(cmd.encode('utf8'))
diff --git a/zuul/cmd/executor.py b/zuul/cmd/executor.py
index aef8c95..ade9715 100755
--- a/zuul/cmd/executor.py
+++ b/zuul/cmd/executor.py
@@ -18,7 +18,6 @@
import logging
import os
import pwd
-import socket
import sys
import signal
import tempfile
@@ -52,15 +51,6 @@
if self.args.command:
self.args.nodaemon = True
- def send_command(self, cmd):
- state_dir = get_default(self.config, 'executor', 'state_dir',
- '/var/lib/zuul', expand_user=True)
- path = os.path.join(state_dir, 'executor.socket')
- s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
- s.connect(path)
- cmd = '%s\n' % cmd
- s.sendall(cmd.encode('utf8'))
-
def exit_handler(self):
self.executor.stop()
self.executor.join()
diff --git a/zuul/cmd/merger.py b/zuul/cmd/merger.py
index 56b6b44..7db1bee 100755
--- a/zuul/cmd/merger.py
+++ b/zuul/cmd/merger.py
@@ -15,8 +15,10 @@
# under the License.
import signal
+import sys
import zuul.cmd
+import zuul.merger.server
# No zuul imports here because they pull in paramiko which must not be
# imported until after the daemonization.
@@ -28,14 +30,28 @@
app_name = 'merger'
app_description = 'A standalone Zuul merger.'
- def exit_handler(self, signum, frame):
- signal.signal(signal.SIGUSR1, signal.SIG_IGN)
+ def createParser(self):
+ parser = super(Merger, self).createParser()
+ parser.add_argument('command',
+ choices=zuul.merger.server.COMMANDS,
+ nargs='?')
+ return parser
+
+ def parseArguments(self, args=None):
+ super(Merger, self).parseArguments()
+ if self.args.command:
+ self.args.nodaemon = True
+
+ def exit_handler(self):
self.merger.stop()
self.merger.join()
def run(self):
# See comment at top of file about zuul imports
import zuul.merger.server
+ if self.args.command in zuul.merger.server.COMMANDS:
+ self.send_command(self.args.command)
+ sys.exit(0)
self.configure_connections(source_only=True)
@@ -45,14 +61,18 @@
self.connections)
self.merger.start()
- signal.signal(signal.SIGUSR1, self.exit_handler)
signal.signal(signal.SIGUSR2, zuul.cmd.stack_dump_handler)
- while True:
- try:
- signal.pause()
- except KeyboardInterrupt:
- print("Ctrl + C: asking merger to exit nicely...\n")
- self.exit_handler(signal.SIGINT, None)
+
+ if self.args.nodaemon:
+ while True:
+ try:
+ signal.pause()
+ except KeyboardInterrupt:
+ print("Ctrl + C: asking merger to exit nicely...\n")
+ self.exit_handler()
+ sys.exit(0)
+ else:
+ self.merger.join()
def main():
diff --git a/zuul/cmd/scheduler.py b/zuul/cmd/scheduler.py
index 539d55b..7722d6e 100755
--- a/zuul/cmd/scheduler.py
+++ b/zuul/cmd/scheduler.py
@@ -22,6 +22,7 @@
import zuul.cmd
from zuul.lib.config import get_default
from zuul.lib.statsd import get_statsd_config
+import zuul.scheduler
# No zuul imports here because they pull in paramiko which must not be
# imported until after the daemonization.
@@ -37,6 +38,18 @@
super(Scheduler, self).__init__()
self.gear_server_pid = None
+ def createParser(self):
+ parser = super(Scheduler, self).createParser()
+ parser.add_argument('command',
+ choices=zuul.scheduler.COMMANDS,
+ nargs='?')
+ return parser
+
+ def parseArguments(self, args=None):
+ super(Scheduler, self).parseArguments()
+ if self.args.command:
+ self.args.nodaemon = True
+
def reconfigure_handler(self, signum, frame):
signal.signal(signal.SIGHUP, signal.SIG_IGN)
self.log.debug("Reconfiguration triggered")
@@ -48,8 +61,7 @@
self.log.exception("Reconfiguration failed:")
signal.signal(signal.SIGHUP, self.reconfigure_handler)
- def exit_handler(self, signum, frame):
- signal.signal(signal.SIGUSR1, signal.SIG_IGN)
+ def exit_handler(self):
self.sched.exit()
self.sched.join()
self.stop_gear_server()
@@ -104,6 +116,10 @@
def run(self):
# See comment at top of file about zuul imports
import zuul.scheduler
+ if self.args.command in zuul.scheduler.COMMANDS:
+ self.send_command(self.args.command)
+ sys.exit(0)
+ # See comment at top of file about zuul imports
import zuul.executor.client
import zuul.merger.client
import zuul.nodepool
@@ -162,14 +178,17 @@
webapp.start()
signal.signal(signal.SIGHUP, self.reconfigure_handler)
- signal.signal(signal.SIGUSR1, self.exit_handler)
- signal.signal(signal.SIGTERM, self.term_handler)
- while True:
- try:
- signal.pause()
- except KeyboardInterrupt:
- print("Ctrl + C: asking scheduler to exit nicely...\n")
- self.exit_handler(signal.SIGINT, None)
+
+ if self.args.nodaemon:
+ while True:
+ try:
+ signal.pause()
+ except KeyboardInterrupt:
+ print("Ctrl + C: asking scheduler to exit nicely...\n")
+ self.exit_handler()
+ sys.exit(0)
+ else:
+ self.sched.join()
def main():
diff --git a/zuul/cmd/web.py b/zuul/cmd/web.py
index 6e5489f..ad3062f 100755
--- a/zuul/cmd/web.py
+++ b/zuul/cmd/web.py
@@ -22,6 +22,7 @@
import zuul.cmd
import zuul.web
+from zuul.driver.sql import sqlconnection
from zuul.lib.config import get_default
@@ -48,6 +49,30 @@
params['ssl_cert'] = get_default(self.config, 'gearman', 'ssl_cert')
params['ssl_ca'] = get_default(self.config, 'gearman', 'ssl_ca')
+ sql_conn_name = get_default(self.config, 'web',
+ 'sql_connection_name')
+ sql_conn = None
+ if sql_conn_name:
+ # we want a specific sql connection
+ sql_conn = self.connections.connections.get(sql_conn_name)
+ if not sql_conn:
+ self.log.error("Couldn't find sql connection '%s'" %
+ sql_conn_name)
+ sys.exit(1)
+ else:
+ # look for any sql connection
+ connections = [c for c in self.connections.connections.values()
+ if isinstance(c, sqlconnection.SQLConnection)]
+ if len(connections) > 1:
+ self.log.error("Multiple sql connection found, "
+ "set the sql_connection_name option "
+ "in zuul.conf [web] section")
+ sys.exit(1)
+ if connections:
+ # use this sql connection by default
+ sql_conn = connections[0]
+ params['sql_connection'] = sql_conn
+
try:
self.web = zuul.web.ZuulWeb(**params)
except Exception as e:
@@ -79,6 +104,8 @@
self.setup_logging('web', 'log_config')
self.log = logging.getLogger("zuul.WebServer")
+ self.configure_connections()
+
try:
self._run()
except Exception:
diff --git a/zuul/configloader.py b/zuul/configloader.py
index 99f10f6..fb1695c 100644
--- a/zuul/configloader.py
+++ b/zuul/configloader.py
@@ -340,7 +340,7 @@
class NodeSetParser(object):
@staticmethod
def getSchema(anonymous=False):
- node = {vs.Required('name'): str,
+ node = {vs.Required('name'): to_list(str),
vs.Required('label'): str,
}
@@ -365,11 +365,13 @@
node_names = set()
group_names = set()
for conf_node in as_list(conf['nodes']):
- if conf_node['name'] in node_names:
- raise DuplicateNodeError(conf['name'], conf_node['name'])
- node = model.Node(conf_node['name'], conf_node['label'])
+ for name in as_list(conf_node['name']):
+ if name in node_names:
+ raise DuplicateNodeError(name, conf_node['name'])
+ node = model.Node(as_list(conf_node['name']), conf_node['label'])
ns.addNode(node)
- node_names.add(conf_node['name'])
+ for name in as_list(conf_node['name']):
+ node_names.add(name)
for conf_group in as_list(conf.get('groups', [])):
for node_name in as_list(conf_group['nodes']):
if node_name not in node_names:
@@ -517,6 +519,7 @@
# "job.run.append(...)").
job = model.Job(name)
+ job.description = conf.get('description')
job.source_context = conf.get('_source_context')
job.source_line = conf.get('_start_mark').line + 1
@@ -1161,8 +1164,8 @@
tenant.config_projects,
tenant.untrusted_projects,
cached, tenant)
- unparsed_config.extend(tenant.config_projects_config)
- unparsed_config.extend(tenant.untrusted_projects_config)
+ unparsed_config.extend(tenant.config_projects_config, tenant=tenant)
+ unparsed_config.extend(tenant.untrusted_projects_config, tenant=tenant)
tenant.layout = TenantParser._parseLayout(base, tenant,
unparsed_config,
scheduler,
diff --git a/zuul/executor/client.py b/zuul/executor/client.py
index a8b94f0..06c2087 100644
--- a/zuul/executor/client.py
+++ b/zuul/executor/client.py
@@ -180,8 +180,7 @@
if (hasattr(item.change, 'newrev') and item.change.newrev
and item.change.newrev != '0' * 40):
zuul_params['newrev'] = item.change.newrev
- zuul_params['projects'] = [] # Set below
- zuul_params['_projects'] = {} # transitional to convert to dict
+ zuul_params['projects'] = {} # Set below
zuul_params['items'] = dependent_changes
params = dict()
@@ -253,7 +252,7 @@
params['projects'].append(make_project_dict(project))
projects.add(project)
for p in projects:
- zuul_params['_projects'][p.canonical_name] = (dict(
+ zuul_params['projects'][p.canonical_name] = (dict(
name=p.name,
short_name=p.name.split('/')[-1],
# Duplicate this into the dict too, so that iterating
@@ -265,12 +264,10 @@
))
# We are transitioning "projects" from a list to a dict
# indexed by canonical name, as it is much easier to access
- # values in ansible. Existing callers are converted to
- # "_projects", then once "projects" is unused we switch it,
- # then convert callers back. Finally when "_projects" is
- # unused it will be removed.
- for cn, p in zuul_params['_projects'].items():
- zuul_params['projects'].append(p)
+ # values in ansible. Existing callers have been converted to
+ # "_projects" and "projects" is swapped; we will convert users
+ # back to "projects" and remove this soon.
+ zuul_params['_projects'] = zuul_params['projects']
build = Build(job, uuid)
build.parameters = params
diff --git a/zuul/executor/server.py b/zuul/executor/server.py
index 016d0e6..7a93f89 100644
--- a/zuul/executor/server.py
+++ b/zuul/executor/server.py
@@ -497,7 +497,8 @@
hosts = {}
for node in nodes:
- hosts[node['name']] = node['host_vars']
+ for name in node['name']:
+ hosts[name] = node['host_vars']
inventory = {
'all': {
@@ -910,7 +911,7 @@
# results in the wrong thing being in interface_ip
# TODO(jeblair): Move this notice to the docs.
ip = node.get('interface_ip')
- port = node.get('ssh_port', 22)
+ port = node.get('connection_port', node.get('ssh_port', 22))
host_vars = dict(
ansible_host=ip,
ansible_user=self.executor_server.default_username,
@@ -958,13 +959,11 @@
"non-trusted repo." % (entry, path))
def findPlaybook(self, path, trusted=False):
- for ext in ['', '.yaml', '.yml']:
- fn = path + ext
- if os.path.exists(fn):
- if not trusted:
- playbook_dir = os.path.dirname(os.path.abspath(fn))
- self._blockPluginDirs(playbook_dir)
- return fn
+ if os.path.exists(path):
+ if not trusted:
+ playbook_dir = os.path.dirname(os.path.abspath(path))
+ self._blockPluginDirs(playbook_dir)
+ return path
raise ExecutorError("Unable to find playbook %s" % path)
def preparePlaybooks(self, args):
@@ -1187,7 +1186,7 @@
callback_path = self.executor_server.callback_dir
with open(jobdir_playbook.ansible_config, 'w') as config:
config.write('[defaults]\n')
- config.write('hostfile = %s\n' % self.jobdir.inventory)
+ config.write('inventory = %s\n' % self.jobdir.inventory)
config.write('local_tmp = %s/local_tmp\n' %
self.jobdir.ansible_cache_root)
config.write('retry_files_enabled = False\n')
@@ -1607,10 +1606,13 @@
self.merger = self._getMerger(self.merge_root)
self.update_queue = DeduplicateQueue()
+ command_socket = get_default(
+ self.config, 'executor', 'command_socket',
+ '/var/lib/zuul/executor.socket')
+ self.command_socket = commandsocket.CommandSocket(command_socket)
+
state_dir = get_default(self.config, 'executor', 'state_dir',
'/var/lib/zuul', expand_user=True)
- path = os.path.join(state_dir, 'executor.socket')
- self.command_socket = commandsocket.CommandSocket(path)
ansible_dir = os.path.join(state_dir, 'ansible')
self.ansible_dir = ansible_dir
if os.path.exists(ansible_dir):
diff --git a/zuul/manager/__init__.py b/zuul/manager/__init__.py
index 6c72c2d..d205afc 100644
--- a/zuul/manager/__init__.py
+++ b/zuul/manager/__init__.py
@@ -853,20 +853,22 @@
if dt:
self.sched.statsd.timing(key + '.resident_time', dt)
self.sched.statsd.incr(key + '.total_changes')
-
- hostname = (item.change.project.canonical_hostname.
- replace('.', '_'))
- projectname = (item.change.project.name.
- replace('.', '_').replace('/', '.'))
- projectname = projectname.replace('.', '_').replace('/', '.')
- branchname = item.change.branch.replace('.', '_').replace('/', '.')
- # stats.timers.zuul.tenant.<tenant>.pipeline.<pipeline>.
- # project.<host>.<project>.<branch>.resident_time
- # stats_counts.zuul.tenant.<tenant>.pipeline.<pipeline>.
- # project.<host>.<project>.<branch>.total_changes
- key += '.project.%s.%s.%s' % (hostname, projectname, branchname)
- if dt:
- self.sched.statsd.timing(key + '.resident_time', dt)
- self.sched.statsd.incr(key + '.total_changes')
+ if hasattr(item.change, 'branch'):
+ hostname = (item.change.project.canonical_hostname.
+ replace('.', '_'))
+ projectname = (item.change.project.name.
+ replace('.', '_').replace('/', '.'))
+ projectname = projectname.replace('.', '_').replace('/', '.')
+ branchname = item.change.branch.replace('.', '_').replace(
+ '/', '.')
+ # stats.timers.zuul.tenant.<tenant>.pipeline.<pipeline>.
+ # project.<host>.<project>.<branch>.resident_time
+ # stats_counts.zuul.tenant.<tenant>.pipeline.<pipeline>.
+ # project.<host>.<project>.<branch>.total_changes
+ key += '.project.%s.%s.%s' % (hostname, projectname,
+ branchname)
+ if dt:
+ self.sched.statsd.timing(key + '.resident_time', dt)
+ self.sched.statsd.incr(key + '.total_changes')
except Exception:
self.log.exception("Exception reporting pipeline stats")
diff --git a/zuul/merger/server.py b/zuul/merger/server.py
index 765d9e0..576d41e 100644
--- a/zuul/merger/server.py
+++ b/zuul/merger/server.py
@@ -19,10 +19,14 @@
import gear
+from zuul.lib import commandsocket
from zuul.lib.config import get_default
from zuul.merger import merger
+COMMANDS = ['stop']
+
+
class MergeServer(object):
log = logging.getLogger("zuul.MergeServer")
@@ -40,9 +44,16 @@
self.merger = merger.Merger(
merge_root, connections, merge_email, merge_name, speed_limit,
speed_time)
+ self.command_map = dict(
+ stop=self.stop)
+ command_socket = get_default(
+ self.config, 'merger', 'command_socket',
+ '/var/lib/zuul/merger.socket')
+ self.command_socket = commandsocket.CommandSocket(command_socket)
def start(self):
self._running = True
+ self._command_running = True
server = self.config.get('gearman', 'server')
port = get_default(self.config, 'gearman', 'port', 4730)
ssl_key = get_default(self.config, 'gearman', 'ssl_key')
@@ -54,6 +65,13 @@
self.worker.waitForServer()
self.log.debug("Registering")
self.register()
+ self.log.debug("Starting command processor")
+ self.command_socket.start()
+ self.command_thread = threading.Thread(
+ target=self.runCommand, name='command')
+ self.command_thread.daemon = True
+ self.command_thread.start()
+
self.log.debug("Starting worker")
self.thread = threading.Thread(target=self.run)
self.thread.daemon = True
@@ -67,12 +85,23 @@
def stop(self):
self.log.debug("Stopping")
self._running = False
+ self._command_running = False
+ self.command_socket.stop()
self.worker.shutdown()
self.log.debug("Stopped")
def join(self):
self.thread.join()
+ def runCommand(self):
+ while self._command_running:
+ try:
+ command = self.command_socket.get().decode('utf8')
+ if command != '_stop':
+ self.command_map[command]()
+ except Exception:
+ self.log.exception("Exception while processing command")
+
def run(self):
self.log.debug("Starting merge listener")
while self._running:
diff --git a/zuul/model.py b/zuul/model.py
index 11fa383..56d08a1 100644
--- a/zuul/model.py
+++ b/zuul/model.py
@@ -383,7 +383,7 @@
self.public_ipv4 = None
self.private_ipv4 = None
self.public_ipv6 = None
- self.ssh_port = 22
+ self.connection_port = 22
self._keys = []
self.az = None
self.provider = None
@@ -498,9 +498,10 @@
return n
def addNode(self, node):
- if node.name in self.nodes:
- raise Exception("Duplicate node in %s" % (self,))
- self.nodes[node.name] = node
+ for name in node.name:
+ if name in self.nodes:
+ raise Exception("Duplicate node in %s" % (self,))
+ self.nodes[tuple(node.name)] = node
def getNodes(self):
return list(self.nodes.values())
@@ -858,6 +859,7 @@
source_line=None,
inheritance_path=(),
parent_data=None,
+ description=None,
)
self.inheritable_attributes = {}
@@ -961,12 +963,12 @@
m = m.matchers[0]
if not isinstance(m, change_matcher.BranchMatcher):
return None
- return m._regex
+ return m
- def addBranchMatcher(self, branch):
+ def addImpliedBranchMatcher(self, branch):
# Add a branch matcher that combines as a boolean *and* with
# existing branch matchers, if any.
- matchers = [change_matcher.BranchMatcher(branch)]
+ matchers = [change_matcher.ImpliedBranchMatcher(branch)]
if self.branch_matcher:
matchers.append(self.branch_matcher)
self.branch_matcher = change_matcher.MatchAll(matchers)
@@ -1121,25 +1123,8 @@
joblist = self.jobs.setdefault(jobname, [])
for job in jobs:
if implied_branch:
- # If setting an implied branch and the current
- # branch matcher is a simple match for a different
- # branch, then simply do not add this job. If it
- # is absent, set it to the implied branch.
- # Otherwise, combine it with the implied branch to
- # ensure that it still only affects this branch
- # (whatever else it may do).
- simple_branch = job.getSimpleBranchMatcher()
- if simple_branch and simple_branch != implied_branch:
- # Job is for a different branch, don't add it.
- continue
- if not simple_branch:
- # The branch matcher could be complex, or
- # missing. Add our implied matcher.
- job = job.copy()
- job.addBranchMatcher(implied_branch)
- # Otherwise we have a simple branch matcher which
- # is the same as our implied branch, the job can
- # be added as-is.
+ job = job.copy()
+ job.addImpliedBranchMatcher(implied_branch)
if job not in joblist:
joblist.append(job)
@@ -1212,8 +1197,8 @@
if soft:
current_parent_jobs = set()
else:
- raise Exception("Dependent job %s not found: " %
- (dependent_job,))
+ raise Exception("Job %s depends on %s which was not run." %
+ (dependent_job, current_job))
new_parent_jobs = current_parent_jobs - all_parent_jobs
jobs_to_iterate |= new_parent_jobs
all_parent_jobs |= new_parent_jobs
@@ -1879,7 +1864,7 @@
result = build.result
finger_url = build.url
# TODO(tobiash): add support for custom web root
- urlformat = 'static/stream.html?' \
+ urlformat = 'stream.html?' \
'uuid={build.uuid}&' \
'logfile=console.log'
if websocket_url:
@@ -2401,14 +2386,25 @@
r.semaphores = copy.deepcopy(self.semaphores)
return r
- def extend(self, conf):
+ def extend(self, conf, tenant=None):
if isinstance(conf, UnparsedTenantConfig):
self.pragmas.extend(conf.pragmas)
self.pipelines.extend(conf.pipelines)
self.jobs.extend(conf.jobs)
self.project_templates.extend(conf.project_templates)
for k, v in conf.projects.items():
- self.projects.setdefault(k, []).extend(v)
+ name = k
+ # If we have the tenant add the projects to
+ # the according canonical name instead of the given project
+ # name. If it is not found, it's ok to add this to the given
+ # name. We also don't need to throw the
+ # ProjectNotFoundException here as semantic validation occurs
+ # later where it will fail then.
+ if tenant is not None:
+ trusted, project = tenant.getProject(k)
+ if project is not None:
+ name = project.canonical_name
+ self.projects.setdefault(name, []).extend(v)
self.nodesets.extend(conf.nodesets)
self.secrets.extend(conf.secrets)
self.semaphores.extend(conf.semaphores)
diff --git a/zuul/rpclistener.py b/zuul/rpclistener.py
index 8c8c783..d40505e 100644
--- a/zuul/rpclistener.py
+++ b/zuul/rpclistener.py
@@ -21,6 +21,7 @@
import gear
from zuul import model
+from zuul.lib import encryption
from zuul.lib.config import get_default
@@ -58,6 +59,8 @@
self.worker.registerFunction("zuul:get_job_log_stream_address")
self.worker.registerFunction("zuul:tenant_list")
self.worker.registerFunction("zuul:status_get")
+ self.worker.registerFunction("zuul:job_list")
+ self.worker.registerFunction("zuul:key_get")
def getFunctions(self):
functions = {}
@@ -283,3 +286,25 @@
args = json.loads(job.arguments)
output = self.sched.formatStatusJSON(args.get("tenant"))
job.sendWorkComplete(output)
+
+ def handle_job_list(self, job):
+ args = json.loads(job.arguments)
+ tenant = self.sched.abide.tenants.get(args.get("tenant"))
+ output = []
+ for job_name in sorted(tenant.layout.jobs):
+ desc = None
+ for tenant_job in tenant.layout.jobs[job_name]:
+ if tenant_job.description:
+ desc = tenant_job.description.split('\n')[0]
+ break
+ output.append({"name": job_name,
+ "description": desc})
+ job.sendWorkComplete(json.dumps(output))
+
+ def handle_key_get(self, job):
+ args = json.loads(job.arguments)
+ source_name, project_name = args.get("source"), args.get("project")
+ source = self.sched.connections.getSource(source_name)
+ project = source.getProject(project_name)
+ job.sendWorkComplete(
+ encryption.serialize_rsa_public_key(project.public_key))
diff --git a/zuul/scheduler.py b/zuul/scheduler.py
index 7dee00d..b978979 100644
--- a/zuul/scheduler.py
+++ b/zuul/scheduler.py
@@ -30,10 +30,13 @@
from zuul import exceptions
from zuul import version as zuul_version
from zuul import rpclistener
+from zuul.lib import commandsocket
from zuul.lib.config import get_default
from zuul.lib.statsd import get_statsd
import zuul.lib.queue
+COMMANDS = ['stop']
+
class ManagementEvent(object):
"""An event that should be processed within the main queue run loop"""
@@ -215,6 +218,9 @@
self.wake_event = threading.Event()
self.layout_lock = threading.Lock()
self.run_handler_lock = threading.Lock()
+ self.command_map = dict(
+ stop=self.stop,
+ )
self._pause = False
self._exit = False
self._stopped = False
@@ -243,6 +249,11 @@
time_dir = self._get_time_database_dir()
self.time_database = model.TimeDataBase(time_dir)
+ command_socket = get_default(
+ self.config, 'scheduler', 'command_socket',
+ '/var/lib/zuul/scheduler.socket')
+ self.command_socket = commandsocket.CommandSocket(command_socket)
+
self.zuul_version = zuul_version.version_info.release_string()
self.last_reconfigured = None
self.tenant_last_reconfigured = {}
@@ -250,6 +261,14 @@
def start(self):
super(Scheduler, self).start()
+ self._command_running = True
+ self.log.debug("Starting command processor")
+ self.command_socket.start()
+ self.command_thread = threading.Thread(target=self.runCommand,
+ name='command')
+ self.command_thread.daemon = True
+ self.command_thread.start()
+
self.rpc.start()
self.stats_thread.start()
@@ -261,6 +280,17 @@
self.stats_thread.join()
self.rpc.stop()
self.rpc.join()
+ self._command_running = False
+ self.command_socket.stop()
+
+ def runCommand(self):
+ while self._command_running:
+ try:
+ command = self.command_socket.get().decode('utf8')
+ if command != '_stop':
+ self.command_map[command]()
+ except Exception:
+ self.log.exception("Exception while processing command")
def registerConnections(self, connections, webapp, load=True):
# load: whether or not to trigger the onLoad for the connection. This
diff --git a/zuul/web/__init__.py b/zuul/web/__init__.py
index 766a21d..e4a3612 100755
--- a/zuul/web/__init__.py
+++ b/zuul/web/__init__.py
@@ -20,11 +20,14 @@
import logging
import os
import time
+import urllib.parse
import uvloop
import aiohttp
from aiohttp import web
+from sqlalchemy.sql import select
+
import zuul.rpcclient
STATIC_DIR = os.path.join(os.path.dirname(__file__), 'static')
@@ -162,6 +165,8 @@
self.controllers = {
'tenant_list': self.tenant_list,
'status_get': self.status_get,
+ 'job_list': self.job_list,
+ 'key_get': self.key_get,
}
def tenant_list(self, request):
@@ -182,6 +187,18 @@
resp.last_modified = self.cache_time[tenant]
return resp
+ def job_list(self, request):
+ tenant = request.match_info["tenant"]
+ job = self.rpc.submitJob('zuul:job_list', {'tenant': tenant})
+ return web.json_response(json.loads(job.data[0]))
+
+ def key_get(self, request):
+ source = request.match_info["source"]
+ project = request.match_info["project"]
+ job = self.rpc.submitJob('zuul:key_get', {'source': source,
+ 'project': project})
+ return web.Response(body=job.data[0])
+
async def processRequest(self, request, action):
try:
resp = self.controllers[action](request)
@@ -194,6 +211,92 @@
return resp
+class SqlHandler(object):
+ log = logging.getLogger("zuul.web.SqlHandler")
+ filters = ("project", "pipeline", "change", "patchset", "ref",
+ "result", "uuid", "job_name", "voting", "node_name", "newrev")
+
+ def __init__(self, connection):
+ self.connection = connection
+
+ def query(self, args):
+ build = self.connection.zuul_build_table
+ buildset = self.connection.zuul_buildset_table
+ query = select([
+ buildset.c.project,
+ buildset.c.pipeline,
+ buildset.c.change,
+ buildset.c.patchset,
+ buildset.c.ref,
+ buildset.c.newrev,
+ buildset.c.ref_url,
+ build.c.result,
+ build.c.uuid,
+ build.c.job_name,
+ build.c.voting,
+ build.c.node_name,
+ build.c.start_time,
+ build.c.end_time,
+ build.c.log_url]).select_from(build.join(buildset))
+ for table in ('build', 'buildset'):
+ for k, v in args['%s_filters' % table].items():
+ if table == 'build':
+ column = build.c
+ else:
+ column = buildset.c
+ query = query.where(getattr(column, k).in_(v))
+ return query.limit(args['limit']).offset(args['skip']).order_by(
+ build.c.id.desc())
+
+ def get_builds(self, args):
+ """Return a list of build"""
+ builds = []
+ with self.connection.engine.begin() as conn:
+ query = self.query(args)
+ for row in conn.execute(query):
+ build = dict(row)
+ # Convert date to iso format
+ if row.start_time:
+ build['start_time'] = row.start_time.strftime(
+ '%Y-%m-%dT%H:%M:%S')
+ if row.end_time:
+ build['end_time'] = row.end_time.strftime(
+ '%Y-%m-%dT%H:%M:%S')
+ # Compute run duration
+ if row.start_time and row.end_time:
+ build['duration'] = (row.end_time -
+ row.start_time).total_seconds()
+ builds.append(build)
+ return builds
+
+ async def processRequest(self, request):
+ try:
+ args = {
+ 'buildset_filters': {},
+ 'build_filters': {},
+ 'limit': 50,
+ 'skip': 0,
+ }
+ for k, v in urllib.parse.parse_qsl(request.rel_url.query_string):
+ if k in ("tenant", "project", "pipeline", "change",
+ "patchset", "ref", "newrev"):
+ args['buildset_filters'].setdefault(k, []).append(v)
+ elif k in ("uuid", "job_name", "voting", "node_name",
+ "result"):
+ args['build_filters'].setdefault(k, []).append(v)
+ elif k in ("limit", "skip"):
+ args[k] = int(v)
+ else:
+ raise ValueError("Unknown parameter %s" % k)
+ data = self.get_builds(args)
+ resp = web.json_response(data)
+ except Exception as e:
+ self.log.exception("Jobs exception:")
+ resp = web.json_response({'error_description': 'Internal error'},
+ status=500)
+ return resp
+
+
class ZuulWeb(object):
log = logging.getLogger("zuul.web.ZuulWeb")
@@ -201,7 +304,8 @@
def __init__(self, listen_address, listen_port,
gear_server, gear_port,
ssl_key=None, ssl_cert=None, ssl_ca=None,
- static_cache_expiry=3600):
+ static_cache_expiry=3600,
+ sql_connection=None):
self.listen_address = listen_address
self.listen_port = listen_port
self.event_loop = None
@@ -212,6 +316,10 @@
ssl_key, ssl_cert, ssl_ca)
self.log_streaming_handler = LogStreamingHandler(self.rpc)
self.gearman_handler = GearmanHandler(self.rpc)
+ if sql_connection:
+ self.sql_handler = SqlHandler(sql_connection)
+ else:
+ self.sql_handler = None
async def _handleWebsocket(self, request):
return await self.log_streaming_handler.processRequest(
@@ -224,12 +332,27 @@
async def _handleStatusRequest(self, request):
return await self.gearman_handler.processRequest(request, 'status_get')
+ async def _handleJobsRequest(self, request):
+ return await self.gearman_handler.processRequest(request, 'job_list')
+
+ async def _handleSqlRequest(self, request):
+ return await self.sql_handler.processRequest(request)
+
+ async def _handleKeyRequest(self, request):
+ return await self.gearman_handler.processRequest(request, 'key_get')
+
async def _handleStaticRequest(self, request):
fp = None
if request.path.endswith("tenants.html") or request.path.endswith("/"):
fp = os.path.join(STATIC_DIR, "index.html")
elif request.path.endswith("status.html"):
fp = os.path.join(STATIC_DIR, "status.html")
+ elif request.path.endswith("jobs.html"):
+ fp = os.path.join(STATIC_DIR, "jobs.html")
+ elif request.path.endswith("builds.html"):
+ fp = os.path.join(STATIC_DIR, "builds.html")
+ elif request.path.endswith("stream.html"):
+ fp = os.path.join(STATIC_DIR, "stream.html")
headers = {}
if self.static_cache_expiry:
headers['Cache-Control'] = "public, max-age=%d" % \
@@ -248,14 +371,24 @@
is run within a separate (non-main) thread.
"""
routes = [
- ('GET', '/console-stream', self._handleWebsocket),
('GET', '/tenants.json', self._handleTenantsRequest),
('GET', '/{tenant}/status.json', self._handleStatusRequest),
+ ('GET', '/{tenant}/jobs.json', self._handleJobsRequest),
+ ('GET', '/{tenant}/console-stream', self._handleWebsocket),
+ ('GET', '/{source}/{project}.pub', self._handleKeyRequest),
('GET', '/{tenant}/status.html', self._handleStaticRequest),
+ ('GET', '/{tenant}/jobs.html', self._handleStaticRequest),
+ ('GET', '/{tenant}/stream.html', self._handleStaticRequest),
('GET', '/tenants.html', self._handleStaticRequest),
('GET', '/', self._handleStaticRequest),
]
+ if self.sql_handler:
+ routes.append(('GET', '/{tenant}/builds.json',
+ self._handleSqlRequest))
+ routes.append(('GET', '/{tenant}/builds.html',
+ self._handleStaticRequest))
+
self.log.debug("ZuulWeb starting")
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
user_supplied_loop = loop is not None
diff --git a/zuul/web/static/README b/zuul/web/static/README
index f17ea5f..e924dc7 100644
--- a/zuul/web/static/README
+++ b/zuul/web/static/README
@@ -50,8 +50,7 @@
</Directory>
# Console-stream needs a special proxy-pass for websocket
- ProxyPass /console-stream ws://localhost:9000/console-stream nocanon retry=0
- ProxyPassReverse /console-stream ws://localhost:9000/console-stream
+ ProxyPassMatch /(.*)/console-stream ws://localhost:9000/$1/console-stream nocanon retry=0
# Then only the json calls are sent to the zuul-web endpoints
ProxyPassMatch ^/(.*.json)$ http://localhost:9000/$1 nocanon retry=0
diff --git a/zuul/web/static/builds.html b/zuul/web/static/builds.html
new file mode 100644
index 0000000..5b9ba35
--- /dev/null
+++ b/zuul/web/static/builds.html
@@ -0,0 +1,78 @@
+<!--
+Copyright 2017 Red Hat
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may
+not use this file except in compliance with the License. You may obtain
+a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations
+under the License.
+-->
+<!DOCTYPE html>
+<html>
+<head>
+ <title>Zuul Builds</title>
+ <link rel="stylesheet" href="../static/bootstrap/css/bootstrap.min.css">
+ <link rel="stylesheet" href="../static/styles/zuul.css" />
+ <script src="../static/js/jquery.min.js"></script>
+ <script src="../static/js/angular.min.js"></script>
+ <script src="../static/javascripts/zuul.angular.js"></script>
+</head>
+<body ng-app="zuulBuilds" ng-controller="mainController"><div class="container-fluid">
+ <nav class="navbar navbar-default">
+ <div class="container-fluid">
+ <div class="navbar-header">
+ <a class="navbar-brand" href="../" target="_self">Zuul Dashboard</a>
+ </div>
+ <ul class="nav navbar-nav">
+ <li><a href="status.html" target="_self">Status</a></li>
+ <li><a href="jobs.html" target="_self">Jobs</a></li>
+ <li class="active"><a href="builds.html" target="_self">Builds</a></li>
+ </ul>
+ <span style="float: right; margin-top: 7px;">
+ <form ng-submit="builds_fetch()">
+ <label>Pipeline:</label>
+ <input name="pipeline" ng-model="pipeline" />
+ <label>Job:</label>
+ <input name="job_name" ng-model="job_name" />
+ <label>Project:</label>
+ <input name="project" ng-model="project" />
+ <input type="submit" value="Refresh" />
+ </form>
+ </span>
+ </div>
+ </nav>
+ <table class="table table-hover table-condensed">
+ <thead>
+ <tr>
+ <th width="20px">id</th>
+ <th>Job</th>
+ <th>Project</th>
+ <th>Pipeline</th>
+ <th>Change</th>
+ <th>Duration</th>
+ <th>Log url</th>
+ <th>Start time</th>
+ <th>Result</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr ng-repeat="build in builds" ng-class="rowClass(build)">
+ <td>{{ build.id }}</td>
+ <td>{{ build.job_name }}</td>
+ <td>{{ build.project }}</td>
+ <td>{{ build.pipeline }}</td>
+ <td><a href="{{ build.ref_url }}" target="_self">change</a></td>
+ <td>{{ build.duration }} seconds</td>
+ <td><a ng-if="build.log_url" href="{{ build.log_url }}" target="_self">logs</a></td>
+ <td>{{ build.start_time }}</td>
+ <td>{{ build.result }}</td>
+ </tr>
+ </tbody>
+ </table>
+</div></body></html>
diff --git a/zuul/web/static/index.html b/zuul/web/static/index.html
index 6747e66..d20a1ea 100644
--- a/zuul/web/static/index.html
+++ b/zuul/web/static/index.html
@@ -17,10 +17,10 @@
<html>
<head>
<title>Zuul Tenants</title>
- <link rel="stylesheet" href="/static/bootstrap/css/bootstrap.min.css">
+ <link rel="stylesheet" href="static/bootstrap/css/bootstrap.min.css">
<link rel="stylesheet" href="static/styles/zuul.css" />
- <script src="/static/js/jquery.min.js"></script>
- <script src="/static/js/angular.min.js"></script>
+ <script src="static/js/jquery.min.js"></script>
+ <script src="static/js/angular.min.js"></script>
<script src="static/javascripts/zuul.angular.js"></script>
</head>
<body ng-app="zuulTenants" ng-controller="mainController"><div class="container-fluid">
diff --git a/zuul/web/static/javascripts/zuul.angular.js b/zuul/web/static/javascripts/zuul.angular.js
index 3152fc0..87cbbdd 100644
--- a/zuul/web/static/javascripts/zuul.angular.js
+++ b/zuul/web/static/javascripts/zuul.angular.js
@@ -30,3 +30,70 @@
}
$scope.tenants_fetch();
});
+
+angular.module('zuulJobs', []).controller(
+ 'mainController', function($scope, $http)
+{
+ $scope.jobs = undefined;
+ $scope.jobs_fetch = function() {
+ $http.get("jobs.json")
+ .then(function success(result) {
+ $scope.jobs = result.data;
+ });
+ }
+ $scope.jobs_fetch();
+});
+
+angular.module('zuulBuilds', [], function($locationProvider) {
+ $locationProvider.html5Mode({
+ enabled: true,
+ requireBase: false
+ });
+}).controller('mainController', function($scope, $http, $location)
+{
+ $scope.rowClass = function(build) {
+ if (build.result == "SUCCESS") {
+ return "success";
+ } else {
+ return "warning";
+ }
+ };
+ var query_args = $location.search();
+ var url = $location.url();
+ var tenant_start = url.lastIndexOf(
+ '/', url.lastIndexOf('/builds.html') - 1) + 1;
+ var tenant_length = url.lastIndexOf('/builds.html') - tenant_start;
+ $scope.tenant = url.substr(tenant_start, tenant_length);
+ $scope.builds = undefined;
+ if (query_args["pipeline"]) {$scope.pipeline = query_args["pipeline"];
+ } else {$scope.pipeline = "";}
+ if (query_args["job_name"]) {$scope.job_name = query_args["job_name"];
+ } else {$scope.job_name = "";}
+ if (query_args["project"]) {$scope.project = query_args["project"];
+ } else {$scope.project = "";}
+ $scope.builds_fetch = function() {
+ query_string = "";
+ if ($scope.tenant) {query_string += "&tenant="+$scope.tenant;}
+ if ($scope.pipeline) {query_string += "&pipeline="+$scope.pipeline;}
+ if ($scope.job_name) {query_string += "&job_name="+$scope.job_name;}
+ if ($scope.project) {query_string += "&project="+$scope.project;}
+ if (query_string != "") {query_string = "?" + query_string.substr(1);}
+ $http.get("builds.json" + query_string)
+ .then(function success(result) {
+ for (build_pos = 0;
+ build_pos < result.data.length;
+ build_pos += 1) {
+ build = result.data[build_pos]
+ if (build.node_name == null) {
+ build.node_name = 'master'
+ }
+ /* Fix incorect url for post_failure job */
+ if (build.log_url == build.job_name) {
+ build.log_url = undefined;
+ }
+ }
+ $scope.builds = result.data;
+ });
+ }
+ $scope.builds_fetch()
+});
diff --git a/zuul/web/static/javascripts/zuul.app.js b/zuul/web/static/javascripts/zuul.app.js
index 7ceb2dd..bf90a4d 100644
--- a/zuul/web/static/javascripts/zuul.app.js
+++ b/zuul/web/static/javascripts/zuul.app.js
@@ -28,8 +28,6 @@
function zuul_build_dom($, container) {
// Build a default-looking DOM
var default_layout = '<div class="container">'
- + '<h1>Zuul Status</h1>'
- + '<p>Real-time status monitor of Zuul, the pipeline manager between Gerrit and Workers.</p>'
+ '<div class="zuul-container" id="zuul-container">'
+ '<div style="display: none;" class="alert" id="zuul_msg"></div>'
+ '<button class="btn pull-right zuul-spinner">updating <span class="glyphicon glyphicon-refresh"></span></button>'
diff --git a/zuul/web/static/jobs.html b/zuul/web/static/jobs.html
new file mode 100644
index 0000000..b27d882
--- /dev/null
+++ b/zuul/web/static/jobs.html
@@ -0,0 +1,55 @@
+<!--
+Copyright 2017 Red Hat
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may
+not use this file except in compliance with the License. You may obtain
+a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations
+under the License.
+-->
+<!DOCTYPE html>
+<html>
+<head>
+ <title>Zuul Builds</title>
+ <link rel="stylesheet" href="../static/bootstrap/css/bootstrap.min.css">
+ <link rel="stylesheet" href="../static/styles/zuul.css" />
+ <script src="../static/js/jquery.min.js"></script>
+ <script src="../static/js/angular.min.js"></script>
+ <script src="../static/javascripts/zuul.angular.js"></script>
+</head>
+<body ng-app="zuulJobs" ng-controller="mainController"><div class="container-fluid">
+ <nav class="navbar navbar-default">
+ <div class="container-fluid">
+ <div class="navbar-header">
+ <a class="navbar-brand" href="../" target="_self">Zuul Dashboard</a>
+ </div>
+ <ul class="nav navbar-nav">
+ <li><a href="status.html" target="_self">Status</a></li>
+ <li class="active"><a href="jobs.html" target="_self">Jobs</a></li>
+ <li><a href="builds.html" target="_self">Builds</a></li>
+ </ul>
+ </div>
+ </nav>
+ <table class="table table-hover table-condensed">
+ <thead>
+ <tr>
+ <th>Name</th>
+ <th>Description</th>
+ <th>Last builds</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr ng-repeat="job in jobs">
+ <td>{{ job.name }}</td>
+ <td>{{ job.description }}</td>
+ <td><a href="builds.html?job_name={{ job.name }}">builds</a></td>
+ </tr>
+ </tbody>
+ </table>
+</div></body></html>
diff --git a/zuul/web/static/status.html b/zuul/web/static/status.html
index 7cb9536..8471fd1 100644
--- a/zuul/web/static/status.html
+++ b/zuul/web/static/status.html
@@ -19,11 +19,11 @@
<html>
<head>
<title>Zuul Status</title>
- <link rel="stylesheet" href="/static/bootstrap/css/bootstrap.min.css">
+ <link rel="stylesheet" href="../static/bootstrap/css/bootstrap.min.css">
<link rel="stylesheet" href="../static/styles/zuul.css" />
- <script src="/static/js/jquery.min.js"></script>
- <script src="/static/js/jquery-visibility.min.js"></script>
- <script src="/static/js/jquery.graphite.min.js"></script>
+ <script src="../static/js/jquery.min.js"></script>
+ <script src="../static/js/jquery-visibility.min.js"></script>
+ <script src="../static/js/jquery.graphite.min.js"></script>
<script src="../static/javascripts/jquery.zuul.js"></script>
<script src="../static/javascripts/zuul.app.js"></script>
</head>
diff --git a/zuul/web/static/stream.html b/zuul/web/static/stream.html
index dbeb66b..f2e7081 100644
--- a/zuul/web/static/stream.html
+++ b/zuul/web/static/stream.html
@@ -73,7 +73,7 @@
} else {
protocol = 'ws://';
}
- path = url['pathname'].replace(/static\/.*$/g, '') + 'console-stream';
+ path = url['pathname'].replace(/stream.html.*$/g, '') + 'console-stream';
params['websocket_url'] = protocol + url['host'] + path;
}
var ws = new WebSocket(params['websocket_url']);