Gitiles
Code Review Sign In
gerrit.cesnet.cz / github / openstack-infra / zuul / 80ac158acd607b5404e36a58db3097801d8a679e / . / doc / source / admin / monitoring.rst
blob: a8b2324f0d84f86b5d6a625079df5a269a02ca9b [file] [log] [blame]
James E. Blaireff5a9d2017-06-20 00:00:37 -0700[diff] [blame]1:title: Monitoring
2
3Monitoring
4==========
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]5
6Statsd reporting
James E. Blaireff5a9d2017-06-20 00:00:37 -0700[diff] [blame]7----------------
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]8
9Zuul comes with support for the statsd protocol, when enabled and configured
Michael Prokop526926a2013-10-24 16:16:57 +0200[diff] [blame]10(see below), the Zuul scheduler will emit raw metrics to a statsd receiver
James E. Blaireff5a9d2017-06-20 00:00:37 -0700[diff] [blame]11which let you in turn generate nice graphics.
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]12
13Configuration
James E. Blaireff5a9d2017-06-20 00:00:37 -0700[diff] [blame]14~~~~~~~~~~~~~
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]15
16Statsd support uses the statsd python module. Note that Zuul will start without
17the statsd python module, so an existing Zuul installation may be missing it.
18
Michael Prokop526926a2013-10-24 16:16:57 +0200[diff] [blame]19The configuration is done via environment variables STATSD_HOST and
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]20STATSD_PORT. They are interpreted by the statsd module directly and there is no
Michael Prokop526926a2013-10-24 16:16:57 +0200[diff] [blame]21such parameter in zuul.conf yet. Your init script will have to initialize both
Paul Belanger174a8272017-03-14 13:20:10 -0400[diff] [blame]22of them before executing Zuul.
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]23
24Your init script most probably loads a configuration file named
25``/etc/default/zuul`` which would contain the environment variables::
26
27 $ cat /etc/default/zuul
28 STATSD_HOST=10.0.0.1
29 STATSD_PORT=8125
30
31Metrics
James E. Blaireff5a9d2017-06-20 00:00:37 -0700[diff] [blame]32~~~~~~~
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]33
David Shrewsbury1c61c712017-08-16 16:02:33 -0400[diff] [blame]34These metrics are emitted by the Zuul :ref:`scheduler`:
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]35
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]36.. stat:: zuul.event.<driver>.event.<type>
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]37 :type: counter
James E. Blaireff5a9d2017-06-20 00:00:37 -0700[diff] [blame]38
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]39 Zuul will report counters for each type of event it receives from
40 each of its configured drivers.
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]41
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]42.. stat:: zuul.<tenant>.pipeline
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]43
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]44 Holds metrics specific to jobs. This hierarchy includes:
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]45
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]46 .. stat:: <pipeline name>
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]47
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]48 A set of metrics for each pipeline named as defined in the Zuul
49 config.
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]50
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]51 .. stat:: all_jobs
52 :type: counter
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]53
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]54 Number of jobs triggered by the pipeline.
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]55
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]56 .. stat:: current_changes
57 :type: gauge
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]58
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]59 The number of items currently being processed by this
60 pipeline.
Antoine Mussoa8eea7d2013-10-05 16:08:00 +0200[diff] [blame]61
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]62 .. stat:: project
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]63
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]64 This hierarchy holds more specific metrics for each project
65 participating in the pipeline.
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]66
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]67 .. stat:: <canonical_hostname>
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]68
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]69 The canonical hostname for the triggering project.
70 Embedded ``.`` characters will be translated to ``_``.
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]71
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]72 .. stat:: <project>
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]73
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]74 The name of the triggering project. Embedded ``/`` or
75 ``.`` characters will be translated to ``_``.
76
77 .. stat:: <branch>
78
79 The name of the triggering branch. Embedded ``/`` or
80 ``.`` characters will be translated to ``_``.
81
82 .. stat:: job
83
84 Subtree detailing per-project job statistics:
85
86 .. stat:: <jobname>
87
88 The triggered job name.
89
90 .. stat:: <result>
91 :type: counter, timer
92
93 A counter for each type of result (e.g., ``SUCCESS`` or
94 ``FAILURE``, ``ERROR``, etc.) for the job. If the
95 result is ``SUCCESS`` or ``FAILURE``, Zuul will
96 additionally report the duration of the build as a
97 timer.
98
99 .. stat:: current_changes
100 :type: gauge
101
102 The number of items of this project currently being
103 processed by this pipeline.
104
105 .. stat:: resident_time
106 :type: timer
107
108 A timer metric reporting how long each item for this
109 project has been in the pipeline.
110
111 .. stat:: total_changes
112 :type: counter
113
114 The number of changes for this project processed by the
115 pipeline since Zuul started.
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]116
117 .. stat:: resident_time
118 :type: timer
119
120 A timer metric reporting how long each item has been in the
121 pipeline.
122
123 .. stat:: total_changes
124 :type: counter
125
David Shrewsbury1c61c712017-08-16 16:02:33 -0400[diff] [blame]126 The number of changes processed by the pipeline since Zuul
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]127 started.
128
129 .. stat:: wait_time
130 :type: timer
131
132 How long each item spent in the pipeline before its first job
133 started.
134
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]135
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]136As an example, given a job named `myjob` in `mytenant` triggered by a
137change to `myproject` on the `master` branch in the `gate` pipeline
138which took 40 seconds to build, the Zuul scheduler will emit the
139following statsd events:
James E. Blair91c9dde2017-08-04 11:10:24 -0700[diff] [blame]140
James E. Blair80ac1582017-10-09 07:02:40 -0700[diff] [blame^]141 * ``zuul.tenant.mytenant.pipeline.gate.project.example_com.myproject.master.job.myjob.SUCCESS`` +1
142 * ``zuul.tenant.mytenant.pipeline.gate.project.example_com.myproject.master.job.myjob.SUCCESS`` 40 seconds
143 * ``zuul.tenant.mytenant.pipeline.gate.all_jobs`` +1