tree e296412593cdea6b0137ffd537e00f9fc8452401
parent 0dbe15993af8359541a6fb19bc5e72797d2e3d42
author Monty Taylor <mordred@inaugust.com> 1464107290 -0500
committer David Shrewsbury <shrewsbury.dave@gmail.com> 1499697148 -0400

Add web-based console log streaming

zuul now provides socket-based console streaming, which is super cool.
In order to have jenkins parity with web streaming, we need to provide a
websocket (javascript in browsers can't really connect to random ports
on servers)

After surveying the existing python websocket options, basically all of
them are based around twisted, eventlet, gevent or asyncio. It's not
just a thing we can easily deal with from our current webob/paste
structure, because it is a change to the fundamental HTTP handling.
While we could write our own websocket server implementation that was
threaded like the rest of zuul, that's a pretty giant amount of work.

Instead, we can run an async-based server that's just for the
websockets, so that we're not all of a sudden putting async code into
the rest of zuul and winding up frankensteined. Since this is new code,
using asyncio and python3 seems like an excellent starting place.

aiohttp supports running a websocket server in a thread. It also
supports doing other HTTP/REST calls, so by going aiohttp we can set
ourselves up for a single answer for the HTTP tier.

In order to keep us from being an open socket relay, we'll expect two
parameters as the first message on the websocket - what's the zuul build
uuid, and what log file do we want to stream. (the second thing,
multiple log files, isn't supported yet by the rest of zuul, but one can
imagine a future where we'd like to support that too, so it's in the
protocol) The websocket server will then ask zuul over gearman for the
IP and port associated with the build and logfile and will start
streaming it to the socket.

Ultimately we'll want the status page to make links of the form:

  /console.html?uuid=<uuid>&logfile=console.log

and we'll want to have apache map the websocket server to something like
/console.

Co-Authored-By: Monty Taylor <mordred@inaugust.com>

Change-Id: Idd0d3f9259e81fa9a60d7540664ce8d5ad2c298f