Idempotent scheduler and QueueItems

Make the scheduler idempotent.  The idea is that after any event,
the scheduler should be able to run and examine the state of every
item in the queue and act accordingly.  This is a change from the
current state where most events are dealt with in context.  This
should ease maintenance as it should facilitate reasoning about
the different actions Zuul might take -- centralizing major
decisions into one function.

Also add a new class QueueItem, which represents a Change(ish)
in a queue.  Currently, Change objects themselves are placed
in the queue, which is confusing information about a change (for
instance: it's number and patchset) as well as information about
the processing of that change in the queue (e.g., the build
history, current build set, merge status, etc.).

Change objects are now cached, which should reduce the number of
queries to Gerrit (except the current algorithm to update them is
very naive and queries Gerrit again on any event relating to a
change).  Changes are expired from the cache when they are not
present or related to any change currently in a pipeline.

There are now two things that need to be asserted at the end of
each test, so use addCleanup in setUp to call a method that
performs those assertions after the test method completes.  Also,
move the existing shutdown method to use addCleanup as well,
because testr experts say that's a best practice.

Change-Id: Id2bf4c484c9e681456c69d99787e7a5b3a247690
Reviewed-on: https://review.openstack.org/34653
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
6 files changed