Joshua Hesketh | 902ea50 | 2013-09-06 11:15:30 +1000 | [diff] [blame] | 1 | :title:Introduction |
Joshua Hesketh | 8b0eb6e | 2013-09-05 13:52:04 +1000 | [diff] [blame] | 2 | |
| 3 | Turbo Hipster |
| 4 | ============= |
| 5 | |
| 6 | Turbo Hipster is a gearman worker designed to run tests using zuul |
| 7 | as the gearman client. It is primarily aimed at running openstack |
Joshua Hesketh | a85e7dd | 2013-09-06 15:55:53 +1000 | [diff] [blame] | 8 | continuous integration tests against pre-existing datasets but can |
| 9 | be used to automate any tests with zuul. |
Joshua Hesketh | 8b0eb6e | 2013-09-05 13:52:04 +1000 | [diff] [blame] | 10 | |
| 11 | Overview |
| 12 | -------- |
| 13 | |
| 14 | The zuul server receives jobs from a trigger requesting particular |
| 15 | jobs/tests to be ran. Turbo Hipster is able to provide a worker for |
| 16 | each of those jobs or a subset and report the success/failure back to |
| 17 | zuul. zuul will then collate responses from multiple workers and |
| 18 | build a report. |
| 19 | |
| 20 | Typical workflow diagram |
| 21 | ------------------------ |
| 22 | |
Joshua Hesketh | 05d0158 | 2013-09-09 15:16:08 +1000 | [diff] [blame] | 23 | **clearly this needs a lot of work, however I believe the structure |
Joshua Hesketh | af26dbb | 2013-09-11 12:30:38 +1000 | [diff] [blame] | 24 | is mostly there... If you know graphviz please help!** |
Joshua Hesketh | 05d0158 | 2013-09-09 15:16:08 +1000 | [diff] [blame] | 25 | |
Joshua Hesketh | 8b0eb6e | 2013-09-05 13:52:04 +1000 | [diff] [blame] | 26 | .. graphviz:: |
| 27 | |
| 28 | digraph overview { |
| 29 | subgraph cluster_1 { |
| 30 | label = "Gerrit" |
| 31 | style = filled; |
| 32 | color = lightgrey; |
| 33 | node [style=filled,color=white]; |
| 34 | |
| 35 | g000 [shape=Mdiamond label="start"]; |
| 36 | g001 [shape=box, label="receive event"]; |
| 37 | g002 [shape=box, label="notify listeners"]; |
| 38 | |
| 39 | g000 -> g001; |
| 40 | g001 -> g002; |
| 41 | g002 -> g001; |
| 42 | } |
| 43 | |
| 44 | subgraph cluster_2 { |
| 45 | label = "Zuul pipeline"; |
| 46 | color = blue |
| 47 | node [style=filled]; |
| 48 | |
| 49 | z000 [shape=Mdiamond label="start"]; |
| 50 | z001 [shape=box, label="register gearman server"]; |
| 51 | z002 [shape=box, label="register launchers"]; |
| 52 | z003 [shape=box, label="listen for events"]; |
| 53 | z004 [shape=box, label="receive event"]; |
| 54 | z005 [shape=box, label="request jobs"]; |
| 55 | z006 [shape=box, label="receive response"]; |
| 56 | z007 [shape=box, label="send report"]; |
| 57 | |
Joshua Hesketh | 902ea50 | 2013-09-06 11:15:30 +1000 | [diff] [blame] | 58 | z000 -> z001 -> z002; |
| 59 | z003 -> z004 -> z005; |
| 60 | z005 -> z006 [dir=none, style=dotted]; |
Joshua Hesketh | 8b0eb6e | 2013-09-05 13:52:04 +1000 | [diff] [blame] | 61 | z006 -> z007; |
| 62 | |
| 63 | } |
| 64 | |
| 65 | subgraph cluster_3 { |
| 66 | label = "Gearman"; |
| 67 | style = filled; |
| 68 | color = lightgrey; |
| 69 | node [style=filled,color=white]; |
| 70 | |
| 71 | gm001 [shape=box, label="receive job method"]; |
| 72 | gm002 [shape=box, label="request worker do method"]; |
| 73 | gm003 [shape=box, label="receive results"]; |
| 74 | gm004 [shape=box, label="return results"]; |
| 75 | |
| 76 | gms000 [label="register client"]; |
| 77 | gms001 [label="register worker"]; |
| 78 | gms002 [label="register method"]; |
| 79 | |
| 80 | gm001 -> gm002; |
Joshua Hesketh | 902ea50 | 2013-09-06 11:15:30 +1000 | [diff] [blame] | 81 | gm002 -> gm003 [dir=none, style=dotted]; |
Joshua Hesketh | 8b0eb6e | 2013-09-05 13:52:04 +1000 | [diff] [blame] | 82 | gm003 -> gm004; |
| 83 | } |
| 84 | |
| 85 | subgraph cluster_4 { |
| 86 | label = "Turbo Hipster"; |
| 87 | color = blue |
| 88 | node [style=filled]; |
| 89 | |
| 90 | th000 [shape=Mdiamond label="start"]; |
| 91 | th001 [shape=box, label="register as worker"]; |
| 92 | th002 [shape=box, label="find available tasks"]; |
| 93 | th003 [shape=box, label="register available job methods"]; |
| 94 | |
| 95 | ths001 [shape=box, label="receive method request"]; |
| 96 | ths002 [shape=box, label="run task"]; |
| 97 | ths003 [shape=box, label="send results"]; |
| 98 | |
| 99 | th000 -> th001 -> th002 -> th003; |
| 100 | ths001 -> ths002 -> ths003; |
| 101 | } |
| 102 | |
| 103 | z001 -> gms000; |
| 104 | z005 -> gm001; |
| 105 | gm004 -> z006; |
| 106 | z003 -> g002 [dir=both, style=dotted]; |
| 107 | th001 -> gms001; |
| 108 | th003 -> gms002; |
| 109 | gm002 -> ths001; |
| 110 | ths003 -> gm003; |
| 111 | |
| 112 | } |
Joshua Hesketh | 05d0158 | 2013-09-09 15:16:08 +1000 | [diff] [blame] | 113 | |
| 114 | zuul integration |
| 115 | ---------------- |
| 116 | |
| 117 | Explain how zuul triggers builds and gates etc and how turbo-hipster |
| 118 | responds to them. Most of this is in the zuul documentation so don't |
| 119 | duplicate. |