Commit graph

20 commits

Author SHA1 Message Date
Lars Karlitski
a91b5640de jobqueue: fail test if PushCompose returns an error 2019-11-10 17:23:14 +01:00
Tom Gundersen
aa404dcb99 worker: move Job type to the jobqueue package
The main purpose of this is to share the structs between the server
and the client, and let the compiler ensure that our marshaling and
unmarshaling matches.

In the future we also want to make it easier to write unittests for
this code.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-29 16:08:54 +01:00
Lars Karlitski
cde1325bc3 blueprint: move tar towards lorax parity
The main difference (according to `rpm -qa`) is an additional package
containing a gpg key which was used to verify packages. The one
generated by lorax-composer doesn't have this, because it doesn't verify
signatures.
2019-10-29 16:01:37 +01:00
Tom Gundersen
2e0deff629 submodule: bump to current head
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-28 14:14:09 +01:00
Tom Gundersen
f055ba2e07 store: change semantics of queue states
A job is now in "WAITING" state exactly when it is in the channel,
once it is popped it enters "RUNNING" state. It is only possible
to update the state of a job that is in the running state.

This mean that updating to "RUNNING" is entirely optional, but in
the future we may want to use this as a watchdog logic, and require
the worker to update at regular intervals to avoid being restarted.

The job queue API is updated to require a POST followed by one
or several PATCH messages to the returned ID. If a patch is sent
to an ID before the POST it is as if the object does not exist
(regarldess of it being in the queue in WAITING state or not).

Once a job has been POSTed it can be PATCHed to update it zero or
more times with (still) RUNNING before exactly oncee with either
FINISHED or FAILED.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-25 10:18:47 +02:00
Ondřej Budai
15b82a15d2 osbuild-composer: Rename module to github.com/osbuild/osbuild-composer
This should be the best practice according to other popular go projects:
- https://github.com/prometheus/prometheus
- https://github.com/syncthing/syncthing
- https://github.com/drone/drone
- https://github.com/hashicorp/terraform

Also, this change fixes go get command (it currently fails due to bad package
name).
2019-10-08 21:44:57 +02:00
Tom Gundersen
22254637f8 blueprint: generate real f30 images for all the output types
This is still just a placeholder, and in particular the live CD support
is not right.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-08 21:03:45 +02:00
Tom Gundersen
7e78b4051d jobqueue: add back tests for the API
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
68ade23fcc store: make state (de)serialization internal
This hides the state hanlding in the store package. As before, it
can be disabled by passing `nil` instead of the path to the state
file.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
9cb140795e store: add {Push,Pop}Compose methods and hide channel
Wrap the channel in Pop and Push methods, so it is not exposed to
the callers. PushCompose replaces the old AddCompose for consistency,
and PopCompose simply reads from the other end of the channel.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
3ff4f59fc7 store: pass the store to the jobqueue API
Drop the jobUpdates channel, and instead add an UpdateCompose method
to the store, which updates the status of a compose directly.

This allows us to report back errors directly, rather than having to
mirror the staet in the jobqueue API.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
0880014edf jobqueue: cleanup API a bit and unify the two job stores
Let the store in weldr be the only one that keeps state, and push
updates directly there. This fixes a bug where there was an ID mismatch.

Change the API to not let the caller pick the UUID, but provide it
in the response. Use the same UUID as is used to identify composes,
this makes it simpler to trace what is going on.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
96f3cbf655 weldr/store/compose: store information exposed in the API
Use the exact same status strings as is used in the API,
making it clearer that they are the same (and avoiding any
translation). Remember the creation/start/finish timestamps.
And store the output type.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-29 14:51:22 +02:00
Tom Gundersen
7625d26ff5 pipeline/target: implement as variant types
Go doesn't really do variants, so we must somehow emulate it. The
json objects we use are essentially tagged unions, with a `name`
field in reverse domain name notation identifying the type and a
type specific 'options' object.

In Go we represent this by having an BarOptions interface, which
implements a private method `isBarOptions()`, making sure that only
types in the same package are able to implement it. Each type FooBar
that should belong to the variant implements the interface, and a
constructor `NewFooBar(options *FooBarOptions) *Bar` that makes sure
the `name` field is set correctly.

This would be enough to represent our types and marshal them into
JSON, but unmarshalling would not work (json does not know about
our tags, so would not know what concrete types to demarshal to).
We therefore must also implement the Unmarshall interface for Bar,
to select the right types for the Options field.

We implement his logic for Target, Stage and Assembler. A handful
of concrete types are also implemented, matching what osbuild
supports.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-28 17:49:07 +02:00
Tom Gundersen
5df6874b94 target: add constructor
We only support local target for now, but this avoids having to
open code it.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-27 15:54:13 +02:00
Tom Gundersen
f880581a14 weldr/store: keep the compose status up-to-date
This way it can be correctly exposed in the API. We listen on a channel
from the job-queue, where status updates are pushed when the worker is
running/finished (or, in the future, failed).

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-27 14:34:51 +02:00
Tom Gundersen
0dc30d7f1b weldr/compose: return the build-id when starting a compose
The return argument was ommitted. Also move to using the uuid package
wherever that makes sense.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-27 00:51:15 +02:00
Lars Karlitski
cfe89eaed5 Add simple osbuild-worker
It doesn't actually build anything yet, but talks to the queue API.
2019-09-26 23:58:03 +02:00
Tom Gundersen
0bdd3a5c89 job-queue: pass a well-formed job object to the worker
For now we will hardcode the org.osbuild.local target, so we might
as well fix this up front.

We do not yet support any target types, but for testing purposes we
claim to support 'tar', and we pass a noop tar pipeline to the worker.

This makes introspecting the job-queu api using curl a bit more
pleasant.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-26 23:48:19 +02:00
Tom Gundersen
3221112d35 composer/job-queue: add scaffolding for the job queue API
This is by no means done, and needs more tests, docs and bugfixes,
but push it early so we have a common base to work on.

Based on work by Martin Sehnoutka.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-26 19:53:06 +02:00