Commit graph

920 commits

Author SHA1 Message Date
Tom Gundersen
b0d9423b73 osbuild-pipeline: add tool to print pipelines to stdout
This tool outputs the pipeline associated with each output type. In
the future this should grow to take a blueprint as an input, and
also allow the arch/os/version/variant to be configured.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-08 21:03:45 +02:00
Tom Gundersen
68ade23fcc store: make state (de)serialization internal
This hides the state hanlding in the store package. As before, it
can be disabled by passing `nil` instead of the path to the state
file.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
ad637b46f8 osbuild-composer: rename the state file state.json
It is no longer specific to the weldr API, so drop the weldr prefix.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
b6c63b6d59 store: don't initialize the store with an example
We probably want to ship in a pristine state, so let's leave this out
for now.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
9cb140795e store: add {Push,Pop}Compose methods and hide channel
Wrap the channel in Pop and Push methods, so it is not exposed to
the callers. PushCompose replaces the old AddCompose for consistency,
and PopCompose simply reads from the other end of the channel.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
3ff4f59fc7 store: pass the store to the jobqueue API
Drop the jobUpdates channel, and instead add an UpdateCompose method
to the store, which updates the status of a compose directly.

This allows us to report back errors directly, rather than having to
mirror the staet in the jobqueue API.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
89fd2e6037 store: move creation to main.go
Let the weldr API take the store as an argument, rather than create it
itself. This will allow us to share the store with the jobqueue API in
follow-up patches.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
0880014edf jobqueue: cleanup API a bit and unify the two job stores
Let the store in weldr be the only one that keeps state, and push
updates directly there. This fixes a bug where there was an ID mismatch.

Change the API to not let the caller pick the UUID, but provide it
in the response. Use the same UUID as is used to identify composes,
this makes it simpler to trace what is going on.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Martin Sehnoutka
ed185b41ce Integration of osbuild composer with systemd
osbuild-composer now uses socket activation instead of hardcoded paths
in the code. osbuild-worker is an http client therefore it uses only
service unit. osbuild-worker must be started after the socket is
created. osbuild-composer service requires osbuild-worker to run, because without
it no jobs can be started.

osbuild-composer is executed as a regular user (newly created
_osbuild-composer user) as opposed to the worker which must run as root
in order to execute osbuild itself
2019-10-02 17:56:59 +02:00
Tom Gundersen
96f3cbf655 weldr/store/compose: store information exposed in the API
Use the exact same status strings as is used in the API,
making it clearer that they are the same (and avoiding any
translation). Remember the creation/start/finish timestamps.
And store the output type.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-29 14:51:22 +02:00
Tom Gundersen
d231694bae worker/target/local: copy to the right location
We were copying the containing directory from our object store, we
only want the contents.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-28 17:49:07 +02:00
Tom Gundersen
7625d26ff5 pipeline/target: implement as variant types
Go doesn't really do variants, so we must somehow emulate it. The
json objects we use are essentially tagged unions, with a `name`
field in reverse domain name notation identifying the type and a
type specific 'options' object.

In Go we represent this by having an BarOptions interface, which
implements a private method `isBarOptions()`, making sure that only
types in the same package are able to implement it. Each type FooBar
that should belong to the variant implements the interface, and a
constructor `NewFooBar(options *FooBarOptions) *Bar` that makes sure
the `name` field is set correctly.

This would be enough to represent our types and marshal them into
JSON, but unmarshalling would not work (json does not know about
our tags, so would not know what concrete types to demarshal to).
We therefore must also implement the Unmarshall interface for Bar,
to select the right types for the Options field.

We implement his logic for Target, Stage and Assembler. A handful
of concrete types are also implemented, matching what osbuild
supports.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-28 17:49:07 +02:00
Lars Karlitski
87bcd7f9d3 worker: run osbuild 2019-09-27 17:42:52 +02:00
Tom Gundersen
f880581a14 weldr/store: keep the compose status up-to-date
This way it can be correctly exposed in the API. We listen on a channel
from the job-queue, where status updates are pushed when the worker is
running/finished (or, in the future, failed).

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-27 14:34:51 +02:00
Lars Karlitski
cfe89eaed5 Add simple osbuild-worker
It doesn't actually build anything yet, but talks to the queue API.
2019-09-26 23:58:03 +02:00
Tom Gundersen
3221112d35 composer/job-queue: add scaffolding for the job queue API
This is by no means done, and needs more tests, docs and bugfixes,
but push it early so we have a common base to work on.

Based on work by Martin Sehnoutka.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-26 19:53:06 +02:00
Lars Karlitski
6576fe45dc main: move http serving to weldr
Also make sure that /run/weldr exists.
2019-09-26 19:53:06 +02:00
Martin Sehnoutka
0861b80c99 create jobs queue for scheduling new builds 2019-09-25 14:31:15 +02:00
Tom Gundersen
da887a43fb tree-wide: format the code
$ go fmt ./...

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-24 01:03:27 +02:00
Tom Gundersen
b60f580d92 tree-wide: use a standard project layout
Keep main.go files under cmd/ and internal libraries under internal/.
This will allow us to add more exutables under cmd/ (whereas only one
was possible when main.go was kept in the root).

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-24 00:03:53 +02:00