Commit graph

45 commits

Author SHA1 Message Date
Martin Sehnoutka
4dbba6398a fix worker service template
It is not properly named so it does not work for starting multiple
instances. This should fix it.
2019-10-04 01:49:16 +02:00
Tom Gundersen
859bc0ad2e pipeline: clean up and document
This finishes the initial version of the pipline package, adding
documentation, but still missing unittests.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-03 12:49:21 +02:00
Tom Gundersen
41c6f5dd0b image-info: include partition table id
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-03 12:35:51 +02:00
Martin Sehnoutka
ed185b41ce Integration of osbuild composer with systemd
osbuild-composer now uses socket activation instead of hardcoded paths
in the code. osbuild-worker is an http client therefore it uses only
service unit. osbuild-worker must be started after the socket is
created. osbuild-composer service requires osbuild-worker to run, because without
it no jobs can be started.

osbuild-composer is executed as a regular user (newly created
_osbuild-composer user) as opposed to the worker which must run as root
in order to execute osbuild itself
2019-10-02 17:56:59 +02:00
Jacob Kozol
d23d57bc31 Update api and store for parity with lorax
Add parameters to the api responses that while not displayed by
cockpit-composer are used in props validation or blueprint updates.
2019-10-02 15:25:02 +02:00
Lars Karlitski
b170ea036c image-info: include output from rpm --verify
This shows the changes an image has relative to what its rpm database
thinks is installed. Output is:

    "rpm-verify": {
        "missing": [ <missing files> ],
        "changed": { <map from filename to rpm attribute octet > }
    }

Alas, this makes running image-info slower.
2019-10-01 17:05:10 +02:00
Lars Karlitski
01b7402ce2 image-info: include partition label and uuid 2019-09-30 22:28:28 +02:00
Lars Karlitski
9867846ff4 tools/image-info: consistent use of - over _
And remove a stray comment and whitespace.
2019-09-30 22:28:28 +02:00
Tom Gundersen
e43955a4c5 image-info: include the image format
Use qemu-img to query the image format and include that in the report.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-30 15:15:14 +02:00
Lars Karlitski
bf3d7fcdca image-info: add basic support for multiple partitions
For each partition, find out if its the root or boot partition and
gather only the relevant information. Make sure that we don't get
information from /boot twice.
2019-09-30 13:32:53 +02:00
Lars Karlitski
172a23bdb2 image-info: bootloader=unknown instead of creashing 2019-09-30 13:32:53 +02:00
Lars Karlitski
457f5f697e image-info: assume bootable=False if key is not present
sfdisk doesn't include the "bootable" key in its output when a partition
is not marked as bootable.
2019-09-30 13:32:53 +02:00
Tom Gundersen
96f3cbf655 weldr/store/compose: store information exposed in the API
Use the exact same status strings as is used in the API,
making it clearer that they are the same (and avoiding any
translation). Remember the creation/start/finish timestamps.
And store the output type.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-29 14:51:22 +02:00
Tom Gundersen
6c49acbd51 pipeline: add locale stage
This was left out from 7625d26ff5.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-29 14:51:22 +02:00
Lars Karlitski
5fbc734a15 tools: add image-info
Rough draft of image-info, a tool that extracts high-level information
about an os image. It prints this information in JSON form on stdout.
Run it like this:

  $ tools/image-info <image>

It supports all images that qemu-ndb supports.
2019-09-29 13:44:56 +02:00
Tom Gundersen
d231694bae worker/target/local: copy to the right location
We were copying the containing directory from our object store, we
only want the contents.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-28 17:49:07 +02:00
Tom Gundersen
7625d26ff5 pipeline/target: implement as variant types
Go doesn't really do variants, so we must somehow emulate it. The
json objects we use are essentially tagged unions, with a `name`
field in reverse domain name notation identifying the type and a
type specific 'options' object.

In Go we represent this by having an BarOptions interface, which
implements a private method `isBarOptions()`, making sure that only
types in the same package are able to implement it. Each type FooBar
that should belong to the variant implements the interface, and a
constructor `NewFooBar(options *FooBarOptions) *Bar` that makes sure
the `name` field is set correctly.

This would be enough to represent our types and marshal them into
JSON, but unmarshalling would not work (json does not know about
our tags, so would not know what concrete types to demarshal to).
We therefore must also implement the Unmarshall interface for Bar,
to select the right types for the Options field.

We implement his logic for Target, Stage and Assembler. A handful
of concrete types are also implemented, matching what osbuild
supports.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-28 17:49:07 +02:00
Lars Karlitski
87bcd7f9d3 worker: run osbuild 2019-09-27 17:42:52 +02:00
Tom Gundersen
5df6874b94 target: add constructor
We only support local target for now, but this avoids having to
open code it.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-27 15:54:13 +02:00
Tom Gundersen
f880581a14 weldr/store: keep the compose status up-to-date
This way it can be correctly exposed in the API. We listen on a channel
from the job-queue, where status updates are pushed when the worker is
running/finished (or, in the future, failed).

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-27 14:34:51 +02:00
Tom Gundersen
cad89c6650 weldr: keep composes in the store
This means they are serialised and restarted in case composer is
crashed/restarted. We also need this in the future to track their
state, and allow them to be listed in the UI.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-27 01:38:41 +02:00
Tom Gundersen
0dc30d7f1b weldr/compose: return the build-id when starting a compose
The return argument was ommitted. Also move to using the uuid package
wherever that makes sense.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-27 00:51:15 +02:00
Lars Karlitski
cfe89eaed5 Add simple osbuild-worker
It doesn't actually build anything yet, but talks to the queue API.
2019-09-26 23:58:03 +02:00
Tom Gundersen
0bdd3a5c89 job-queue: pass a well-formed job object to the worker
For now we will hardcode the org.osbuild.local target, so we might
as well fix this up front.

We do not yet support any target types, but for testing purposes we
claim to support 'tar', and we pass a noop tar pipeline to the worker.

This makes introspecting the job-queu api using curl a bit more
pleasant.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-26 23:48:19 +02:00
Tom Gundersen
3221112d35 composer/job-queue: add scaffolding for the job queue API
This is by no means done, and needs more tests, docs and bugfixes,
but push it early so we have a common base to work on.

Based on work by Martin Sehnoutka.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-26 19:53:06 +02:00
Lars Karlitski
fa7068b59b weldr: act as if we can produce a tar output
We can't yet, but this is useful for testing.
2019-09-26 19:53:06 +02:00
Lars Karlitski
6576fe45dc main: move http serving to weldr
Also make sure that /run/weldr exists.
2019-09-26 19:53:06 +02:00
Jacob Kozol
e24bfcf7d0 Add blueprint workspace diff
The list of changes between the committed blueprint and its workspace
are returned when a user requests the diff. Each change includes the
new(added) or the old(removed) package.
2019-09-26 19:52:28 +02:00
Jacob Kozol
e14c48ff61 Fix package name-version for wildcard version
If a package had version "*" and its name-version would be "name-*".
This would match all packages containing "name" in their name. Instead
the name-version is "name-*-*" so it matches any version of the package.
2019-09-26 13:13:03 +02:00
Martin Sehnoutka
0861b80c99 create jobs queue for scheduling new builds 2019-09-25 14:31:15 +02:00
Martin Sehnoutka
7df735e36b minimal travis configuration 2019-09-24 17:50:48 +02:00
msehnout
b700cd7a51
Merge pull request #1 from teg/layout
tree-wide: use a standard project layout
2019-09-24 16:32:57 +02:00
Tom Gundersen
3533597dde weldr: add missing arguments to constructor calls
In the test we don't need to serialize the state, so pass nil to
the costructor.
2019-09-24 01:09:58 +02:00
Tom Gundersen
da887a43fb tree-wide: format the code
$ go fmt ./...

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-24 01:03:27 +02:00
Tom Gundersen
b60f580d92 tree-wide: use a standard project layout
Keep main.go files under cmd/ and internal libraries under internal/.
This will allow us to add more exutables under cmd/ (whereas only one
was possible when main.go was kept in the root).

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-09-24 00:03:53 +02:00
Lars Karlitski
11af3f0c0b main: save state of weldr server to /var/lib 2019-09-15 18:51:21 +02:00
Lars Karlitski
73655f600e weldr: add support for the blueprint workspace 2019-09-15 14:08:46 +02:00
Lars Karlitski
6089ebe077 .gitignore: ignore osbuild-composer binary 2019-09-14 13:59:08 +02:00
Lars Karlitski
59161eb310 weldr: start testing the API 2019-09-14 13:58:49 +02:00
Lars Karlitski
8cde1b6bd5 weldr: always return original offset and limit values 2019-09-14 13:58:21 +02:00
Lars Karlitski
f0e2e3786f weldr: use source.Id instead of .Name for keys 2019-09-14 13:57:50 +02:00
Lars Karlitski
2ee12e2b51 main: add -v command line flag
Only log accesses when -v is given.
2019-09-14 13:56:55 +02:00
Lars Karlitski
a63a408c06 main: remove stale socket, if it exists 2019-09-14 08:44:52 +02:00
Lars Karlitski
dd3b837d8d weldr: remove Composer interface
No need to have an interface we don't really use yet. This moves
fetching the list of packages out of weldr, which will ease testing.
2019-09-14 07:41:39 +02:00
Lars Karlitski
1adbf1a4a6 osbuild-composer 2019-09-13 18:50:13 +02:00