Commit graph

221 commits

Author SHA1 Message Date
Lars Karlitski
740e4da733 rpmmd: move SourceToRepo to store
This removes the `store` dependency from `rpmmd`, which is the more
generic package.

It's also nicer to have this on a method of `SourceConfig`.
2019-11-18 12:22:10 +01:00
Jacob Kozol
1b46a83522 store: add getBlueprintChange update blueprint commit message
Add function to get a single blueprint change given the blueprint name
and the commit hash. Update the PushBlueprint function to allow a commit
message to be passed to it which will be used when adding a change.
2019-11-14 20:36:17 +01:00
Jacob Kozol
080bd4968c store: add function to get blueprint changes
For a given blueprint name the store returns a list of the changes made
to that blueprint.
2019-11-13 14:51:06 +01:00
Jacob Kozol
7d2650a2e3 store: add blueprint changes
When a blueprint is pushed to the store it will also add a change to the
BlueprintChanges map. Each blueprint has a list of changes mapped to
their commit. This commmit is a hex string based off of the sha1 hash of
the current time so every commit is unique. The message and timestamp
follow the format of lorax-composer.
2019-11-13 14:51:06 +01:00
Lars Karlitski
f9cbc8593f distro: add DetectHost
lorax-composer uses the host's repositories for building images. This is
prone to accidental configuration errors and duplicates functionality
(adding custom repositories via the source API is much more explicit).

However, blueprints don't specify the distribution they're based on.
This is something they should do in the future to enable cross-distro
image builds. Until then, read `/etc/os-release` to detect the host OS
and use that as the base distro for a blueprint.

Detection works by concatenating the ID field with a `-` and the
VERSION_ID field. This mandates how additional distributions should
register themselves to the `distro` package.

If composer cannot build the detected distro, fall back to fedora-30.

Use this detection everywhere that fedora-30 was hard-coded before.
2019-11-10 17:23:14 +01:00
Lars Karlitski
b33ed9e5d2 blueprint: move pipeline generation into its own package
Introduce the `distro` package, which contains an interface for OS
implementations. Its main purpose is to convert a blueprint to a
distro-specific pipeline.

Also introduce the `distro/fedora30` package. It is the first
implementation of the distro interface. Most of its code has been copied
with minimal modifications from the blueprint package.

The `blueprint` package is now back to serving a single purpose:
representing a weldr blueprint. It does not depend on the `pipeline`
package anymore.

Change osbuild-composer and osbuild-pipeline to use the new API,
hard-coding "fedora-30". This looks a bit weird now, but is the same
behavior as before.

All test cases now also take an "distro" key in the "compose" object.
2019-11-07 17:13:20 +01:00
Jacob Kozol
29ebb9ff51 store: add sources to store
The api is initialized with a base repo but users may want to add their
own custom repos. Based on lorax's implementation, the store now
supports sources. The source object is similar to but not the same as
the repos used by dnf. The store can now add, delete, and get info about
sources.
2019-11-06 21:40:31 +00:00
Ondřej Budai
0feb4e4d44 api: improve error handling when compose push fails
There was missing condition for the case when compose push fails.
As the previous commit introduces customizations which can return even more
errors it is important to report such errors.

For now the API is returning only 500 HTTP code. In future we should
distinguish between failures and blueprint validation errors.
2019-11-02 14:10:13 +01:00
Ondřej Budai
93e7a26785 blueprint: fix groups definition
Group has no version field, remove it.
2019-10-31 21:49:09 +01:00
Jacob Kozol
4366e390cf store: add image size to compose status response
All composes now include the image size in their status response. The
image size will be 0 except for finished composes where it will be the
file length in bytes.
2019-10-28 02:02:16 +01:00
Jacob Kozol
4d94207488 store: update Image to include size
Alongside File, Name, and Mime type, each Image in the store will
contain the image size. This will default to 0 unless the compose's
status is FINISHED. Then, it  will be the length in bytes of the file.
2019-10-28 02:02:16 +01:00
Jacob Kozol
320eaf4b0c store: update ComposeEntry to include version and type
Each compose now includes the blueprint version and compose type it is
based off of.
2019-10-28 02:02:16 +01:00
Tom Gundersen
f055ba2e07 store: change semantics of queue states
A job is now in "WAITING" state exactly when it is in the channel,
once it is popped it enters "RUNNING" state. It is only possible
to update the state of a job that is in the running state.

This mean that updating to "RUNNING" is entirely optional, but in
the future we may want to use this as a watchdog logic, and require
the worker to update at regular intervals to avoid being restarted.

The job queue API is updated to require a POST followed by one
or several PATCH messages to the returned ID. If a patch is sent
to an ID before the POST it is as if the object does not exist
(regarldess of it being in the queue in WAITING state or not).

Once a job has been POSTed it can be PATCHed to update it zero or
more times with (still) RUNNING before exactly oncee with either
FINISHED or FAILED.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-25 10:18:47 +02:00
Ondřej Budai
15b82a15d2 osbuild-composer: Rename module to github.com/osbuild/osbuild-composer
This should be the best practice according to other popular go projects:
- https://github.com/prometheus/prometheus
- https://github.com/syncthing/syncthing
- https://github.com/drone/drone
- https://github.com/hashicorp/terraform

Also, this change fixes go get command (it currently fails due to bad package
name).
2019-10-08 21:44:57 +02:00
Tom Gundersen
4845826048 blueprint: add proper error handling for querying invalid output types
Otherwise an invalid output type would crash composer.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-08 21:03:45 +02:00
Tom Gundersen
94e1e6f42b store: add error types and rudimentary error handling
This will allow us to better distinguish between different error
conditions in the API.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
2e4063c78b job: drop the job package
This was only used internally in the store package, so moved the
one type that was still used and remove the rest.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
68ade23fcc store: make state (de)serialization internal
This hides the state hanlding in the store package. As before, it
can be disabled by passing `nil` instead of the path to the state
file.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
9cb140795e store: add {Push,Pop}Compose methods and hide channel
Wrap the channel in Pop and Push methods, so it is not exposed to
the callers. PushCompose replaces the old AddCompose for consistency,
and PopCompose simply reads from the other end of the channel.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
3ff4f59fc7 store: pass the store to the jobqueue API
Drop the jobUpdates channel, and instead add an UpdateCompose method
to the store, which updates the status of a compose directly.

This allows us to report back errors directly, rather than having to
mirror the staet in the jobqueue API.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00
Tom Gundersen
4fcb5d66fa store: make the Store a package of its own
This encapsulates all the state of osbuild, no longer just the state specific
to the weldr API. Make it a first class type, preparing for direct access from
both the weldr and jobqueue APIs.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-07 10:37:43 +02:00