Commit graph

135 commits

Author SHA1 Message Date
Jacob Kozol
5fe42f13d7 blueprint: add change object for blueprints
Add a struct to store changes made to a blueprint. Each change contains
a commit which is a hex string based off of an sha1 hash, a message
describing the change, a revision which will usually be null, a
timestamp, and the blueprint at the time of the change.
2019-11-13 14:51:06 +01:00
Lars Karlitski
b33ed9e5d2 blueprint: move pipeline generation into its own package
Introduce the `distro` package, which contains an interface for OS
implementations. Its main purpose is to convert a blueprint to a
distro-specific pipeline.

Also introduce the `distro/fedora30` package. It is the first
implementation of the distro interface. Most of its code has been copied
with minimal modifications from the blueprint package.

The `blueprint` package is now back to serving a single purpose:
representing a weldr blueprint. It does not depend on the `pipeline`
package anymore.

Change osbuild-composer and osbuild-pipeline to use the new API,
hard-coding "fedora-30". This looks a bit weird now, but is the same
behavior as before.

All test cases now also take an "distro" key in the "compose" object.
2019-11-07 17:13:20 +01:00
Ondřej Budai
913dc8d6c9 blueprint: integrate customizations with all output types
Customizations are ready to be used, this commit enables them. There are still
some unresolved TODOs to be solved in future. Also, tests are missing.
2019-11-02 14:10:13 +01:00
Ondřej Budai
4b546f1bbf blueprint: don't create group with the same as one of the users 2019-10-31 21:51:02 +01:00
Ondřej Budai
dc359fca3a blueprint: add support for extra packages, "modules" and groups
"modules" because they're not actual modules. Currently, it's just
a synonym to packages. This is the same behavior as lorax-composer has.
2019-10-31 21:49:09 +01:00
Ondřej Budai
93e7a26785 blueprint: fix groups definition
Group has no version field, remove it.
2019-10-31 21:49:09 +01:00
Ondřej Budai
705338c5b3 blueprint: add support for unencrypted passwords in user customization 2019-10-31 21:46:39 +01:00
Lars Karlitski
e7ae3b4c47 test: move image-info tests to test/
These tests (will) test more than just image-info: they'll take a
blueprint, verify that `osbuild-pipeline` generates the correct
pipeline, run osbuild with that pipeline and verify that the resulting
image has the expected image-info output.

This change only includes the latter half (i.e., only moves the already
existing tests).

Also drop python's unittest. It was hard to control output (important
for quickly spotting failures and to make travis happy). This introduces
test/run, which runs all test cases in test/cases or the ones given on
the command line.

When a failure occurs, it prints a diff of the actual and the expected
image info.
2019-10-30 01:25:44 +01:00
Lars Karlitski
fabd40da1c blueprint: always run selinux stage last
A few output types had it before the fix-bls stage.
2019-10-29 16:01:37 +01:00
Lars Karlitski
6a6144c2d9 blueprint: always exclude the rescue kernel
Its BLS entry always expects a separate boot partition. Skip it on all
images until the rescue kernel package is usable.
2019-10-29 16:01:37 +01:00
Lars Karlitski
930347c56e blueprint: move vmdk towards lorax parity
The main difference (according to image-info) is an additional package
containing a gpg key which was used to verify packages. The one
generated by lorax-composer doesn't have this, because it doesn't verify
signatures.

Also, lorax generates an image with a separate /boot partition. This is
not yet addressed here, because osbuild doesn't support it yet.
2019-10-29 16:01:37 +01:00
Lars Karlitski
cde1325bc3 blueprint: move tar towards lorax parity
The main difference (according to `rpm -qa`) is an additional package
containing a gpg key which was used to verify packages. The one
generated by lorax-composer doesn't have this, because it doesn't verify
signatures.
2019-10-29 16:01:37 +01:00
Lars Karlitski
d4e1ecea7e blueprint: move partitioned-disk towards lorax parity
The main difference (according to image-info) is an additional package
containing a gpg key which was used to verify packages. The one
generated by lorax-composer doesn't have this, because it doesn't verify
signatures.

Also, lorax generates an image with a separate /boot partition. This is
not yet addressed here, because osbuild doesn't support it yet.
2019-10-29 16:01:37 +01:00
Lars Karlitski
fe46fd3a89 blueprint: move ext4-filesystem towards lorax parity
The main difference (according to image-info) is an additional package
containing a gpg key which was used to verify packages. The one
generated by lorax-composer doesn't have this, because it doesn't verify
signatures.
2019-10-29 16:01:37 +01:00
Tom Gundersen
8c0561c054 blueprint/test: express test-cases in json
A test case is now expressed purely by dropping in a json document in
`tools/test_image_info/pipelines`. It should contain the input compose
(output format and blueprint) as well as the expected pipeline. All the
existing tests are moved over to this format.

This shares the same infrastructure as the image tests, ideally we want
to run the blueprint tests and the image tests against the same pipelines.

For now, test cases are skipped from the blueprint tests if they do not
contain a 'compose' section, and from the image tests if they do not
cotain an 'expected' section. In the future we may want to make both
mandatory.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-28 17:32:45 +01:00
Tom Gundersen
2e0deff629 submodule: bump to current head
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-28 14:14:09 +01:00
Ondřej Budai
8d010c0466 blueprint: add support for kernel customization 2019-10-25 14:37:21 +02:00
Ondřej Budai
967b0e8ce0 blueprint: add initial support for customizations
List of currently unsupported ones:
- [[repos.git]]
- [customizations.kernel]
- [[groups]]
- [[packages]] and [[modules]]

Some of customizations have unimplemented behaviour, see TODOs
2019-10-24 15:13:47 +02:00
Martin Sehnoutka
ed2dd9829c Adjust Openstack output to the Lorax one 2019-10-24 15:12:21 +02:00
Martin Sehnoutka
bf73ee019e Adjust VHD output to the Lorax one 2019-10-23 20:52:22 +02:00
Martin Sehnoutka
4ef4112a12 Adjust AMI output to the Lorax one 2019-10-23 01:05:14 +02:00
Martin Sehnoutka
96d401b212 QCOW2 image generation and output test 2019-10-22 13:52:36 +02:00
Tom Gundersen
09ac84926e blueprint: add unit tests for ex4 output type
Also fix a bug where the RawFs assembler was not correctly unmarshaled.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-21 23:17:49 +02:00
Tom Gundersen
32be0d05cb blueprint: add unit tests to public methods
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-21 23:17:49 +02:00
Tom Gundersen
b63cda530e blueprint: document error type
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-21 23:17:49 +02:00
Tom Gundersen
c51f1833f9 blueprint/output: add missing test files
These files were left out of 0272fb8815.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-21 23:17:49 +02:00
Tom Gundersen
0272fb8815 blueprint/output: add unit tests
These all follow the same scheme as was introduced for the qcow2
output type.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-21 13:35:05 +02:00
Tom Gundersen
5a82c95c9a blueprint/qcow2: move wanted pipeline into sepratate file
This will allow us to run tests on the generated images from the
same files in the future.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-21 13:35:05 +02:00
Tom Gundersen
d302ac6d09 blueprint/output/qcow2: add unit tests
Use the autogenerated test framework from VSCode (with minor fixes to
make them compile).

As we do not yet support customizations in blueprint, all the tests are
trivial.

I was not entirely sure about the best way to encode the wanted pipeline
output. I currently represent it as a JSON string and unmarshal it into
the object to compare with.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-18 15:57:31 +02:00
Ondřej Budai
6e790bc62d osbuild-composer: Fix imports broken by new module name
We've merged some stuff with old module name used in imports. This
commit fixes it.
2019-10-09 08:06:07 +02:00
Ondřej Budai
15b82a15d2 osbuild-composer: Rename module to github.com/osbuild/osbuild-composer
This should be the best practice according to other popular go projects:
- https://github.com/prometheus/prometheus
- https://github.com/syncthing/syncthing
- https://github.com/drone/drone
- https://github.com/hashicorp/terraform

Also, this change fixes go get command (it currently fails due to bad package
name).
2019-10-08 21:44:57 +02:00
Tom Gundersen
22254637f8 blueprint: generate real f30 images for all the output types
This is still just a placeholder, and in particular the live CD support
is not right.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-08 21:03:45 +02:00
Tom Gundersen
4845826048 blueprint: add proper error handling for querying invalid output types
Otherwise an invalid output type would crash composer.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-08 21:03:45 +02:00
Tom Gundersen
b1688cda76 blueprint: add infrastructure for translating to blueprints
Each output type will have its own translator from a blueprint to
a pipeline.

In the future, we may also want to distinguish between different
base distributions and architectures, but for now we keep it
simple and hardcode to a given one.

Some placeholder output formats are added, these have not been
tested or reserached, but are included to show how the it will
all fit together.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-05 20:44:32 +02:00
Tom Gundersen
949d3464b5 weldr: split blueprint into its own package
This will contain all the translation logic from blueprints to pipelines,
which will constitute the main part of composer, so let's separate that
out from the API.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-10-05 20:44:32 +02:00