Commit graph

652 commits

Author SHA1 Message Date
Tom Gundersen
a385d91565 osbuild-pipeline: print more helpful info in case invalid input is given
Use the List*() functions to print the valid options in case an invalid
one is given.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 15:46:57 +01:00
Tom Gundersen
f903601ec4 distro: add ListArchs() to Distro interface
Similar to ListImageTypes() add a helper to enumerate the supported
architectures.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 15:46:57 +01:00
Tom Gundersen
4a7519807e common/Architecture: drop unused code
The intended use for this will be repalced by Arch objects, currently
it was unused, so drop it.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 15:46:57 +01:00
Tom Gundersen
c66db42677 weldr/upload: uploadRequestToTarget() cannot fail
Drop the error return and simplify code accordingly.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 14:13:03 +01:00
Tom Gundersen
87b0bb6e5d distro: mass cleanup
Delete unused methods and make types and fields private where
possible. Some code is moved around, but apart from that there
is no change in behavior.

The naming of the distros were moved back into the distro
packages as the common types now only had one user, and this
allowed us to drop some redundant error checking.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 14:13:03 +01:00
Tom Gundersen
cf2ad51243 compose/ImageBuild: drop the distro field
This likely needs to be reintroduced in some fashion, but it was
unused, and when we reintroduced it it should be as a real Distro
object.

For now, drop it.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 14:13:03 +01:00
Jiri Kortus
f5cf649511 Fix typo in RootFilesystemUUID
RootFilesystemUUDI -> RootFilesystemUUID
2020-03-24 11:01:20 +01:00
Jakub Rusz
6603ae8d74 spec: add testify to BuildRequires 2020-03-24 10:59:15 +01:00
Alexander Todorov
5a0ccd55db tests: Add sanity tests for sources. Refs #315, #356 2020-03-23 21:10:46 +01:00
Brian C. Lane
3af34fba3f test: Move temporary repo setup into test/helpers.go
This allows it to be shared between the dnf-json tests and weldrcheck.
2020-03-23 21:08:01 +01:00
Brian C. Lane
222d09499c weldr: Improve the error message from sourceNewHandler 2020-03-23 21:08:01 +01:00
Brian C. Lane
a87c0ee44d Makefile: Fix osbuild-image-tests build 2020-03-23 21:08:01 +01:00
Brian C. Lane
6e40729cd3 weldr: Update the sourceNewHandler to use the correct struct
The store supports different fields than the API, this switches to using
an API specific struct with support for proxy and gpgkey_files.
2020-03-23 21:08:01 +01:00
Brian C. Lane
01aca4ecf7 weldrcheck: Adding integration tests for projects/source 2020-03-23 21:08:01 +01:00
Brian C. Lane
52febbdfa6 weldrcheck: Setup a temporary repository for testing 2020-03-23 21:08:01 +01:00
Brian C. Lane
3005486cb4 client: Add projects/source API support 2020-03-23 21:08:01 +01:00
Brian C. Lane
971bafbc09 Move the API struct definitions into internal/weldr
This will prevent problems with import loops. client already imports
weldr, so weldr cannot import client.
2020-03-23 21:08:01 +01:00
Brian C. Lane
0f8e40bdf1 weldrcheck: Fix typo in socket failure 2020-03-23 21:08:01 +01:00
Tom Gundersen
d642fd2c69 distro: drop redundant parts of the interfaces
Usage of these have now been entirely replaced with the newly
introduced interfaces. The individual distros should be cleaned up
and dead code removed, buth that is left for a future patch.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
7fe3421afb distro/test: use new interfaces
Test in terms of the newly introduced interfaces. This should not
have any functional effect.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
5de98458e5 ditro/registry: index by name
This is purely internal, and there is no benefit to translating
to a tag, a string identifier will do just fine.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
1ceb504738 osbuild-pipeline: resolve argumnets to Arch and ImageType objects
This gives us verification at the point of use for free and simplifies
the code a bit.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
d7aad58897 store/PushCompose: take ImageType and Arch as argumnet rather than strings
Resolve eagerly into real objects, and avoid having to resolve and
error-check again.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
7d4ad6b649 rcm/depsolve: take ImageType as argumnet rather than strings
Use the already resolved real object, and avoid having to resolve
and error-check again.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
1597ae2ed6 rcm/submit: resolve into real Arch and ImageType objects
Rather than using the `common` package, parse the JSON as strings,
then resolve into real Arch and ImageType objects.

This should not be a behavioural change, but will simplify the code
in subsequent commits.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
96b4850554 rcm/submit: fix error handling
We were not correctly catching the case where more than one arch
or more than one image type was specified.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
8e5b79df96 weldr/uploadRequestToTarget: take ImageType as argumnet rather than a string
Resolve image type into a real object eagerly, and avoid having to
resolve and error-check again.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
bd86eaf6a5 weldr/composeTypesHandler: list image types per-arch
Currently all image types are supported on all arches, but in the
future we may want to restrict this. In that case, return the
image types that are valid for the arch in qusetion, rather than
all the possible ones.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
95203e0fff weldr/depsolveBlueprint: take ImageType as argumnet rather than strings
Resolve image type into a real object eagerly, and avoid having to
resolve and error-check again.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
090d01602b weldr/composeHandler: invert error handling
Check for errors and return early if they are found, rather than
check for the absence of errors.

This is not a functional change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
f201fc84b7 weldr: pin a real Arch object, rather than a string
weldr needs to know the host architecture. Rather than pinning
a string, pin a real Arch object, and query its name when we
need it.

This verifies the validitiy of the architecture for the given
distro before it is passed to weldr, rather than lazily on
demand.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
bbd8dc338d distro: introduce Arch and ImageType interfaces
Objects implementing these interfaces will represent the
architecture support for a given distro and the image type
support for a given architecture distro combination, respectively.

The idea is to always resolve to these objects early, and drop
the equilavent methods from the distro interface. This means that
we convert our input strings to real objects once, and then never
have to verify their correctness again.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
ff0ec01eb6 distro: drop NewDefaultRegistry()
Require the caller to pass in the required distros explicitly. This
would allow us to easily add distros in osbuild-pipeline and tests
before exposing them in composer itself, for instance.

This means there is no longer a dependency from the distro package
to each of the individual distros, so the distros are now able
to depend on the distro packag for types and interfaces.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
08ff8ab81b distro: the distro constructors cannot fail
Simplify the code by dropping the potential error return. The
constructor simply instantiates some maps, this cannot fail.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
1345ca77fd store: drop the distro registry
This was now unused.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-22 22:26:09 +01:00
Tom Gundersen
0656fdcd38 store: don't query the distro for the filename
The LocalTarget contains the filename, so look it up there instead.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-22 22:26:09 +01:00
Tom Gundersen
839b22026e target: pass the image filename in each target
Rather than having to assume that we only ever produce one
artifact, have each upload target contain the filename it expects
to upload from the osbuild output.

An image file is always explicitly named in the manifest, and we
leave it up to each distro to decide how this is done, but the
convention is to use the same image filename as used when
downloading the image through weldr.

Now make this policy explicit, by quering the distro for the image
name and inserting it into each upload target.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-22 22:26:09 +01:00
Tom Gundersen
61836a7079 weldr/upload: no longer reuse the structs from the target package
For serializeation, make our own private structs. The structs
in the target package are not exactly the same as the ones used by
weldr, so in order to avoid too many compromises, let's just do
an explicity translation.

As a general principle, we aim to only use private types for
serialization and rather translate than reuse for different
purposes.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-22 22:26:09 +01:00
Tom Gundersen
440a5acb41 weldr/upload: make translations between upload options explicit
The same types are used in the weldr API as internally. We want
to avoid sharing serialized types like this, as it easily leads
to layering vialotions.

For now just make the translation explicity, in a follow-up
we will introduce types dedicated to serialization in the weldr
API.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-22 22:26:09 +01:00
Lars Karlitski
1bb8f7eee0 osbuild-image-tests: add missing build constraints
Build constraints must be added to every file. We only want to build
this package when the `integration` tag is set.

Without this, every build prints this warning:

    # github.com/osbuild/osbuild-composer/cmd/osbuild-image-tests
    runtime.main_main·f: function main is undeclared in the main package
2020-03-22 13:50:28 +01:00
Tom Gundersen
d808ab484c gitignore: don't ignore files we do not generate
The rpms and tarballs generated by the Makefile all end up under
rpmbuild/, so there is no reason to ignore them explicitly.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-21 19:50:21 +01:00
Lars Karlitski
069ccba894 store: pass repos to PushCompose
Repositories were not passed down into `Distro.Manifest()`. This makes
all builds fail, because these repositories are use to access gpgkeys.
2020-03-21 11:08:16 +01:00
Jacob Kozol
afa4975a15 tools: add script to generate multiple test cases
This script generates json test cases for each supported output format.
It requires an architecure, distro, osbuild store, and output directory
as input. There is also a json object which maps from output format to a
test case request which includes the compose request and boot type.
The script uses these test case requests to call the generate-test-case
script and then outputs the resulting json test cases to files in the
specified output directory.
2020-03-20 13:55:36 +01:00
Jacob Kozol
1909620660 tools: add boot parameters as input to generate-test-case
The generate-test-cases no longer maps from output format to boot
type. Instead, the boot data is passed in through the standard input
alongside the composer-request object.
2020-03-20 13:55:36 +01:00
Jacob Kozol
ab930e7a0c tools: move and rename generate-test-case
The script generate-test-cases only creates a single test case per run.
Therefore, the name should be singular. Also, this script has been moved
to a directory which will contain all test case generators.
2020-03-20 13:55:36 +01:00
Ondřej Budai
7c1de17300 tests/image: use %#v when printing errors
When using %#v instead of %v, printing an error gives a slightly more
informative error message, therefore this commit switches to %#v in all
error prints.
2020-03-19 14:25:08 +01:00
Ondřej Budai
1b49755a7e tests/image: remove all distro restrictions
The distro argument and restrictions to run only tests for the same distro
as the host's one were confusing. This commit removes them. Now the behaviour
is following:

By default all the test cases in test case directory are run.
If test cases are given by arguments, they are all run, and test case
directory is ignored.
2020-03-19 14:25:08 +01:00
Ondřej Budai
832fb44eb8 tests/image: give the test VM only 1 GB of memory
The VM doesn't need 2 GBs to start Fedora and ssh server. This fix enables
us to run the tests in more constrained environment.
2020-03-19 14:25:08 +01:00
Ondřej Budai
3cb2a5f930 tests/image: remove the hack for fedora-30
Previously we had only tests for Fedora 30, therefore I added a hack to treat
every Fedora as Fedora 30, so the tests would run. Now, we have tests for
multiple Fedora versions, therefore the hack is no longer needed.
2020-03-19 14:25:08 +01:00
Ondřej Budai
acfb461aa5 tests/image: switch the implementation to go testing framework
We're currently rewriting all the integration tests to use the Go
testing framework. This commit does the switch for the image tests.

I decided not to use the testing framework in functions which are
not directly tight to testing (booting images, running osbuild). I think
it's reasonable to use classic error handling there and propagate the errors
to places directly tight to testing and use the testing library.
This enables us to reuse the code in different part of projects if needed.
2020-03-19 14:25:08 +01:00