Commit graph

679 commits

Author SHA1 Message Date
Ondřej Budai
247e8cb205 update readmes to use the new osbuild-composer package 2020-03-25 11:43:39 +01:00
Ondřej Budai
b45119c471 spec: sync the line order and whitespace changes
In the new spec file, the tests-related code is always at the bottom of the
containing block to make the conditional code easier to read. Do the same
in the old spec file.

Also, in the new spec file, some whitespaces are different (rpmlint reported
mix used of tabs and spaces), let's do the same changes in the old spec file.
2020-03-25 11:43:39 +01:00
Ondřej Budai
cdc4248909 spec: use the standard way to represent a go dependency
This isn't a functional change.
2020-03-25 11:43:39 +01:00
Ondřej Budai
64181e7ba0 ci/rpm: use the old spec file for f31
Prior this commit the rpm for fedora 31 was built using the new spec file.
The old spec file should be used for that Fedora version.
2020-03-25 11:43:39 +01:00
Ondřej Budai
48a51611ec makefile: use the new spec file for rpm targets
This commit switches the rpm and srpm targets to use the new spec file.
Also, it introduces old-rpm and old-srpm targets to build an rpm from
the old spec file.

Note that the rpm target builds also the tests subpackages (they are not
built by default).
2020-03-25 11:43:39 +01:00
Ondřej Budai
0b08eb1904 spec: add spec file for osbuild-composer package
We successfully renamed the package in Fedora to osbuild-composer
(RHBZ #1815577). This commit adds the new spec file.

Changes: the tests subpackage is not built by default, to build it
--with tests parameter must be passed to rpmbuild.

Let's hope both spec files stay in sync.
2020-03-25 11:43:39 +01:00
Lars Karlitski
ee752b0ab8 tree-wide: panic when json marshalling fails
According to the new guidelines in docs/errors.md.

Note that this does not include code that marshals to a writer that
might fail (when a connection drops, for example).
2020-03-25 10:22:16 +01:00
Lars Karlitski
ad1a12f7aa tree-wide: use uuid.MustParse() when possible
According to the new guidelines in docs/errors.md.
2020-03-25 10:22:16 +01:00
Lars Karlitski
9625c589b0 docs: add errors.md
docs/errors.md are guidelines for how we handle errors. It's meant to be
expanded as we figure out more rules.
2020-03-25 10:22:16 +01:00
Brian C. Lane
899d38422c weldrcheck: Adding integration tests for modules 2020-03-25 10:01:14 +01:00
Brian C. Lane
b5da15e2d8 client: Add /modules/ support 2020-03-25 10:01:14 +01:00
Brian C. Lane
d530d9a87e weldr: Fix projects/depsolve route handling
Return a 400 with json error message when no packages are included, with
or without trailing /
2020-03-25 10:01:14 +01:00
Brian C. Lane
3bddfd3449 weldrcheck: Adding integration tests for projects 2020-03-25 10:01:14 +01:00
Brian C. Lane
271d27a41d client: Add /projects/ support 2020-03-25 10:01:14 +01:00
Brian C. Lane
5ac3cb3f46 rpmmd: Fix the format of BuildTime to match the weldr API
The format of the BuildTime returned by /projects/list and /modules/list
does not include the 'Z' at the end. This fixed the format and adjusts
the tests.
2020-03-25 10:01:14 +01:00
Lars Karlitski
ce6dc53a12 weldr: also pass custom sources to Store.PushCompose()
Also move assembly of a list of all repositories (base + custom sources)
into its own function.
2020-03-24 22:29:41 +01:00
Lars Karlitski
3544590036 store: move adding the local target to weldr
The automatic local target is only needed when accessing the API via
weldr.

In the store, the target was only added when `stateDir` was not `nil`.
This is only used for testing which doesn't exercise the branch in
weldr. Thus, the same check is not needed there.
2020-03-24 22:29:41 +01:00
Lars Karlitski
7594cb262e store: remove job.ImageType
It is not used.

We probably need it at some point to decide which worker to give which
job to.
2020-03-24 22:29:41 +01:00
Lars Karlitski
25e926bc36 store: use Job.ImageBuildID to fetch image build
ImageBuildID is an index into Compose.ImageBuilds. Use that directly
instead of looping over ImageBuilds and matching on OutputType.
2020-03-24 22:29:41 +01:00
Lars Karlitski
f386ae43dc jobqueue: remove Job.OutputType
It is not used. The worker does not need to know the name of the
originating output type. It has all information it needs in the
manifest.
2020-03-24 22:29:41 +01:00
Martin Sehnoutka
e43a30e4cf osbuild-dnf-json-tests: Introduce depsolving test
This test makes sure we can run depsolve for all build packages sets and
base packages sets for all image types for all architectures for all
defined Fedora versions. In could run the same for RHEL, but I haven't
yet implemented it, because such tests cannot run in public Internet.
2020-03-24 20:45:30 +01:00
Martin Sehnoutka
e1e5c1bb0b distro: include xz in distro build packages
xz compression is used for the ami and tar output types, it should be in
the buildroot. This fixes Weldr integration test (/cmd/osbuild-tests) on
ARM architecture (aarch64).

The fact that it worked on x86_64 was just a happy coincidence because
we require grub2-pc which in turn requires dracut which requires xz. We
should not rely on these implicit dependencies because we need xz
unconditionaly, therefore adding it to build packages for all platforms.
2020-03-24 20:45:30 +01:00
Martin Sehnoutka
7e69299259 dnf-json: allow passing arch as an argument
dnf can do cross-arch depsolving, but in case repositories for multiple
arches are provided and it is running on a different architecture than
the image build requires, it can lead to errors.

This patch makes sure that we only include packages from the
architecture we want.

It uses substitutions as defined in dnf documentation:
https://dnf.readthedocs.io/en/latest/api_conf.html#dnf.conf.Conf.substitutions
Unfortunately the docs are very sparse on details and the Fedora docs
are not updated any more:
https://docs.fedoraproject.org/en-US/Fedora/26/html/System_Administrators_Guide/sec-Using_DNF_Variables.html
Also dnf team is migrating the code to libdnf:
https://github.com/rpm-software-management/libdnf
which does not yet have any documentation.
2020-03-24 20:45:30 +01:00
Martin Sehnoutka
9d2dacbcab internal: pass architecture from the APIs to dnf-json
The following commit will introduce support for forced architecture in
dnf-json. The APIs already have this kind of information, so we can
simply pass it to the Depsolve and FetchMetadata functions.
2020-03-24 20:45:30 +01:00
Jakub Rusz
6f1c461c32 tests: use testify for existing blueprint_test functions 2020-03-24 20:37:10 +01:00
Jakub Rusz
833e798f1b tests: test for blueprint.GetPackages()
This also tests Package with "*" specified as version
2020-03-24 20:37:10 +01:00
Alexander Todorov
4c39223497 tests: Add coverage notes and link to Coveralls in docs
also updates the path to the resulting RPMs b/c that seems to have
chagned.
2020-03-24 20:27:16 +01:00
Tom Gundersen
a385d91565 osbuild-pipeline: print more helpful info in case invalid input is given
Use the List*() functions to print the valid options in case an invalid
one is given.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 15:46:57 +01:00
Tom Gundersen
f903601ec4 distro: add ListArchs() to Distro interface
Similar to ListImageTypes() add a helper to enumerate the supported
architectures.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 15:46:57 +01:00
Tom Gundersen
4a7519807e common/Architecture: drop unused code
The intended use for this will be repalced by Arch objects, currently
it was unused, so drop it.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 15:46:57 +01:00
Tom Gundersen
c66db42677 weldr/upload: uploadRequestToTarget() cannot fail
Drop the error return and simplify code accordingly.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 14:13:03 +01:00
Tom Gundersen
87b0bb6e5d distro: mass cleanup
Delete unused methods and make types and fields private where
possible. Some code is moved around, but apart from that there
is no change in behavior.

The naming of the distros were moved back into the distro
packages as the common types now only had one user, and this
allowed us to drop some redundant error checking.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 14:13:03 +01:00
Tom Gundersen
cf2ad51243 compose/ImageBuild: drop the distro field
This likely needs to be reintroduced in some fashion, but it was
unused, and when we reintroduced it it should be as a real Distro
object.

For now, drop it.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 14:13:03 +01:00
Jiri Kortus
f5cf649511 Fix typo in RootFilesystemUUID
RootFilesystemUUDI -> RootFilesystemUUID
2020-03-24 11:01:20 +01:00
Jakub Rusz
6603ae8d74 spec: add testify to BuildRequires 2020-03-24 10:59:15 +01:00
Alexander Todorov
5a0ccd55db tests: Add sanity tests for sources. Refs #315, #356 2020-03-23 21:10:46 +01:00
Brian C. Lane
3af34fba3f test: Move temporary repo setup into test/helpers.go
This allows it to be shared between the dnf-json tests and weldrcheck.
2020-03-23 21:08:01 +01:00
Brian C. Lane
222d09499c weldr: Improve the error message from sourceNewHandler 2020-03-23 21:08:01 +01:00
Brian C. Lane
a87c0ee44d Makefile: Fix osbuild-image-tests build 2020-03-23 21:08:01 +01:00
Brian C. Lane
6e40729cd3 weldr: Update the sourceNewHandler to use the correct struct
The store supports different fields than the API, this switches to using
an API specific struct with support for proxy and gpgkey_files.
2020-03-23 21:08:01 +01:00
Brian C. Lane
01aca4ecf7 weldrcheck: Adding integration tests for projects/source 2020-03-23 21:08:01 +01:00
Brian C. Lane
52febbdfa6 weldrcheck: Setup a temporary repository for testing 2020-03-23 21:08:01 +01:00
Brian C. Lane
3005486cb4 client: Add projects/source API support 2020-03-23 21:08:01 +01:00
Brian C. Lane
971bafbc09 Move the API struct definitions into internal/weldr
This will prevent problems with import loops. client already imports
weldr, so weldr cannot import client.
2020-03-23 21:08:01 +01:00
Brian C. Lane
0f8e40bdf1 weldrcheck: Fix typo in socket failure 2020-03-23 21:08:01 +01:00
Tom Gundersen
d642fd2c69 distro: drop redundant parts of the interfaces
Usage of these have now been entirely replaced with the newly
introduced interfaces. The individual distros should be cleaned up
and dead code removed, buth that is left for a future patch.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
7fe3421afb distro/test: use new interfaces
Test in terms of the newly introduced interfaces. This should not
have any functional effect.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
5de98458e5 ditro/registry: index by name
This is purely internal, and there is no benefit to translating
to a tag, a string identifier will do just fine.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
1ceb504738 osbuild-pipeline: resolve argumnets to Arch and ImageType objects
This gives us verification at the point of use for free and simplifies
the code a bit.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
d7aad58897 store/PushCompose: take ImageType and Arch as argumnet rather than strings
Resolve eagerly into real objects, and avoid having to resolve and
error-check again.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00