Commit graph

2361 commits

Author SHA1 Message Date
Tomas Hozza
a9b676e43e Weldr API: return the actual getImageType() error in composeHandler()
Refactor the `composeHandler()` method to send the actual error
returned by `getImageType()` as an API response.

Modify tests to handle the changed error message in API calls.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-02 18:51:03 +02:00
Tomas Hozza
dc8a93b3f6 Weldr API: rename checkImageTypeDenylist() and return bool
Rename the `checkImageTypeDenylist()` method to `isImageTypeAllowed()`
and return boolean value instead of error.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-02 18:51:03 +02:00
Tomas Hozza
b150d57c18 Weldr API: make Image Type denylist distribution-specific
Change the Image Type denylist in Weldr API from being applied to all
distributions to being distribution-specific. A special name `*`
can be used in the configuration to match any distribution
or any image type.

Modify NEWS entry and unit tests to reflect this change.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-02 18:51:03 +02:00
Tomas Hozza
076bbc5456 Weldr API: introduce Image Type denylist for filtering exposed images
Extend Weldr API to accept a list of denied image types, which should
not be exposed via API for any supported distribution. This
functionality will be needed to not expose image types which can't be
successfully built outside of Red Hat VPN. Example of such images are
the official RHEL EC2 images, which include RHUI client packages not
available publicly.

Image Types are filters when listing available compose types and
creating a new compose using Weldr API.

Extend osbuild-composer configuration to allow specifying the list of
denied Image Types for Weldr API.

Add unit tests for implemented changes.

Add NEWS entry describing the newly introduced functionality.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-02 18:51:03 +02:00
Alexander Todorov
c70e437272 Enable additional aarch64 test runners
Signed-off-by: Alexander Todorov <atodorov@redhat.com>
2021-07-30 22:03:00 +02:00
Achilleas Koutsou
248632eaba test/data: update manifests
The previous manifests were generated inside a RHEL VM which means that
the selinux context mismatch check didn't run properly
(see https://bugzilla.redhat.com/show_bug.cgi?id=1973754).

Regenerating them on F34 shows some mismatches for certain image types.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-29 16:45:20 +02:00
Achilleas Koutsou
51cb4ae043 distro/rhel85: move selinux stage to last
In the main OS pipeline created by osPipeline(), there is often a set of
stages that are added after the common ones. The SELinux stage must
always be last in the pipeline, so it is added externally by the calling
function.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-29 16:45:20 +02:00
Achilleas Koutsou
daae2bbfca distro/rhel85: set build root for main os pipeline
OS image tree was being built on the host instead.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-29 16:45:20 +02:00
Juan Abia
c4fa4e4bb7 test: Replace hard coded ssh keys on some test
Generate a new ssh key each time. Only implemented on bash test scripts,
go tests pending.
2021-07-29 09:38:03 +03:00
Lars Karlitski
9c2c92f729 jobqueue: Introduce jobqueue backed by a postgres database
Co-authored-by: sanne <sanne.raymaekers@gmail.com>
2021-07-28 21:52:31 +01:00
Lars Karlitski
871c6e9cbb fsjobqueue: make canceling a finished job an error
This mirrors FinishJob(), which also errors when the job is already
finished.
2021-07-28 21:52:31 +01:00
Lars Karlitski
30492bfc60 jobqueue: move fsjobqueue's generic tests into new package
fsjobqueue_test contained tests that are generically testing the
JobQueue interface. Split those out into its own package `jobqueuetest`.

These tests will be useful when implementing a new package that conforms
to the JobQueue interface.
2021-07-28 21:52:31 +01:00
Gianluca Zuccarelli
bf62f3868e weldr: add filesystem configuration support 2021-07-28 21:44:03 +01:00
Gianluca Zuccarelli
fa15541ea5 blueprint: test minsize and sector-size validation 2021-07-28 21:44:03 +01:00
Gianluca Zuccarelli
90733e79f3 blueprint: add filesystem minsize function 2021-07-28 21:44:03 +01:00
Gianluca Zuccarelli
7408be580c client: test supported and unsupported mountpoints 2021-07-28 21:44:03 +01:00
Gianluca Zuccarelli
c671a0dab1 distro: test filesystem errors 2021-07-28 21:44:03 +01:00
Gianluca Zuccarelli
f4ee0bfabf distro: add root filesystem support 2021-07-28 21:44:03 +01:00
Gianluca Zuccarelli
d0e4558b2b blueprint: add filesystem customisations support 2021-07-28 21:44:03 +01:00
Martin Sehnoutka
700afb5af9 test/data: remove rhel-8-beta repository override
It contains beta repositories for RHEL 8.3, which is far outdated and
the file is not used anywhere any more.
2021-07-28 13:31:44 +02:00
Thomas Lavocat
4729990ac0
cloudapi: Add User customization
Cloud api now exposes user customization that let a customer able to add
a new user with a set of groups and a ssh key.

Testing:
* adds 2 users to the AWS image, accessible with a temp ssh key.
* the first one is in the group wheel, the other is not

Fixes #1574
2021-07-28 10:40:25 +02:00
Thomas Lavocat
5e127de303 Worker: report stages status after RunOSBuild
To help along with debugging, this commit makes the worker able to print
the status of the different stages with a oneliner for each successfull
stages and a detailed message for failed ones.

Sample output:
Jul 23[..]: Build stages results:
Jul 23[..]: org.osbuild.rpm  success
Jul 23[..]: org.osbuild.selinux  success
Jul 23[..]: Stages results:
Jul 23[..]: org.osbuild.rpm  success
Jul 23[..]: org.osbuild.fix-bls  success
Jul 23[..]: org.osbuild.fstab  success
Jul 23[..]: org.osbuild.grub2  success
Jul 23[..]: org.osbuild.locale  success
Jul 23[..]: org.osbuild.timezone  success
Jul 23[..]: org.osbuild.users failure:
Jul 23[..]:  [/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping
Jul 23[..]: All rules containing unresolvable specifiers will be skipped.
Jul 23[..]: Failed to create file /sys/fs/selinux/checkreqprot: Read-only file system
Jul 23[..]: useradd: group 'toto' does not exist

Fixes #1584
2021-07-27 18:13:10 +02:00
Achilleas Koutsou
df1eddbf07 test/data: update aarch64 manifests
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-27 16:53:24 +01:00
Achilleas Koutsou
b113cf0f8a test/data: update x86-64 manifests
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-27 16:53:24 +01:00
Achilleas Koutsou
9bb2ae09bd distro/rhel85: fix pkg set for grub2 stage
The package set is needed by the GRUB 2 stage options function to find
the Kernel package in order to add the saved entry option. The Kernel
package is defined in the blueprints package set, not the OS package
set.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-27 16:53:24 +01:00
Martin Sehnoutka
b950d6e062 weldr: accept rhsm parameter in sources
The system sources allow specification of the rhsm parameter, but it
isn't available in the sources configured over the Weldr API. This patch
implements support for it.
2021-07-27 15:39:36 +02:00
Ondřej Budai
05ffdc70aa ci: run only koji.sh for rhel_90 distro
Previously, all sorts of tests that provided no value were run for RHEL 9.0.
This commit limits its testing to the only test that makes sense: koji.sh.
See #1461 for more details.

Fixes #1461

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-27 10:18:05 +03:00
Tomas Hozza
928131003c osbuild2: fix incorrect function name in dracut.conf stage
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-26 22:01:33 +02:00
Tomas Hozza
cdc65f82da osbuild2: extract unit drop-ins from systemd -> systemd.unit
The part creating Systemd unit drop-ins was extracted from
`org.osbuild.systemd` stage to `org.osbuild.systemd.unit`,
before the osbuild v30 release. Update the composer implementation
to reflect the change.

Related to https://github.com/osbuild/osbuild/pull/739.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-26 22:01:33 +02:00
Tomas Hozza
49aebf1c68 osbuild2: change systemd-logind stage to match osbuild v30
The `org.osbuild.systemd-logind` stage was reworked before the osbuild
v30 release. Update the composer implementation to match the expected
schema.

Related to https://github.com/osbuild/osbuild/pull/739.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-26 22:01:33 +02:00
Tomas Hozza
6cd626d462 osbuild2: change dracut.conf stage to match osbuild v30
The `org.osbuild.dracut.conf` stage was reworked before the osbuild v30
release. Update the composer implementation to match the expected
schema.

Related to https://github.com/osbuild/osbuild/pull/739.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-26 22:01:33 +02:00
Tomas Hozza
b56642d6dd osbuild2: change modprobe stage to match osbuild v30
The `org.osbuild.modprobe` stage was reworked before the osbuild v30
release. Update the composer implementation to match the expected
schema.

Related to https://github.com/osbuild/osbuild/pull/739.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-26 22:01:33 +02:00
Tomas Hozza
d4de95ab5a osbuild2: change cloud-init stage to match osbuild v30
The `org.osbuild.cloud-init` stage was reworked before the osbuild v30
release. Update the composer implementation to match the expected
schema.

Related to https://github.com/osbuild/osbuild/pull/739.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-26 22:01:33 +02:00
Ondřej Budai
1abd9a1192 ci: drop all remaining 8.4 nightly jobs
RHEL 8.4 is GA for a long time so we no longer need 8.4 nightly jobs.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-25 20:26:29 +02:00
Achilleas Koutsou
c27ca818ae schutzbot/deploy.sh: dependency bug workaround 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
15ca644a3f news: add entry about RHEL 8.5 images 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
33b7fee194 test/data: generate aarch64 test manifests 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
d3d56dc6e6 test/data: generate x86_64 test manifests 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
74d61b9ae2 distro/rhel85: fix edge-installer build packages
The edge-installer build root requires the installer build package set.
2021-07-24 15:54:00 +02:00
Ondřej Budai
3ef9a397bf Schutzfile: bump osbuild for RHEL 8.5
Requires support for new features not yet released in RHEL 8.5
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
e0178bd76e distro/rhel85: add kernel to main OS package set
If there's no kernel in the main package set, the standard/default
kernel will be added while depsolving. This causes issues when an
alternative kernel is selected in the blueprint. Both kernels will be
installed (one from the blueprint and one from the main OS set) which
causes issues with ostree image types.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
e062fe3c44 test/api.sh: suppress shellcheck for sudo redirect 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
359100e1a1 test/api.sh: more robust edge commit validation
Instead of inspecting the tarball directly, extract it and use ostree to
verify the ref and commit ID.

Adds some data to the CI artifacts directory:
- Build manifest
- Tarball file list for s3 edge commit with s3 upload
- Build metadata
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
fe1ed9f869 distro/rhel85: remove broken arch-imgType combo check
The specific combination isn't broken in RHEL 8.5. The condition check
was added accidentally when copying the tests from RHEL 8.4.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
67c5e099ed tools: flush print in test case generator
When not flushed, the line often doesn't get printed until after the job
is done. Printing it before the job is useful for knowing the progress
of a multi-job run.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
347fcf74a6 osbuild2: base device options are optional 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
2b558e49ab distro/rhel85: init mounts array to 0 elements
Since partitions without a filesystem are skipped, we need to
dynamically append to create the mounts array instead of pre-allocating
to the number of partitions.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
6313532c90 distro/rhel85: add blueprint pkgs to ostree pipeline
Blueprint packages are now defined and passed into Manifest()
separately. The main osPipelines() already has an argument for
explicitly passing the blueprint packages. Added the same for the ostree
pipelines.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
4048383e07 distro/rhel85: package set key consts in pipelines
Use the package set key constants defined in distro in the pipeline
definitions as well.
2021-07-24 15:54:00 +02:00
Ondřej Budai
57e8ca0e5f distro/rhel85: sort mounts
Previously, /boot/efi mount was specified before /. This obviously doesn't
work because we need to mount / firstly.

This commit adds explicit ordering of the mounts.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-24 15:54:00 +02:00