Commit graph

6288 commits

Author SHA1 Message Date
Mario Cattamo
e6a7cb2544 ci: skip ostree-remount check in CS9 due to bug RHEL-25249 2024-02-27 17:11:03 +01:00
schutzbot
51a6b08591 Post release version bump
[skip ci]
2024-02-26 08:15:12 +00:00
Tomáš Hozza
2f087f1a6c Update osbuild/images to v0.40.0
In addition, simplify the SPEC file to not have to update the minimum
required osbuild version gazillion times, but just once.

Update the minimum required osbuild version to v109, due to changes in
grub2 stages required by the new osbuild/images version.

Update osbild SHA in Schutzfile to v109.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-26 09:08:50 +01:00
Sanne Raymaekers
c138ea6939 tools/appsre-build-fedora: wait until rpms are built upstream
The fedora build script should wait until the rpms are released upstream
before starting the build, otherwise the rpms will come from the fedora
repos.
2024-02-22 16:05:46 +01:00
Amelia Crate
b3bb851863 Tag rhel 9.2+ with SEV_LIVE_MIGRATABLE_V2
SEV-SNP capable kernels containing commit ac3f9c9f are compatible.
SEV_LIVE_MIGRATABLE indicated compatibility with an older version of SEV live migration, without ac3f9c9f.
See: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?id=ac3f9c9f1b37edaa7d1a9b908bc79d843955a1a2
2024-02-22 15:45:39 +01:00
Florian Schüller
bb4ee36cdc Makefile: remove dnfjson tests COMPOSER-2068 2024-02-22 15:22:52 +01:00
Florian Schüller
d900a5315e blueprints_test: Simplify check of the error message RHEL-16006 2024-02-22 15:22:52 +01:00
Florian Schüller
fe338cc475 blueprints_test: change to exact error message validation RHEL-16006 2024-02-22 15:22:52 +01:00
Florian Schüller
7bfd3ea464 blueprints_test: change to more readable table driven tests 2024-02-22 15:22:52 +01:00
Florian Schüller
42da3cb143 blueprint: rename package variable for readability 2024-02-22 15:22:52 +01:00
Florian Schüller
fb8634a991 blueprint: improve error message on missing name
the error should not infer that a version is mandatory.
Also the error message is now explicitly checked
2024-02-22 15:22:52 +01:00
Florian Schüller
e31ec13c87 Makefile: implement push-check
Tries to replicate github actions as close as possible
2024-02-22 15:22:52 +01:00
Florian Schüller
8e98884bff Makefile: implement support for BUILDDIR in "build"
also document "build" and implement the corresponding "clean"
2024-02-22 15:22:52 +01:00
Florian Schüller
cd9367f964 README: improve dependency documentation 2024-02-22 15:22:52 +01:00
Florian Schüller
99e84abc66 api: add error message for package without name RHEL-16006 2024-02-22 15:22:52 +01:00
Florian Schüller
80f49a526c README: add a hint how to run the unit-tests locally RHEL-16006 2024-02-22 15:22:52 +01:00
Ondřej Budai
242df9ab75 cloudapi: Add a smoke test for GetImageRequests
The focus of this test is to ensure that the image type enumeration in the
public Cloud API is correctly translated to the image type names as defined
in the images library.

Additionally, it covers that the default target is correctly set.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
9b965b0912 cloudapi: Add forgotten iot-simplified-installer to getDefaultTarget
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
504a51089c cloudapi: split GetImageRequests off PostCompose
This allows us to do some testing of the ComposeRequest => []imageRequest
conversion without actually enqueueing any jobs (which requires us to
do quite a lot of setup in the testsuite).

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
28ef0bc855 cloudapi: move manifest seed into an image request
The goal of this commit is primarily to simplify the API of the enqueue
methods. This way, basically everything needed to generate manifests
is in the imageRequest structure, which simplifies the amount of structures
that we need to think about.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
7385cab165 cloudapi: move blueprint into an image request
The goal of this commit is primarily to simplify the API of the enqueue
methods. This way, basically everything needed to generate manifests
is in the imageRequest structure, which simplifies the amount of structures
that we need to think about.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
ab8c1ae4f7 clouadpi: remove distribution from enqueue methods
They can just find it from the image type.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
c48bf0a143 cloudapi: remove arch from imageRequest
You can absolutely just get it from imageType, let's not duplicate
information, that's error-prone.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
08aabe1bef cloudapi: add minimal-raw 2024-02-21 12:09:01 +01:00
Ondřej Budai
eb3bd9dc43 cloudapi: sort image types in getDefaultTarget
Not a functional change.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Tomáš Hozza
e561ba0854 templates/composer: set DISTRO_ALIASES for composer
Set the RHEL release names without the minor version to point to the
latest GA release. Set the 'rhel-10.0' to the latest RHEL-9 minor
release in development, so that one can start building RHEL-10 images
without referencing RHEL-9.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-21 12:06:33 +01:00
Tomáš Hozza
f57e88d034 osbuild-composer: support setting distro_aliases via ENV variable
This will allow us to configure the distro aliases in the service by
setting ENV variable.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-21 12:06:33 +01:00
Tomáš Hozza
218a784778 Test/base_tests.sh: workaround missing osbuild-depsolve-dnf in nightly
The RHEL nightly composes still contain old osbuild-composer version,
which does not have an explicit dependency on osbuild-depsolve-dnf
package.

As a result, running the latest upstream base_tests.sh on RHEL nightly
composes fails due to osbuild-depsolve-dnf not being installed.

Workaround this in the test case by explicitly installing the
osbuild-depsolve-dnf package in case the osbuild-composer version is
lower than the one which has an explicit dependency on
osbuild-depsolve-dnf.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-20 15:55:47 +01:00
Diaa Sami
82add84c31 Test/old-worker-new-composer: use latest composer for verification
To avoid incompatibility between older image-info and latest osbuild
2024-02-20 15:55:47 +01:00
Diaa Sami
76e686df10 switch to images/pkg/dnfjson and remove internal copy
COMPOSER-2068
2024-02-20 15:55:47 +01:00
Diaa Sami
98e3dab9c7 Schutzfile: bump version of osbuild to 619a64f0 2024-02-20 15:55:47 +01:00
Sanne Raymaekers
5025ec31d3 cloud/awscloud: describe security groups using filters
Using the group names option only works for the default VPC, the workers
are not running in the default VPC. For non-default VPCs filters should
be used.
2024-02-20 15:23:52 +01:00
Sanne Raymaekers
bb43f330b8 cmd/osbuild-jobsite-builder: shutdown http server gracefully 2024-02-16 15:21:20 +01:00
Sanne Raymaekers
7fce482baa cloud/awscloud: create secure instance in the same subnet
This reduces network costs as transferring data between AZs is not free.
2024-02-16 15:21:20 +01:00
Sanne Raymaekers
1abd887c23 templates/packer: rename executor log group
In app-interface the output resource names need to be unique, and the
log group name is already shared with the role.
2024-02-16 11:46:03 +01:00
Simon de Vlieger
7533f58fb2 jobsite/manager: turn off compression
We're seeing some unexpected EOFs in staging deployment; going on a
hunch I've seen these before when gzip gets involved in transfering
large files so let's disable that.
2024-02-16 11:45:51 +01:00
Sanne Raymaekers
ab6fd9e53e cmd/osbuild-jobsite-builder: actually assign the stdout buffer
The buffer needs to be assigned to the process' stdout for the buffer to
fill up.
2024-02-15 18:28:07 +01:00
Simon de Vlieger
b9584099ab jobsite/manager: create export directory
Be a bit more friendly and actually create the export directory instead
of assuming it exists.
2024-02-15 16:23:55 +01:00
Sanne Raymaekers
1150f0f27e cmd/osbuild-jobsite: capture osbuild's stdout
Write osbuild's stdout in the progress step. The manager can just copy
it to stdout and the executor will be able to parse the output into an
osbuild result.
2024-02-15 14:31:24 +01:00
Sanne Raymaekers
ee6b198b0a cloud/awscloud: remove restricting egress rule from SG
The machine still needs to be able to fetch sources, so just keep the
default 0.0.0.0/0 rule.
2024-02-15 14:23:18 +01:00
Sanne Raymaekers
3400c5c20d templates/packer: setup vector in osbuild-executor 2024-02-15 12:05:17 +01:00
Sanne Raymaekers
a9010479e3 templates/packer: set -builder-path to /var/cache/osbuild-builder
The builder uses `/run/osbuild` as a default path for this argument. Yet
this directory doesn't exist when the builder writes the manifest. But
osbuild should own this directory, not the builder.

Furthermore `/run` is a tmpfs, so the executor might run into memory
issues if we use `/run` as the store and output directory (on the "host"
workers these are in `/var/cache`).

While `/tmp` might seem like a good candidate on RHEL, it's a tmpfs on
Fedora, so it's also to be avoided.
2024-02-15 01:02:10 +01:00
Sanne Raymaekers
04ce056b6a templates/packer: let the executor listen on all interfaces 2024-02-14 22:55:54 +01:00
Sanne Raymaekers
bfd1978ad8 osbuildexecutor/aws.ec2: pass the manifest to the job manager 2024-02-14 20:28:22 +01:00
Sanne Raymaekers
8e6717fa1b cloud/awscloud: take instance type from host
InstanceRequirements is very flakey, the create fleet request fails
almost consistently with the same error.

To continue with testing use a fixed instance type for now. As a
followup we can expand the instance type selection logic or figure out
what was wrong with the InstanceRequirements.
2024-02-14 18:15:25 +01:00
Sanne Raymaekers
8a1d66a0bd cloud/awscloud: max 4 overrides are allowed when creating a fleet
```
InvalidParameterValue: Your request contains more than the maximum allowed number of InstanceRequirements (4)
```
2024-02-14 15:24:42 +01:00
Sanne Raymaekers
f03dac5834 templates/packer: don't subscribe fedora executor 2024-02-14 13:45:52 +01:00
Sanne Raymaekers
7fd150b938 cloud/awscloud: specify subnets when creating secure instance
For non-default VPCs, AWS needs the subnets it can launch the instance
in, otherwise it will try to launch the instance in the default VPC,
even if the supplied security groups are attached to a non-default VPC.

Furthermore there can only be 1 subnet specified per availability zone,
so query the subnets in the VPC of the host (as the instance needs to be
launched in the same network), and pick 1 of the VPC's subnets per AZ.
2024-02-14 13:45:52 +01:00
Sanne Raymaekers
d5fd1bbbf0 templates/packer: add worker-executor service to image 2024-02-14 13:45:52 +01:00
Sanne Raymaekers
a9066ee793 templates/packer: remove worker_builder script from init service
This script doesn't actually exist.
2024-02-14 13:45:52 +01:00