Add `--build-rpms` option which will make the script build osbuild-composer
RPMs on the remote runner from the sources which were copied over. These
RPMs are then installed on the system, before any image test cases are
generated.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Exclude s390x for now as it is emulated in COPR and `go` binary
segfaults when building osbuild-composer.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Script is run with the image type to build as the argument.
The target / cloud service is selected based on the image type
specified. This is how the API actually works now: Only an image type
can be specified.
The script now supports all the blobby image types for testing:
- edge-commit
- edge-container
- edge-installer
- image-installer
- guest-image (qcow2)
- vsphere (vmdk)
These are image types that are uploaded to S3 and provided to the user
as an object to download rather than a VM image on a cloud provider.
To verify the cloud api compose request options for the qcow2 and vmdk
image types, download the object and inspect it using image-info.
Checks if postgresql is installed and that user1 and user2 exist in the
passwd file.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Because of the way json encoding works in golang we haven't run into any
issues. But if we add automated validation based on the spec it will
break, the image-builder-crc client for instance doesn't specify these
fields.
Temporarily switch to the nightly devel composes for RHEL 9.0.
We're testing new packages and changes right now (like coreos-installer)
and it takes time to make new snapshots every time.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Remove commented out lines and some comments, kept only as a reference
when we moved away from using the `@core` group.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Don't use the `@core` package group in image definitions, because it is
not intended as the minimal package set for virtual / cloud images. In
addition, its content is changing without us knowing, which has
consequences such as the recent discovery of the fact that TuneD is no
longer installed by default on RHEL images, while it definitely should be.
Replace the `@core` package group with the `coreOsCommonPackageSet`
package set. The content of it is based on the latest `@core` group
definition with a few modifications, so that image package sets
never end up having the same package listed in the `Include` and `Exclude`
package set at the same time. All additions have been accompanied with a
comment and all removals have been kept commented out with a comment.
The fact that the change does not have any effect on image package sets
was verified by regenerating all RHEL-9.0 image test cases. There is
however one change in the VMDK image. Specifically the
`python3-libselinux` package have been added. The reason is that the
latest `@core` group definition was used when defining the content of
`coreOsCommonPackageSet`, however the `@core` group definition in the
RPMRepo snapshot used for the image test case didn't include the package
yet.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Listing a single package per line in the package set definitions makes
it much more easier to review diffs in code changes and spot potential
issues.
Align EC2 package set functions to use the structure's `.Append()`
method as it is used by all the other package set functions.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `@core` package group used to include TuneD package by default on
RHEL-8. It has been removed from the group in Fedora as part of [1] and
inherited into RHEL-9. As a result, TuneD is no longer installed by
default on RHEL images.
After a discussion on rhel-devel there seems to be an agreement, that
TuneD should be installed by default on all RHEL virtual images. At
least we should keep the consistency in this regard with RHEL-8.
Regenerate all RHEL-9.0 image test cases.
Related to https://bugzilla.redhat.com/show_bug.cgi?id=2026709
[1] https://pagure.io/fork/adelton/fedora-comps/c/a5d4f1b6c9fcbe20cb0c38eac5048d7d45d1dd17
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Every schutzbot run will now build an AMI image that contains the worker.
The composer RPM is taken from the mockbuild phase. The osbuild version
is taken from Schutzfile for RHEL 8 GA.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Previously, monit and vector RPMs were embedded directly in the
image-builder-packer repository. This was not ideal because hosting big
binary files in git is always ugly.
This commit brings back monit and vector:
- monit is installed from EPEL
- vector is installed from the upstream RPM repository
Ansible was dropped because we don't need it in the image.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
We want an absolute path, otherwise packer doesn't know where to find the
playbook if called from a wrong directory.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
osbuild and composer commit SHAs now must be passed into packer using
variables, no defaults are defined. Also, packer is no longer responsible
for naming the AMIs, the name is also passed as a variable.
imagebuilder_packer_sha was dropped entirely as the packer configuration
now lives directly in osbuild-composer repository.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Console support in AWS EC2 is very basic. We now use vector that works much
better than console so we can just drop the forwarding and rely on vector
dumping the logs into cloudwatch.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This is currently not working because workers in aoc no longer use mTLS.
Definitely something we want to fix in the future I think.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
/templates/packer now contains a copy of image-builder-packer repository
as of b8a4b45f93890090de24e3d043e2d958948fc3c5
Changes:
- LICENSE file was dropped (it was redundant)
- README file was dropped (no longer needed)
- GitHub workflows were removed (will be replaced by schutzbot)
- RPMs were removed (they were huge, will be installed in a different way)
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Related to: https://github.com/osbuild/osbuild/pull/866/
Introduce new fields and move structure validation into the constructor.
This will fail faster and hopefully provide less space for programming
errors. Another advantage is simplified code with less type aliases and
lines.
We have been actually unmarshalling into a wrong datatype for a year, by
fixing this, we should get much more logging in Brew.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Jobs that don't interact with clouds can be canceled at any time without
the risk of leaving unused resources in the cloud. This enables the use
of "automatic cancellation of redundant pipelines" which means that if
and update is pushed to an open PR the current running pipeline is
canceled. This is done by adding an "interruptible" flag to the jobs.
Default value is false so only jobs that have it explicitly set to true
will be canceled.
It often takes a lot of time to synchronize the compose across all mirrors,
so it's safer to use just one, otherwise we can get hit by 404s if the
compose isn't available yet by a mirror.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
