Let's explain how RPMs for RHEL are built:
We use a subscribed RHEL 8.x machine and mock build these on it. Mock
initializes its own buildroot based on the latest RHEL 8 CDN content, see[1].
This means that the minor version of the buildroot is independent of the minor
version of the host.
However, we currently upload RPMs to a directory whose name consists also of
the minor version of the host. Our hosts are currently running RHEL 8.3 so
the RPMs are uploaded into rhel-8.3 directory despite them being built in the
RHEL 8.4 buildroot (RHEL 8 CDN buildroot specifically). This means that
we cannot guarantee that they are installable on RHEL 8.3 which is weird.
This commit adds a special case for hosts that run on subscribed RHEL and
thus build RPMs in a buildroot constructed from RHEL CDN. These RPMs are
now uploaded into rhel-8-cdn directory. This change more accurately reflects
the way we build our RPMs and removes some confusion.
Also, we need to bump osbuild commit so we have a version that already has
the rhel-8-cdn change in it.
[1]: https://github.com/rpm-software-management/mock/blob/main/mock-core-configs/etc/mock/templates/rhel-8.tpl#L37
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Previously a bad error code was returned, fixes#1477.
Testing:
I have two test cases to test the solution. The first is a request that
makes depsolve crash by changing the dnf-json script by an almost empty
one that only throws an exception. The second one fails because it
requests a non existing package. The former ends with a 500 error and
the later with a 400.
----8<-----
HTTP/1.1 500 Internal Server Error
Failed to depsolve base packages for ami/x86_64/centos-8: ailed to
depsolve base packages for ami/x86_64/centos-8: unexpected end of JSON
input
----8<-----
HTTP/1.1 400 Bad Request
Content-Length: 226Failed to depsolve base packages for
ami/x86_64/centos-8: DNF error occured: MarkingErrors: Error occurred
when marking packages for installation: Problems in request:
missing packages: jesuisunpaquetquinexistepas_idonotexist
Introduce Dockerfile and devcontainer.json specifying how to build and
run a containerized development environment in VSCode.
The configuration is inspired by the osbuild repo where a similar
configuration directory already exists.
Some osbuild stages added as part of PR#1525 declare unexported types
which are complete copies of types with custom MarshalJSON method,
to prevent recursion when marshalling to JSON.
Modify relevant osbuild stages to use type alias instead of declaring a
complete type copy.
https://github.com/osbuild/osbuild-composer/pull/1525
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `api.sh` test currently always defaults to "<REGION>-a" zone when
creating instance using the built image. The resources in a zone may get
exhausted and the solution is to use a different zone. Currently even a
CI job retry won't help with mitigation of such error during a CI run.
Modify `api.sh` to pick random GCP zone for a given region when creating
a compute instance. Use only GCP zones which are "UP".
The `cloud-cleaner` relied on the behavior of `api.sh` to always choose
the "<REGION>-a" zone. Guessing the chosen zone in `cloud-cleaner` is
not viable, but thankfully the instance name is by default unique for
the whole GCP project. Modify `cloud-cleaner` to iterate over all
available zones in the used region and try to delete the specific
instance in each of them.
Make `ComputeZonesInRegion` method from the `internal/cloud/gcp` package
exported and use it in `cloud-cleaner` for getting the list of available
zones in a region.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `test/cases/api.sh` script relied on environment variables specific
to Jenkins for detecting it if is running in a CI environment. If this
was the case, it used other environment variables to construct a
predictable `TEST_ID` which could be used for names of resources created
in cloud-provider environment as part of the test. This is important to
ensure that `cloud-cleaner` can "guess" resource names and delete them
in case the test script fails to clean up after itself.
With the move from Jenkins to GitLab CI, this stopped to work and the
script started to generate random `TEST_ID`, which can not be guessed by
the `cloud-cleaner` tool.
Modify the `test/cases/api.sh` to detect the CI environment using the
`CI` environment variable, which is always predefined in the GitLab CI
environment [1].
[1] https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Some images with ESP, e.g. the `rhel-ec2-aarch64`, have the `/boot` on
a separate partition. `image-info` currently produces traceback on such
images, e.g.:
Traceback (most recent call last):
File "/home/thozza/devel/osbuild-composer/./tools/image-info", line 1997, in <module>
main()
File "/home/thozza/devel/osbuild-composer/./tools/image-info", line 1991, in main
report = analyse_image(target)
File "/home/thozza/devel/osbuild-composer/./tools/image-info", line 1863, in analyse_image
append_partitions(report, device, loctl)
File "/home/thozza/devel/osbuild-composer/./tools/image-info", line 1849, in append_partitions
append_filesystem(report, tree)
File "/home/thozza/devel/osbuild-composer/./tools/image-info", line 1809, in append_filesystem
with open(f"{tree}/grub2/grubenv") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp3i__6m1w/grub2/grubenv'
The reason is that `grub2/grubenv` on the `/boot` partition is a symlink
to `../efi/EFI/redhat/grubenv`. However the `efi` directory on the
`/boot` partition is empty and the ESP must be mounted to it for the
expected path to exist.
Modify `image-info` to mount the ESP to `efi` directory if it exists on
the inspected partition.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Because the state directory, which is still currently used, only allows
for a single pod to mount it, allow for 0 available pods when updating
the deployment. Otherwise it will block updating until the state
directory is available for mounting, which will be never.
This commit also makes aarch64 and x84_64 consistent with each other
and updates the updates repos for x86_64
Fixes#1551
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This commit adds RELEASING.md guide describing the process of making a new
upstream release and pushing it into Fedora and CentOS Stream 9.
Additionally, a new helper is added to the repository - update-distgit.py.
The purpose of this helper is to simplify work with the dist-git.
Random thoughts:
I decided to left off RHEL 8 because the guide would be full of internal
URLs and tools. I will probably write a similar guide for it and put it
into internal guides.
I decided to just reference the RHEL Developer Guide for CentOS Stream 9.
It's pretty well written and I didn't feel like duplicating the effort.
We should definitely switch to PR-based approach for Fedora and implement
at least some smoke tests. I believe that the current guide is good enough
and we can iterate later.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
mirrors.kernel.org are sometimes desynchronized, and thus causing issues with
the image tests. This commit switches the image tests for F33 to use RPMRepo.
All test cases were regenerated.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Some distributions do not have repositories and therefore cannot be
built. This filters the list of supported distributions by checking for
repos when starting up. All other requests use the api.distros list or
api.getDistro() function.
The name of the distro you get from distros.FromHost() may not match any of
the names in the registry's list. Use the actual name of the distro
instead of the mangled name.
Also removes api.distro which is unused.
