For backward compatibility, revert changes related to hybrid boot mode
for RHEL (RHUI) EC2 images before 9.3 release.
This change does not affect CentOS Stream 9 AMI images nor the RHEL AMI
build by the service or on-premise.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
For backward compatibility, revert changes related to hybrid boot mode
for RHEL (RHUI) EC2 images before 8.9 release.
This change does not affect CentOS Stream 8 AMI images nor the RHEL AMI
build by the service or on-premise.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The post install script customises the selinux policy, but is gated on
`selinuxenabled`, which will fail inside the buildroot container.
As a result it's never executed.
Add Fedora iot-raw-image test manifests with supported customizations,
specifically:
- custom users and groups
- custom enabled services
- custom files and directories
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the `ostree-raw-image.sh` to use the directories and files
customizations for the raw image and test their effect using the ansible
playbook.
Custom files and directories are currently tested only on Fedora,
because they are allowed by policy only there.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Reformat calls of `ansible-playbook` by splitting them into multiple
lines. This makes it easier to read.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Break the execution of `ansible-playbook` into multiple lines to make it
easier to read.
Add a comment on top of the second `ansible-playbook` execution if
Ignitions is availble, explaining what differs. Otherwise this is hard
to spot.
There are 3 places in the script, where there are always 2 executions of
the `ansible-playbook`. First the user from the BP is used. On the
second run, the user created by Ignition is used. This was not the case
for the last 2 executions of `ansible-playbook` at the end of the
script. Adjust the order for consistency with the rest of the script.
The changes should have no effect on the test case.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
One of the IOT use cases for custom files and directories is to be able
to enable a custom systemd unit. This was previously not possible to do
via BP Customizations.
In addition, the test case for custom files and directories in
ostree-based images (currently only build-time images) takes advantage
of enabling a custom systemd unit file, this this will enable using the
same test scenario also for deploy-time ostree image types.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Set the user provided BP customizations related to custom files and
directories to the iot raw-image type, to ensure that these get
created while deploying a commit.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the `OSTreeRawImage` with a slice of custom directories and
files, that can be created in the image. Pass these down to the ostree
deployment pipeline, so that it can add necessary osbuld stages if any
directories or files were specified by user.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the `OSTreeDeployment` with a list of custom files and
directories, that can be created in the deployed ostree commit. This
functionality is equivalent to the one that was added for the OS
pipeline.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
When a blueprint containing package name globs is frozen it was failing
because it could not find the string in the dependency list. This fixes
it by replacing the package glob with all of the matching packages from
the dependency list.
This removes the setPkgEVRA function and adds a new expandBlueprintGlobs
function that handles the package name glob expansion, and replacement of
the version globs with the dependency EVRA.
Also includes testing for the new function.
GetEVRA will return the Epoch:Version-Release.Arch string, and
GetNEVRA will return Name-Epoch:Version-Release.Arch
with Epoch being left off if it is zero.
Also includes tests.
This is currently failing (fixed in the next commit). It tests to make
sure that a blueprint with package name globs can be frozen. The
resulting blueprint should replace the glob entries with the expanded
list of packages.
This tests to make sure that package name globs are working during
integration test runs. dnf supports this, and users have been using it,
so testing to make sure it keeps working is important.
Extend the Cloud API AWS test case to verify the uploaded AMI boot mode.
For informational purposes, also log the boot mode of the instance
created using the AMI by the test case.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
On RHEL-8, the x86_64 AMI / EC2 images used a BIOS-only partition table
layout, because the base partition table unification happened in the
past only on RHEL-9 and Fedora (inherited from RHEL-9).
To make things consistent and uniform across RHEL-8 and RHEL-9, I copied
the base partition table used by RHEL-9 AMI / EC2 images to RHEL-8. This
has a side-effect for aarch64 AMI / EC2, where the `/boot` partition
size changed from 512 MiB to 500 MiB, together with the partition GUID
to "Extended Boot Loader Partition GUID".
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The image already used base partition table with necessary layout to
support hybrid boot mode, so the change was just a matter of modifying
the associated platform.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The image already used base partition table with necessary layout to
support hybrid boot mode, so the change was just a matter of modifying
the associated platform.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Fedora distro definition contained an empty `s390x` architecture with no
image types added to it. Let's remove it from the distro definition,
since it's adding no value in its current form.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Explicitly specify the AMI boot mode in AWS upload target in Cloud API
compose handler. The value is determined based on image type's boot
mode.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Explicitly specify the AMI boot mode in AWS upload target in Weldr API
compose handler. The value is determined based on image type's boot
mode.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add an optional `BootMode` field to the AWS target options.
This allows to signal to worker the intended boot mode to use when
registering the AMI in AWS. If not specified, the default behavior is
preserved, specifically that the boot mode will be determined by the
default boot mode of the instance provisioned from the AMI.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
When the AMI is being registered from a snapshot, the caller can
optionally specify the boot mode of the AMI. If no boot mode is
specified, then the default behavior is to use the boot type of the
instance that is launched from the AMI.
The default behavior (no boot type specified) is preserved after this
change.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
As a preparation to be able to determine the image boot mode when
importing it to the target environment (e.g. AWS), expose the
information on the `ImageType` level.
The image boot mode is determined based on the platform associated with
it.
The new method is not yet used by any code, but will be eventually used
by osbuild-composer server to set the proper value in the upload target
options for the worker. The worker will be then able to import the image
in the proper way to the cloud environment.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This was the intention since the beginning (based on images built by
Google. Clean up code and mark the platform associated with GCE image
types as UEFI-only.
The only missing part is the default partition table used by the GCE
image, which is shared with other image types and still contains the
BIOS boot partition. I added a TODO comment to preserve this
information, but kept things as they are for now to not have to
introduce a new set of GCE-specific base partition tables.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Do not extend the image base package set with list of packages needed
for booting the OS, returned by `bootPackageSet()` based on the specific
image type, architecture and its boot type. This duplicated
functionality that is already handled by the platform associated image
and all the necessary packages are provided by the platform's
`GetPackages()` method and added to the base package list.
This reflects changes which were done in Fedora when it was ported to
the "new" image definitions, but were not ported to RHEL.
RHEL-8 GCE image type note:
After a previous change, the image boot type is now determined by the
associated platform and as a result, the GCE image type is marked as
supporting hybrid boot type, although it was meant to be UEFI only. As a
result, the package list returned by `bootPackageSet()` and previously
appended would contain grub2 BIOS-related packages. This is still the
case after this change, because the platform's `GetPackages()` method
will return the same list of packages in this case. However, the
platform used by RHEL-8 GCE image type has its `GetPackages()`
overridden by a different implementation not containing grub2 BIOS
related packages. For some reason, this change is not present in RHEL-9.
As a result, the grub2 BIOS related packages disappeared from the RHEL-8
GCE image package set, while there was no change in RHEL-9.
Keep the GCE image as is for now and make it an UEFI-only in a follow
up.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Remove the `legacy` from `architecture` struct, since this information
is already contained in the platform associated with the image.
This reflects changes which were done in Fedora when it was ported to
the "new" image definitions, but were not ported to RHEL.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Remove the bootType from imageType and architecture structures and
determine the image boot type based on its associated platform.
This reflects changes which were done in Fedora when it was ported to
the "new" image definitions, but were not ported to RHEL.
GCE image type note:
This change has a side-effect on the GCE image type. It was meant to be
UEFI only, but the previous mixture of bootType set in the imageType and
the platform used for it made it a weird combination of almost hybrid
boot type, but not completely. For now, the grub2 BIOS-related packages
are added to the image content as a result. Eventually, the platform
used for the image should be changed to not support BIOS and the image
should also not have BIOS partition at all.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The s390x platform definition would previously always return `false`
when calling its `GetZiplSupport()` method. This was obviously not
correct.
The method is meant to suite a similar purpose as `GetBIOSPlatform()`
and `GetUEFIVendor()` on BIOS / UEFI enabled platforms.
Change the S390X platform struct to contain `Zipl` member instead of
`BIOS`, which is technically more correct. Make sure that the value
set in the `Zipl` struct member is returned by `GetZiplSupport()`.
Ensure that `FirmwarePackages` from `BasePlatform` are added to the list
of packages returned by `GetPackages()`.
Adjust distro definitions using the `S390X` platform.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
We sometimes see the following error in the logs:
Fault(1000): upload path exists: /mnt/koji/work/osbuild-cg/osbuild-composer-koji-082e1c88/Fedora-IoT-38.raw.xz.
I think this happens when we retry the upload call of the first chunk due to
random network issues. The solution is to always upload in the overwriting
mode, which ignores the already existing file.
See https://pagure.io/koji/blob/175ecb5e8f3d45a1d244b227eb889321e5dd0a29/f/kojihub/kojihub.py#_15522
This is safe because:
1) We use UUIDs in the filename, which means that there should never be a real
conflict.
2) The overwriting mode is actually the default mode in koji, see
https://pagure.io/koji/blob/175ecb5e8f3d45a1d244b227eb889321e5dd0a29/f/koji/__init__.py#_3342
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
CGImport quite often fails with the following error:
Fault(1000): File size 735051776 for Fedora-IoT-38.raw.xz (expected 738785372)
doesn't match. Corrupted upload?
When I inspect the file manually, everything seems fine, though.
I believe that this because of NFS inconsistency when multiple DNS-balanced
kojihubs are used in the setup (which is what Fedora uses). The addded
loop implements a retrying mechanism for the CGImport call to try again
whenever we see this issue.
Note that this isn't caught by other HTTP retrying mechanism because a failed
XMLRPC call returns code 200.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
When `go install` is called, go tries to get the git commit hash and embed it
into the built binary. Internally, go just calls the git executable.
The newer go-toolset seems to be based on RHEL 9.2 that ships a newer version
of git (2.39.1). This version contains the safe directory patch that
disallows git from operating on repositories owned by different users.
Thus, we need to chown the files when copying.
See
https://git-scm.com/docs/git-config/2.35.2#Documentation/git-config.txt-safedirectory
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Remove firewalld from the base package set for Azure and add it to all
the image-specific package sets except EAP (and explicitly exclude it).
Remove firewalld from the base image config for Azure and add it to all
the image-specific configs.
Test manifests updated.
Manifest changes for non-EAP image types is only the enabled-services
reordering: firewalld is last because it is appended to the base config.
See COMPOSER-1859
Sorted lists of strings make it easier to add and remove elements
without needing to think about the order, making diffs easier.
The sorting was done using the 'sort' coreutils command with LC_ALL=C.
