Google repositories use RSA/SHA1 for signing packages. However the SHA1
has been disabled by default on el9/c9s. Since osbuild-composer imports
GPG keys specified in the repository definition unconditionally, this
creates issues when installing rpms signed with the key by osbuild [1].
Remove GPG keys in all el9/c9s GCP repo definitions and disable GPG
signature verification until [2] is resolved.
[1] https://github.com/osbuild/osbuild/issues/991
[2] https://issuetracker.google.com/issues/223626963
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add support for importing the GCE image into GCP using Weldr API. The
credentials to be used can be specified in the upload settings and will
be then used by the worker to authenticate with GCP.
The GCP target credentials are passed to Weldr API as base64 encoded
content of the GCP credentials JSON file. The reason is that the JSON
file contains many values and its format could change in the future.
This way, the Weldr API does not rely on the credentials file content
format in any way.
Add a new test case for the GCP upload via Weldr and run it in CI.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Modify the Cloud API test case for GCP to use `gcloud` and GCP guest
tools installed in the image to connect to the VM instance over SSH.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Filter the list of repositories passed in compose request based on the
`image_type_tags` object member. This is the same approach used by the
Weldr API. If the `image_type_tags` does not exist, the repo is added to
the list. If the `image_type_tags` exists, the repo is added to the list
only if the image type name is in the tags array.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
jobimpl-osbuild
---------------
Add GenericS3Creds to struct
Add method to create AWS with Endpoint for Generic S3 (with its own credentials file)
Move uploading to S3 and result handling to a separate method (along with the special VMDK handling)
adjust the AWS S3 case to the new method
Implement a new case for uploading to a generic S3 service
awscloud
--------
Add wrapper methods for endpoint support
Set the endpoint to the AWS session
Set s3ForcePathStyle to true if endpoint was set
Target
------
Define a new target type for the GenericS3Target and Options
Handle unmarshaling of the target options and result for the Generic S3
Weldr
-----
Add support for only uploading to AWS S3
Define new structures for AWS S3 and Generic S3 (based on AWS S3)
Handle unmarshaling of the providers settings' upload settings
main
----
Add a section in the main config for the Generic S3 service for credentials
If provided pass the credentials file name to the osbuild job implementation
Upload Utility
--------------
Add upload-generic-s3 utility
Makefile
------
Do not fail if the bin directory already exists
Tests
-----
Add test cases for both AWS and a generic S3 server
Add a generic s3_test.sh file for both test cases and add it to the tests RPM spec
Adjust the libvirt test case script to support already created images
GitLabCI - Extend the libvirt test case to include the two new tests
We want to ensure that cloud images connect to Red Hat[1] independently
of how the content was acquired (PAYG, BYOS, or marketplace).
This auto-registration feature is already enabled for AWS and this
patch enables it for Azure with the same recommended settings:
Services:
rhsmcertd: Enabled (already done, so not changed in the patch)
/etc/rhsm/rhsm.conf:
auto_registration: enabled
auto_registration_interval: 60 (the default, so not explicitly set)
manage_repos: false
The latter value `manage_repos` is left enabled (the default) in case
the user explicitly requested to have the system subscribed, i.e. the
`RHSMConfigWithSubscription` code path.
Regenerate the relevant test manifests and image information.
[1] https://cloud.redhat.com
[2] https://docs.google.com/document/d/1VeZFJxNUlyZMQJh6s3NA3RLvadqATsGxVet6uuP87_4
The users anaconda module enables users to create user accounts at
install time if one is not already created in the payload. This is
required for the cloud API (Image Builder service) for the image
installer where user customizations are not supported. Without it, user
creation isn't possible on the installed system.
The module also enables user creation at install time through the
kickstart file for both the image-installer and the edge-installer
(Anaconda only).
Therefore, for the image-installer, the users and groups are no longer
created as part of the payload.
This commit adapts the changes from the following commits (originally
made in the RHEL 8.6 and RHEL 9.0 distros) to the rest of the RHEL
distro definitions:
ebc3330cbd5825294dad
Use single NewGroupsStageOptions() from osbuild1 and osbuild2 instead of
implementing in each distro.
- Followup from 2eef6e6e2d, copied to the
rest of the RHEL distro definitions.
- Added NewGroupsStageOptions() to osbuild1 for rhel8 and rhel84.
NB: The change was not made in the Fedora distro definitions as they are
currently being rewritten.
Followup from, f34380d5b5 and
3a1765a5a8, copied to the rest of the RHEL
distro definitions.
For now, these customizations have no effect on the manifest.
The new `with-users` variants of the edge-installer test cases include
the user customizations in the blueprint, but the manifests are
(currently) the same as the corresponding base cases.
Regression test suite has grown considerably and is taking too long to
run with a single wrapper. Splitting them into individual standalone
tests instead and making them run in parallel.
The VMDK image must be in stream-optimized format in order to be
imported to VSphere. osbuild-composer does not produce VMDK by default
as stream-optimized. Instead, it is converted on the fly when the image
build job has been submitted via Weldr API.
Since we are aiming mainly for the VSphere use case with the VMDK image
in the service, the image should be ready for importing to VSphere.
Implement a temporary workaround for the Cloud API and AWS S3 target to
upload stream-optimized VMDK image.
Adjust the `api.sh` test case to not convert the VMDK image downloaded
form S3, before importing it to VSphere.
Ensure that the content of the database is not printed to the console
when dumped at the end of the test case. The output is still preserved
as a CI run artifact.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Kill and remove the DB container as part of the test case cleanup.
Without this change, running the test case more than once fails.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Extend the `api.sh` test to verify the VMDK images uploaded to S3 in
VSphere by booting them and configuring using cloud-init.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Install `cloud-init` by default on the VMDK image on RHEL-85/86/90 and
as a result also CentOS Stream 8/9.
Regenerate image test cases.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
test cases fixup
RHEL 9.0 will ship a brand new auxiliary key. Let's use it everywhere in our
RHEL 9 stuff. Taken from current RHEL 9.0's redhat-release package.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
We should honour `pkg.CheckGPG` when creating the file inputs for the
rpm stage. This was lost in the transition from v1 to v2 manifests.
Regenerate image test manifests.
Co-authored-by: Tomas Hozza <thozza@redhat.com>
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Remove comment out code
Use three different IP address for different test scenarios
Move /boot/device-credentials file checking into playbook
Some shell script improvements
Enable the user module unconditionally for the image-installer:
- If users are specified for the kickstart file, the module is required
to set up the users.
- If no users are specified, the module can be used at install time to
create users.
Updated relevant test cases (manifests).
