Commit graph

2842 commits

Author SHA1 Message Date
Achilleas Koutsou
0eeb7e47b0 ostree: rename ResolveParams to Resolve 2023-05-31 16:40:07 +02:00
Achilleas Koutsou
fc5461e9e3 ostree: rename RequestParams to SourceSpec
Same as with the container SourceSpec, the struct specifies the required
information to resolve an ostree commit from a source (URL, ref, and
optional parent).
Renaming for consistency.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
00d473c802 container: create a SourceSpec for the resolve parameters
Create a struct to conveniently hold the source parameters for a
container resolve request and use it in the Add() function of the
resolver.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
30dc8cb87a manifest: expand the manifest package doc
Expand the manifest package documentation and move it to the manifest.go
file.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
c7aecdf31c distro: delete distro.Manifest type
Delete the distro.Manifest type and its tests.
Change all occurrences of distro.Manifest to manifest.OSBuildManifest.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
5498764725 manifest: serialize to OSBuildManifest instead of distro.Manifest
Copy the Marshal and Unmarshal functions from distro.Manifest to
manifest.OSBuildManifest to keep the same behaviour.

The Version() function isn't used, so let's drop it.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
8f69088af1 distro: remove architecture names and use platform enum
Removing the dependence of the manifest package on the distro package to
import manifest into distro.
Wherever arch names are needed, we use the enums from the platform
package instead.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
0f925c445c subscription: new package for subscription options
Move the subscription options from distro to its own package.
Now we can import the manifest package into the distro package (instead
of the other way around) so we can work with the manifest.Manifest type
in distro.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
ea305d9c0f distro: simplify checkOptions() arguments
Make checkOptions() take the whole blueprint and options.  There is no
need to pass in the resolved containers separately since we only care
whether there are any containers defined for image types that don't
support them.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
e4e411a4c1 distro/rhel7: move imageType implementation to separate file 2023-05-31 16:40:07 +02:00
Achilleas Koutsou
31fd40f359 distro/fedora: move imageType implementation to separate file 2023-05-31 16:40:07 +02:00
Achilleas Koutsou
3d9dcff7f6 distro: inline initializeManifest
Inline the initializeManifest() function so we can start simplifying the
PackageSets() and Manifest() (the two callers) separately.
2023-05-31 16:40:07 +02:00
Gianluca Zuccarelli
388e2572d7 internal/osbuild: add missing sslverify field
The blueprint customizations enabled the `sslverify`
field. However, this has not been implemented yet in
the yum repos stage.
2023-05-31 16:24:36 +02:00
Gianluca Zuccarelli
ce299dfa0e internal/rpmmd: change ignoressl to pointer
Change the `IgnoreSSL` field in `rpmmd.RepoConfig`
to a pointer. This will be later used to configure
the `SSLVerify` field in the yum repo stage.
2023-05-31 16:24:36 +02:00
Sanne Raymaekers
8a8607cdf6 internal/vmware: add support for the GOVC_FOLDER option
When importing the ova it also creates a VM, and users don't always have
permission to register in the default folder.
2023-05-25 10:14:32 +02:00
Sanne Raymaekers
967306bc47 internal/upload: add import.ova support to vmware 2023-05-25 10:14:32 +02:00
Sanne Raymaekers
e91dcae110 fedora: add ova image type 2023-05-25 10:14:32 +02:00
Sanne Raymaekers
a7c02245ee rhel8: add ova image type 2023-05-25 10:14:32 +02:00
Sanne Raymaekers
39f1e74d73 rhel9: add ova image type 2023-05-25 10:14:32 +02:00
Sanne Raymaekers
1687937c51 internal/image: add ova support to live image 2023-05-25 10:14:32 +02:00
Sanne Raymaekers
53fa47f104 internal/osbuild: add ovf stage 2023-05-25 10:14:32 +02:00
Tomáš Hozza
0292725ce4 internal/GCP: remove all remaining uses of cloudbuild
Some uses of `cloudbuild` GCP API have been left in our internal cloud
API implementation for GCP. We do not use `cloudbuild` to import GCE
images into GCP any more.

Do not request the `cloudbuild` authentication scope when getting new
GCP client.

Update vendored packages accordingly.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-24 19:28:06 +02:00
Tomáš Hozza
f335096f66 distro/rhel9: revert hybrid boot on RHEL EC2 (RHUI) images before 9.3
For backward compatibility, revert changes related to hybrid boot mode
for RHEL (RHUI) EC2 images before 9.3 release.

This change does not affect CentOS Stream 9 AMI images nor the RHEL AMI
build by the service or on-premise.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-23 13:04:31 +02:00
Tomáš Hozza
3eb5779e8b distro/rhel8: revert hybrid boot on RHEL EC2 (RHUI) images before 8.9
For backward compatibility, revert changes related to hybrid boot mode
for RHEL (RHUI) EC2 images before 8.9 release.

This change does not affect CentOS Stream 8 AMI images nor the RHEL AMI
build by the service or on-premise.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-23 13:04:31 +02:00
Sanne Raymaekers
8e637848e4 internal/manifest: execute rhc's post install script
The post install script customises the selinux policy, but is gated on
`selinuxenabled`, which will fail inside the buildroot container.

As a result it's never executed.
2023-05-22 21:39:14 +02:00
Paul Whalen
cd5cc08ce6 iot: add fedora-release-iot to iot-installer
Signed-off-by: Paul Whalen <pwhalen@fedoraproject.org>
2023-05-22 17:15:56 +02:00
Tomáš Hozza
9ab20f72b0 distro/fedora: restrict allowed 'iot-raw-image' type customizations
Previously, the rest was mostly silently ignored without errors.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
Tomáš Hozza
76cbec2149 distro/el8: restrict allowed 'edge-raw-image' type customizations
Previously, the rest was mostly silently ignored without errors.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
Tomáš Hozza
2cd7074b23 distro/el9: restrict allowed 'edge-raw-image' type customizations
Previously, the rest was mostly silently ignored without errors.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
Tomáš Hozza
f24e2cb175 image/ostree-raw: pass services from workload to the pipeline
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
Tomáš Hozza
c53471a3f6 manifest/ostree-deployment: support services customization
One of the IOT use cases for custom files and directories is to be able
to enable a custom systemd unit. This was previously not possible to do
via BP Customizations.

In addition, the test case for custom files and directories in
ostree-based images (currently only build-time images) takes advantage
of enabling a custom systemd unit file, this this will enable using the
same test scenario also for deploy-time ostree image types.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
Tomáš Hozza
edf3a38b29 distro/fedora: hook up custom dirs / files for iot raw-image
Set the user provided BP customizations related to custom files and
directories to the iot raw-image type, to ensure that these get
created while deploying a commit.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
Tomáš Hozza
bca3f72657 image/ostree-raw: pass directories and files customization to pipeline
Extend the `OSTreeRawImage` with a slice of custom directories and
files, that can be created in the image. Pass these down to the ostree
deployment pipeline, so that it can add necessary osbuld stages if any
directories or files were specified by user.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
Tomáš Hozza
141611f698 manifest/ostree-deployment: support directories and files customization
Extend the `OSTreeDeployment` with a list of custom files and
directories, that can be created in the deployed ostree commit. This
functionality is equivalent to the one that was added for the OS
pipeline.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
dependabot[bot]
60e55b5ed3 build(deps): bump cloud.google.com/go/compute from 1.10.0 to 1.19.3
Bumps [cloud.google.com/go/compute](https://github.com/googleapis/google-cloud-go) from 1.10.0 to 1.19.3.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/documentai/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/kms/v1.10.0...compute/v1.19.3)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/compute
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Migrated to the new version by following
https://github.com/googleapis/google-cloud-go/blob/main/migration.md

Co-authored-by: Tomáš Hozza <thozza@redhat.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 11:51:42 +02:00
Brian C. Lane
468c63d433 weldr: Expand package name globs in the frozen blueprint
When a blueprint containing package name globs is frozen it was failing
because it could not find the string in the dependency list. This fixes
it by replacing the package glob with all of the matching packages from
the dependency list.

This removes the setPkgEVRA function and adds a new expandBlueprintGlobs
function that handles the package name glob expansion, and replacement of
the version globs with the dependency EVRA.

Also includes testing for the new function.
2023-05-19 08:14:33 -07:00
Brian C. Lane
f378ff6367 rpmmd: Add NEVRA helper functions to PackageSpec
GetEVRA will return the Epoch:Version-Release.Arch string, and
GetNEVRA will return Name-Epoch:Version-Release.Arch
with Epoch being left off if it is zero.

Also includes tests.
2023-05-19 08:14:33 -07:00
Brian C. Lane
c7bc25cead tests: Add a test for freezing a blueprint with globs
This is currently failing (fixed in the next commit). It tests to make
sure that a blueprint with package name globs can be frozen. The
resulting blueprint should replace the glob entries with the expanded
list of packages.
2023-05-19 08:14:33 -07:00
Brian C. Lane
73ab18a501 tests: Add a test for blueprint package name globs
This tests to make sure that package name globs are working during
integration test runs. dnf supports this, and users have been using it,
so testing to make sure it keeps working is important.
2023-05-19 08:14:33 -07:00
Tomáš Hozza
1d01833153 distro/rhel8: use hybrid boot mode for AMI / EC2 image types
On RHEL-8, the x86_64 AMI / EC2 images used a BIOS-only partition table
layout, because the base partition table unification happened in the
past only on RHEL-9 and Fedora (inherited from RHEL-9).

To make things consistent and uniform across RHEL-8 and RHEL-9, I copied
the base partition table used by RHEL-9 AMI / EC2 images to RHEL-8. This
has a side-effect for aarch64 AMI / EC2, where the `/boot` partition
size changed from 512 MiB to 500 MiB, together with the partition GUID
to "Extended Boot Loader Partition GUID".

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Tomáš Hozza
2c9b46876e distro/rhel9: use hybrid boot mode for AMI / EC2 image types
The image already used base partition table with necessary layout to
support hybrid boot mode, so the change was just a matter of modifying
the associated platform.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Tomáš Hozza
e7fdfbaea1 distro/fedora: use hybrid boot mode for AMI image type
The image already used base partition table with necessary layout to
support hybrid boot mode, so the change was just a matter of modifying
the associated platform.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Tomáš Hozza
1d37647506 distro/fedora: remove empty s390x architecture
Fedora distro definition contained an empty `s390x` architecture with no
image types added to it. Let's remove it from the distro definition,
since it's adding no value in its current form.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Tomáš Hozza
7c9d671eae Cloud API: explicitly specify boot mode in AWS upload target
Explicitly specify the AMI boot mode in AWS upload target in Cloud API
compose handler. The value is determined based on image type's boot
mode.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Tomáš Hozza
a9b2964ea6 Weldr: explicitly specify boot mode in AWS upload target
Explicitly specify the AMI boot mode in AWS upload target in Weldr API
compose handler. The value is determined based on image type's boot
mode.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Tomáš Hozza
403b1e4692 AWS: extend target options with the AMI boot mode
Add an optional `BootMode` field to the AWS target options.
This allows to signal to worker the intended boot mode to use when
registering the AMI in AWS. If not specified, the default behavior is
preserved, specifically that the boot mode will be determined by the
default boot mode of the instance provisioned from the AMI.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Tomáš Hozza
e13f0a1ae2 AWS: allow specifying the AMI boot mode when registering the image
When the AMI is being registered from a snapshot, the caller can
optionally specify the boot mode of the AMI. If no boot mode is
specified, then the default behavior is to use the boot type of the
instance that is launched from the AMI.

The default behavior (no boot type specified) is preserved after this
change.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Tomáš Hozza
594778a230 distro: extend the ImageType interface with BootMode()
As a preparation to be able to determine the image boot mode when
importing it to the target environment (e.g. AWS), expose the
information on the `ImageType` level.

The image boot mode is determined based on the platform associated with
it.

The new method is not yet used by any code, but will be eventually used
by osbuild-composer server to set the proper value in the upload target
options for the worker. The worker will be then able to import the image
in the proper way to the cloud environment.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Tomáš Hozza
2c686cc988 distro/rhel: make the GCE image UEFI-only
This was the intention since the beginning (based on images built by
Google. Clean up code and mark the platform associated with GCE image
types as UEFI-only.

The only missing part is the default partition table used by the GCE
image, which is shared with other image types and still contains the
BIOS boot partition. I added a TODO comment to preserve this
information, but kept things as they are for now to not have to
introduce a new set of GCE-specific base partition tables.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Tomáš Hozza
dd26769abc distro/rhel: don't extend image pkg sets with boot packages
Do not extend the image base package set with list of packages needed
for booting the OS, returned by `bootPackageSet()` based on the specific
image type, architecture and its boot type. This duplicated
functionality that is already handled by the platform associated image
and all the necessary packages are provided by the platform's
`GetPackages()` method and added to the base package list.

This reflects changes which were done in Fedora when it was ported to
the "new" image definitions, but were not ported to RHEL.

RHEL-8 GCE image type note:
After a previous change, the image boot type is now determined by the
associated platform and as a result, the GCE image type is marked as
supporting hybrid boot type, although it was meant to be UEFI only. As a
result, the package list returned by `bootPackageSet()` and previously
appended would contain grub2 BIOS-related packages. This is still the
case after this change, because the platform's `GetPackages()` method
will return the same list of packages in this case. However, the
platform used by RHEL-8 GCE image type has its `GetPackages()`
overridden by a different implementation not containing grub2 BIOS
related packages. For some reason, this change is not present in RHEL-9.
As a result, the grub2 BIOS related packages disappeared from the RHEL-8
GCE image package set, while there was no change in RHEL-9.

Keep the GCE image as is for now and make it an UEFI-only in a follow
up.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00