Commit graph

5059 commits

Author SHA1 Message Date
Achilleas Koutsou
4bd2c0cb1d test: regenerate manifests
Regenerate manifests with the added RHSM fact.
RHEL 9 manifests haven't changed because the stage is missing in the OS
pipeline (bug).
2022-11-30 10:04:16 +01:00
Achilleas Koutsou
e3e71f01bf cmd/gen-manifests: add package-sets to repo config
Was missing the package sets key from the repo config struct, which
means that the option was being ignored and wasn't being serialised into
the test manifest either.
2022-11-30 10:04:16 +01:00
Achilleas Koutsou
de7bae02f3 cmd/gen-manifests: add RHSM fact to options
Add RHSM fact to image options when generating test manifests.
We add the value "test-manifest" to the API type to indicate it's a test
manifest.  This should never be registered and therefore shouldn't show
up in our data, but it's useful to detect changes and regressions in the
fact creation in the pipelines.
2022-11-30 10:04:16 +01:00
schutzbot
f7334287af Post release version bump
[skip ci]
2022-11-30 08:28:40 +00:00
Tomáš Hozza
c6570f6c94 CloudAPI: add description for Repository definition
Specifically describe that at least one of `baseurl`, `mirrorlist` or
`metalink` properties must be specified. Although this is not enforced
by the schema, it is the reality. Modifying the schema to enforce this
results in the generated data structure for `Repository` to be simply
an `interface{}`, which would just make the code more ugly, Therefore
this is covered by the API implementation logic.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-28 17:30:56 +01:00
Achilleas Koutsou
b3f4d75699 image: panic if unknown compression option is specified
Currently we only support "xz", but keeping it as a `switch` to easily
support more types in the future.
The empty string is also supported as a no-op.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
6374ec022d distro/rhel9: define qcow2 image type in function
The global qcow2ImgType was unconfigured and meant to be used by calling
the mkQcow2ImgType() function.  In the distro initialisation code, the
new variable created from the mkQcow2ImgType() function with the
configuration was shadowing the global variable.  This can lead to
errors in the future where it's not clear which variable is used where.

Putting the definition in a function makes it impossible to use without
configuring.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
7da0cde855 test: regenerate image-installer and edge-installer manifests 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
9ae3d03f88 tools: add package to image-installer manifest request
Add podman to the image installer's packages when generating test
manifests.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
fa948b9e0b distro: set the workload for the image installer pipelines 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c6bfb22f54 manifest: support specifying kickstart file location
Add support for specifying where the kickstart file will be created:
either in the default location in the anaconda tree or at the root of
the ISO like we did so far in RHEL.

From the distro level, this is controlled by a boolean that when set to
true puts the kickstart file in the root of the ISO tree at /osbuild.ks
(the old behaviour) and if unset falls back to the default location
/usr/share/anaconda/interactive-defaults.ks.

Changed the name of the kickstart file for ostree back to osbuild.ks
Was ostree.ks for a while since a1bfcfe91c.

For Fedora we use the default and for RHEL we maintain the old behaviour
for now.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
d4a8775751 test: update installer manifests
- Added users module.
- Set squashfs compression to xz for RHEL 9.x and CentOS Stream 9.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
5d55ccf109 manifest: support specifying squashfs compression for ISOs
Add support for specifying the squashfs compression method for ISOTree
pipelines from the caller.
Build Fedora ISOs with lz4 compression and RHEL with xz.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
cb0280c8c9 image: always enable users module in anaconda
Unconditionally enable the users module in Anaconda.
The module serves two purposes:
- It will create a user that's defined in the blueprint by adding the
  user creation stage in the kickstart file.
- It allows users to create user accounts interactively during
  installation.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
11b08da7a5 distro/rhel9: cleanup: delete unused functions 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
42274fd6a7 distro_test_common: refactor the kernel count test
- Add a list of image type names to skip with explanations.
- Run the test in two configurations: empty blueprint and blueprint with
  named kernel (kernel, kernel-debug).
- Count only unique kernel names.  Specifying the same kernel twice
  isn't an issue.  We might change this in the future, but for now the
  kernel selection logic is a bit messy.
- Temporarily skip the image-installer until we settle on a common rule
  for both Fedora and RHEL.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
ff494d66f6 test: update RHEL 9.x and CS9 tar manifests 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
d66bc11bc7 distro/rhel9: update tar image type to new definition
Add the image function and remove the pipeline function.
Remove the build package set.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
e100c57e58 image: new ImageKind: archive 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c4af0a1886 test: update RHEL 9.x and CS9 image-installer manifests 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
108ac3931b distro/rhel9: update image-installer to new definition
Add the image function and remove the pipeline function.
Remove the build package set.

The image function (imageInstallerImage) was adapted from the same one
in Fedora.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
e031b45377 distro: set ostree ref during package collection
Similar to 194351e681 and
9ae47b6843cace3c9bef55d40a33fa11d8de391a.

Manifest initialisation during package collection uses options and
customizations to determine build root packages.  Just like adding a
fake ostree checksum and an unresolved container list, we also need to
add the ostree ref, which is used to determine if a pipeline is being
generated for an ostree-based image type.

This fixes manifest generation when an ostree commit or container is
being built with an embedded container through the cloud API.  Without
this change, the python toml module is not installed in the build root
and the container storage stage fails.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
60d2a88d4f distro/test: add containers to manifest test
Read the containers from the test manifests when regenerating for the
test.
Also move the ostree option handling higher and make it cleaner.  Only
apply the values if they're specified and needed.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
8612131351 store: add containers to test 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
3400c27153 test: update manifests with containers
Update test manifests that have embedded containers with the new info.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
1fc1178dea gen-manifests: save resolved containers
Store resolved container info in the test manifests so that they can be
read in the manifest unit test.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
bb75cc0947 gen-manifests: change default output directory
Change the default output directory to the one in the repo.
Originally it was set to a different directory to avoid overwriting the
manifests that had image-info, but those are long gone.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
5c8b5995bf test: update manifests with embedded containers
Updated manifests (and generated new ones) that embed a container:
- edge-commit on RHEL 9.x and CS9
- qcow2 on RHEL 9.1
- iot-commit on all Fedora versions (new)
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c5f6aca53b format-request-map: iot-commit with container
Add manifests for container embedding in iot-commits for Fedora.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
e192cc4a18 gen-manifests: set default ref before resolving packages
Package collection reads options and customizations to determine which
build packages to pull, so we must set the default ref early if it's
needed.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
09903bd0db manifest: support for container embedding
Adding support for container embedding.
The containers need to be specified in the image function (imageFunc)
arguments and when specified, propagate down to the OS pipeline
generator to add the necessary stages.

Support is added for RHEL 9.x and Fedora.

Requires a temporary container spec array with the info from the
blueprint for the first initialization of the manifest that's needed
when collecting required packages.
This should be simplified in the future.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c4b5c229fb distro/rhel9: delete unused pipeline functions 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
5086f1e413 test: update RHEL 9.x and CS9 gce manifests
Regenerate manifests for the gce and gce-rhui image types with the
rewritten definitions:
- remove unnecessary RPMs in the build root
  artifact
- some stages changed order, but the pipelines be functionally
  equivalent
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
2369869db7 distro/rhel9: update gce images to new definitions
add the image function and remove the pipeline function.
Remove the build package set.

Parameterise image config creation functions so that RHSM is added
conditionally based on distro name, like we did for AMI/EC2.

image: set the raw filename for the GCE image type

GCP requires that the raw image file inside the archive be named
'disk.raw'.  We set it on the imagePipeline while instantiating the
manifest as a workaround for now.

This should be changed to be configurable on the image type when
necessary, the same way the final filename is defined.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
a1332c5b82 manifest: add more customizations to live image
Extend the OSCustomizations to include options for dnf-automatic,
yum.repos, and gcp.guest-agent.
Propagate these options from the image configs in rhel9 down to the
OSCustomizations and add the stages to the os pipeline when they're
specified.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
581dde4215 manifest: use osbuild firewall options on os customization
OSCustomizations defined firewall options as blueprint firewall
customizations.  Changed it to instead be osbuild firewall stage
options.
This is swapping out one messy thing for another, but at least now we're
consistent in our mess.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
f0e29ab697 image: handle the GCE format in the live image
Add the options required by GCE to a tar pipeline at the end of the
manifest for images defined on the GCE format.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
8c361e510d platform: add special GCE platform format
GCE requires an archive with specific (tar) options, so we should have a
platform format defined separately for it.
Add the GCE image types to the x86_64 arch using the new platform
format.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
3fc8fd2cb9 manifest: support all tar options in tar pipeline
Make the tar pipeline take any implementation of the Pipeline interface
as an input argument.
Add support for all the tar stage options.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
1a0a16bc46 test: update RHEL 9.x and CS9 azure and vhd manifests
Regenerate manifests for the Azure and VHD image types with the
rewritten definitions:
- remove unnecessary RPMs in the build root
- changed order of rhsm stage: this should have no effect functionally
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
1d1523828a distro/rhel9: update azure images to new definitions
Add the new image function and remove the pipeline function.
Remove the build package set.
Rename the last pipeline to xz for the RHUI image type.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
ae37ebb286 image: support compression on all live image types
Apply "xz" compression to any artifact pipeline if it's specified in the
image type.  The image filename should be applied to the pipeline only
if it's the last one, so we need to skip this assignment if we're going
to add a compression pipeline at the end.

This is a bit dirty but the plan is to remove artifact compression from
manifests and perform it during the export in osbuild-composer, so this
should go away soon.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
458f51128a manifest: set default VPC filename to image.vhd 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
4f60aa3c1d distro/rhel9: move azure image definitions to the top of the file 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
db88c734c1 manifest: add support for waagent and udevrules to liveImage
Extend the OSCustomizations to include the WAAgentConfig and UdevRules
options.
Propagate those options from the image's config down to the
OSCustomizations and add the stages to the os pipeline when they're
specified.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
f3e43a01bc test: update RHEL 9.x and CS9 manifests
Adding python3-pyyaml to the build root for manifests that contain
cloud-init in the OS pipeline.
Adding python3-iniparse to the build root for manifests that contain
rhsm or dnf.config in the OS pipeline.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
370a0d1b29 manifest: add packages required by osbuild stages
Add python3-pyyaml to the build root when the OS pipeline includes the
cloud-init stage.
Add python3-iniparse to the build root when the OS pipeline includes
the dnf.config or rhsm stage.
The addition of the stages is detected using the corresponding
customizations.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
b0574daa90 test: update RHEL 9.x ec2 and ec2-ha manifests
Regenerate manifests for the EC2 and EC2-HA image types with the
rewritten definitions:
- remove unnecessary RPMs in the build root
- added dofstools to the image
- internal raw image filename changed: this has no effect on the final
  artifact, since it's compressed and renamed before exporting.
- rhsm stage order changed.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c35c4784a4 distro/rhel9: update ec2 and ec2-ha to new definitions
Add the image function and remove the pipeline function.
Remove the build package set.
Add "xz" compression and rename the last pipeline.
Propagate the compression option down to the liveImage() image creation
function.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
6fed422972 image: xz compress a pipeline if the image specifies it
If an image specifies xz compression, add an xz pipeline to the end of
the manifest.
2022-11-28 17:20:49 +01:00