Commit graph

5305 commits

Author SHA1 Message Date
Sanne Raymaekers
8f9f0d03c5 repositories: fix rhel-90 repositories
The rhel-90 repos should point to the rhel-90 minor version repos, the
rhel-9 repos should point to the rolling release.
2023-02-06 17:35:54 +01:00
Antonio Murdaca
e34b88d6e4 .github/workflows: pin golangci-lint to v1.51.0
The new "latest" aka v1.51.1 started throwing unused errors during runs,
blocking all the other tests, to quickly unblock that until a proper
fix/rewrite is found, pin the version to the previous one.

Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-06 11:04:06 +01:00
Ondřej Budai
9633dce3bb osbuild-pipeline: add support for RHSM when defining repos
I use this tool quite a lot and I often want to use the CDN content, so
I would very much appreciate RHSM support. :)

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-02-06 10:02:04 +01:00
Achilleas Koutsou
d877b7419e osbuild-dnf-json-tests: remove distro loop
We only test one distro, CentOS 9, so there's no need to loop and spawn
a new t.Run().
2023-02-02 19:01:20 +01:00
Brian C. Lane
356f261bcd dnf-json: disable pylint warnings
The _dnfrepo is full of branches, turn off warning.

The keyfile doesn't use 'with' because the file needs to remain
available when the function exits. Cleanup of persistdir will clean up
the temporary file used for the key.
2023-02-01 10:27:58 +01:00
Brian C. Lane
d5e666cc9c tests: Add tests for new repo gpg support
Add testing for gpgkeys and check_repogpg.
2023-02-01 10:27:58 +01:00
Brian C. Lane
c2577eaea8 Add gpgkey and check_repogpg support to dnf-json
This allows verification of repository metadata signatures.

The gpgkeys field is a list of key urls, or the gpg key itself, starting
with '-----BEGIN PGP PUBLIC KEY BLOCK-----'. These will be written to a
temporary file, and that file:// url will be passed to dnf.
2023-02-01 10:27:58 +01:00
Brian C. Lane
d12447408b Change RepoConfig.GPGKey to an array of key strings
DNF supports more than one GPG key. It is possible that one may be used for
signing packages, and another to sign the repository metadata. This
renamed GPGKey to GPGKeys internally. It does not change the on-disk
repository json format.
2023-02-01 10:27:58 +01:00
Brian C. Lane
a47375a10d Rename gpgkey_urls to gpgkeys
This has never been supported in osbuild-composer, so it is safe to
rename in preparation for adding repo metadata gpg signature checking.
2023-02-01 10:27:58 +01:00
Achilleas Koutsou
7a1d7ae1d7 test: update installer manifests for RHEL 9.x and CS9
Dracut modules and drivers fixed for the dracut stage in all RHEL 9.x
and CS9 Anaconda installers (image-installer and edge-installer).
2023-01-31 11:03:26 +01:00
Achilleas Koutsou
2f96cc7051 manifest: fix dracut modules and drivers for Anaconda installers
When the image definitions were updated to the new framework, I failed
to update the dracut modules for Anaconda installers to match the
existing ones.

The changes in the manifest are at commit
c4af0a1886.
The nvdimm module and the additional drivers were removed.

The nvdimm module in particular is required for http boot but should
only be specified for RHEL 9.  In RHEL 8 it is part of the default set
of modules.

See 02bb7a0b4f and
dc95382ba3 for the original commits that
introduced these changes.
2023-01-31 11:03:26 +01:00
Irene Diez
1dfc82ae4a test/data/manifests: regen for edge container + commit
Signed-off-by: Irene Diez <idiez@redhat.com>
2023-01-30 19:26:44 +01:00
Irene Diez
3b62e15d01 internal/distro/rhel8/edge: add sos package for rhel8.6+ and CS8
Similarly to the change made for rhel9, adding the sos package
gives users a built-in way to gather system logs and debug info.

Signed-off-by: Irene Diez <idiez@redhat.com>
2023-01-30 19:26:44 +01:00
Irene Diez
18aee4e465 internal/distro/rhel9/edge: add sos package
Adds the sos package to the edge commit package set so that
users have built-in way to gather system logs and debug info.

Signed-off-by: Irene Diez <idiez@redhat.com>
2023-01-30 19:26:44 +01:00
Achilleas Koutsou
2bdf6c724f distro/rhel8: update distro unit tests 2023-01-30 17:40:18 +01:00
Achilleas Koutsou
9706de4f44 test: generate new manifests: RHEL 8 Azure aarch64 2023-01-30 17:40:18 +01:00
Achilleas Koutsou
5e27e01e5e test-case-generators: enable rhel8 azure aarch64 manifest generation
Add the rhui-azure rpmrepo snapshots to aarch64 repositories.
The repos are labelled as x86_64 but they're actually 'noarch' and we
name them x86_64 when creating the snapshots.
2023-01-30 17:40:18 +01:00
Achilleas Koutsou
2723554b2a distro/rhel8: add Azure aarch64 platform and include image types
The Azure image types require hyperv-daemons which isn't available on
RHEL 8 prior to 8.6.
2023-01-30 17:40:18 +01:00
Achilleas Koutsou
c3d5fd516f distro/rhel8: add partition table for aarch64 azure-rhui
Copy of the partition table for the same image type in RHEL 9.
2023-01-30 17:40:18 +01:00
Achilleas Koutsou
f35d37188b distro/rhel8: clarify partition table error message
The previous error didn't make it clear where the issue was coming from.
Now it explains that the problem is that a partition table for a given
architecture isn't specified on the image type.
2023-01-30 17:40:18 +01:00
Achilleas Koutsou
e56445593c distro/rhel8: remove arch-dependent packages from Azure common 2023-01-30 17:40:18 +01:00
dependabot[bot]
70c61d4c24 build(deps): bump ludeeus/action-shellcheck
Bumps [ludeeus/action-shellcheck](https://github.com/ludeeus/action-shellcheck) from 6d3f514f44620b9d4488e380339edc0d9bbe2fba to 00cae500b08a931fb5698e11e79bfbd38e612a38.
- [Release notes](https://github.com/ludeeus/action-shellcheck/releases)
- [Commits](6d3f514f44...00cae500b0)

---
updated-dependencies:
- dependency-name: ludeeus/action-shellcheck
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 12:26:29 +01:00
Tomáš Hozza
1667e6e859 osbuild: rework RPM stage inputs
Rework the stage to not reimplement `FilesInput` as
`RPMStageInput`, but instead use the one common
`FilesInput` implementation and its supported
references.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-30 11:24:08 +01:00
Tomáš Hozza
b25e4bfa8b osbuild: rework Ignition stage inputs
Rework the stage to not reimplement `FilesInput` as
`IgnitionStageInput`, but instead use the one common
`FilesInput` implementation and its supported
references.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-30 11:24:08 +01:00
Tomáš Hozza
645956d203 osbuild: rework QEMU stage inputs
Rework the stage to not reimplement `FilesInput` as `QEMUStageInput`,
but instead use the one common `FilesInput` implementation and its
supported references.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-30 11:24:08 +01:00
Tomáš Hozza
3f90a32661 osbuild: rework FDO stage inputs
Rework the stage to not reimplement `FilesInput` as
`FDOStageInput`, but instead use the one common
`FilesInput` implementation and its supported
references.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-30 11:24:08 +01:00
Tomáš Hozza
dd59ce6a16 osbuild: rework XZ stage inputs
The `FilesInputs` was since the beginning an XZ-specific implementation
of the input, but it was implemented in the `files_input.go` in a false
hope that it could be used as a generic stage inputs by any stages. It
turned out that various stages require different implementation of
its input. Specifically there is usually a stage-specific key, which has
assigned a common input type. For XZ stage, the key is `file`.

Remove `FilesInputs` and instead implement `XzStageInputs` which is now
accepted by the XZ stage.

Fix all affected pipeline implementations that use XZ stage.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-30 11:24:08 +01:00
Tomáš Hozza
413538a18e osbuild: rework files input
Rework files input implementation to support all reference types
supported by the input schema. Also implement helper functions to
generate supported reference types. In some cases, the reference
supports e.g. referencing multiple pipelines in the stage inputs, but
this is currently not implemented, since no pipeline in composer uses
it.

Rework the files input unit tests to cover the new functionality.

Adjust all code affected by the changes made to files input.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-30 11:24:08 +01:00
Diaa Sami
081cc37973 osbuild-worker: Add log statements for Splunk dashboard 2023-01-27 12:02:12 +01:00
Simon de Vlieger
a8b6fd45ec iot-installer: enable anaconda modules
As noted in #3141 I've enabled the extra modules for Anaconda for the
iot-installer image type.
2023-01-27 10:46:27 +01:00
Achilleas Koutsou
6c503e4edc test: update rhel-91 manifests
Updated manifests for RHEL 9.1.

The changes added here would have been part of commit
73882b7289 but the lack of repositories
defined for the distro prevented test case generation.
2023-01-26 20:32:38 +01:00
Achilleas Koutsou
db97690b84 tools: add RHEL 9.1 repos for test case generators
In commit 7c8b9f4da8 I intended to make a
copy of the RHEL 9.1 repos under the key "rhel-9" so that the
RHEL 9 distribution, which is currently an alias to RHEL 9.1 (the
current GA) would have valid test repositories available for the test
case generator.  Instead, I accidentally replaced the "rhel-91"
repositories with the ones for "rhel-9".
This commit fixes that mistake.
2023-01-26 20:32:38 +01:00
Diaa Sami
7a67e7b7e8 osbuild-worker: annotate log statements used by splunk
So that they are not changed by mistake
2023-01-26 11:34:20 +01:00
Achilleas Koutsou
fe7860778c test: update RHEL 7 manifests
Changes:
- Removed unnecessary packages from build roots.  Build packages are
  added on-demand.
- Some stages changed order in the OS pipeline for azure-rhui.  This
  will have no functional effect.
- RHSM Facts stage added to all RHEL 7 manifests.  This wont be used
  since RHEL 7 is not built in the cloud service.  The RHSM fact is
  added by the test case generator to all image types to test that the
  option is working properly.
- Azure archive pipeline renamed to xz like in all the other rewritten
  image types that use the xz pipeline.
- Azure image filename before archiving has changed from 'disk.vhd' to
  'image.vhd'.  Again, this will have no functional effect because the
  name of the final artifact is unchanged (disk.vhd.xz).
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
450c44aaff manifest: include yumconfig options in OS customizations
YUMConfig is supported in ImageConfig for RHEL 7.  We now copy the
options over to OSCustomizations and create the stage when necessary.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
d2114d0f9e manifest: support force-relabel for SELinux in image config
The RHEL 7 images need to enable the force_autorelabel option for the
SELinux stage in osbuild.  This option should almost never be used but
it was added specifically for RHEL 7.  With the rewrite to the new
definitions and the sharing of pipeline code between all distros, we
need to add support to all stages of the pipeline generation to be able
to enable it.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
822571e28e manifest: support specifying force_size for VPC
The RHEL 7 vpc subformat in qemu does not support force_size so we need
to be able to disable it.  The parameter in all parts is defined as a
pointer because the default value is 'true'.  Not specifying it will
keep the option in the osbuild stage as 'nil', falling back to 'true' in
osbuild.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
b39d802155 manifest: add support for selecting grub2.legacy
Older OS versions (RHEL 7) with older versions of grub2 don't support
BLS entries.  Setting NoBLS to true configures the bootloader with
traditional menu entries through the grub2.legacy osbuild stage.  This
requires specifying extra information for the OS to the pipeline:
version, product, and nick.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
83fc8218d6 manifest: make the partition tool for raw images configurable
Add the partition tool as an option on the Raw pipeline.  Set it to the
old value (sfdisk) by default.

Expose the option up through the liveImage image kind so that the
distribution can set it if needed.
For RHEL 7, set it to sgdisk.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
1cc53f00f4 distro/rhel7: delete unused functions, types, and constants 2023-01-25 20:37:12 +01:00
Achilleas Koutsou
05094ea5c8 distro/rhel7: update image types to new definitions
- Replace Manifest() and PackageSets() imageType methods with (adapted)
  copies from RHEL 8.
- Replace pipeline functions with liveImage image function.
- Specify xz compression for Azure RHUI.
- Add similar package name overrides as we did in RHEL 8.  For RHEL 7,
  we need to modify the capitalisation of python3-pyyaml.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
79a0d36bf4 distro/rhel7: move qcow2 image type definition to top of file
Moved the qcow2 image type definition to the top of the file for
consistency with the other image type files.
Separated the default image config struct from the base image type
definition to make it easier to read.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
34a38008d9 distro/rhel7: move azure image type definition to top of file
Moved the Azure image type definition to the top of the file for
consistency with the other image type files.
Separated the default image config struct from the base image type
definition to make it easier to read.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
318fdf5ffd distro/rhel7: add osCustomizations() and liveImage()
Copied osCustomizations() and liveImage() functions from RHEL 8 and remove
unneeded customizations and options.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
b5795d18b2 distro/rhel7: update imageType implementation
Add environment, compression, and image fields and define the imageFunc
function type.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
cef5f3d3bd runner: add platform-python only for RHEL 8+
No platform-python for RHEL 7.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
c12f2c61e2 distro/rhel7: and add platforms to image types 2023-01-25 20:37:12 +01:00
Achilleas Koutsou
6fdf917271 distro/rhel7: update distribution implementation
Update the implementation of the distro.Distro interface to match the
one in RHEL 8, 9, and Fedora.  The main change is that the runner is a
runner.Runner and not a string.

The runner name is now rhel79 (changed from rhel7).  This is
functionally equivalent based on osbuild's runner version fallback
logic.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
a93ae83c5c distro/rhel7: add packageset chains to qcow2
The qcow2 image type for RHEL 7 doesn't have packageset chains defined.
This means that the blueprint packages are never merged into the os
pipeline.

This is unnecessary right now because of the upcoming rewrite, but it
will minimise the differences that will show up in the manifest.

The qcow2-customize manifest has an added block of options for the
grub2.legacy stage because now the dracut-config-rescue package is being
installed in the image.
2023-01-25 20:37:12 +01:00
Sarita Mahajan
0d16e2310e image: add new image type "minimal-raw" for fedora
Signed-off-by: Sarita Mahajan <sarmahaj@redhat.com>
2023-01-25 19:19:45 +01:00