Commit graph

5405 commits

Author SHA1 Message Date
Tomáš Hozza
f8b39cd34e manifest/os: fix SUPPRESSED_ERROR issue reported by Coverity
Fix the following error reported by Coverity:
```
Defect type: SUPPRESSED_ERROR
1. osbuild-composer-75/_build/src/github.com/osbuild/osbuild-composer/internal/manifest/os.go:546: suppressed_error: The error returned by the function "GetVerStrFromPackageSpecList(p.packageSpecs, "dracut-config-rescue")" is not explicitly checked.
2. osbuild-composer-75/_build/src/github.com/osbuild/osbuild-composer/internal/manifest/os.go:546: remediation: Explicitly check the return error.
   544|   				}
   545|
   546|-> 				rescueVer, _ := rpmmd.GetVerStrFromPackageSpecList(p.packageSpecs, "dracut-config-rescue")
   547|   				hasRescue := rescueVer != ""
   548|   				bootloader = osbuild.NewGrub2LegacyStage(
```

Change the code to check if the called function returned an error or
not. If not error is returned, then the function returns a package
version string.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-01 20:20:52 +01:00
Sanne Raymaekers
75e6333b9e Revert "tools/appsre-ansible: workaround missing authselect-compat"
This reverts commit 4833f99070.
2023-03-01 20:05:38 +01:00
Sanne Raymaekers
0096ff3689 Revert "Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos"
This reverts commit 0a4a75e19e.
2023-03-01 20:05:38 +01:00
Sanne Raymaekers
d9f2ac14bc tools/appsre-ansible: don't subscribe machines used for rpmbuild
We actually use rhui/cloud access images for rpmbuild as well. And the
sync issue between cdn and rhui repos can cause issues when rpms are
build against older packages.
2023-03-01 20:05:38 +01:00
Sanne Raymaekers
56aa40ea5e test/data/manifests: regenerate fedora manifests 2023-03-01 14:35:56 +01:00
Sanne Raymaekers
b06863e468 distro/fedora: add open-vm-tools to vmdk package set 2023-03-01 14:35:56 +01:00
schutzbot
12e5dbd531 Post release version bump
[skip ci]
2023-03-01 12:06:53 +00:00
Antonio Murdaca
ed4d12fe33 ignition: enable systemd firstboot condition through kargs
This is a workaround to make the systemd believe it's firstboot
when ignition runs on real firstboot.
Right now, since we ship /etc/machine-id, systemd thinks it's not firstboot
and ignition depends on it to run on the real firstboot to enable services from presets.
Since this only applies to artifacts with ignition and changing machineid-compat at
commit creation time may have undesiderable effect, we're doing it here as a stopgap.
We may revisit this in the future.
This patch also pins the version of osbuild because it depends on a fix
for the ignition stage.

Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-28 18:35:45 +01:00
Alexander Todorov
72b223c064 Remove duplicate function and use the shared implementation 2023-02-28 11:15:49 +01:00
Alexander Todorov
e064c12449 Update test suite after rebase to weldr-client-35.9
in order to avoid failures in nightly pipelines b/c JSON format has
changed
2023-02-28 11:15:49 +01:00
Juan Abia
a761184725 Modify CIV options
With the previous method of passing options, we encountered some
problems when sending the options via an array.
2023-02-28 10:26:01 +01:00
Achilleas Koutsou
f11589367e internal: FIXME notes for container package set
Tracing the package set and repository journey, I found that the
container package defined on the image type is never really used.

Added notes to fix later.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
d2a5c9aafa test: regenerate affected manifests
RHEL 7 manifests included rel-eng packages in their OS pipelines.
These have now been removed.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
b8fc241755 gen-manifests: copy whole repo config when generating tests
The bug wasn't caught because the PackageSets field of the repository
wasn't being copied after parsing the compose request for the test
manifest.

This should now catch future occurrences of this bug.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
b0a6b486a1 distro/test: set dnf-json path for solver
The value comes from the command line args of the test binary
(osbuild-composer-manifest-tests).  It works in our tests because we use
the default value, but if it was set differently it would have been
ignored.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
114ed20fab distro/test: include package sets field in manifest test repos
The test previously ignored the package-sets field in the repos in the
compose request.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
2a4cd1966d distro: pass all repos to initializeManifest
Pass through all repos to the initalizeManifest() function.  Each
pipeline will then select which repositories it needs based on the
PackageSets field of each repository.

Before, we only passed global repos down to the manifest generators and
pipeline-specific repositories would only be used if they were attached
to package sets and were handled explicitly by a pipeline generator.

The repositories of the "blueprint" package set are explicitly added to
the workload and returned by the "os" pipeline.
The repositories of the "installer" package set are explicitly added to
the "anaconda-tree" pipeline.

If a repository was specified for any other pipeline, for example
"build", the repositories for the that package set were never added to
the pipeline.

Fixes #3290
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
c7e0b69704 manifest: add only necessary repositories to each pipeline
Each pipeline constructor receives the full list of repositories and
filters the list based on its own internal name.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
a66003e513 manifest: helper function for collecting pipeline repos
Function that filters a list of repositories to return only the global
repos and any that define a given pipeline or package set name in their
list of PackageSets.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
50f578924c distro: rename osPkgsKey from "packages" to "os"
Match the key for the OS packages to the name of the OS pipeline ("os").
We will use this key to identify package sets that are returned from the
OS pipeline since the manifest returns package sets indexed by each
pipeline's name.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
942d807877 weldr: fix payload repos package sets assignment
The PayloadPackageSets() was being added to the wrong field in each
payload repository.
Also updated the outdated docstring.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
b918dc8a8c distro/test: add new test for pipeline-specific repos
New test that runs through a few different scenarios of assigning
repositories to specific package sets.  Each scenario defines a set of
global repositories (or none) and a set of pipeline-specific
repositories (or none) and an expected result.  It then calls the
distro's PackageSets() method.  The test checks that the package sets
that are returned are assigned the correct repositories based on the
test case's expected result.

A basic string-set struct is implemented to help with merging and
comparing string slices.

This test should currently fail due to #3290.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
777d27efbf distro/test: remove RHEL 7 exception from PipelineNames test
RHEL 7 rewrite makes this exception obsolete.
2023-02-23 16:22:42 +01:00
schutzbot
cd8898842c Post release version bump
[skip ci]
2023-02-22 14:01:35 +00:00
Xiaofeng Wang
eaee7870ba test: Add user in simplified installer blueprint
Run ansible playbook with user configured in simplified installer
blueprint
2023-02-22 12:18:12 +01:00
Irene Diez
38b6b9cdcf distro/rhel8+9: allow User/Group customizations on edge-si
Signed-off-by: Irene Diez <idiez@redhat.com>
2023-02-22 12:18:12 +01:00
Irene Diez
dbadd42a01 distro/rhel8+9: deprecate User/Group customi. on edge-commit/container
Signed-off-by: Irene Diez <idiez@redhat.com>
2023-02-22 12:18:12 +01:00
Tomáš Hozza
f5580b1f3e Test: regenerate image manifest that use files customization
Regenerate image manifests which use the files customization and are
affected by the recent change to `osbuild/fsnode.go`, specifically to
always setting the `RemoveDestination` to `true`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
9bebd98d5b osbuild/fsnode: default to removing destination for custom files
Default to always remove the destination before copying when generating
the copy stage options for custom files in the image. This will ensure
that if the destination is an existing symlink to another file, it won't
be followed.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
d9f8dcdd44 Schutzfile: pin osbuild v80 on all distros
Commit is taken from:
https://github.com/osbuild/osbuild/releases/tag/v80

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
145e89211d SPEC: depend on osbuild v80
Set the dependency to osbuild version 80, which contains the
`remove_destination` option in the copy stage.

https://github.com/osbuild/osbuild/releases/tag/v80

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
ad860a6dd4 osbuild/copy: add support for remove_destination option
Add support for newly added `remove_destination` option in the copy
stage.

Related to https://github.com/osbuild/osbuild/pull/1241

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
19e38621c9 Test/ostree-ng: test dirs and files customizations
Test the directories and files customization with the ostree container
image type deployed using the (Anaconda-based) installer image.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
3cf77187d6 Test: reformat parts of ostree-ng.sh
Reformat calls of `ansible-playbook` by splitting them into multiple
lines. This makes it easier to read.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
6cd7892c47 Test/ostree: test dirs and files customizations
Test the directories and files customization with the ostree commit
image type.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
50fc6cc29c Test: reformat part of ostree.sh
Split the call to `ansible-playbook` to multiple lines.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
55c3abe842 Test/ansible/check_ostree: test dirs and files customization
Extend the `check_ostree.yaml` to check the result of directories
and files customizations on a running ostree-based system. The test
also checks if a custom systemd unit `custom.service` got enabled
and started on boot.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
c528948b71 Test/API: use directories and files customization for supported images
Use directories and files customization in the compose request for image
types that support this customization (only ostree installer and raw
image do not support it).

Extend the instance verification to check for the custom directories and
files.

Extend the ostree commit verification to check for the custom
directories and files.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
d1325aaf45 Test/API/GCP: don't set gcloud format to JSON by default
Previously, the `--format=json` was always set in the `$GCP_CMD`. This
was working OK in general, but it is creating issues when using
`$GCP_CMD compute ssh` to execute commands on the provisioned VM. In
such case, an extra `[]` is added to the output, which then causes
various checks to fail, since the captured stdout is different than
expected.

Instead remove the `--format=json` by default from `$GCP_CMD` and use it
explicitly only in cases when `jq` is used to parse the command's
output. In all other cases, it should be OK to not set the output format
to JSON and it is very much desirable to do so when SSH-ing to the VM.

Also by default pass the `--quiet` option to `gcloud` when using it from
the container. This was previously done only when using the tool
installed on the system.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
1c64d15a61 Test/fedora: add iot-container test manifests with customizations
Add `iot-container` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
df26be35cc Test/fedora: add iot-commit test manifests with customizations
Add `iot-commit` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
e16fdeb7ed Test/el/cs: add edge-container test manifests with customizations
Add `edge-container` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
5a7092bfd5 Test/el/cs: add edge-commit test manifests with customizations
Add `edge-commit` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
1ea58566ff tools/tc-generators/format-req-map: drop all rhel-edge-* requests
Drop unused `rhel-edge-*` requests, which are not used any more. They
have been replaced by `edge-*` requests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
aa9373cb18 Test: use dir and file customizations in qcow2 image manifests
Add the new directories and files customization to the customized qcow2
image manifests used for testing.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
e9af3bda64 Cloud API: expose directory and file customizations
Expose the Directory and File customizations in the Cloud API. Also
validate the provided customizations while processing the compose
request by trying to convert them to internal representations
`fsnode.File` and `fsnode.Directory`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
15f1962c34 Cloud API: add error for failed customization validation
Add new error `ErrorInvalidCustomization` used in situations when the
user-provided customization values don't pass validation. This will be
used by the Directory  / File customization.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
ffd0bdb7ad distro: add dir / files customizations policy check
Add a default policy for custom directories and files to constrain what
users can do. The intention is to ensure that directories and files can
be created only in `/etc` and also that none of the important
configuration files can be overwritten by this customization.

Add the policy validation to all distro implementation.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
3ee973c8ee blueprint: add functions checking dir / file customizations policy
Add helper functions for checking directory and file blueprint
customizations against the policy of allowed paths.

These functions are not yet used in the distro definitions.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
b98f5dad70 distro: set dir / file BP customizations in OS customizations
Hook up the custom BP directories and files implementation with OS
pipeline implementation. The user-provided values are now set in the OS
customizations structure and will be used by the OS pipeline generator
when adding stages to the pipeline.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00