Commit graph

6498 commits

Author SHA1 Message Date
Alexander Todorov
39370abce4 Define rhel-9.5 repos
Related: COMPOSER-2227
2024-06-07 10:31:54 +02:00
Michael Vogt
a691df2353 osbuild-worker-executor: fix order of assert.Equal() in tests
The `assert.Equal()` expects that the "expected" value is put
first. Which is not what I'm used to. It's also slightly inconsistent
because `assert.EqualError()` expects the "actual" err first and
then the expected string. But this commit is not about ranting :)

This commit fixes the order in the tests assert.Equal() so that
mismatches actually are displayed correctly.
2024-06-07 08:17:32 +02:00
Michael Vogt
61bf0c3235 osbuild-worker: do not use error in clienterror.Error.Details
This is an alternative/complementary fix for PR#4137. It is very
simple so should be uncontroverisal.

It fixes an issue that @schuellerf discovered, i.e. that when an error
interface is passed into clienterrors.Error.Details the details get
lost because the json.Marshaler will not know how to handler an
error interface.

To find the problematic uses of `error` a custom vet checker was
build in https://github.com/mvo5/osbuild-cvet. With that the
result is:
```
$ go run github.com/mvo5/osbuild-cvet@latest ./...
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-depsolve.go:93:26: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:404:31: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:519:31: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:556:31: do not pass '[]error' to WorkerClientError() details, use []string instead
```
and once this commit is in no more errors.

Just like PR#4137 this is not perfect because it will not do a
recursive check for the passed argument.
2024-06-07 01:19:11 +02:00
Michael Vogt
8ebefbdbc9 main: rework the way the mock logger is passed
Pass the mock logger directly to `run()` instead of mocking
`logrus.New`. Doing the later leads to a data race when multiple
parallel tests modify the (global) `var logrusNew logrus.New`.

Thanks to Tomas Hozza for reporting.
2024-06-06 21:14:31 +02:00
Michael Vogt
95b4a9e250 osbuild-worker-executor: make test output silent again
Do not use the global logger but pass instead the locally created
logger. This means the test output is silent again.

Sadly using the global logger is difficult because it is a global
resource so replacing it in tests means all tests (that are
potentially run in parallel) will write to it which makes testing
specific log output hard.
2024-06-06 16:16:33 +02:00
Michael Vogt
a634868793 all: run ./tools/prepare-source.sh 2024-06-05 18:26:08 +02:00
Michael Vogt
e34728b466 osbuild-worker-executor: appease errcheck 2024-06-05 18:26:08 +02:00
Michael Vogt
138bc73e37 osbuild-worker-executor: appease gosec
Note that gosec IMHO is a bit silly here, the heuristics used are
note very good, i.e. the code is already validating the external
inputs and it's not clear to me that "filepath.Clean()" will help
but it seems to supress the error. I hope gosec provides value
in other places, here it seems to be adding work :/

I also excluded "gosec" from any _test.go files, I do not see
why we should gosec tests?
2024-06-05 18:26:08 +02:00
Sanne Raymaekers
b0543e89f4 osbuild-worker-executor: fix lint warnings/errors
The osbuild-composer linting found a bunch of issues that this
commit fixes.
2024-06-05 18:26:08 +02:00
Sanne Raymaekers
01cfb93956 osbuild-worker-executor: integrate into build system/spec files
Add the osbuild-worker-executor to the build system and add it
to the spec file.
2024-06-05 18:26:08 +02:00
Michael Vogt
0a0e8e27ae vendor: add vendor/github.com/sirupsen/logrus/hooks/test
This is needed by the new `osbuild-worker-executor`.
2024-06-05 18:26:08 +02:00
Michael Vogt
cbb8d79baf c/osbuild-worker-executor: update to match new name
Update the imports/names to match the new name
"osbuild-worker-executor".
2024-06-05 18:26:08 +02:00
Michael Vogt
372d9f07dd cmd/osbuild-worker-executor: import verbatim from mvo5/oaas
This commit imports the repository https://github.com/mvo5/oaas
without keeping the history. The history is largely unimportant
and can be looked up in https://github.com/mvo5/oaas/commits/main
if needed.
2024-06-05 18:26:08 +02:00
Tomáš Hozza
fa416e4545 Test: re-enable snapshot URL check (COMPOSER-2263)
Enumerating snapshots now works after the cleanup of EOL releases.

Fixes https://issues.redhat.com/browse/COMPOSER-2263

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 17:17:19 +02:00
Tomáš Hozza
f3e83a3838 Tools/provision.sh: adjust conditional EPEL installation for el10
EPEL-10 can't be installed on any of c10s and el10. Previously, the
script would try to install EPEL repositories package on el10 and fail.

In addition, use our "workaround" repository with c10s builds of some
packages (such as koji) also on el10.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 17:17:19 +02:00
Mario Cattamo
ba60b22104 test: update ostree osname in pending test cases 2024-06-04 17:01:53 +02:00
Tomáš Hozza
4267c28ffb Tests: temporarily disable checking of valid snapshot URLs
rpmrepo snapshot enumeration is timing out, which is effectively
blocking GitLab CI from running. Disable the check for now to unblock
CI.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
0174cf5ee3 SPEC/repos: drop EOL c8s
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
88c6e63a8a Schutzfile: drop EOL c8s
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
67f383f3eb .gitlab-ci.yml: stop testing on c8s
c8s is EOL and repos don't work any more.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
6e3a41ae07 Don't run TestMultilibBlueprintDepsolveV0 on el10 / c10s
TestMultilibBlueprintDepsolveV0 unit test relies on 'gsl' package and
the availability of its 32 bit and 64 bit versions in the distribution
repositories. However, the package is no longer available in c10s /
el10. Moreover, there are no 32 bit RPMs in the repositories.
Conditionally compile the test only if 'rhel10' build tag is not
specified. Modify the SPEC file to define 'rhelX' build tag when
compiling tests on RHEL, where X is the major version.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
f9079ae39c Tools/provision.sh: add workarounds for c10s
There is no EPEL for RHEL-10 at this moment. Use our own COPR repository
with c10s builds of some packages required by our tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
607b65c67f Templates: update RHEL distro aliases
The latest GA releases are 8.10 and 9.4.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
b01b080565 Update 'rhel-9' distro alias to 'rhel-9.5'
New releases are landing in 9.5 already, so the on-prem version should
reflect that. The service can and does override this using a
configuration.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
695febf39c Define distro alias for 'rhel-10'
Set it to RHEL-10.0.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
f87f555362 Packit: remove EOL c8s
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
6cb4cd9012 Packit: build c9s in COPR on all arches
Build c9s RPMs in COPR for all available architectures.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
5b95e956a6 Packit: add c10s COPR builds on all arches
Build RPMs for c10s using Packit in COPR for all available
architectures.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
80f9410e7f Add c10s and el10.0 repo definitions.
Add the GA repos (which don't obviously work yet for el10.0) and RPMrepo
snapshots for testing.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
c8e8bffd99 SPEC: take c10s and el10 into consideration for repo installation
Previously, the SPEC file was written in the way that c9s and el9 were
the latest major releases of the respective distributions and as such,
they could build images for all previous major versions of the same
distro. With c10s and el10, this is no longer true. Amend the section
which installs repository definitions.

The idea is that:
 - c10s / el10 are the latest major version and can build any previous
   major version of the same distro.
 - Any cs and el distro version can build images for the same major
   version.
 - c9s can build also c8s / el9 can build also el8 images.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Tomáš Hozza
cdf0a5b8cc SPEC/-tests: don't require podman-plugins on F40+ and c10s
The sub-package has been deprecated in podman v5.0.0, which is shipped
in these distributions.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-06-04 13:03:37 +02:00
Mario Cattamo
eb3884f71a test: skip ostree-remount due to issue RHEL-25249 2024-05-31 21:10:51 +02:00
Sanne Raymaekers
4629a31f22 templates/packer: use python3.10 on fedora
Ansible on fedora 40 seems broken, the default python 3.12 interpreter
doesn't work, 3.10 works but then the dnf module breaks.

Use 3.10 and stop using the dnf module.
2024-05-31 13:55:58 +02:00
Sanne Raymaekers
22e15da73c templates/packer: use import_tasks instead of include_tasks
The tags don't get inherited through the dynamic `include_tasks`
command. Use `import_tasks` to preserve the tags.
2024-05-31 13:55:58 +02:00
Tomáš Hozza
8f36b6f26e Explicitly pass rpmlint config when calling it
New version of rpmlint stopped picking the config automatically.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-05-30 19:58:34 +02:00
Tomáš Hozza
71b9c0ad65 Fix pylint issues
Add timeout to `requests.post()` and `requests.get()` calls to make
pylint happy.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-05-30 19:58:34 +02:00
Tomáš Hozza
666baff944 Update F37 to F40
Fedora 37 has been EOL for some time.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-05-30 19:58:34 +02:00
Sanne Raymaekers
a96f1b6d31 templates/packer: switch to fedora-40
Fedora 38 is EOL, and packit no longer builds rpms for it.

The current python3.12 + ansible 2.12 combination which is the default
on fedora 40 doesn't work, so switch to python3.9.
2024-05-29 19:36:31 +02:00
Sanne Raymaekers
5d1094a94e test/cases/ubi-wsl: test rhel9 ga 2024-05-29 16:05:01 +02:00
schutzbot
0ad74e0f34 Post release version bump
[skip ci]
2024-05-29 08:14:09 +00:00
Mario Cattamo
cab72ff914 test: execute dmesg task as privileged user 2024-05-29 09:05:44 +02:00
Achilleas Koutsou
580903fa27 gitlab: run the RHEL 8 alias test on RHEL 8.8 (GA) 2024-05-29 00:27:31 +02:00
Achilleas Koutsou
d6e78fa3b1 test/dot-notation-aliases: typo fixes and cleanups
- Change the alias example at the top of the file to use RHEL 9 -> 9.4
  instead of 8 -> 8.10 which is no longer true for the test.
- Fix some comment typos.
- Clean trailing whitespaces.
2024-05-29 00:27:31 +02:00
Achilleas Koutsou
a6a5daa209 test/dot-notation-aliases: override rhel-8 alias
The test relies on the existence of the /etc/dnf/vars/releasever file
and the value it specifies.  In SAP images, this file specifies the
locked release version for the system and it is written by the image
definition itself.  This makes it a good candidate for ensuring we are
using the correct distro version when building the image.

However, RHEL 8.10 SAP images don't lock the distro version because 8.10
is the last minor version in the RHEL 8 cycle.

Let's override the alias for RHEL 8 to point to 8.8, which is a version
that should contain a releasever.  The exact value we use isn't
important.  What we are testing here is the mechanism for resolving
aliases.
2024-05-29 00:27:31 +02:00
Achilleas Koutsou
4241f5bb39 tools/gen-ssh: upgrade openssh before generating key
openssl gets installed as a dependency of the osbuild-composer-tests but
it might not update openssh at the same time, which can cause a version
mismatch when running ssh-keygen:

  OpenSSL version mismatch. Built against 30000000, you have 30200010

This is currently happening in CentOS Stream 9 so let's make sure
openssh is up to date before running ssh-keygen.
2024-05-29 00:27:31 +02:00
Achilleas Koutsou
f09aab365a Schutzfile: bump pinned osbuild version to v119 2024-05-29 00:27:31 +02:00
Achilleas Koutsou
f5c6cdd9cf blueprint: add new kickstart.contents option 2024-05-29 00:27:31 +02:00
Achilleas Koutsou
a3a539abd1 Update osbuild/images v0.63.0
Pulling in (among others):
- https://github.com/osbuild/images/pull/700
    - Blueprint User customization change required.
- https://github.com/osbuild/images/pull/705
2024-05-29 00:27:31 +02:00
dependabot[bot]
c1d56a50c2 build(deps): bump the go-deps group across 1 directory with 11 updates
---
updated-dependencies:
- dependency-name: cloud.google.com/go/compute
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/hashicorp/go-retryablehttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/vmware/govmomi
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-29 00:27:31 +02:00
schutzbot
295dddca37 schutzfile: Update snapshots to 20240514 2024-05-29 00:27:31 +02:00