Commit graph

5590 commits

Author SHA1 Message Date
Ondřej Budai
ce5e41f980 tools/run-koji-container: allow passing a custom certificate dir
This helps with running the script locally.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-18 09:25:22 +02:00
Ondřej Budai
128f56169e Dockerfile*: chown the copied files
When `go install` is called, go tries to get the git commit hash and embed it
into the built binary. Internally, go just calls the git executable.

The newer go-toolset seems to be based on RHEL 9.2 that ships a newer version
of git (2.39.1). This version contains the safe directory patch that
disallows git from operating on repositories owned by different users.

Thus, we need to chown the files when copying.

See

https://git-scm.com/docs/git-config/2.35.2#Documentation/git-config.txt-safedirectory
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-17 13:08:29 +02:00
Ondřej Budai
7ce51e0ab5 distribution: update to ubi9
This commit also adds `-y` to `microdnf install`. Microdnf shipped in RHEL 9
apparently requires it.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-17 13:08:29 +02:00
schutzbot
ab09feee59 Post release version bump
[skip ci]
2023-05-17 08:20:38 +00:00
Achilleas Koutsou
584ae47eca rhel8/azure: remove and disable firewalld from EAP
Remove firewalld from the base package set for Azure and add it to all
the image-specific package sets except EAP (and explicitly exclude it).
Remove firewalld from the base image config for Azure and add it to all
the image-specific configs.

Test manifests updated.
Manifest changes for non-EAP image types is only the enabled-services
reordering: firewalld is last because it is appended to the base config.

See COMPOSER-1859
2023-05-15 21:26:34 +02:00
Achilleas Koutsou
9132dd469e rhel8/azure: sort the common package set
Sorted lists of strings make it easier to add and remove elements
without needing to think about the order, making diffs easier.

The sorting was done using the 'sort' coreutils command with LC_ALL=C.
2023-05-15 21:26:34 +02:00
Achilleas Koutsou
5337b746e4 test: update RHEL 8 EAP manifests
Azure RHUI snapshot was updated but manifests for Azure EAP weren't
updated, probably due to a merge race.
2023-05-15 21:26:34 +02:00
jabia99
7224c896b6 Pass JIRA_PAT env variable to container 2023-05-15 12:18:40 +03:00
Ondřej Budai
156c411836 ostree-simplified-installer.sh: fix waiting loop for fdouser
`ssh grep fdouser /etc/passwd` returns 1 if the user doesn't exist yet.
We run this script with set -euo pipefail, which cause the script to exit
immediately. Thus, the waiting loop isn't actually waiting for anything.

By adding `|| true`, we ensure that the loop is indeed waiting for the fdouser
to be available.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-12 11:02:49 +02:00
Ondřej Budai
25161a8316 libvirt_test: stop setting nvram in virt-install on RHEL 9
We use CDN repositories when running tests on GA RHEL. Thus, our RHEL 9.1
machines are actually pulling packages from RHEL 9.2. Therefore, we are
now getting virt-install >= 4. This version brings firmware auto-detection
and doesn't support nvram_template anymore. Let's remove it then.

See 966049ec3c

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-12 11:02:49 +02:00
Sanne Raymaekers
c7d2eaa5f7 test: use non-deprecated gitlab CI variables
CI_BUILD_ID was replaced with CI_JOB_ID in gitlab 9+.
2023-05-12 11:02:49 +02:00
Sanne Raymaekers
0335db645b internal/manifest: restorecon after registering
When the first boot service runs insights-client, /root/.gnupg gets
created with the wrong selinux context.
2023-05-12 11:02:49 +02:00
Sanne Raymaekers
9f40e960c7 internal/manifest: install rhc-worker-playbook when using rhc
This isn't pulled in as a dependency, and is required for services like
remediations.
2023-05-12 11:02:49 +02:00
dependabot[bot]
a48c9d7b00 build(deps): bump github.com/stretchr/testify from 1.8.1 to 1.8.2
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-09 10:00:33 +02:00
dependabot[bot]
4948d2142b build(deps): bump golang.org/x/sys from 0.3.0 to 0.8.0
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.3.0 to 0.8.0.
- [Commits](https://github.com/golang/sys/compare/v0.3.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-05 12:26:29 +02:00
Ondřej Budai
dce2ced50b packer: bump the amazon plugin to 1.2.3
Since the previous commit removed the associate_public_ip_address, we should
not be hitting the new behaviour introduced in 1.2.3, thus everything will
hopefully work as before.
2023-05-05 11:07:05 +02:00
Ondřej Budai
a2a5618149 packer: remove associate_public_ip_address
The documentation for this option says the following:

> If using a non-default VPC, public IP addresses are not provided by default.
> If this is true, your new instance will get a Public IP. default: unset

We don't specify a VPC in the packer build, thus we are using the default
one. Therefore, I don't think we actually need this option as it's useful
only for non-default VPCs.

See
https://developer.hashicorp.com/packer/plugins/builders/amazon/ebs#run-configuration

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-05 11:07:05 +02:00
Gianluca Zuccarelli
c18f2962ae internal/blueprint: repo filename validation
Validate custome repository filenames in order to
avoid unexpected `5xx` errors when building an image.

Before this the filename was only validated at the
yum repo stage, which was causing unexpected errors.
2023-05-03 18:27:06 +02:00
Gianluca Zuccarelli
79284f3447 internal/cloudapi: fix custom repo fields
Minor fixes to cloudapi custom repos:
- add missing `priority` field
- rename `repo_check_gpg` to `check_repo_gpg`
  to match `payload_repositories`
2023-05-03 18:27:06 +02:00
Jakub Rusz
bc28daf6b6 Update terraform SHA
Centos-stream-9 images are gone for some reason, let's use new ones.
2023-05-03 15:25:34 +02:00
schutzbot
19bfd7c9b2 Post release version bump
[skip ci]
2023-05-03 08:20:58 +00:00
Sanne Raymaekers
0a7fd52466 internal/manifest: remove redundant insights-client call
RHC automatically connects to insights already.
2023-04-27 21:40:14 +02:00
Mario Cattamo
54ce49a852 Test fedora minimal embedded container 2023-04-27 14:01:44 +02:00
Alexander Todorov
70949fddf0 Enable more regression tests for nightly pipeline 2023-04-26 19:11:09 +02:00
Gianluca Zuccarelli
1c25d2792c tests/api: custom repository tests
- remove `custom-repos.sh` integratoin test
- add custom repositories check to `api` tests for supported
  images
- verify custom repositores are added to /etc/yum.repos.d
- verify gpg key is saved to /etc/pki/rpm-gpg (for inline keys)
2023-04-26 14:16:16 +01:00
Gianluca Zuccarelli
fb63d7ad9a internal/cloudapi: expose custom repos
Expose the custom repository customizations for
the cloudapi.
2023-04-26 14:16:16 +01:00
Gianluca Zuccarelli
e313a76103 dnf-json: replace dnf hash function
Replace the dnf-json `Hash()` function in
favour of a hash calculated using the
`rpmmd.RepConfig.Hash()` function. The
`repoHash` field is populated when converting
a `rpmmd.RepoConfig` to `dnfjson.repoConfig`
object. The `dnfson.repoConfig.Hash()` function
then returns the `repoHash` field instead of
re-calculating the hash.
2023-04-26 14:16:16 +01:00
Gianluca Zuccarelli
f59e248bc8 blueprint: add gpgkey validation
Add custom repo gpg keys validation checks to
make sure that a gpg key is either a valid key
or a valid url.
2023-04-26 14:16:16 +01:00
Alexander Todorov
72d33c8651 Don't execute test on osbuild-composer < 81
New functionality introduced in
https://github.com/osbuild/osbuild-composer/pull/3228
2023-04-25 18:33:37 +02:00
Gianluca Zuccarelli
6326d072e9 schutzbot: add gzuccare ssh-key 2023-04-25 14:05:09 +02:00
Tomáš Hozza
808118431f mockbuild.sh: use dnf to install local package, not rpm
DNF has more elaborate locking system and can wait for other instances of
itself when installing packages. Using rpm directly to install local
package is causing failures in CI due to it not being able to acquire
lock on `/var/lib/rpm/.rpm.lock`.

Using DNF should improve the situation, although there is no good
documentation to link and support this claim for sure.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-04-24 20:40:58 +02:00
Antonio Murdaca
01e7370c08 test/simplified-installer: fix yq installation on rhel8
Basically use the latest 3.2.1 version because of https://github.com/kislyuk/yq/issues/165

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2023-04-24 14:14:15 +02:00
Tomáš Hozza
7785330fb5 Good bye Lars and David
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-04-24 09:31:03 +02:00
Gianluca Zuccarelli
e271d1d30d test/cases: add custom repos integration test
Add an integration test for the custom repos feature.
The file resolve feature is also tested as part of this
test.
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
376199467c internal/distro: rhel9 repo customizations
Wire up the repository customizations for
rhel9 images.
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
febe018733 internal/distro: rhel8 repo customizations
Wire up the repository customizations for
rhel8 images.
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
71fd5dafdc internal/distro: rhel7 repo customizations
Wire up the repository customizations for
rhel7 images.
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
08c10b51a0 internal/distro: fedora repo customizations
Wire up the repository customizations for
fedora images.
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
1284642ab2 tools/test-case-generator: add custom repos 2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
f29e3e9b0e osbuild: yum repo stage from RepoConfig
Create an osbuild yum repository from
`rpmmd.RepoConfig`. Additionally, remove
pointers from the `YumRepository` struct,
since this will add values for fields that
weren't explicitly set by the user in the
repo customizations.
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
29643c2e06 customizations: custom repo utility functions
Create some utility functions that will be used for implementing
custom repo configuration files. This commit adds these functions:
- a helper to get the filename of a custom repo, or the
  `<repo-id>.repo` if the filename is empty
- a function to convert the custom repos to a map of `RepoConfig`.
  This function also creates an `fsnode.File` for each inline gpg
  key set in the customizations and swaps the inline key for the
  file path. The function returns the map of `RepoConfig` and a list
  of `fsnode.File` containing the inline gpg keys.
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
3b6fddb14a internal/rpmmd: add missing fields to RepoConfig
Further align the RepoConfig object to the dnf
spec and add missing fields.
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
d44703cdc8 rpmmd/repository: repoconfig pointers
Convert some of the fields in the `RepoConfig` struct
to pointers. Since `RepoConfig` will be used to convert
custom repositories to an array of `osbuild.YumRepository`,
we need to ensure that fields that are not set explicitly
are not saved to the `/etc/yum.repos.d` repository files.
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
75e2138878 customizations: add custom repositories
Define blueprint custom repositories. These
definitions have been based on the upstream
definitions of a DNF repo[1].

[1] See dnf.conf
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
4d42808b6a internal/rpmmd: RepoConfig baseurl change
Update the internal RepoConfig object to
accept a slice of baseurls rather than a
single field. This change was needed to
align RepoConfig with the dnf spec [1].

Additionally, this change adds custom json
marshal and unmarshal functions to ensure
backwards compatibility with older workers.
Add json tags to the internal rpmmd config
since this is serialized in dnfjson.
Add unit tests to check the serialization
is okay.

[1] See dnf.config
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
17d730593c internal/cloudapi: fix test
Minor fix to a test to correct the
expected vs actual fields.
2023-04-21 17:40:00 +02:00
Alexander Todorov
2c23894e2a Conditionally skip container test when using older osbuild version
Introduced in
https://github.com/osbuild/osbuild-composer/pull/3336
2023-04-20 13:22:29 +02:00
Alexander Todorov
3686223a40 Execute ContainerEmbedding CI job for RHEL and include in nightly pipeline
Achilleas says it should be triggered for RHEL as well
2023-04-20 13:22:29 +02:00
Ondřej Budai
edf4f7e879 packer: pin the version of the amazon plugin to 1.2.2
Version 1.2.3 made changes to how the plugin handles auto-selection of a
subnet when it's not specified, see

f1ec287c77

Sadly, the new algorithm selects us-east-1e for us that doesn't support
the machine types we use (c6*.large) which causes the build to fail.
I reported it here:
https://github.com/hashicorp/packer-plugin-amazon/issues/368

One workaround might be to pin a working subnet, but that's apparently also
broken in 1.2.3, see
https://github.com/hashicorp/packer-plugin-amazon/issues/367

Therefore, I decided to pin the plugin to 1.2.2 for now, and see what's
the recommended approach from terraform guys.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-04-20 13:02:34 +02:00
dependabot[bot]
05eb1a95b6 build(deps): bump github.com/golang-jwt/jwt/v4 from 4.4.2 to 4.5.0
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.4.2 to 4.5.0.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v4.4.2...v4.5.0)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-20 11:20:05 +02:00