Commit graph

2725 commits

Author SHA1 Message Date
Achilleas Koutsou
66b76c2ace gitleaks: add allow list for test passwords and keys
Add a .gitleaks.toml file that lists all the test passwords and paths to
keys that should be ignored by the security scanner. This avoids false
positives so we can enable the scanner to catch actual leaks.

Closes #1747

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-01 16:56:26 +02:00
Ondřej Budai
af784a11b1 mockbuild: use download.devel of download.eng.bos
download.eng.bos.redhat.com is down, use the mirroring service instead.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-01 15:23:53 +02:00
Ondřej Budai
2070747876 ci: install ansible-core instead of ansible for EL9
ansible-core no longer provides ansible so we need to require specifically
ansible-core.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-01 15:23:53 +02:00
Ondřej Budai
4f8dc76ca7 dnf-json: disable zchunk
See the comment

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-01 15:23:53 +02:00
Diaa Sami
cc54c4deee composer: improve logging 2021-09-30 18:34:05 +02:00
Diaa Sami
27ca55372a cloudapi: Add extra logging & improve existing
Use different logging levels depending on situation
Log HTTP 5xx errors to stdout with internal error details when possible
2021-09-30 18:34:05 +02:00
Diaa Sami
179009fec4 cloudapi: use Logrus as default logger in Echo
And remove log.Logger references
2021-09-30 18:34:05 +02:00
Diaa Sami
75372b98e1 logging: add logrus dependency
Just add the dependency and update relevant files
2021-09-30 18:34:05 +02:00
Jakub Rusz
c112e9d12d ci: don't run nightly pipeline on ga releases
It makes no sesnse to run nightly testing on ga composes. This updates
the regex to exclude ga runners from the scheduled nightly run.
2021-09-30 16:03:14 +02:00
Ondřej Budai
5ed954bcf5 distro/rhel90: use qcow2 compat 1.1 for qcows
There's no reason to use 0.10 since we don't need to support running
this image on RHEL 6 anymore.
See https://bugzilla.redhat.com/show_bug.cgi?id=2008910

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-30 15:36:15 +02:00
sanne
106e9b3fc2 tools: Push ubi container to quay.io/app-sre/composer 2021-09-29 22:37:25 +02:00
Gianluca Zuccarelli
b8c14a5689 blueprints: change minsize from int to uint64
thozza pointed out that `int` is platform dependent which results in
a fs size that is too small for 32-bit machines. This commit changes
the filesystem custimizations to use `uint64` instead of `int`
2021-09-29 15:08:12 +02:00
sanne
5239e264bb test/regression: Adapt to new rhel-84-ga runners 2021-09-29 15:07:35 +02:00
sanne
042eb75de6 schutzbot: Also build container on branch pushes
The regression test failed to find the container when not running from a
PR.
2021-09-29 15:07:35 +02:00
Ondřej Budai
9403c6e795 schutzbot: pin osbuild to >=37 on all distros
The bootloader option in ostree.config stage is new in osbuild 37. However,
osbuild 37 contains a bug that prevents it from working in certain cases,
thus we need to pin to a slightly newer commit that contains a fix for it:

bug: https://github.com/osbuild/osbuild/issues/822
fix: https://github.com/osbuild/osbuild/pull/824

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-27 20:10:04 +02:00
Ondřej Budai
1b59b2973e spec: bump osbuild depedendency to 37
The bootloader option in ostree.config stage is new in osbuild 37.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-27 20:10:04 +02:00
Christian Kellner
7a4488e5c4 distro/rhel85: set bootloader to none for edge
Use the new bootloader config for OSTree deployments to set the
bootloader backend to `none` since we use grub2 and BLS in all
currentlysupported architectures. In fact, not setting it to
`none`, will use the default `auto`, which will result in the
run of `grub2-mkconfig`, which is not what we want and might
even fail for us.
2021-09-27 20:10:04 +02:00
Christian Kellner
f6d4228b00 osbuild2: add bootloader option to ostree.config
Expose the new (osbuild > 36) `sysroot.bootloader` option to
configure the bootloader backend for OSTree deployments.
Normally, this should be set to `none` for modern bootloader
that use the BLS, since the BLS snippets are always generated.
2021-09-27 20:10:04 +02:00
Christian Kellner
920863df70 osbuild2: small doc fix for ostree.config 2021-09-27 20:10:04 +02:00
dependabot[bot]
0091d66b95 build(deps): bump github.com/labstack/echo/v4 from 4.5.0 to 4.6.0
Bumps [github.com/labstack/echo/v4](https://github.com/labstack/echo) from 4.5.0 to 4.6.0.
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.5.0...v4.6.0)

---
updated-dependencies:
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-27 14:35:27 +02:00
dependabot[bot]
352128f831 build(deps): bump github.com/aws/aws-sdk-go from 1.40.46 to 1.40.49
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.46 to 1.40.49.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.40.46...v1.40.49)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-27 14:34:48 +02:00
sanne
2f328b0e97 workers: Backwards compatible api.openshift.com spec compliance
The main changes are:
- Kind, Href, Id fields for every object returned
- Attach operationIds to each request, return it for errors
- Errors are predefined and queryable
2021-09-27 13:10:05 +01:00
Alexander Todorov
5e206322a2 Disable regression-composer-works-behind-satellite.sh, Refs #1834 2021-09-24 17:03:53 +02:00
Ondřej Budai
3c9c9f81e7 test/ostree: use 8.4 when testing on 8.4
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
3991827f68 ci: add workaround for missing redhat.repo in EC2
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
e64cc2effe schutzfile: remove rhel-8.3
We don't run on RHEL 8.3 anymore.

Note that the koji-osbuild rev-deps test is not run anymore, therefore
it's fine to drop dependants, see:
https://github.com/osbuild/osbuild-composer/issues/1559

We should reintroduce it though.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
aff680542d test/koji: always build the latest RHEL
We use the koji API to build RHEL nightlies, therefore we should build
only development versions in the testsuite.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
fbb091c8c8 mockbuild: use REPO_BUCKET when defining the base URL
Just deduplication of code.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
27693b1949 test: use hyphen in DISTRO_CODE instead of underscore
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
4c5d1ae45a test: move env variables into set-env-variables helper
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
f3a58d699a test/koji: remove the workaround for rhel-8
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
45417091cd test/api: don't source os-release again
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
8ec7f39a0f test: use cdn repos for 8.3 and 8.4
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
0737136bc8 mockbuild: reuse nightly repos from redhat.repo
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
c25e58bc9d ci: assume subscribed machines
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
e4d912f348 terraform: update
- run only koji.sh for rhel-90 target, Fixes #1461

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
dependabot[bot]
d1c48b4e84 build(deps): bump github.com/Azure/azure-sdk-for-go
Bumps [github.com/Azure/azure-sdk-for-go](https://github.com/Azure/azure-sdk-for-go) from 57.2.0+incompatible to 57.4.0+incompatible.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/v57.2.0...v57.4.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-24 15:30:03 +02:00
Martin Sehnoutka
e310d4e4f0 distribution: Use After=network.target instead of multi-user
There is no need for osbuild-composer to wait until the whole multi-user
target is completed. It can be started earlier as it doesn't have any
dependencies in the target.

This can be a problem if there is a unit in the target that is not starting
and still is unrelated to osbuild-composer. There was a bug like this
with Plymouth where the service didn't finish and it was hanging. That
prevented osbuild-composer from starting and the user was left with
working SSH connection and shell, but composer-cli and systemctl start
osbuild-composer.service were both unresponsive.

Replace After=multi-user.target with After=network.target to start
osbuild-composer earlier.
2021-09-24 14:15:10 +02:00
Martin Sehnoutka
d7d4e02c8c internal/common: introduce git revision and rpm version
The variables are set to the git revision from which the build is
triggered and rpm version from the spec file, if it is build using RPM.

This can be later used to query exact source version while
running osbuild-composer.

It is necessary to use both, because none of them is available in all
possible scenarios.

Use either git-rev (preferably) or RPM version (NEVRA) instead of the
"devel" build type. It was just a placeholder.
2021-09-24 14:13:23 +02:00
Achilleas Koutsou
c8032c7bbc test/ostree: support weldr-client output structure 2021-09-24 13:27:21 +02:00
Achilleas Koutsou
e6cff1c59a test/data: update test manifests for RHEL 9.0 edge
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Achilleas Koutsou
5368ecf7fe distro/rhel90: explicitly enable greenboot services for edge
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Achilleas Koutsou
23109df500 distro/rhel90: add gnome-kiosk to installer package set
Required for graphical Anaconda installation

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Achilleas Koutsou
ab5d560b1c tools/provision: install community.general ansible collection
Required for the `json_query` which we use in our playbooks.
Also requires python3-jmespath.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Achilleas Koutsou
829ae87241 test/ostree: remove debug callback from ansible calls
The callback is no longer supported and the default output format
appears to be adequate.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Achilleas Koutsou
bdc428fbb7 test/ostree: install python3 instead of specific version
Better cross-distro-version compatibility.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Achilleas Koutsou
0c5a0d6763 distro/rhel90: remove unavailable packages from edge-installer
Packages that are no longer not available in RHEL 9.0.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Achilleas Koutsou
0ea491e9bd ci: enable ostree tests on RHEL 9.0-beta
The ostree-ng test was updated to accommodate a change in the
edge-container image type
(see https://github.com/osbuild/osbuild-composer/pull/1595).
RHEL 9.0-beta uses the old edge-container configuration, however RHEL
9.0 GA should eventually adopt the new container.

Copying the old ostree-ng test and running it separately is better than
having several conditions in the single test script to accommodate the
old behaviour and then reverting them when the changes land in RHEL 9.

Modified from the old version to assume we're using 'weldr-client'
(which has a different json output schema) and to use RHEL 9
naming/versioning where necessary.

The ansible checks were also copied to remove the kernel-rt check.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Simon Steinbeiss
1d0f979525 NEWS: Drop title line
Make the NEWS.md file more consistent with osbuild's NEWS.md, which
makes it easier to update it in an automated fashion.
2021-09-24 13:16:22 +02:00
Diaa Sami
60e403e53e cloudapi: use Recover middleware to handle panics
recover from panics such as out-of-bounds array access & nil
pointer access, print a stack trace and return 5xx error
instead of the service crashing and relying on Execution
framework to handle crashes
2021-09-24 12:11:04 +01:00