Commit graph

3287 commits

Author SHA1 Message Date
Achilleas Koutsou
82eedf5b82 DepsolveJob: rename struct field for consistency
We have two fields, `Repos` and `PackageSets`.  Renaming
`PackageSetsRepositories` to `PackageSetsRepos` for consistency.
The struct is for internal use only so the rename has no impact as long
as the serialised name is the same (json tag).

Also it's shorter.

Added docstring to the struct that explains the arguments in the same
way as they are described for the `depsolve()` function.

Changing the name of the argument in the internal `depsolve()` function
for the same reasons.
2022-02-14 17:38:41 +01:00
Achilleas Koutsou
70f83775b2 osbuild-worker: small changes to internal function
Change order of arguments for depsolve function:
Put the two similar arguments (repos and packageSetsRepositories) next
to each other since they serve similar purposes.

Add docstring for depsolve function:
It is useful to clarify how the arguments are used even if it's an
unexported function.
2022-02-14 17:38:41 +01:00
Achilleas Koutsou
0e5bb66516 cloudapi: small comment for readability
Comment the code that reads the user-defined repositories to explaine
what's happening.
2022-02-14 17:38:41 +01:00
Tomas Hozza
ece6452e2a Use the latest RPMRepo snapshot for RHEL-9.0 image tests
This is needed to satisfy RPM dependencies of GCP guest tools, which
will be installed on the `gce` image type.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-14 12:39:12 +01:00
Jakub Rusz
f4b5c0956d tests: update IDs in Openstack image boot test
Update flavor and network ID for booting Openstack images in rhos-01
cloud.
2022-02-14 12:13:18 +02:00
Christian Kellner
b18b4e80a0 distro/{rhel86,rhel90}: specify a remote for deployments
When deploying an ostree commit, specify a remote, currently hard-
coded to `rhel-edge`, so that updates work automatically, if they
are served from the same location as the initial commit is pulled
from.

NB: now that the remote is specified in the raw image, remove the
corresponding bits form the tests.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-02-11 12:30:44 +01:00
Christian Kellner
4b84a3c70f osbuild2: add 'remote' option for ostree.pull
Add support for the `remote` option of the `ostree.pull` stage.
This can be used to tie a commit to a remote while pulling the
commit.
2022-02-11 12:30:44 +01:00
Christian Kellner
bebf046f84 osbuild2: add 'remote' option for ostree.deploy
Add support for the `remote` option of the `ostree.deploy` stage.
This can be used to deploy a commit that is tied to a remote.
2022-02-11 12:30:44 +01:00
lavocatt
984e58ede4 Post release version bump
[skip ci]
2022-02-11 09:12:23 +00:00
Ondřej Budai
5e639cba6f github: adapt the github trigger to work the same way as in osbuild
I basically just copied
7c9944ee18/.github/workflows/trigger-gitlab.yml

and:
- changed osbuild to osbuild-composer
- add a SKIP_CI mechanism

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-10 16:16:30 +01:00
Diaa Sami
c1ae5b0881 Relax TCP timeouts for koji connections
See COMPOSER-1354 and linked tickets
2022-02-10 14:58:10 +01:00
Ondřej Budai
631bd21ffe cloudapi/v2: add support for aws-*-rhui image types
For the brew use-case, we also need to build AWS images containing RHUI. This
commit is thus adding them.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-10 11:26:50 +00:00
Roy Golan
439699fcec Add oracle-oci.md under image-types/rhel8 2022-02-10 10:58:46 +01:00
Sanne Raymaekers
b3796a7319 tools/appsre-ansible: Explicitly specific mock config 2022-02-09 18:33:14 +01:00
Sanne Raymaekers
a173a3513d tools/appsre-build-worker-packer: Run on subscribed 8.5 machine 2022-02-09 16:54:22 +01:00
Juan Abia
3699c52b59 test: update govc env variables documentation
govc cli no longer uses GOVMOMI_* enviroment variables. As the govc
documentation indicates, this variables should be GOVC_*.

https://github.com/vmware/govmomi/blob/master/govc/README.md#usage

[skip-ci]
2022-02-09 16:00:37 +01:00
Tomas Hozza
f43d9f4744 Add missing F34 and F35 image test cases
Add F34 and F35 image test cases for all remaining image types, which
were previously not tested. With this PR, image test cases are now
generated for all image types on all architectures as supported by the
Fedora distro definition.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-09 15:29:33 +01:00
Gianluca Zuccarelli
e8d7519c7d templates/dashboard: worker metric queries
The prometheus queries have been updated with
the correct namepsace for the job metrics
Additionally, this commit fixes some of the
queries to add fallback values when the
query results are returned empty.
2022-02-09 14:09:50 +01:00
Ondřej Budai
b1a40c1040 distro: add an alias for RHEL 8.7
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-08 17:37:36 +01:00
Gianluca Zuccarelli
80f24dbd61 metrics: change job metrics namespace
Currently the job metrics are namespaced with the composer
subsystem, i.e. `composer_worker`. Since we plan to split
the components to their own namespaces in app interface,
the worker subsystem should be split too.
2022-02-08 15:57:12 +01:00
Sanne Raymaekers
a739151c71 Revert "templates: Add dnf-json template"
This reverts commit 8cb3900dd6.
2022-02-08 14:05:48 +01:00
Jakub Rusz
a4c19eee85 Schutzfile: add RPMREPO to repo titles
This will make it easier to distinguish nightly rpmrepo snapshots from
actual nightly repositories in the test logs.
2022-02-08 14:28:06 +02:00
Jakub Rusz
29a8810d6e tests/ci: fix sonarqube analysis for main branch
Only use the branch options if it's actually running in a branch and
analyse main without them, sonarqube otherwise doesn't recognize the
main branch.
2022-02-08 14:28:06 +02:00
Tomas Hozza
1401a7a659 test: add image test cases for Fedora 34 and 35
Delete Fedora 33 image test cases.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-08 10:19:52 +01:00
Tomas Hozza
d9d5a6c1a6 Schutzfile: don't pin osbuild on Fedora and drop F33
Drop unnecessary pinning of osbuild release on Fedora. The latest version
available in all supported Fedora releases should be good enough for
osbuild-composer testing.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-08 10:19:52 +01:00
Tomas Hozza
617572ad6c test/api.sh: remove parts related to Fedora 33
We no longer test Cloud API on Fedora and Fedora 33 is EOL anyway.
Remove all Fedora 33 related lines from the test case.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-08 10:19:52 +01:00
Tomas Hozza
2c04329645 distro: drop support for Fedora 33
Fedora 33 is already EOL, therefore there is no point in supporting
image builds for it. Drop F33 from the distroregistry list and remove
F33 repositories definition.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-08 10:19:52 +01:00
Ondřej Budai
f66740fdbd github: fix gitlab trigger
With the quoting, we are passing an empty variable instead of no argument.
Also, if the skip is required, we are passing "-o ci.skip" instead of
"-o" "ci.skip".

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-08 10:03:59 +01:00
Roy Golan
e8dd3654d5 Allow skipping CI for draft PR or WIP label
Save resources and needless CI runs by skipping the trigger-gitlab job if a PR is 'draft' (can be set in the pull request page at any time) or by setting a label "WIP".

Asside from saving resources, WIP PRs that don't trigger CI are a good message to a contributers to keep testing it locally and to only get the project resources after serious testing been made on their side.
2022-02-08 08:24:23 +01:00
Sanne Raymaekers
4956e48a0b service-maintenance: Skip db cleanup
Let's enable the cloud cleanup first, and then move on to the db.
2022-02-07 20:42:45 +01:00
Gianluca Zuccarelli
dbf396db2b templates/dashboards: worker error metrics
Update the grafana dashboard for the workers
to show information on the success rate for
osbuild and depsolve jobs.
2022-02-07 20:40:37 +01:00
Tomas Hozza
91097fa007 test: regenerate RHEL 8.5 and 8.6 EC2 image test cases
Fix missing missing `image-info` report updates after PR#2235.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-07 20:33:54 +01:00
dependabot[bot]
c7cba177d9 build(deps): bump cloud.google.com/go/cloudbuild from 0.2.0 to 1.0.0
Bumps [cloud.google.com/go/cloudbuild](https://github.com/googleapis/google-cloud-go) from 0.2.0 to 1.0.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/v0.2.0...dlp/v1.0.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/cloudbuild
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-07 19:08:06 +01:00
Juan Abia
cf72a8963e test: update README with cloud cleaner info
update testing documentation with the purpose of 'cloud cleaner' and
'scheduled cloud cleaner'
2022-02-07 10:51:47 +01:00
Jakub Rusz
0d64331ffe tests/ci: run all integration tests on fedora and centos 2022-02-07 11:37:13 +02:00
Jakub Rusz
409eb3f33b tests/cross-distro: filter out beta and stream repos
Currently these repositories are not recognized by composer which is a
bug described in issue #2257. This commit will be reverted when related
issue is fixed.
2022-02-07 11:37:13 +02:00
Jakub Rusz
17f84ad6cc tests: expand cross-distro tests
This test now checks if all the installed repositories are recognized,
there are no extra or missing and that blueprint push with unsupported
distro fails gracefuly.
2022-02-07 11:37:13 +02:00
Tomas Hozza
eb36b86161 RHEL-90: Remove deprecated crashkernel=auto option
Setting of the `crashkernel` option to the appropriate value is now done
by the `kexec-tools` package when installed and when any new kernel is
installed.

Regenerate relevant image test cases.

Fix #1819
Fix rhbz#2006692

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-07 09:29:07 +01:00
Sanne Raymaekers
8cb3900dd6 templates: Add dnf-json template 2022-02-06 14:48:32 +00:00
Gianluca Zuccarelli
a8f20811f6 worker: add koji build worker validation
Validate the results for a koji build job
and check for empty osbuild output. This commit
ensures that there aren't any gaps in the error
handling. Additionally this commit adds a few
extra error cases rather than a generic error.
2022-02-05 00:13:59 +00:00
Gianluca Zuccarelli
3f8bd5da67 worker: check for empty osbuild job output
Add an additional check to ensure empty
osbuild output is marked as an error.
Add an error case for failed parsing of
dynamic args.
2022-02-05 00:13:59 +00:00
Alexander Todorov
0fb96345f2 tests: Disable SonarQube for nightly pipelines 2022-02-04 23:43:43 +01:00
Alexander Todorov
966692be11 tests: Conditionally enable osbuild-dnf-json-tests
based on package version
2022-02-04 23:43:43 +01:00
Alexander Todorov
8b3a1367be tests: Add nvr checking into a shared library
can be used for conditionally enabling parts of the test scripts:

source /usr/libexec/tests/osbuild-composer/shared_lib.sh

if nvrGreaterOrEqual "osbuild-composer" "41"; then
    .... do some more testing ...
fi
2022-02-04 23:43:43 +01:00
Alexander Todorov
c5d62909e8 tests: Don't run cloud API tests for nightly trees
https://coreos.slack.com/archives/C0235DZB0DT/p1643969527395979?thread_ts=1643968867.786629&cid=C0235DZB0DT

Note: koji.sh already checks the $NIGHTLY variable and just exits
2022-02-04 23:43:43 +01:00
Alexander Todorov
b31f61649a ci: Overwrite REPO_PATH based on the value of $NIGHTLY
Override only for osbuild-composer, not other projects

will produce different repositories/rpms for the same commit when
executed from a PR and from the scheduled pipelines. We need the
distinction b/c osbuild-composer-tests RPMs end up slightly different in
these two cases. For more info see:
https://github.com/osbuild/osbuild-composer/pull/2093#pullrequestreview-854328838
https://github.com/osbuild/osbuild-composer/pull/2093#issuecomment-1014469915
2022-02-04 23:43:43 +01:00
Alexander Todorov
43738d6caa tests: Fix for unbound variable 2022-02-04 23:43:43 +01:00
Alexander Todorov
8a46796cf7 ci: Don't account for SCHUTZ_OSBUILD variable
if you want to use a custom osbuild.rpm you can supply that in REPO_URL,
no?

Note: this removal of .repo files breaks building the -tests RPM from
source in downstream branches/nightlies b/c the repo is defined in
osbuild-composer.repo
2022-02-04 23:43:43 +01:00
Alexander Todorov
e07bd4bebf Don't override osbuild-composer-tests-multi-arch repo if not needed
drop the discovery mechanism from Brew b/c we're building the tests RPM
from main but overwrite the repository if REPO_URL is configured in ENV.
2022-02-04 23:43:43 +01:00
Alexander Todorov
63bca237cf ci: Check that installed osbuild-composer RPM matches SUT 2022-02-04 23:43:43 +01:00