Commit graph

3300 commits

Author SHA1 Message Date
Ondřej Budai
11893485cd github: split checks into 3 jobs
To speed it up a bit.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-16 10:00:25 +01:00
Sanne Raymaekers
d492e8f702 cmd/osbuild-service-maintenance: GCP deletes by image name 2022-02-15 18:22:39 +01:00
Tomas Hozza
1273ef7b35 Regenerate all image test cases with the enhanced image-info
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-15 15:56:56 +01:00
Tomas Hozza
2182996aae tools/image-info: extend the gathered information
Extend the information gathered by `image-info`. This is needed to
properly inspect GCE images.

Enhancements:
 - read all ssh client configuration files
 - read all sshd configuration files
 - read all YUM / DNF repos
 - read DNF Automatic configuration
 - fix reading of DNF configuration

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-15 15:56:56 +01:00
Brian C. Lane
9f29f8eb25 store: Filter out blank blueprint name
Filter out listing blueprints with no name. This prevents old state.json
data from continuing to cause problems with the UI.

Resolves: rhbz#1922845
2022-02-15 10:10:21 +01:00
Brian C. Lane
a186fd4705 weldr: Check for missing undo blueprint
When the server is restarted the blueprint changes, which are only held
in memory, are lost. This checks for missing changes and returns an
error.

The test is also adjusted for the new error.

Related: rhbz#1922845
2022-02-15 10:10:21 +01:00
Brian C. Lane
aa8cf41ac2 blueprint: Make sure name is not empty
The blueprint name should never be empty, as it can cause other problems
like with the blueprints list results. Return an error if one is pushed
to the store, either as a blueprint commit or as a blueprint workspace.

Also adjusts the new test for the new error.

Related: rhbz#1922845
2022-02-15 10:10:21 +01:00
Brian C. Lane
a3b415d1df test: Add fixture support for blueprint changes
There is a problem with blueprint changes, once the server is restarted
the previous changes are all lost because they are not serialized to
disk.

This adds test fixture support so that new tests can be added before
fixing the problem. It adds store.FixtureOldChanges with blueprints
changes and empty blueprints.

Related: rhbz#1922845
2022-02-15 10:10:21 +01:00
Achilleas Koutsou
346486cd3f cloudapi: test repository config conversion function 2022-02-14 17:38:41 +01:00
Achilleas Koutsou
679028bb8e cloudapi: copy all Repository fields to RepoConfig 2022-02-14 17:38:41 +01:00
Achilleas Koutsou
2d0755b924 cloudapi: test new repository collection function 2022-02-14 17:38:41 +01:00
Achilleas Koutsou
3320f2c57d cloudapi: separate repositories based on package_sets
- Any repository without package_sets is added to the general `Repos`
  field of the DepsolveJob, just like before.
- Repositories with package_sets are added to the `PackageSetsRepos`
  map, indexed by the package set names.
- Repositories defined in the customizations as `PayloadRepositories`
  are considered to be associated only with the `PayloadPackageSets`
  names from the image type and are added to the `PackageSetsRepos`
  under the payload sets.

The repository collection and conversion of repository structs (from
Repository to RepoConfig) has been moved to a separate function.
2022-02-14 17:38:41 +01:00
Achilleas Koutsou
2dde6d7e33 cloudapi: add tags to repositories in image request
Tags is an array of strings that associates repositories with package
sets.  A repository tagged with a package set name will be used only for
the named package sets.
2022-02-14 17:38:41 +01:00
Achilleas Koutsou
82eedf5b82 DepsolveJob: rename struct field for consistency
We have two fields, `Repos` and `PackageSets`.  Renaming
`PackageSetsRepositories` to `PackageSetsRepos` for consistency.
The struct is for internal use only so the rename has no impact as long
as the serialised name is the same (json tag).

Also it's shorter.

Added docstring to the struct that explains the arguments in the same
way as they are described for the `depsolve()` function.

Changing the name of the argument in the internal `depsolve()` function
for the same reasons.
2022-02-14 17:38:41 +01:00
Achilleas Koutsou
70f83775b2 osbuild-worker: small changes to internal function
Change order of arguments for depsolve function:
Put the two similar arguments (repos and packageSetsRepositories) next
to each other since they serve similar purposes.

Add docstring for depsolve function:
It is useful to clarify how the arguments are used even if it's an
unexported function.
2022-02-14 17:38:41 +01:00
Achilleas Koutsou
0e5bb66516 cloudapi: small comment for readability
Comment the code that reads the user-defined repositories to explaine
what's happening.
2022-02-14 17:38:41 +01:00
Tomas Hozza
ece6452e2a Use the latest RPMRepo snapshot for RHEL-9.0 image tests
This is needed to satisfy RPM dependencies of GCP guest tools, which
will be installed on the `gce` image type.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-14 12:39:12 +01:00
Jakub Rusz
f4b5c0956d tests: update IDs in Openstack image boot test
Update flavor and network ID for booting Openstack images in rhos-01
cloud.
2022-02-14 12:13:18 +02:00
Christian Kellner
b18b4e80a0 distro/{rhel86,rhel90}: specify a remote for deployments
When deploying an ostree commit, specify a remote, currently hard-
coded to `rhel-edge`, so that updates work automatically, if they
are served from the same location as the initial commit is pulled
from.

NB: now that the remote is specified in the raw image, remove the
corresponding bits form the tests.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-02-11 12:30:44 +01:00
Christian Kellner
4b84a3c70f osbuild2: add 'remote' option for ostree.pull
Add support for the `remote` option of the `ostree.pull` stage.
This can be used to tie a commit to a remote while pulling the
commit.
2022-02-11 12:30:44 +01:00
Christian Kellner
bebf046f84 osbuild2: add 'remote' option for ostree.deploy
Add support for the `remote` option of the `ostree.deploy` stage.
This can be used to deploy a commit that is tied to a remote.
2022-02-11 12:30:44 +01:00
lavocatt
984e58ede4 Post release version bump
[skip ci]
2022-02-11 09:12:23 +00:00
Ondřej Budai
5e639cba6f github: adapt the github trigger to work the same way as in osbuild
I basically just copied
7c9944ee18/.github/workflows/trigger-gitlab.yml

and:
- changed osbuild to osbuild-composer
- add a SKIP_CI mechanism

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-10 16:16:30 +01:00
Diaa Sami
c1ae5b0881 Relax TCP timeouts for koji connections
See COMPOSER-1354 and linked tickets
2022-02-10 14:58:10 +01:00
Ondřej Budai
631bd21ffe cloudapi/v2: add support for aws-*-rhui image types
For the brew use-case, we also need to build AWS images containing RHUI. This
commit is thus adding them.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-10 11:26:50 +00:00
Roy Golan
439699fcec Add oracle-oci.md under image-types/rhel8 2022-02-10 10:58:46 +01:00
Sanne Raymaekers
b3796a7319 tools/appsre-ansible: Explicitly specific mock config 2022-02-09 18:33:14 +01:00
Sanne Raymaekers
a173a3513d tools/appsre-build-worker-packer: Run on subscribed 8.5 machine 2022-02-09 16:54:22 +01:00
Juan Abia
3699c52b59 test: update govc env variables documentation
govc cli no longer uses GOVMOMI_* enviroment variables. As the govc
documentation indicates, this variables should be GOVC_*.

https://github.com/vmware/govmomi/blob/master/govc/README.md#usage

[skip-ci]
2022-02-09 16:00:37 +01:00
Tomas Hozza
f43d9f4744 Add missing F34 and F35 image test cases
Add F34 and F35 image test cases for all remaining image types, which
were previously not tested. With this PR, image test cases are now
generated for all image types on all architectures as supported by the
Fedora distro definition.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-09 15:29:33 +01:00
Gianluca Zuccarelli
e8d7519c7d templates/dashboard: worker metric queries
The prometheus queries have been updated with
the correct namepsace for the job metrics
Additionally, this commit fixes some of the
queries to add fallback values when the
query results are returned empty.
2022-02-09 14:09:50 +01:00
Ondřej Budai
b1a40c1040 distro: add an alias for RHEL 8.7
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-08 17:37:36 +01:00
Gianluca Zuccarelli
80f24dbd61 metrics: change job metrics namespace
Currently the job metrics are namespaced with the composer
subsystem, i.e. `composer_worker`. Since we plan to split
the components to their own namespaces in app interface,
the worker subsystem should be split too.
2022-02-08 15:57:12 +01:00
Sanne Raymaekers
a739151c71 Revert "templates: Add dnf-json template"
This reverts commit 8cb3900dd6.
2022-02-08 14:05:48 +01:00
Jakub Rusz
a4c19eee85 Schutzfile: add RPMREPO to repo titles
This will make it easier to distinguish nightly rpmrepo snapshots from
actual nightly repositories in the test logs.
2022-02-08 14:28:06 +02:00
Jakub Rusz
29a8810d6e tests/ci: fix sonarqube analysis for main branch
Only use the branch options if it's actually running in a branch and
analyse main without them, sonarqube otherwise doesn't recognize the
main branch.
2022-02-08 14:28:06 +02:00
Tomas Hozza
1401a7a659 test: add image test cases for Fedora 34 and 35
Delete Fedora 33 image test cases.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-08 10:19:52 +01:00
Tomas Hozza
d9d5a6c1a6 Schutzfile: don't pin osbuild on Fedora and drop F33
Drop unnecessary pinning of osbuild release on Fedora. The latest version
available in all supported Fedora releases should be good enough for
osbuild-composer testing.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-08 10:19:52 +01:00
Tomas Hozza
617572ad6c test/api.sh: remove parts related to Fedora 33
We no longer test Cloud API on Fedora and Fedora 33 is EOL anyway.
Remove all Fedora 33 related lines from the test case.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-08 10:19:52 +01:00
Tomas Hozza
2c04329645 distro: drop support for Fedora 33
Fedora 33 is already EOL, therefore there is no point in supporting
image builds for it. Drop F33 from the distroregistry list and remove
F33 repositories definition.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-08 10:19:52 +01:00
Ondřej Budai
f66740fdbd github: fix gitlab trigger
With the quoting, we are passing an empty variable instead of no argument.
Also, if the skip is required, we are passing "-o ci.skip" instead of
"-o" "ci.skip".

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-08 10:03:59 +01:00
Roy Golan
e8dd3654d5 Allow skipping CI for draft PR or WIP label
Save resources and needless CI runs by skipping the trigger-gitlab job if a PR is 'draft' (can be set in the pull request page at any time) or by setting a label "WIP".

Asside from saving resources, WIP PRs that don't trigger CI are a good message to a contributers to keep testing it locally and to only get the project resources after serious testing been made on their side.
2022-02-08 08:24:23 +01:00
Sanne Raymaekers
4956e48a0b service-maintenance: Skip db cleanup
Let's enable the cloud cleanup first, and then move on to the db.
2022-02-07 20:42:45 +01:00
Gianluca Zuccarelli
dbf396db2b templates/dashboards: worker error metrics
Update the grafana dashboard for the workers
to show information on the success rate for
osbuild and depsolve jobs.
2022-02-07 20:40:37 +01:00
Tomas Hozza
91097fa007 test: regenerate RHEL 8.5 and 8.6 EC2 image test cases
Fix missing missing `image-info` report updates after PR#2235.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-07 20:33:54 +01:00
dependabot[bot]
c7cba177d9 build(deps): bump cloud.google.com/go/cloudbuild from 0.2.0 to 1.0.0
Bumps [cloud.google.com/go/cloudbuild](https://github.com/googleapis/google-cloud-go) from 0.2.0 to 1.0.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/v0.2.0...dlp/v1.0.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/cloudbuild
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-07 19:08:06 +01:00
Juan Abia
cf72a8963e test: update README with cloud cleaner info
update testing documentation with the purpose of 'cloud cleaner' and
'scheduled cloud cleaner'
2022-02-07 10:51:47 +01:00
Jakub Rusz
0d64331ffe tests/ci: run all integration tests on fedora and centos 2022-02-07 11:37:13 +02:00
Jakub Rusz
409eb3f33b tests/cross-distro: filter out beta and stream repos
Currently these repositories are not recognized by composer which is a
bug described in issue #2257. This commit will be reverted when related
issue is fixed.
2022-02-07 11:37:13 +02:00
Jakub Rusz
17f84ad6cc tests: expand cross-distro tests
This test now checks if all the installed repositories are recognized,
there are no extra or missing and that blueprint push with unsupported
distro fails gracefuly.
2022-02-07 11:37:13 +02:00