Commit graph

3300 commits

Author SHA1 Message Date
Jakub Rusz
c173d88061 test/ci: add sonarqube scan
Adding sonarqube scan to the pipeline.
2022-02-01 16:38:43 +02:00
Tom Gundersen
0b24099751 jwt: support multiple key providers
We may need to use several SSO providers, so extend our
configuration to allow that.

Based on PoC from Sanne:

```
package main

import (
	"net/http"
	"log"

	"github.com/openshift-online/ocm-sdk-go/authentication"
	"github.com/openshift-online/ocm-sdk-go/logging"
)

type H struct{}

func (h *H) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	log.Println("HURRAY")
}

func main() {

	logBuilder := logging.NewGoLoggerBuilder()
	logger, err := logBuilder.Build()
	if err != nil {
		panic(err)
	}

	aH, err := authentication.NewHandler().
		KeysURL("https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/certs").
		KeysURL("https://identity.api.openshift.com/auth/realms/rhoas/protocol/openid-connect/certs").
			Logger(logger).Next(&H{}).Build()
	if err != nil {
		panic(err)
	}

	log.Fatal(http.ListenAndServe(":8080", aH))

}
```
2022-01-31 20:40:22 +00:00
Christian Kellner
f3d0a4ac89 mock-openid: ability to set scope field
Add a new `-scope` command line flag that allows to set the scope
of the token.
2022-01-31 12:09:29 +01:00
Christian Kellner
03c21364f9 mock-openid: reply with token_type and exipres_in
Include the `token_type` (a necessary field) and the `expires_in`
field in the response.
2022-01-31 12:09:29 +01:00
Christian Kellner
669d3e0734 distro/rhel86: fix ec2 boot partition for arm64
It should be `1048576` (exactly 512 MiB), like it is for all other
distributions. It somehow got mingled in when the distribution was
forked off from 8.5/9.0 beta (1048676 to 1048576 strongly suggests
a sed command was involved, so we blame that).
2022-01-31 11:46:09 +01:00
Juan Abia
17e8a570a3 scheduled-cloud-cleaner: clean vmware resources
remove every object that has the tag "gitlab-ci-test" and it's old
enough
2022-01-31 09:26:51 +01:00
Juan Abia
2b69ad633e gitlab-ci: add internal network to runner
Scheduled cloud cleaner runner needs access to internal network to
interact with vmware.
2022-01-31 09:26:51 +01:00
Juan Abia
653cc76408 test/vmware: add test tag to vmware objects
add tag "gitlab-ci-test" to the created vm inside the test, so it can be
cleaned later.
2022-01-31 09:26:51 +01:00
Christian Kellner
5e2617ed49 kojiapi: successful osbuild jobs must have output
When we compute the overall status of a koji compose, the individual
build jobs are checked. Currently, a job is considered a failure, if
a build job has output (`OSBuildOutput`) and the output's `Success`
field is `false`. But `OSBuildOutput` will be `nil` when osbuild
crashed or refused the manifest input. Therefore the job status is
a failure if `OSBuildOutput` is `nil`, since if osbuild was run,
and the run was successful we must have a non-`OSBuildOutput` field.
2022-01-28 20:37:23 +00:00
Christian Kellner
987f4c4795 worker: fix job error check for koji-osbuild job
This check got inverted during the work on "Worker errors backwards
compatibility". As a consequence, osbuild was never run and the
result structure `buildResult.OSBuildOutput` was `nil` Since the
overall status reporting is not complete, and does not take this,
i.e. `buildResult.OSBuildOutput`, being `nil` as an error case,
the overall status was reported as "success". See the function
`composeStatusFromJobStatus` in `internal/kojiapi/server.go`.
2022-01-28 20:37:23 +00:00
Achilleas Koutsou
edfdcf0be6 osbuild-worker: change error handling for OCI upload
Issues caused by rebase-merge:
- Error handling was changed in one PR
- Errors using the old method were introduced in another
2022-01-28 17:33:48 +01:00
Achilleas Koutsou
022a52d56b distro/rhel90: special case root user for ssh keys
Add a special cases for the root user to the work-around for ssh keys in
OSTree commits.

See 93e54cd872 for the original,
equivalent change in RHEL 8.6.
2022-01-28 15:16:56 +01:00
Roy Golan
bee932e222 Add support for OCI upload provider
Signed-off-by: Roy Golan <rgolan@redhat.com>
2022-01-28 15:16:47 +01:00
Roy Golan
d9051c23e6 Add build requirement in README.md
koji depends on uccr/kerbi go module which depends on kerberos C header
files. Make sure new commers know about what to install in order to
compile the project.

Signed-off-by: Roy Golan <rgolan@redhat.com>
2022-01-28 15:16:47 +01:00
Roy Golan
da17cfe872 Add rpm build requierments
Signed-off-by: Roy Golan <rgolan@redhat.com>
2022-01-28 15:16:47 +01:00
Roy Golan
932e0269f5 OCI: vendor dependencies
Signed-off-by: Roy Golan <rgolan@redhat.com>
2022-01-28 15:16:47 +01:00
Christian Kellner
c5feb93279 distro/rhel90: no uuids in dos partition table
The unification of the partition table also introduced uuids and
types in uuid form for partition tables in dos layout, sill used
on PPC64LE and s390x. The org.osbuild.sfdisk stage did work with
that but produced a `/boot` partition with the wrong type, which
grub2 refused to read from and thus prevented boot. Fix this by
removing uuids from the dos partition tables.

Reported-by: Jakub Rusz <jrusz@redhat.com>
2022-01-28 12:16:39 +01:00
Gianluca Zuccarelli
88b5529cc4 osbuild-worker: test error backwards compatability
Since the workers will use structured error messages
going forward, it is necessary to maintain backwards
compatability for there errors in composer. Tests have
been added to the various apis to ensure that each api
checks for both kinds of errors, old and new.
2022-01-27 16:45:14 +01:00
Gianluca Zuccarelli
cc981b887a osbuild-worker: implement structured errors
Implement the structured errors as defined by the worker client.
Every error for each of the job types now returns a structured
error with a reason and a specific error code.  This will make
it possible to differentiate between 4xx errors and 5xx errors.

This commit refactors the way errors are implemented in the workers,
but maintains backwards compatability in composer by checking for
both kinds of errors.
2022-01-27 16:45:14 +01:00
Gianluca Zuccarelli
daf24f8db3 worker: define worker errors
Define worker errors to give more structured
error messages. The error api is:
id: VALIDATION_ERROR_NUMBER, reason: STRING, details: { issues: [{...}, {...}] }

The api was agreed upon with osbuild so that,
in future, osbuild errors will share the same
structure
2022-01-27 16:45:14 +01:00
sanne
1dbbc37cac ostree: Use stable installer image for PRs 2022-01-27 14:07:23 +01:00
sanne
8f8e871e3b schutzfile: Define nightly repos in schutzfile 2022-01-27 14:07:23 +01:00
Ondřej Budai
32651a610e test/koji: remove the koji-cli patch
The fix is already in Koji 1.27.1 which should be available in all downstreams
we are testing against.

See https://src.fedoraproject.org/rpms/koji

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-01-26 14:44:27 +01:00
Alexander Todorov
864a1a2330 tests: Don't execute koji.sh inside nightly CI pipelines 2022-01-26 11:39:43 +01:00
lavocatt
3899acbf84 Post release version bump
[skip ci]
2022-01-26 09:33:32 +00:00
Alexander Todorov
1bd852ceef tests: Add missing /var/log/audit to explicit checks
- the /var/log/audit mountpoint has been added previously to the
  blueprint but wasn't explicitly checked
- reordered the list of mountpoints to match the blueprint
  (alphabetically)
- added logging of successfull results we that QE can see what has been
  tested in the logs and use it for verification purposes.

Related: rhbz#2002727, rhbz#2001891
2022-01-25 11:59:20 +01:00
Alexander Todorov
e6212cced6 Add Image Tests to nightly pipelines 2022-01-25 11:53:16 +01:00
Jakub Rusz
1c2c97c900 tests/api: Make cleanup best effort
We should not fail on missing variables or errors during cleanup. The
test can fail in any stage and the cleanup should be run full each time.
2022-01-19 19:56:28 +01:00
sanne
6b716a4bbc tools/appsre-build-worker-packer: Fix branch parsing on jenkins 2022-01-19 12:48:14 +01:00
sanne
8a8ed14319 templates/dashboards: Fixed grafana uids
This way we get a nice URL `.../d/image-builder-(composer|worker)`.
2022-01-19 12:27:33 +01:00
Jakub Rusz
98f5623914 test/cases: properly kill journal for installers.sh
CI gets stuck if we don't kill the journal watcher on exit.
See bc1a7dc30c for more info.
2022-01-19 09:59:09 +01:00
Jakub Rusz
35fd20c0de ci: don't run iostats temporarily
CI job execution gets stuck if it fails and waits for timeout because
isostat are running in background preventing the main scrip from
exiting. Removing temporarily until a better solution is found.
2022-01-19 09:59:09 +01:00
Jakub Rusz
18ee1607a1 ci: change openstack runners to rhos-01
We can use both Openstack clouds now so switch to the new rhos-01
2022-01-19 09:59:09 +01:00
Jakub Rusz
1b951c5377 ci: update terraform SHA 2022-01-19 09:59:09 +01:00
sanne
ef6c5df9fa templates/packer: Make cdn host check less sensitive 2022-01-18 17:00:17 +01:00
sanne
68e98244b9 templates/packer: Correct priority for worker rpms
Lower priority means higher, currently the images built through AppSRE's
infra install the worker from epel.
2022-01-17 14:30:11 +01:00
sanne
a6f40f807d tools/appsre-build-worker-packer: Run entire workflow in container
Avoids the issue with artifacts that jenkins can't clean up.
2022-01-13 15:05:52 +01:00
sanne
2d0bfedc9d tools/appsre-ansible: Correct rpm destination when pulling rpms 2022-01-12 19:47:45 +01:00
sanne
e8b790e8d4 tools/appsre-build-worker-packer: Disable ansible debug output 2022-01-12 19:47:45 +01:00
sanne
6b43594840 tools/appsre-build-worker-packer: Ansible rpmbuild debug output 2022-01-12 17:33:50 +01:00
sanne
e6fd492372 tools/appsre-build-worker-packer: Prevent ssh from timing out 2022-01-12 17:33:50 +01:00
sanne
a21061fc5c tools/appsre-ansible: Don't use /tmp for rpmbuilds 2022-01-12 15:27:27 +01:00
lavocatt
b303e95827 Post release version bump
[skip ci]
2022-01-12 13:46:04 +00:00
sanne
93ec151c60 tools/appsre-build-worker-packer: Verbose ssh check 2022-01-12 12:33:04 +01:00
sanne
8c514308e7 tools/appsre-build-worker-packer: Correct user in ssh check 2022-01-12 12:13:41 +01:00
sanne
01163a1384 tools/appsre-build-worker-packer: Use ssh instead of ssh-keyscan
`ssh-keyscan` on appsre's infra has a bug where it doesn't exit 1 in
case of failure.
2022-01-12 12:02:58 +01:00
sanne
a83cf95d5b go.mod: Update oapi-codegen and kin-openapi 2022-01-12 11:35:06 +01:00
Ondřej Budai
add17bba45 spec: drop Fedora 33 hacks
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-01-12 11:35:06 +01:00
Ondřej Budai
b667df4443 bump the minimum Go version to 1.16
Fedora 34 ships 1.16
RHEL 8.5 also ships 1.16

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-01-12 11:35:06 +01:00
Ondřej Budai
f605fe9feb ci: drop Fedora 33
It's EOL.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-01-12 11:35:06 +01:00