Github actions have a peculiar behaviour: The "job root" directory is owned
by user with ID 1001 whereas many containers (like the Fedora one) run as root
by default. This causes git to freak out because repositories owned by
different users aren't considered safe anymore and must be explicitly allow-
listed.
This becomes a problem when we switch to Go 1.18 because it stamps build with
Git information which fails because the repository technically isn't safe.
Let's fix this by marking the repository as a safe one.
Note that the actions/checkout action has a set-safe-directory feature, but
this one is not permanent. After the action is done, the setting is removed.
See https://github.com/actions/checkout/pull/762
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This prepares the usage of the `internal/container` from composer
directly, as opposed to the existing use in the worker. Said pkg
uses the `containers/image/v5`, which uses `proglottis/gpgme` and
the latter needs the gpgme C library. We therefore install it and
its dependencies.
Add a new generic container registry client via a new `container`
package. Use this to create a command line utility as well as a
new upload target for container registries.
The code uses the github.com/containers/* project and packages to
interact with container registires that is also used by skopeo,
podman et al. One if the dependencies is `proglottis/gpgme` that
is using cgo to bind libgpgme, so we have to add the corresponding
devel package to the BuildRequires as well as installing it on CI.
Checks will follow later via an integration test.
Run unit tests in GitHub workflows in a Fedora container to enable the
dnf-json tests. Run the tests alone with the `force-dnf` flag to make
sure the tests pass and are not skipped.
Install Go using dnf instead of the GH action. The action seems to
cause issues with the $PATH.
Use the registry.fedoraproject.org container for both unit tests and
pylint on dnf-json.
Requires some reordering of the steps in each workflow and the addition
of `git-core` as a dependency.
Using Fedora 35 instead of latest because of changes in the go build
tool: The new -buildvcs flag causes issues on GitHub actions.
Fedora 36 ships pylint 2.13 that newly reports:
dnf-json:436:20: E0601: Using variable 'cache_state' before assignment (used-before-assignment)
As dnf-json is pending a big rewrite
(https://github.com/osbuild/osbuild-composer/pull/2537),
I decided to pin fedora to 35 and let the other PR decide how to proceed in
order to prevent any conflicts.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The oldest distributions that we now support are RHEL 8.4 and Fedora 33.
They both support go 1.15, let's bump.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Separate the loading of repo definitions from JSON file from
`LoadRepositories()` to a standalone function
`loadRepositoriesFromFile()`, to make it easy to reuse it in the future.
Add unit tests for `LoadRepositories()` function.
Exclude github.com/osbuild/osbuild-composer/internal/rpmmd/test package
from test coverage. Package with just tests and no other code makes `go
test` to fail. This should be fixed in go 1.17.
See https://github.com/golang/go/issues/27333
Signed-off-by: Tomas Hozza <thozza@redhat.com>
By default, the checkout action checkouts the merge commit. This is different
from what Schutzbot currently does - it runs the test on the PR HEAD commit.
Let's change the GitHub workflows behaviour to the same one as Schutzi
uses.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
When unit tests succeed, no one cares about them.
When unit tests fail, it's hard to find which one failed. This commit removes
the verbose flag, so it's easy to spot what failed.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
We can now send webhook data to an SQS queue at AWS without signing the
request with credentials. This allows us to trigger Schutzbot from
forks and from branches on the main repository.
Signed-off-by: Major Hayden <major@redhat.com>
The oldest distros we support are Fedora 32 and RHEL 8.3. As both have
Go 1.14, we're safe to upgrade.
Also, I had to change prepare-source.sh because go fmt now refuses to run on
a project which has issues in go.mod, go.sum or modules.text. I think this
should be a harmless change.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
As the team obsessed with immutable test dependencies, how could we use
the ubuntu-latest VM which can change at any time?
Speaking of changes to ubuntu-latest... It will soon be updated from ubuntu
18.04 to 20.04 [1].
This commit switches our testing pipeline to use ubuntu-20.04 to:
1) make our test dependencies immutable (or at least slightly more immutable)
2) make us prepared for the ubuntu-latest changes.
[1]: https://github.com/actions/virtual-environments/issues/1816
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
rpmlint errors are usually not a big deal but it's always nice when we can
catch them early. This commit introduces a new Github workflow that builds
an SRPM and runs rpmlint against it.
The Koji test in Github actions was always a bit quick and dirty solution.
I think it's much nicer solution to run it on Schutzbot.
Therefore, this commit moves the koji_test.go to a new osbuild-koji-tests
executable. This new test isn't run in the base test suite as one would
anticipate but inside the koji.sh test. This is needed because
osbuild-koji-tests requires a running koji instance. This might change
in the future but I think it works for now.
Also install it is part of he tests subpackage. This a helper-tool, not
golang code, so should not live in `internal`. We need access to this
from the integration tests, so install it onto the tests system.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This job uses shellcheck to analyze all our bash scripts except for
those in /vendor directory.
It also allows sourcing /etc/os-release without checking it, because
it doesn't make any sense to check that the CI machine has correct
/etc/os-release file. Also allow useless cats because I don't think they
cause any harm.
Rather than using the arbitrary port 8701, use the standard 443. The
worker API will remain on a separate port, and as long as the two APIs
are exposed by the same binary that will have to remain separate at
8700.
Move the test instance of koji on localhost from 443 to 4343, to avoid a
conflict.
In a follow-up we should also give this API a prefix, so the cloud API
can share the same port with it.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Contrary to our assumption, we cannot initialize the build with the
link to the task. We can only update the link once the build has
completed.
This seems like a bug in koji, but we keep it like this for now.
Prior this commit we only had support for username/password authentication
in the koji integration. This wasn't particularly useful because this
auth type isn't used in any production instance.
This commit adds the support for GSSAPI/Kerberos authentication.
The implementation uses kerby library which is very lightweight wrapper
around C gssapi library.
Also, the koji unit test and the run-koji-container script were modified
so the GSSAPI auth is fully tested.
Services in Github Actions are cool but have some drawbacks:
1) We want to be able to use the container setup locally, therefore there's
run-koji-container script which does exactly the same setup as it's defined
in Github Actions. We don't want duplicities though.
2) In the near future, we will need more complicated setup - generating
certificates before a container is started. This is not possible with
the current Github Actions capabilities.
This commit removes the container setup from Github Actions and just reuses
the run-koji-container script in the GH Actions environment. This way we
have only one setup which is also more flexible.
All our downstream platforms now support Go 1.13:
RHEL 8.2: golang-1.13.4
Fedora 31: golang-1.13.14
There's no reason anymore to stay on 1.12, therefore this commit bumps
the minimal required Go version to 1.13
For reasons unknown, golangci-lint's default 1m0s timeout is *slightly*
too short for CI runs occasionally. Extend it to 5 minutes to ensure the
job always has enough time to run.
Signed-off-by: Major Hayden <major@redhat.com>
We are now building RPMs via mock in Jenkins, so we don't need these RPM
build jobs in GitHub Workflows.
Signed-off-by: Major Hayden <major@redhat.com>
