Commit graph

409 commits

Author SHA1 Message Date
Christian Kellner
94d0a5f57a test: add httpretty & requests to test container
Add the requests and httpretty packages to the test environment.
The former will be used in the builder plugin and the latter for
mocking http requests.
2020-09-15 18:07:25 +02:00
Christian Kellner
385d51b2ba ci: send coverage data to codecov.io
Now that we have coverage data, send it to codecov.io, the service
already used by osbuild-composer, for visualization.
2020-09-14 17:37:35 +02:00
Christian Kellner
edb05b1683 ci: use pytest and generate coverage data
Instead of the standard unit tests module use pytest and generate
coverage data in XML form, later to be used with another service.
2020-09-14 17:37:35 +02:00
Christian Kellner
ed4e8795e5 gitignore: ignore coverage files
The intermediate as well as the output.
2020-09-14 17:37:35 +02:00
Christian Kellner
e5f563c9a1 run-tests: use pytest and generate coverage report
Use the more powerful pytest framework together with pytest-cov
to generate coverage support.
2020-09-14 17:37:35 +02:00
Christian Kellner
0efe8941c6 test/cli: check a normal invocation
Check a normal, i.e. successful, invocation and ensure relevant
functions are called with the right parameters.
2020-09-14 17:37:35 +02:00
Christian Kellner
0af2d958e3 test: add pytest to test env container
Add pytest and coverage support for it to the koji test environment
container.
2020-09-14 17:02:39 +02:00
Christian Kellner
b92e19aa1d test/cli: add unit test skeleton
Add the skeleton to run the cli plugin unit tests. As first check
ensure that exceptions are thrown for the build target checks.
2020-09-14 13:06:52 +01:00
Christian Kellner
9e98f10afc test/unit: extract common code
Create a new plugintest.PluginTest class that shares the plugin
loading code that is common to all plugin testing. Adapt the
existing (hub, builder) tests.
Also correct the name for test_builder.TestHubPlugin to
TestBuilderPlugin.
2020-09-14 13:06:52 +01:00
Christian Kellner
d8107f2347 test/builder: add unit test skeleton
Add the skeleton to run builder plugin unit tests and check that
exceptions are throw for unknown build target and unsupported
architectures.
2020-09-14 10:00:56 +02:00
Christian Kellner
c39f1d6d59 plugin/builder: properly raise exceptions
In two cases the exception was merely instantiated but not
actually thrown.
2020-09-14 10:00:56 +02:00
Christian Kellner
8b9143aae9 ci: run unit tests
Use the koji test environment container to run the unit tests.
2020-09-13 13:20:19 +02:00
Christian Kellner
7568437437 test/hub: simple input validation check
Check that an exception of `koji.ParameterError` gets raised for
invalid input.
2020-09-13 13:20:19 +02:00
Christian Kellner
5ba7028340 plugin/hub: raise ParameterError for invalid input
Convert the jsonschema.exceptions.ValidationError into the more
appropriate koji.ParameterError.
2020-09-13 13:20:19 +02:00
Christian Kellner
51f55998da run-test.sh: simple script to execute unit tests
Builds the test environment container and executes unit tests in
it.
2020-09-13 13:20:19 +02:00
Christian Kellner
272d7591ef test: add koji hub plugin unit test skeleton
Load the module, mock session and kojihub, check basic calling
osbuildImage will result in `kojihub.make_task` with the right
parameters.
2020-09-13 13:20:19 +02:00
Christian Kellner
5e074e83b9 gitignore: ignore __pycache__
Should never end up in git.
2020-09-13 13:20:19 +02:00
Christian Kellner
6d0a298492 ci: run pylint on the source
Use the new github action to run pylint on all the source code.
2020-09-12 17:22:51 +02:00
Christian Kellner
734d039089 test: add github test environment action
Create a github action that accepts any command with args and runs
it in a container that is prepared with all the dependencies for
koji (hub, builder, command line), so all tests, like pylint or
unit tests can run in it.
2020-09-12 16:00:54 +02:00
Christian Kellner
0180bce383 test: use python based integration test
Instead of creating a compose from integration.sh, use the new
python based integration tests. The integration.sh script will
this just setup the environment and the actual tests are all
found in test/integration/
2020-09-11 23:18:04 +01:00
Christian Kellner
6fbdcf505f test: add unknown tag check test
Test that we fail a compose for an unknown tag.
2020-09-11 23:18:04 +01:00
Christian Kellner
3908e5f568 test: add python based compose test 2020-09-11 23:18:04 +01:00
Christian Kellner
fa1561a9b4 pylintrc: copied from osbuilid
Only modification is the removal of an osbuild specific type
check exception.
2020-09-11 23:18:04 +01:00
Christian Kellner
0bf3c47113 plugin/hub: clean up imports
Remove an unused import and sort the imports properly.
Also silence the for the kojihub import.
2020-09-11 23:18:04 +01:00
Christian Kellner
899c2233c7 plugin/cli: silence pylint warning
We use `_running_in_bg` for now, as other plugins do. Silence the
warning for that.
2020-09-11 23:18:04 +01:00
Christian Kellner
01cf643516 plugin/cli: return a value in all code paths
Be consistent in all code paths.
2020-09-11 23:18:04 +01:00
Christian Kellner
4eeb998a60 plugin/cli: provide a bit documentation
Better describe the plugin in the file level doc-string.
2020-09-11 23:18:04 +01:00
Christian Kellner
e3ce5f7c7b plugin/builder: silence some pylint warnings
The 'koji' parameter for the compose request is called that way to
follow the OpenAPI spec, so silence the "redefined-outer-name"
warning.

The handler function needs have the actual task arguments and thus
on purpose deviates from the argument list of the base class'
handler method. So silence that.
2020-09-11 23:18:04 +01:00
Christian Kellner
6f1d685d9b plugin/builder: move argparse into main
The argparse import is only needed when the plugin is used as a
stand alone composer koji API client, therefore only include it
in main().
2020-09-11 23:18:04 +01:00
Christian Kellner
f7794daa13 plugin/builder: fix indent in cmd line arguments 2020-09-11 23:18:04 +01:00
Christian Kellner
1715558c8f plugin/builder: add more documentation
Describe the plugin itself and how it can be used as a stand-alone
executable.
2020-09-11 23:18:04 +01:00
Christian Kellner
20831a99bb plugin/builder: fix BuildError msg to include name
Fix the error message string to be a proper f-string in the
OSBuildImage.arches_for_config method, so it actually includes
the name of the tag.
2020-09-11 23:18:04 +01:00
Christian Kellner
d5a80cf3cf test: manually install container-selinux
Seems to not be pulled in. Don't disable selinux.
2020-09-10 20:12:39 +01:00
Christian Kellner
b292458771 test: create a compose via koji
Try testing a compose via the koji, which involves creating using
the koji command line plugin to make the XMLRPC call to koji hub,
where the osbuild koji hub plugin verifies the parameters and then
creates the task. The osbuild koji plugin for the builder is then
picking up the task, and uses composer's koji API to request a
compose. Once this is successful it will be imported by composer
into koji via the 'CGImport' method.
The `koji osbuild-image` command waits for all this and reports
whether the task was successful or not via its exit code (and
on stdout).

This uses a fleet of containers: a database one, a kerberos kdc
one, another one for the koju hub and finally one for the koji
builder. The pre-build RPMs are used to install the plugins.

NB: On RHEL we need to manually install the `dnsname` podman
plugin, since it is missing, but required so that containers
can address each other by hostnames.
See [schutzbot/vendor/README.md](schutzbot/vendor/README.md)
2020-09-10 18:41:13 +01:00
Christian Kellner
3fdf66a61c run-builder: use jq instead of inspect --format
Because podman on RHEL 8.2 does not understand --format, so we
basically do the same now via 'jq'.
2020-09-10 18:41:13 +01:00
Christian Kellner
dd2d3cf0a9 run-builder: argument checking
Check we have at least one positional argument, and it is either
"start", "stop" or "fg".
2020-09-10 18:41:13 +01:00
Christian Kellner
928d2278be plugin/cli: return task result when waiting
When waiting for the task, return the result from the `watch_task`
call, which will indicate if the task was successful. This will
then be reflected in the error code of the koji call.
2020-09-10 18:41:13 +01:00
Christian Kellner
1db41a067f run-koji-container: be more verbose
Show logs and running containers, should help track down issues.
2020-09-10 18:41:13 +01:00
Christian Kellner
10e691d8c3 run-builder: support background execution
The run-builder script now takes a command line argument, which is
either
  start - run the container in the background
  stop  - stop the running container
  fg    - run the container in the foreground (old behavior)

This should prove useful in CI.
2020-09-10 18:41:13 +01:00
Christian Kellner
1effdc7a2c container/hub: sync with script via file
Create a file in the hub container the share directory that will
signal to the outside that we are done with the setup.
2020-09-10 18:41:13 +01:00
Christian Kellner
7a366eca7a run-koji-container: don't fail if share dir exists
We need to drop in the RPMs into the share-dir before the hub
container is started, so don't fail if the directory already
exists.
2020-09-10 18:41:13 +01:00
Christian Kellner
2682911556 kerberos: add dns realm map for osbuild-local.conf 2020-09-10 18:41:13 +01:00
Christian Kellner
b778133e89 container: support installing via RPMs
If the share dir contains rpms in /share/rpms, install those,
instead of using the plugin from the container.
2020-09-10 18:41:13 +01:00
Christian Kellner
af4e66d2b2 plugin/hub: use jsonschema to validate input
Use jsonschema to validate the input to the XMLRPC call to catch
mistakes early, i.e. before creating the task.
2020-09-10 18:41:13 +01:00
Christian Kellner
3dc463c8dd plugin/cli: pretty-print the options dict
Also show the options that we are using.
2020-09-10 18:41:13 +01:00
Christian Kellner
60516fc811 gitignore: remove old ssl dirs, add build/
The old container related ssl dir ignores are no longer necessary.
Add 'build/' for RPM builds though.
2020-09-10 18:41:13 +01:00
Tom Gundersen
a416570ea2 schutzbot: add CI integration
The CI is in two stages, for each supported distro.

First the RPMs are generated from the spec file in the repo for the
given distro and architecture.

Once all the RPM builds have succeeded successfully, a test machine is
provisioned with osbulid-composer installed, and koji API enabled.

The repository containing the RPMs of the code being tested is also
enabled on the test machine, and the cli client is installed.

Finally, the test/integration.sh script is executed, which currently
does nothing.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-09-10 00:05:44 +02:00
Tom Gundersen
ec033ed623 spec/cli: depend on koji
This should depend on the cli client, which is shipped in the koji package,
not the builder.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-09-10 00:05:44 +02:00
Tom Gundersen
d2e81fbf79 spec: don't use pycached macro
This does not work on RHEL8.2.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-09-10 00:05:44 +02:00
Christian Kellner
857ae4ad77 README.md: add general description 2020-09-09 16:12:26 +02:00