Commit graph

192 commits

Author SHA1 Message Date
Christian Kellner
0df038c8c5 container: add internal repos
One for the latest koji and the other one for RHEL. They are only
accessible from within the internal network.
2020-09-17 10:48:37 +02:00
Christian Kellner
14003137fc run-builder: supply --rm only for "fg"
When run in the background, remove the container in "builder_stop",
this should make it easier to print logs between starting and
stopping the container.
2020-09-17 10:48:37 +02:00
Christian Kellner
5b237ffe84 run-builder: set sensible bash flags
Do the usual "set -euo pipefail".
2020-09-17 10:48:37 +02:00
Christian Kellner
955c531000 test/builder: check --cert command line option
Check that passing `--cert` as a command line option works, for
this we use real working certificates because this will actually
be parsed by requests.
2020-09-17 10:48:37 +02:00
Christian Kellner
15f645e6fa test/builder: more ssl_cert config check
Check we can handle a single cert properly, i.e. pass it as a
plain string to requests. Also check that if three components
are specified, an ValueError is thrown.
2020-09-17 10:48:37 +02:00
Christian Kellner
94f7b35967 test/builder: check ssl_verify can be a string
The ssl_verify config option can be a string, i.e. a path to the
certificate authority for the server side certificate. Check
that we handle that properly.
2020-09-17 10:48:37 +02:00
Christian Kellner
bae564a1ab plugin/builder: new helper to parse cert strings
The requests cert argument accepts a tuple with the key and the
cert, or one string as a path to a combination of both. Create
a new helper and use that to parse the "ssl_cert" configuration
option as well as the command line option. Change the latter
into a string.
2020-09-17 10:48:37 +02:00
Christian Kellner
c2d73b431a test/builder: check config parsing
Check that options defined in the config file will be reflected
in the plugin. test all known options.
2020-09-17 10:48:37 +02:00
Christian Kellner
9eeb810e3f plugin/builder: use global constant for cfg files
Use a global constant `DEFAULT_CONFIG_FILES` for the default
plugin configuration files. Besides giving it more visibility
it also makes it easier to test, because in the unit test,
this can easily be overwritten.
2020-09-17 10:48:37 +02:00
Christian Kellner
cde65d1f74 CI: run rhel8 integration test in PSI
We need access to internal repos, only available in PSI.
2020-09-17 10:48:37 +02:00
Christian Kellner
78cef84f67 test/builder: check repo configuration
In the test_compose_success check, after the build is "done",
check the repositories in compose request match with what
we expect.
2020-09-17 10:48:37 +02:00
Christian Kellner
41ee1f1708 test/integration: build image for current distro
Add a separate unit test to do a compose for a RHEL based image,
but then restrict the execution of test_compose_{disto} to the
distribution the test is running on, since currently cross-
distribution builds can be problematic.
2020-09-17 10:48:37 +02:00
Christian Kellner
565cecc07f test/integration: wait for compose result
In the "test_compose" integration test, explicitly wait for the
compose result, instead of being ok with having created the
request for it.
2020-09-17 10:48:37 +02:00
Christian Kellner
cf56696d90 test/builder: check stand-alone usage
Check the basic usage of the plugin as a stand-alone client for
the osbuild-composer API.
2020-09-17 10:48:37 +02:00
Christian Kellner
f2d73f018b plugin/builder: return 0 from sub-commands
In the functions that implement the sub-commands (for stand-alone
use), always explicitly return 0, so that main() will always
return an int.
2020-09-17 10:48:37 +02:00
Christian Kellner
d4f0b169a0 composer: update to master/8ccbde8
This is right when PR-952 landed, which is what we are updating
from. This will use SSL certification and also re-enable auto
kerberos authorization via config files for composer.
2020-09-17 10:48:37 +02:00
Christian Kellner
a660b39e00 test/integration: use copy-cred.sh
Instead of manually copying credentials around, use the new
copy-creds.sh script. Also don't manually initialize kerberos,
because that is now done automatically by the worker and the
builder.
2020-09-17 10:48:37 +02:00
Christian Kellner
c782f7f350 test/integration: make composer certificates
Use test/make-certs.sh to generate osbuild composer ssl/tls certs.
2020-09-17 10:48:37 +02:00
Christian Kellner
660713a39b container/builder: configure ssl for plugin
Configure the ssl/tls cert verification. Use make-certs.sh and
copy-creds.sh from test/ to put all the ducks in the row for
this to work.
2020-09-17 10:48:37 +02:00
Christian Kellner
2fed0b7c5b test: add copy-creds.sh to copy credentials
This script is intended to be called between "run-koji-container"
and "run-builder". It will copy the kerberos keytabs out of the
temporary directory generated by "run-koji-container" as well as
copying the SSL/TLS certificates from host to the share dir so
that the plugin ca use it to authenticate itself to composer.
2020-09-17 10:48:37 +02:00
Christian Kellner
481243e628 test: add make-certs script to generate SSL certs
This will create the a certificate authority (CA) and then create
a cert for composer and another one for the worker. The worker
one can also be used by the koji plugin.
The configuration file is needed to get subjectAltName working.
2020-09-17 10:48:37 +02:00
Christian Kellner
db82105eb0 plugin/builder: TLS/SSL support (client side certs)
Add support for client side certificates via a new configuration
option: "ssl_cert", which should be a path to the certificate or
a comma separated list of paths to certificates.
Additionally, the "ssl_verify" option controls how the server
certificate is validated. This can either be a boolean ("True",
"False") or a path to a CA file.
NB: The url for the composer host must contain "https" for the
ssl/tls engine to kick in.

When used as a stand-alone koji API client, two new command
line options are provided, --cert and --ca. See their help
text for a description.
2020-09-17 10:48:37 +02:00
Christian Kellner
790462e967 plugin/builder: sync status enum with upstream
Align our status enums with the one from upstream, i.e. composer's
koji API openAPI spec as of commit 8ccbde8.
2020-09-17 10:48:37 +02:00
Christian Kellner
6b4632385a test/hub: validate the jsonschema
Validate the json schema used to validate the input arguments. So
much validation!
2020-09-17 10:48:37 +02:00
Christian Kellner
d0d167d2f8 plugin/hub: fix typo
It is schema, not schmea. That is not even a word.
2020-09-17 10:48:37 +02:00
Christian Kellner
2f5bea53f0 test: move kerberos cfg for LOCAL to test/data
This is a test specific configuration file that should not be at
the top level directory. Move it to a new test/data directory.
2020-09-17 10:48:37 +02:00
Christian Kellner
9a21c00f81 test/builder: use composer url from the plugin
Use the new DEFAULT_COMPOSER_URL constant from the plugin. This
will follow changes to e.g. SSL automatically now.
2020-09-17 10:48:37 +02:00
Christian Kellner
d4b5cae604 plugin/builder: global constants for default urls
Use a global constant for the composer and kojib hub URLs instead
of defining them in the source twice.
2020-09-17 10:48:37 +02:00
Christian Kellner
1ee0d80538 run-tests: also lint the plugins
Use pylint also for the plugins not only the tests.
2020-09-15 18:59:03 +02:00
Christian Kellner
a0c562e5c6 fixup! plugin/builder: use requests python package 2020-09-15 18:59:03 +02:00
Christian Kellner
e1edb12d11 test/builder: check a successful compose
Use the standard session and options mock object together with the
new composer mock object to simulate a successful compose:
the mock composer will accept the request and also status calls
for it, which will immediately return with success. The plugin
should then return and indicate a successful compose.
2020-09-15 18:59:03 +02:00
Christian Kellner
31885efab8 test/builder: mock composer, check a bad request
Use httpretty to mock composer responses. Create a helper method
to configure a session object that should result in a valid
compose request. Simulate a bad request by asking for an arch
that the mock composer will refuse and check the correct exception
is thrown in the builder plugin.
2020-09-15 18:59:03 +02:00
Christian Kellner
569d53c2d9 plugin/builder: use requests python package
Use the requests python package, which is more convenient to use,
and also supports sessions, where global options, like TLS certs
can be set.
Adapt the container and the spec file to install and require
the python package.
2020-09-15 18:59:03 +02:00
Christian Kellner
94d0a5f57a test: add httpretty & requests to test container
Add the requests and httpretty packages to the test environment.
The former will be used in the builder plugin and the latter for
mocking http requests.
2020-09-15 18:07:25 +02:00
Christian Kellner
385d51b2ba ci: send coverage data to codecov.io
Now that we have coverage data, send it to codecov.io, the service
already used by osbuild-composer, for visualization.
2020-09-14 17:37:35 +02:00
Christian Kellner
edb05b1683 ci: use pytest and generate coverage data
Instead of the standard unit tests module use pytest and generate
coverage data in XML form, later to be used with another service.
2020-09-14 17:37:35 +02:00
Christian Kellner
ed4e8795e5 gitignore: ignore coverage files
The intermediate as well as the output.
2020-09-14 17:37:35 +02:00
Christian Kellner
e5f563c9a1 run-tests: use pytest and generate coverage report
Use the more powerful pytest framework together with pytest-cov
to generate coverage support.
2020-09-14 17:37:35 +02:00
Christian Kellner
0efe8941c6 test/cli: check a normal invocation
Check a normal, i.e. successful, invocation and ensure relevant
functions are called with the right parameters.
2020-09-14 17:37:35 +02:00
Christian Kellner
0af2d958e3 test: add pytest to test env container
Add pytest and coverage support for it to the koji test environment
container.
2020-09-14 17:02:39 +02:00
Christian Kellner
b92e19aa1d test/cli: add unit test skeleton
Add the skeleton to run the cli plugin unit tests. As first check
ensure that exceptions are thrown for the build target checks.
2020-09-14 13:06:52 +01:00
Christian Kellner
9e98f10afc test/unit: extract common code
Create a new plugintest.PluginTest class that shares the plugin
loading code that is common to all plugin testing. Adapt the
existing (hub, builder) tests.
Also correct the name for test_builder.TestHubPlugin to
TestBuilderPlugin.
2020-09-14 13:06:52 +01:00
Christian Kellner
d8107f2347 test/builder: add unit test skeleton
Add the skeleton to run builder plugin unit tests and check that
exceptions are throw for unknown build target and unsupported
architectures.
2020-09-14 10:00:56 +02:00
Christian Kellner
c39f1d6d59 plugin/builder: properly raise exceptions
In two cases the exception was merely instantiated but not
actually thrown.
2020-09-14 10:00:56 +02:00
Christian Kellner
8b9143aae9 ci: run unit tests
Use the koji test environment container to run the unit tests.
2020-09-13 13:20:19 +02:00
Christian Kellner
7568437437 test/hub: simple input validation check
Check that an exception of `koji.ParameterError` gets raised for
invalid input.
2020-09-13 13:20:19 +02:00
Christian Kellner
5ba7028340 plugin/hub: raise ParameterError for invalid input
Convert the jsonschema.exceptions.ValidationError into the more
appropriate koji.ParameterError.
2020-09-13 13:20:19 +02:00
Christian Kellner
51f55998da run-test.sh: simple script to execute unit tests
Builds the test environment container and executes unit tests in
it.
2020-09-13 13:20:19 +02:00
Christian Kellner
272d7591ef test: add koji hub plugin unit test skeleton
Load the module, mock session and kojihub, check basic calling
osbuildImage will result in `kojihub.make_task` with the right
parameters.
2020-09-13 13:20:19 +02:00
Christian Kellner
5e074e83b9 gitignore: ignore __pycache__
Should never end up in git.
2020-09-13 13:20:19 +02:00