particle-os/debian-koji-osbuild
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
90 commits 1 branch 0 tags 1.8 MiB
Commit graph

90 commits

This branch
This branch
All branches
Author SHA1 Message Date
Christian Kellner
0180bce383 test: use python based integration test 
Instead of creating a compose from integration.sh, use the new
python based integration tests. The integration.sh script will
this just setup the environment and the actual tests are all
found in test/integration/
 2020-09-11 23:18:04 +01:00
Christian Kellner
6fbdcf505f test: add unknown tag check test 
Test that we fail a compose for an unknown tag.
 2020-09-11 23:18:04 +01:00
Christian Kellner
3908e5f568 test: add python based compose test 2020-09-11 23:18:04 +01:00
Christian Kellner
fa1561a9b4 pylintrc: copied from osbuilid 
Only modification is the removal of an osbuild specific type
check exception.
 2020-09-11 23:18:04 +01:00
Christian Kellner
0bf3c47113 plugin/hub: clean up imports 
Remove an unused import and sort the imports properly.
Also silence the for the kojihub import.
 2020-09-11 23:18:04 +01:00
Christian Kellner
899c2233c7 plugin/cli: silence pylint warning 
We use `_running_in_bg` for now, as other plugins do. Silence the
warning for that.
 2020-09-11 23:18:04 +01:00
Christian Kellner
01cf643516 plugin/cli: return a value in all code paths 
Be consistent in all code paths.
 2020-09-11 23:18:04 +01:00
Christian Kellner
4eeb998a60 plugin/cli: provide a bit documentation 
Better describe the plugin in the file level doc-string.
 2020-09-11 23:18:04 +01:00
Christian Kellner
e3ce5f7c7b plugin/builder: silence some pylint warnings 
The 'koji' parameter for the compose request is called that way to
follow the OpenAPI spec, so silence the "redefined-outer-name"
warning.

The handler function needs have the actual task arguments and thus
on purpose deviates from the argument list of the base class'
handler method. So silence that.
 2020-09-11 23:18:04 +01:00
Christian Kellner
6f1d685d9b plugin/builder: move argparse into main 
The argparse import is only needed when the plugin is used as a
stand alone composer koji API client, therefore only include it
in main().
 2020-09-11 23:18:04 +01:00
Christian Kellner
f7794daa13 plugin/builder: fix indent in cmd line arguments 2020-09-11 23:18:04 +01:00
Christian Kellner
1715558c8f plugin/builder: add more documentation 
Describe the plugin itself and how it can be used as a stand-alone
executable.
 2020-09-11 23:18:04 +01:00
Christian Kellner
20831a99bb plugin/builder: fix BuildError msg to include name 
Fix the error message string to be a proper f-string in the
OSBuildImage.arches_for_config method, so it actually includes
the name of the tag.
 2020-09-11 23:18:04 +01:00
Christian Kellner
d5a80cf3cf test: manually install container-selinux 
Seems to not be pulled in. Don't disable selinux.
 2020-09-10 20:12:39 +01:00
Christian Kellner
b292458771 test: create a compose via koji 
Try testing a compose via the koji, which involves creating using
the koji command line plugin to make the XMLRPC call to koji hub,
where the osbuild koji hub plugin verifies the parameters and then
creates the task. The osbuild koji plugin for the builder is then
picking up the task, and uses composer's koji API to request a
compose. Once this is successful it will be imported by composer
into koji via the 'CGImport' method.
The `koji osbuild-image` command waits for all this and reports
whether the task was successful or not via its exit code (and
on stdout).

This uses a fleet of containers: a database one, a kerberos kdc
one, another one for the koju hub and finally one for the koji
builder. The pre-build RPMs are used to install the plugins.

NB: On RHEL we need to manually install the `dnsname` podman
plugin, since it is missing, but required so that containers
can address each other by hostnames.
See [schutzbot/vendor/README.md](schutzbot/vendor/README.md)
 2020-09-10 18:41:13 +01:00
Christian Kellner
3fdf66a61c run-builder: use jq instead of inspect --format 
Because podman on RHEL 8.2 does not understand --format, so we
basically do the same now via 'jq'.
 2020-09-10 18:41:13 +01:00
Christian Kellner
dd2d3cf0a9 run-builder: argument checking 
Check we have at least one positional argument, and it is either
"start", "stop" or "fg".
 2020-09-10 18:41:13 +01:00
Christian Kellner
928d2278be plugin/cli: return task result when waiting 
When waiting for the task, return the result from the `watch_task`
call, which will indicate if the task was successful. This will
then be reflected in the error code of the koji call.
 2020-09-10 18:41:13 +01:00
Christian Kellner
1db41a067f run-koji-container: be more verbose 
Show logs and running containers, should help track down issues.
 2020-09-10 18:41:13 +01:00
Christian Kellner
10e691d8c3 run-builder: support background execution 
The run-builder script now takes a command line argument, which is
either
  start - run the container in the background
  stop  - stop the running container
  fg    - run the container in the foreground (old behavior)

This should prove useful in CI.
 2020-09-10 18:41:13 +01:00
Christian Kellner
1effdc7a2c container/hub: sync with script via file 
Create a file in the hub container the share directory that will
signal to the outside that we are done with the setup.
 2020-09-10 18:41:13 +01:00
Christian Kellner
7a366eca7a run-koji-container: don't fail if share dir exists 
We need to drop in the RPMs into the share-dir before the hub
container is started, so don't fail if the directory already
exists.
 2020-09-10 18:41:13 +01:00
Christian Kellner
2682911556 kerberos: add dns realm map for osbuild-local.conf 2020-09-10 18:41:13 +01:00
Christian Kellner
b778133e89 container: support installing via RPMs 
If the share dir contains rpms in /share/rpms, install those,
instead of using the plugin from the container.
 2020-09-10 18:41:13 +01:00
Christian Kellner
af4e66d2b2 plugin/hub: use jsonschema to validate input 
Use jsonschema to validate the input to the XMLRPC call to catch
mistakes early, i.e. before creating the task.
 2020-09-10 18:41:13 +01:00
Christian Kellner
3dc463c8dd plugin/cli: pretty-print the options dict 
Also show the options that we are using.
 2020-09-10 18:41:13 +01:00
Christian Kellner
60516fc811 gitignore: remove old ssl dirs, add build/ 
The old container related ssl dir ignores are no longer necessary.
Add 'build/' for RPM builds though.
 2020-09-10 18:41:13 +01:00
Tom Gundersen
a416570ea2 schutzbot: add CI integration 
The CI is in two stages, for each supported distro.

First the RPMs are generated from the spec file in the repo for the
given distro and architecture.

Once all the RPM builds have succeeded successfully, a test machine is
provisioned with osbulid-composer installed, and koji API enabled.

The repository containing the RPMs of the code being tested is also
enabled on the test machine, and the cli client is installed.

Finally, the test/integration.sh script is executed, which currently
does nothing.

Signed-off-by: Tom Gundersen <teg@jklm.no>
 2020-09-10 00:05:44 +02:00
Tom Gundersen
ec033ed623 spec/cli: depend on koji 
This should depend on the cli client, which is shipped in the koji package,
not the builder.

Signed-off-by: Tom Gundersen <teg@jklm.no>
 2020-09-10 00:05:44 +02:00
Tom Gundersen
d2e81fbf79 spec: don't use pycached macro 
This does not work on RHEL8.2.

Signed-off-by: Tom Gundersen <teg@jklm.no>
 2020-09-10 00:05:44 +02:00
Christian Kellner
857ae4ad77 README.md: add general description 2020-09-09 16:12:26 +02:00
Christian Kellner
3f5de8a68d plugin/cli: eager check of target input 
Check the target with name exists and has a destination tag, so
we can abort early.
 2020-09-09 15:56:35 +02:00
Christian Kellner
3934382337 plugin/cli: honor global --quiet flag 
If --quiet was requested globally, then don't print anything.
 2020-09-09 15:51:11 +02:00
Christian Kellner
8099d79aa1 plugin/cli: support --wait, --nowait option 
Can be used to explicitly opt in or out of waiting for the task to
finish. The help text is taken from the standard koji client.
 2020-09-09 15:28:36 +02:00
Christian Kellner
79387f1f2c plugins/cli: use upstream description help text 
For --repo and --release, use the help text from koji build-image.
 2020-09-09 15:07:36 +02:00
Christian Kellner
81aebbc681 plugin/cli: use koji_cli.lib.OptionParser 
Use koji's OptionParser instead of argparse to be better aligned
with the formatting of standard koji options.
 2020-09-09 15:06:53 +02:00
Christian Kellner
6087e2d90e run-builder: inspect network to get gateway IP 
Instead of inferring the gateway ip via the the network device for
the podman bridge, where the name can change, use podman network
inspect.

```js
The json looks like this:
[
  {
    "cniVersion": "0.4.0",
    "name": "org.osbuild.koji",
    "plugins": [
      {
        "bridge": "cni-podman1",
        "hairpinMode": true,
        "ipMasq": true,
        "ipam": {
          "ranges": [
            [
              {
                "gateway": "10.89.0.1",
                "subnet": "10.89.0.0/24"
              }
            ]
          ],
          "routes": [
            {
              "dst": "0.0.0.0/0"
            }
          ],
          "type": "host-local"
        },
        "isGateway": true,
        "type": "bridge"
      },
      {
        "capabilities": {
          "portMappings": true
        },
        "type": "portmap"
      },
      {
        "backend": "",
        "type": "firewall"
      },
      {
        "domainName": "dns.podman",
        "type": "dnsname"
      }
    ]
  }
]
```

So the podman format for this is very obviously:

  {{ (index (index (index .plugins 0).ipam.ranges 0) 0).gateway }}
 2020-09-09 14:35:58 +02:00
Christian Kellner
dda203419c plugin/osbuild: adapt for composer API changes 
The task id can and must now be passed to composer via the `koji`
object in the `ComposeRequest`. Do that.
 2020-09-09 14:12:25 +02:00
Christian Kellner
c54a817da2 spec: package koji client plugin 
Add the new plugin to a new sub-package.
 2020-09-08 17:51:35 +02:00
Christian Kellner
49a6974e18 plugins/cli: initial version of the cli plugin 
Re-uses code from compose.py. Needs to be installed into
  /usr/lib/python3.$/site-packages/koji_cli_plugins
Yes indeed.
 2020-09-08 17:46:21 +02:00
Christian Kellner
4508a6f980 spec: only have one spec file 
To avoid confusion, lets only use the template and the out-generated
spec file.
 2020-09-08 17:10:02 +02:00
Christian Kellner
df628f26aa Support for generating rpms via meson 
Add a meson script that can be used to create rpms from the source
tree. It can also be used to install the plugins.
 2020-09-08 16:58:06 +02:00
Christian Kellner
8fb1342631 Make image_type and distro required arguments 
Distro, in composer terms "distribution", and "image_type" are
required for composer, so make that explicit everywhere in the
code.
 2020-09-08 16:13:48 +02:00
Christian Kellner
c735ebc6d0 container: support for koji web 
Install and configure koji web.
 2020-09-07 20:38:12 +02:00
Christian Kellner
c5db26596f plugin/builder: nicer errors on compose fail 
When creating the compose requests fails, catch it and transform
it into a koji.GenericError, which will avoid showing the full
backtrace in the UI.
 2020-09-07 20:35:32 +02:00
Christian Kellner
a9e187d38a plugin/hub: rename method to osbuildImage 
Remove the 'Test' suffix.
 2020-09-07 19:03:15 +02:00
Christian Kellner
cb7e4d76c5 plugins/builder: change location of config file 
Make the path reflect the name of the package / project.
 2020-09-07 18:39:13 +02:00
Christian Kellner
ccf6d60dac plugin/osbuild: simple whitespace changes 
No semantic change.
 2020-09-07 18:29:22 +02:00
Christian Kellner
0bbcef29e3 client.py: use compose.py instead 2020-09-07 18:28:47 +02:00
Christian Kellner
bfde0dfa14 spec: rename file to match the package name 2020-09-07 18:28:13 +02:00