particle-os/debian-koji-osbuild
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
227 commits 1 branch 0 tags 1.8 MiB
Commit graph

227 commits

This branch
This branch
All branches
Author SHA1 Message Date
Christian Kellner
d5a80cf3cf test: manually install container-selinux 
Seems to not be pulled in. Don't disable selinux.
 2020-09-10 20:12:39 +01:00
Christian Kellner
b292458771 test: create a compose via koji 
Try testing a compose via the koji, which involves creating using
the koji command line plugin to make the XMLRPC call to koji hub,
where the osbuild koji hub plugin verifies the parameters and then
creates the task. The osbuild koji plugin for the builder is then
picking up the task, and uses composer's koji API to request a
compose. Once this is successful it will be imported by composer
into koji via the 'CGImport' method.
The `koji osbuild-image` command waits for all this and reports
whether the task was successful or not via its exit code (and
on stdout).

This uses a fleet of containers: a database one, a kerberos kdc
one, another one for the koju hub and finally one for the koji
builder. The pre-build RPMs are used to install the plugins.

NB: On RHEL we need to manually install the `dnsname` podman
plugin, since it is missing, but required so that containers
can address each other by hostnames.
See [schutzbot/vendor/README.md](schutzbot/vendor/README.md)
 2020-09-10 18:41:13 +01:00
Christian Kellner
3fdf66a61c run-builder: use jq instead of inspect --format 
Because podman on RHEL 8.2 does not understand --format, so we
basically do the same now via 'jq'.
 2020-09-10 18:41:13 +01:00
Christian Kellner
dd2d3cf0a9 run-builder: argument checking 
Check we have at least one positional argument, and it is either
"start", "stop" or "fg".
 2020-09-10 18:41:13 +01:00
Christian Kellner
928d2278be plugin/cli: return task result when waiting 
When waiting for the task, return the result from the `watch_task`
call, which will indicate if the task was successful. This will
then be reflected in the error code of the koji call.
 2020-09-10 18:41:13 +01:00
Christian Kellner
1db41a067f run-koji-container: be more verbose 
Show logs and running containers, should help track down issues.
 2020-09-10 18:41:13 +01:00
Christian Kellner
10e691d8c3 run-builder: support background execution 
The run-builder script now takes a command line argument, which is
either
  start - run the container in the background
  stop  - stop the running container
  fg    - run the container in the foreground (old behavior)

This should prove useful in CI.
 2020-09-10 18:41:13 +01:00
Christian Kellner
1effdc7a2c container/hub: sync with script via file 
Create a file in the hub container the share directory that will
signal to the outside that we are done with the setup.
 2020-09-10 18:41:13 +01:00
Christian Kellner
7a366eca7a run-koji-container: don't fail if share dir exists 
We need to drop in the RPMs into the share-dir before the hub
container is started, so don't fail if the directory already
exists.
 2020-09-10 18:41:13 +01:00
Christian Kellner
2682911556 kerberos: add dns realm map for osbuild-local.conf 2020-09-10 18:41:13 +01:00
Christian Kellner
b778133e89 container: support installing via RPMs 
If the share dir contains rpms in /share/rpms, install those,
instead of using the plugin from the container.
 2020-09-10 18:41:13 +01:00
Christian Kellner
af4e66d2b2 plugin/hub: use jsonschema to validate input 
Use jsonschema to validate the input to the XMLRPC call to catch
mistakes early, i.e. before creating the task.
 2020-09-10 18:41:13 +01:00
Christian Kellner
3dc463c8dd plugin/cli: pretty-print the options dict 
Also show the options that we are using.
 2020-09-10 18:41:13 +01:00
Christian Kellner
60516fc811 gitignore: remove old ssl dirs, add build/ 
The old container related ssl dir ignores are no longer necessary.
Add 'build/' for RPM builds though.
 2020-09-10 18:41:13 +01:00
Tom Gundersen
a416570ea2 schutzbot: add CI integration 
The CI is in two stages, for each supported distro.

First the RPMs are generated from the spec file in the repo for the
given distro and architecture.

Once all the RPM builds have succeeded successfully, a test machine is
provisioned with osbulid-composer installed, and koji API enabled.

The repository containing the RPMs of the code being tested is also
enabled on the test machine, and the cli client is installed.

Finally, the test/integration.sh script is executed, which currently
does nothing.

Signed-off-by: Tom Gundersen <teg@jklm.no>
 2020-09-10 00:05:44 +02:00
Tom Gundersen
ec033ed623 spec/cli: depend on koji 
This should depend on the cli client, which is shipped in the koji package,
not the builder.

Signed-off-by: Tom Gundersen <teg@jklm.no>
 2020-09-10 00:05:44 +02:00
Tom Gundersen
d2e81fbf79 spec: don't use pycached macro 
This does not work on RHEL8.2.

Signed-off-by: Tom Gundersen <teg@jklm.no>
 2020-09-10 00:05:44 +02:00
Christian Kellner
857ae4ad77 README.md: add general description 2020-09-09 16:12:26 +02:00
Christian Kellner
3f5de8a68d plugin/cli: eager check of target input 
Check the target with name exists and has a destination tag, so
we can abort early.
 2020-09-09 15:56:35 +02:00
Christian Kellner
3934382337 plugin/cli: honor global --quiet flag 
If --quiet was requested globally, then don't print anything.
 2020-09-09 15:51:11 +02:00
Christian Kellner
8099d79aa1 plugin/cli: support --wait, --nowait option 
Can be used to explicitly opt in or out of waiting for the task to
finish. The help text is taken from the standard koji client.
 2020-09-09 15:28:36 +02:00
Christian Kellner
79387f1f2c plugins/cli: use upstream description help text 
For --repo and --release, use the help text from koji build-image.
 2020-09-09 15:07:36 +02:00
Christian Kellner
81aebbc681 plugin/cli: use koji_cli.lib.OptionParser 
Use koji's OptionParser instead of argparse to be better aligned
with the formatting of standard koji options.
 2020-09-09 15:06:53 +02:00
Christian Kellner
6087e2d90e run-builder: inspect network to get gateway IP 
Instead of inferring the gateway ip via the the network device for
the podman bridge, where the name can change, use podman network
inspect.

```js
The json looks like this:
[
  {
    "cniVersion": "0.4.0",
    "name": "org.osbuild.koji",
    "plugins": [
      {
        "bridge": "cni-podman1",
        "hairpinMode": true,
        "ipMasq": true,
        "ipam": {
          "ranges": [
            [
              {
                "gateway": "10.89.0.1",
                "subnet": "10.89.0.0/24"
              }
            ]
          ],
          "routes": [
            {
              "dst": "0.0.0.0/0"
            }
          ],
          "type": "host-local"
        },
        "isGateway": true,
        "type": "bridge"
      },
      {
        "capabilities": {
          "portMappings": true
        },
        "type": "portmap"
      },
      {
        "backend": "",
        "type": "firewall"
      },
      {
        "domainName": "dns.podman",
        "type": "dnsname"
      }
    ]
  }
]
```

So the podman format for this is very obviously:

  {{ (index (index (index .plugins 0).ipam.ranges 0) 0).gateway }}
 2020-09-09 14:35:58 +02:00
Christian Kellner
dda203419c plugin/osbuild: adapt for composer API changes 
The task id can and must now be passed to composer via the `koji`
object in the `ComposeRequest`. Do that.
 2020-09-09 14:12:25 +02:00
Christian Kellner
c54a817da2 spec: package koji client plugin 
Add the new plugin to a new sub-package.
 2020-09-08 17:51:35 +02:00
Christian Kellner
49a6974e18 plugins/cli: initial version of the cli plugin 
Re-uses code from compose.py. Needs to be installed into
  /usr/lib/python3.$/site-packages/koji_cli_plugins
Yes indeed.
 2020-09-08 17:46:21 +02:00
Christian Kellner
4508a6f980 spec: only have one spec file 
To avoid confusion, lets only use the template and the out-generated
spec file.
 2020-09-08 17:10:02 +02:00
Christian Kellner
df628f26aa Support for generating rpms via meson 
Add a meson script that can be used to create rpms from the source
tree. It can also be used to install the plugins.
 2020-09-08 16:58:06 +02:00
Christian Kellner
8fb1342631 Make image_type and distro required arguments 
Distro, in composer terms "distribution", and "image_type" are
required for composer, so make that explicit everywhere in the
code.
 2020-09-08 16:13:48 +02:00
Christian Kellner
c735ebc6d0 container: support for koji web 
Install and configure koji web.
 2020-09-07 20:38:12 +02:00
Christian Kellner
c5db26596f plugin/builder: nicer errors on compose fail 
When creating the compose requests fails, catch it and transform
it into a koji.GenericError, which will avoid showing the full
backtrace in the UI.
 2020-09-07 20:35:32 +02:00
Christian Kellner
a9e187d38a plugin/hub: rename method to osbuildImage 
Remove the 'Test' suffix.
 2020-09-07 19:03:15 +02:00
Christian Kellner
cb7e4d76c5 plugins/builder: change location of config file 
Make the path reflect the name of the package / project.
 2020-09-07 18:39:13 +02:00
Christian Kellner
ccf6d60dac plugin/osbuild: simple whitespace changes 
No semantic change.
 2020-09-07 18:29:22 +02:00
Christian Kellner
0bbcef29e3 client.py: use compose.py instead 2020-09-07 18:28:47 +02:00
Christian Kellner
bfde0dfa14 spec: rename file to match the package name 2020-09-07 18:28:13 +02:00
Christian Kellner
faea80f73b spec: rename the package 
Rename the package to `koji-osbuild`, which follows the same scheme
as the OSBS one (`koji-containerbuild`).
The upstream repo has been changed accordingly as well.
 2020-09-07 18:26:46 +02:00
Christian Kellner
b94a4b398d compose: support --release 
Use this to explicitly set the release.
 2020-09-07 18:26:21 +02:00
Christian Kellner
0783e7491e compose: add distro argument 
The plugin expects this, or it falls back to name-version, which is
not the right thing in most cases.
 2020-09-07 18:25:27 +02:00
Christian Kellner
3ef4d5e566 plugin/osbuild: make release optional 
If release was not in `opts`, use `session.getNextRelease` to
obtain the next release.
 2020-09-07 18:23:28 +02:00
Christian Kellner
dbf2960792 plugins/osbuild: sync with composer's koji API 
Send the NVR as part of the compose request. The result now also
contains the koji build id. The distro argument is taken from
the options.
 2020-09-07 18:15:30 +02:00
Christian Kellner
750bf5421b container/builder: configure plugin via cfg file 
Matches the config of the containers.
 2020-09-06 20:11:37 +02:00
Christian Kellner
d31cadd304 plugins/builder: make urls configurable 
Can't really hardcode those.
 2020-09-06 20:10:59 +02:00
Christian Kellner
5eeb251882 spec: add spec file draft 
Initial attempt at packaging.
 2020-09-06 16:15:47 +02:00
Christian Kellner
a17b683f03 LICENSE: add ASL 2.0 2020-09-06 16:02:54 +02:00
Christian Kellner
5173d697e5 README.md: add development links 2020-09-05 19:38:30 +02:00
Christian Kellner
0749472d8c README.md: add make-tags.sh instructions 2020-09-05 19:34:07 +02:00
Christian Kellner
318c53d84b README.md: add compse.py instructions 2020-09-05 19:31:35 +02:00
Christian Kellner
f58ecb97ac plugin/builder: support repo overwrites 
Allow the repository information to be overwritten by the user
via opts.
 2020-09-05 19:28:17 +02:00