Follow the worker API so we standardise on one library. This simplifies
the code quite a bit.
No functional change.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This sets up containers running koji and supporting infrastructure, and
calls the osbuild-composer-koji API to build and image and push it into
our testing instance.
koji-compose.py and various fixes by Christian Kellner.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Introduce a target for Koji and hooked it up in the worker, so if koji
target is specified, the image is uploaded to koji.
[teg: use system kerberos config rather than reading from env]
Add a systemd socket for Koji API. If enabled when osbuild-composer.service
is started, the service will also listen on the socket and serve Koji API
there.
Note that Koji API doesn't upload to Koji yet, this still needs to be hooked
up.
Based on a patch from Tom Gundersen, thanks!
This just translates between the OpenAPI spec and our internal
API.
This still lacks tests, but a follow-up commit adds integration tests.
`internal/kojiapi/openapi.gen.go` was automatically generated from
`internal/kojiapi/openapi.yml`. To regenerate use `go generate ./...`.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Write an openapi spec for the worker API and use `deepmap/oapi-codegen`
to generate scaffolding for the server-side using the `labstack/echo`
server.
Incidentally, echo by default returns the errors in the same format that
worker API always has:
{ "message": "..." }
The API itself is unchanged to make this change easier to understand. It
will be changed to better suit our needs in future commits.
vCenter requires images to be uploaded as vmdk StreamOptimized. Lorax
always produced images on this format, so we should make sure to do the
same for our VMWare images.
Allow LocalTarget to request the images produced by osbuild be converted
to be streamOptimized before saving in composer, and hook the weldr API
up to enable this option for vmdk images.
Ideally this should simply be an option in osbuild, but that would
require some more work, which we will not manage in time for RHEL8.3.
Therefore do this minimal fix.
Note that that means the images produced by our manifests (including in
our image-test test cases) are not on the format that the weldr API
returns, so the tests we run on them would also, for now, need to
convert before uploading to vCenter.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This is not shipped in RHEL, so use the library directly to query the IP
address. This is a massive hack, but let us revisit this after the next
release.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The osbuild-composer-rcm package was never finished, not in use and will be replaced by osbulid-composer-koji.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The composer-tests use systemd-nspawn, so make sure the `*-tests` RPM
depends on `systemd-container`.
Right now this dependency is recursively fulfilled by the `osbuild`
RPM. However, this might change if we switch to `bubblewrap` as a
sandbox. Furthermore, we should not fulfill our dependencies
recursively, but list them directly.
Currently, RHEL 8.3 ships lorax-composer 28.14.49, which we want to
replace. 28 <= 28.14.49, so this did not work, bump it to 29, so we do
not get tripped up by bugfix releases of lorax-composer.
In the event that this obsoletes should one day be reverted, we do not
bump it to "infinity", so a sufficiently new version of lorax, would still
be able to obolte osbuild-composer again.
Signed-off-by: Tom Gundersen <teg@jklm.no>
osbuild-composer is a drop-in replacement of lorax-composer, and
side-by-side installation is not supported. It is still possible to
explicitly install lorax-composer and exclude osbuild-composer, but the
default is to replace lorax with osbuild on upgrades.
This change is for now only for RHEL8.3, but we should consider doing
the same for F33.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1836844
Signed-off-by: Tom Gundersen <teg@jklm.no>
The subpackages are not really meant to be used with a different version of
the osbuild-composer package. This commit enforces.the usage of a matching
version in the spec file.
RHEL 8 doesn't have golang in the i686 buildroot anymore. Fedora doesn't have
i686 images and kernel anymore. Also, osbuild-composer doesn't have support
for building i686.
Let's exclude i686 for these reasons, it's dead.
The new Fedora IoT image type uses org.osbuild.ostree.commit assembler with
tar option that was introduced in osbuild 15, therefore the dependency version
must be bumped.
This also bumps the submodule to 9cbedc04. That's basically version 15 of
osbuild with one fix related to secrets.
Currently, removing osbuild-composer produces these warnings:
Failed to stop osbuild-worker@.service: Unit name
osbuild-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-worker@.service' for details.
Failed to stop osbuild-remote-worker@.service: Unit name
osbuild-remote-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-remote-worker@.service' for
details.
%systemd_preun calls on a package upgrade this:
systemctl --no-reload disable --now [ARGS...]
I tried using a wildcard:
systemctl --no-reload disable --now "osbuild-worker@*.service"
But this gives:
Invalid unit name "osbuild-worker@*.service" was escaped as
"osbuild-worker@\x2a.service" (maybe you should use systemd-escape?)
Failed to disable unit: Unit file osbuild-worker@\x2a.service does not exist.
In the end I decided to use two commands - disable and stop. Disable works
without a wildcard, stop requires one. This solution should mimic systemctl
disable --now pretty well.
Currently, upgrading osbuild-composer produces these warnings:
Failed to try-restart osbuild-worker@.service: Unit name
osbuild-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-worker@.service' for details.
Failed to try-restart osbuild-remote-worker@.service: Unit name
osbuild-remote-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-remote-worker@.service'
for details.
%systemd_postun_with_restart calls on a package upgrade this:
systemctl try-restart [ARGS...]
However, try-restart requires using a wildcard when restarting all
the services created from a template unit. This commit therefore adds the
wildcards.
Previously, vhd images were tested using QEMU. This commit changes that to
boot them in the actual Azure infrastructure.
Azure VMs have quite a lot of dependencies - a network interface, a virtual
network, a network security group, a public ip address and a disk. Azure CLI
and Azure Portal handle the creation of all these resources internally.
However, when using the API, the caller is responsible to create all these
resources before creating an actual VM.
To handle the creation of all the resources in the right order, a deployment
is used. A deployment is a set of resources defined in a JSON document.
It can optionally take parameters to customize each deployment. After the
deployment is finished, the VM is up and ready to be tested using SSH.
Sadly, the deployments are a bit hard to clean-up. One would expect that
deleting a deployment removes all the deployed resources. However, it doesn't
work this way and therefore it's needed to clean up all resources "manually".
For this reason, our deployment sets a unique tag on all the resources created
by the deployment. After this test is finished, the API is queried for all
the resources with the tag and then, they're deleted in the right order.
We will soon need the support for vhdx format, which is supported since
osbuild 12.
This commit bumps the dependency in the spec file and also updates the
submodule.
The osbuild-image-tests don't currently support boot test for any
alternative architecture because the qemu-system-x86_64 command is
hardcoded. This patch introduces a branch specific to aarch64, but
without a KVM support as I was unable to make it run in Beaker, which is
currently the only offering we have with ARM machines. As a workaround
the boot tests will be skipped if kvm kernel module is not found and only
image-info tests will run.
The newest osbuild is needed for support for aarch64. The aarch64 images
are using GPT which requires stable partuuid which was included in the
latest osbuild release (11). This will be used to produce stable
image-info tests.
On Fedora, the `%gobuild` macro turns off go modules and sets gopath to
installed libraries (in `/usr/share/gocode`).
When building the test binaries, we used `go test -c` directly, which
downloaded the libraries into GOPATH. That's wasteful and doesn't work
in mock (without internet). Replicate what `%gobuild` does by setting
`GO111MODULE=off` and `GOPATH` to the correct value.
RHEL uses modules as intended. No change is necessary.
With this change the integration tests can now also be run as unit tests
against the mocked server. The way it works is this:
internal/client/unit_test.go sets up the mock server and is built
when the `integration` build tag is *not* included.
internal/client/integration_test.go sets up the connection to an
existing server and is built when the `integration` build tag *is*
included.
The test code is built and run for both cases.
Currently they all pass for the integration test run. The unit test
cases need some work because the mocked server isn't a real server with
real depsolving and package lists. A future commit will fix this.
The packaging guidelines requires that these provides and obsoletes stays
in the spec file for 2 fedora versions. The name change was introduced in
Fedora 32, there the lines should be removed in F34.
We successfully renamed the package in Fedora to osbuild-composer
(RHBZ #1815577). This commit adds the new spec file.
Changes: the tests subpackage is not built by default, to build it
--with tests parameter must be passed to rpmbuild.
Let's hope both spec files stay in sync.
