./test/run test suite has served us well over the last months. However,
there is currently a major effort to run the better defined integration
test suite on a CI. Nonetheless, two very important parts are still missing
from the integration test suite: inspecting the image with image-info
and booting the image. This commit begins the work on this matter by porting
a part of ./test/run suite to Go. Currently, only image-info tests work, the
rest will come in the following commits.
comparing to lorax-composer test suite only ext4-filesystem and
partitioned-disk are built without asserting anything other than
the build succeeds. For the rest of the images we usually try to
boot them and verify the resulting VM works somehow.
Right now the implementation expects the RCM socket to live in the same
unit file as other osbuild-composer sockets. This would require a
solution where we ship the osbuild-composer.socket in two different
versions: one for regular usage, one for rcm. But that is very
inconvenient and it would probably require some weird scriptlets (and
scriptlets are bad!).
After this change, the RCM API socket lives in a separate file and only
if the socket unit is activated, the API runs. The unit file itself was
introduced in previous commits.
There's a usecase for running workers at a different machine than
the composer. For example when there's need for making images for
architecture different then the composer is running at. Although osbuild has
some kind of support for cross-architecture builds, we still consider it
as experimental, not-yet-production-ready feature.
This commit adds a support to composer and worker to communicate using TCP.
To ensure safe communication through the wild worlds of Internet, TLS is not
only supported but even required when using TCP. Both server and client
TLS authentication are required. This means both sides must have their own
private key/certificate pair and both certificates must be signed using one
certificate authority. Examples how to generate all this fancy crypto stuff
can be found in Makefile.
Changes on the composer side:
When osbuild-remote-worker.socket is started before osbuild-composer.service,
osbuild-composer also serves jobqueue API on this socket. The unix domain
socket is not affected by this changes - it is enabled at all times
independently on the remote one. The osbuild-remote-worker.socket listens
by default on TCP port 8700.
When running the composer with remote worker socket enabled, the following
files are required:
- /etc/osbuild-composer/ca-crt.pem (CA certificate)
- /etc/osbuild-composer/composer-key.pem (composer private key)
- /etc/osbuild-composer/composer-crt.pem (composer certificate)
Changes on the worker side:
osbuild-worker has now --remote argument taking the address to a composer
instance. When present, the worker will try to establish TLS secured TCP
connection with the composer. When not present, the worker will use
the unix domain socket method. The unit template file osbuild-remote-worker
was added to simplify the spawning of workers. For example
systemctl start osbuild-remote-worker@example.com
starts a worker which will attempt to connect to the composer instance
running on the address example.com.
When running the worker with --remote argument, the following files are
required:
- /etc/osbuild-composer/ca-crt.pem (CA certificate)
- /etc/osbuild-composer/worker-key.pem (worker private key)
- /etc/osbuild-composer/worker-crt.pem (worker certificate)
By default osbuild-composer.service will always spawn one local worker.
If you don't want it you need to mask the default worker unit by:
systemctl mask osbuild-worker@1.service
Closing remarks:
Remember that both composer and worker certificate must be signed by
the same CA!
This is needed for unit tests, because it wasn't possible to mock the
rpmmd module before. This also requires that the checksum is moved to
the compose request and evaluated in the endpoint handler instead of
push compose. I think it makes sense to have the checksum in the compose
request directly.
Also a "module platform ID" is required now, but we don't have the
"global" distribution any more, so this patch introduces mapping from a
distribution to the module platform ID.
the name was misleading because the function could do more than just
download package list. In PushComposeRequest it is also used to fetch
checksums for the repositories, therefore I decided to rename it to
reflect this usage.
When creating a pipeline with the default image size, the size should no
longer be set to 0. Instead, the size is fetched using the distro
function GetSizeForOutputType which can return the default image size
for a given image type. This size can then be passed into the pipeline.
This is unused for now, but will allow us to generate pipelines with
the pre-depsolved NEVRAs, so osbuild does not need to depsolve again.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This will detect inconsistent blueprints, and in the future will
allow us to use the returned packages for generating the pipeline.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We must avoid depending on the host's state in any way. This achieves
isolation in the following ways:
- rather than the default config file /dev/null is used
- rather than sharing the host persistent state dir a temporary one
is used and thrown away for each call
- the module_platform_id is set explicitly per supported distro, rather
than taken from /etc/os-release.
Optionally, the cache directory can be configured, as we may want to keep
this separate from the host, if for no other reason than accounting.
However, the cache appears to be well-behaved, so we can keep sharing
it between calls (or even with the host). This speeds up things
considerably, so this is definitely what we want.
Signed-off-by: Tom Gundersen <teg@jklm.no>
In our base distro definitions we exclude packages in addition to
including them. Extend dnf-json to support this, so we can depsolve
the base package set as well as the packages added in blueprints.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Prior this commit local target copied the image from a worker to a composer
using cp(1) command. This prevented the local target to work on remote
workers.
This commit switches the local target implementation to using the jobqueue
API introduced in the previous commit. I had some concerns about speed
of this solution (imho nothing can beat pure cp(1) implementation) but
ad hoc sanity tests showed the copying of the image using the jobqueue API
when running the worker on the same machine as the composer is still
more or less instant.
Imho it makes more sense from REST perspective. Also, in the future there
will be ROUTE for uploading image to image build. As it's not a good idea
transport file inside JSON, all the parameters (compose id and image
build id) need to be inside the URL. Therefore for the sake of consistency,
all these routes should have compose id and image build id in the URL.
There is another solution to embedding multiple values inside http body
which allows file transport - multipart/form-data. I think using form-data
is worth when doing more complex stuff, for our usecase transporting all
the metadata in the URL is more appropriate solution.
Everything that this field contained can be computed in another way:
- path: just lookup the local target and read the path from there
- mime: can be derived from distribution and compose output type
- size: can be derived from the path
Therefore it imho doesn't make much sense to store these information multiple
times.
In the future remote workers will be introduced. Obviously, the remote worker
cannot support the local target. Unfortunately, the current implementation of
storing the osbuild result is dependant on it.
This commit moves the responsibility of storing osbuild result to the
composer process instead of the worker process. The result is transferred from
a worker to a composer using extended HTTP API.
We treat the osbuild-pipeline as a tool run only within the source, therefore
we should search for the repository configs only in the current directory.
In the current state, osbuild-pipeline exits with random golang error,
such as goroutine failed, which is not at all helpful. This PR
introduces CLI arguments validation and helful error messages that use
the newly introduced types so that we don't waste time guessing what
was the right way to invoke this tool.
When creating a pipeline the assembler includes an image size. This
image size can be set when creating the pipeline but if it is 0 then a
default image size will be used. The default is 2 GB except for ami
images which are 6 GB.
Osbuild-composer expects two or three listeners and fails if there is
an unexpected number of listening sockets. Checking if there are not two
or if there are not three listeners always returns true even if there are
the desired number of listeners. Therefore, osbuild-composer always
crashes. The check now only crashes if there are a number of listeners
other than 2 or 3.
It contains two basic endpoints:
* POST /v1/compose
* GET /v1/compose/<uuid>
It passes all the tests, but cannot be used for the intended use case
because the store API does not (yet) support distributions and
architectures as a parameters.
Introduce a new osbuild-tests command and ship it in the -tests
sub-package.
The intended usecase is to install the -tests subpackage into an
otherwise pristine VM, and call the osbuild-tests binary over ssh
from the outside of the booted VM. If the binary exits with a return
code of 0, the tests passed, otherwise they failed. The VM should
not be reused after running the tests.
Signed-off-by: Tom Gundersen <teg@jklm.no>
During development of a new distro, we need to test composer against
nightly or beta repositories, but we cannot ship composer itself
with the nightly repository information hardcoded in. At the same
time, we want to distinguish between the system repositories of the
host and the repositories we use to generate images (the host may not
use the same distro/version/architecture as the target, and it may
include custom repositories that the target should not).
We therefore ship per distro repository information that can be
overriden (typically in testing) by dropping files in /etc.
For now use the latest nightlies for RHEL-8.2, we may want to
replace these with the official mirrors for GA eventually.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Introduce a DistroRegister object. For now this does not introduce
any functional changes, as the object is always instantited to be
the same. However, in follow-up patches it will get options.
Signed-off-by: Tom Gundersen <teg@jklm.no>
dnf-json relies on dnf's ability to cache repository metadata. This is
important, because the API calls it quite often to serve requests for
package lists and depsolves.
However, osbuild's dnf stage always fetches new metadata, because it
doesn't have access to the host's cache. Since metadata is valid for
some time, even after a repository changed, the checksum we put in
the pipeline might be old.
Force a new metadata download when producing the pipeline. This is still
not perfect, but greatly reduces the probability of putting stale
metadata into the pipeline.
Allow bootloader specific packages to be defined per architecture,
and allow repositories to depend on the architecture.
This does not altert he pipelines we produce, part from the ami
image now contains the grub2-pc package, rather than the grub2
package. This should make no difference.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Only panic on compile-time errors (e.g., built for unsupported
architecture). Otherwise, use log.Fatalf(), which is equivalent to
printing and exiting with return code 1. Only ever do this from
main(), in all other cases pass on the error object.
This is mostly relevant when the server disconects, in which case
we'll get EOF, and will now restart cleanly instead of panicing.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The pipeline generation now takes the architecture as an argument.
Currently only x86_64 is supported. The architecture is detected
at start-up, and passed down to each pipeline translation.
For osbuild-pipeline we now requrie the architecture to be passed
in.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Instead of having a static repository checksum, set it dynamically from
the metadata that osbuild-composer last saw. This is implemented in
dnf-json, which returns the checksums for each repository on every call.
This enables the use of repositories that change over time, such as
fedora-updates. Note that the osbuild pipeline will break when such a
repository changes. This is intentional: pipelines have to be
reproducible.
Prior to this commit outputs directory used by local target was owned by root.
This made impossible for osbuild-composer to delete images. (osbuild-composer
doesn't run as root).
This commit introduces state directory in which osbuild-composer creates
outputs directory. Because this directory is owned by osbuild-composer, it's
able to delete files inside.
As a part of f4991cb1 ComposeEntry struct was removed from store package.
This change made sense because this struct is connected more with API than
with store - store uses its own Compose struct. In addition, converters
between Compose and ComposeEntry were added. Unfortunately, ComposeEntry
contains ImageSize which was not stored in Compose but retrieved from store
using GetImage method. This made those converters dependent on the store,
which was messy.
To solve this issue this commit adds image struct into Compose struct.
The content of image struct is generated on the worker side - when the worker
sets the compose status to FINISHED, it also sends Image struct with detailed
information about the result.
Make osbuild-composer use FromHost() directly. Everywhere else needs to
specify the distro explicitly.
Also don't panic when a distro doesn't exist. Instead, return nil. Make
sure all callers check for that.
Automatically registering on `init()` is clever, but a bit too magical
and easy to get wrong, because every binary must include all distros
somewhere.
Flip this inside out: distros now have a `New()`, which returns
something that implements the `Distro` interface. The distro package
explicitly creates all of them.
This means that distros cannot import distro itself anymore, because go
forbids import cycles. This only affected `InvalidOutputFormatError`.
Return a generic error for now.
Return the error code of the osbuild run, and an array of errors,
one for each target provided. If a target fails, all other targets
are still attempted.
If either osbuild or one of the targets retursn an error, the worker
notifies osbuild-composer that the job failed.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This takes a different approach to outputs and customizations, which is
much shorter and duplicates less code.
This uses links to internal repositories for now, because 8.2 hasn't
been released yet.
Add a `distro` flag to `osbuild-pipeline`.
