systemd >= 240 sets this variable to `/var/cache/` + the value of
CacheDirectory. osbuild-composer must run on earlier versions though
(specifically RHEL 8.2).
Without passing in a cachedir, dnf would create a random one for every
invocation. This meant that caches were never reused, nor cleaned up
properly.
Let systemd create a cache directory for us in /var/cache/ and use
that via the environment variable systemd sets for us.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This is not currently used. Drop it for now, until we use it, and
when we do, it should be reintroduced with the right name, to avoid
clashing with osbuild-composer (they are owned by different users,
so cannot be shared).
Signed-off-by: Tom Gundersen <teg@jklm.no>
There's a usecase for running workers at a different machine than
the composer. For example when there's need for making images for
architecture different then the composer is running at. Although osbuild has
some kind of support for cross-architecture builds, we still consider it
as experimental, not-yet-production-ready feature.
This commit adds a support to composer and worker to communicate using TCP.
To ensure safe communication through the wild worlds of Internet, TLS is not
only supported but even required when using TCP. Both server and client
TLS authentication are required. This means both sides must have their own
private key/certificate pair and both certificates must be signed using one
certificate authority. Examples how to generate all this fancy crypto stuff
can be found in Makefile.
Changes on the composer side:
When osbuild-remote-worker.socket is started before osbuild-composer.service,
osbuild-composer also serves jobqueue API on this socket. The unix domain
socket is not affected by this changes - it is enabled at all times
independently on the remote one. The osbuild-remote-worker.socket listens
by default on TCP port 8700.
When running the composer with remote worker socket enabled, the following
files are required:
- /etc/osbuild-composer/ca-crt.pem (CA certificate)
- /etc/osbuild-composer/composer-key.pem (composer private key)
- /etc/osbuild-composer/composer-crt.pem (composer certificate)
Changes on the worker side:
osbuild-worker has now --remote argument taking the address to a composer
instance. When present, the worker will try to establish TLS secured TCP
connection with the composer. When not present, the worker will use
the unix domain socket method. The unit template file osbuild-remote-worker
was added to simplify the spawning of workers. For example
systemctl start osbuild-remote-worker@example.com
starts a worker which will attempt to connect to the composer instance
running on the address example.com.
When running the worker with --remote argument, the following files are
required:
- /etc/osbuild-composer/ca-crt.pem (CA certificate)
- /etc/osbuild-composer/worker-key.pem (worker private key)
- /etc/osbuild-composer/worker-crt.pem (worker certificate)
By default osbuild-composer.service will always spawn one local worker.
If you don't want it you need to mask the default worker unit by:
systemctl mask osbuild-worker@1.service
Closing remarks:
Remember that both composer and worker certificate must be signed by
the same CA!
Image generation is very much a batch job, latency is not important
and because it is so CPU and IO intensive, it really affects the
overall responsiveness of the system if left on the default settings.
Set scheduling to idle/batched for the worker. The composer service
is still left in the normal scheduling classes as its latency is
important for the UI.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The intention is for the sockets always to stay around, even when
the service is restarted. This way the worker can always connect.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Rather than enabling the worker when composer is enabled, make sure
it is started whenever composer is.
This means that one worker instance is always available, and more can
be enabled if required. They are then also started whenever composer
is.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This makes no difference, so let's just put them where the Fedora
guidelines say they should be.
Also, make sure to own the containing directory.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Unify into one make file, with two targets:
build:
This unconditionally builds the binaries that wolud be shipped in the package,
this is marked as a phony target, and relies on the go compiler to know what
is up-to-date and what needs to be rebuilt.
install:
This installs binaries and config files into the file system and runs any
scripts to set things up correctly, as if the rpm had been installed.
This drops the old `run` target, which could still be done from the commandline,
but we want to be able to rely on systemd features, so it is not something we
want to test or support.
To run osbuild-composer from a checkout one should now do:
```
$ make build
$ sudo make install
```
This has the same effect as installing the RPM would have, starting and stopping
the sokcets/services is still left to the user.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This directory is currently only used for unix-domain sockets but they are created by systemd with the socket unit and therefore this line is redundant and maybe even causes problems.
We want the sockets to live even when the service is down (either because of error or intentional suspension) and it seems that setting the RuntimeDirectory to the directories that contain the sockets makes systemd remove both directories and unlink the sockets when the service fails and that is exactly what we don't want.
osbuild-composer now uses socket activation instead of hardcoded paths
in the code. osbuild-worker is an http client therefore it uses only
service unit. osbuild-worker must be started after the socket is
created. osbuild-composer service requires osbuild-worker to run, because without
it no jobs can be started.
osbuild-composer is executed as a regular user (newly created
_osbuild-composer user) as opposed to the worker which must run as root
in order to execute osbuild itself
