The underlying filesystem was mounted in __init__ and unmonuted in
__exit__/__del__. This meant that if the same object was reused in
several `with` clauses, only the first one would work as intended.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Support the LOOP_SET_DIRECT_IO ioctl, which alows us to control
whether or not a loopback device should perform its own buffering
or rely on the one done by the underlying backing file.
Enabling this should improve both throughput and memory consumption,
it is not currently hooked up as more testing would be required.
Stop guessing if we're in the source directory by looking if a `stages`
subdirectory exists. Instead, assume that osbuild is installed on the
host.
If `--libdir` is given, mount the libdir into `/run/osbuild/lib` (alas,
we can't overwrite `/usr/libexec/osbuild`) and run osbuild from there.
Thus, running from source must now be done like this:
# python3 -m osbuild --libdir . [other args]
This really only makes sense if we are running systemd as PID1
inside the container, but we are not booting a system, just using
it as a glorified chroot.
This means entering the namespaces from the outside will be a bit
more cumbersome, but that was not used much and was never reliable
to begin with.
Signed-off-by: Tom Gundersen <teg@jklm.no>
loop.py is a simple wrapper around the kernel loop API. remoteloop.py
uses this to create a server/clinet pair that communicates over an
AF_UNIX/SOCK_DGRAM socket to allow the server to create loop devices
for the client.
The client passes a fd that should be bound to the resulting loop
device, and a dir-fd where the loop device node should be created.
The server returns the name of the device node to the client.
The idea is that the client is run from whithin a container without
access to devtmpfs (and hence /dev/loop-control), and the server
runs on the host. The client would typically pass its (fake) /dev
as the output directory.
For the client this will be similar to `losetup -f foo.img --show`.
[@larskarlitski: pylint: ignore the new LoopInfo class, because it
only has dynamic attributes. Also disable attribute-defined-outside-init,
which (among other problems) is not ignored for that class.]
Signed-off-by: Tom Gundersen <teg@jklm.no>
Add a directory to each BuildRoot potentially containing a set of
sockets. Also add a helper to create a named bound socket in a given
BuildRoot.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Compute a hash based on the content of a stage, together with the
hash of its parent stage.
The output of a pipeline is saved by the id of the last stage.
This is largely equivalent to the current logic, where it is the
pipeline that contains the id, but this means that the ids are
indepedent of how pipelines are split, the only thing that matters
is the sequence of stages, not whether or not they are in one or
several interdependent pipelines.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This removes the possibility of passing in arbitrary input data. We
now restrict ourselves to explicitly specified files/directories or
a base tree given by its pipeline id.
This drops the tar/tree stages/assemblers, as the tree/untree ones
are implicit in osbuild, and if we wish to also support compressed
trees, then we should add that to osbuild core as an option.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Whenever an assembler is not specified, the output tree is instead
saved to the content store, in a directory named after the pipeline
id.
This should render the io.weldr.tree assembler redundant.
In order to build the samples as before, specify the content store
as the input directory to build any pipeline that uses the
io.weldr.untree stage.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This uniquely identifies a pipeline based on its content. Pipelines
are considered equal modulo whitespace and the order of object
elements.
The intention is that two runs of a pipeline with the same id
generates functionaly equivalent ids. It is up to the writers
of stages and pipelines to ensure this property holds.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We wanted to force an empty output dir to avoid assembly stages using
previous output when creating their new one, and hence creating
dependencies between osbuild runs. We may still do that, but for now
let's remove the restriction as it seems rather arbitrary to protect
people from themselves to this extent.
Signed-off-by: Tom Gundersen <teg@jklm.no>
It's not really useful because it's at the wrong place, after a stage
has torn down all mounts. It also makes the code more complex for too
little benefit.
Don't print systemd-nspawn's messages about starting and stopping
containers.
Also supress a ldconfig warning and only show output from
systemd-sysusers when it fails.
All stages must be able to handle an input_dir argument, as we now
either pass it to all or none for agiven run. Simply set it to
'None' if it is not provided.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The new arguments are passed to the first, respectively last, stage
and are both directories. --input is read only and can be used to
initialize the first stage. --output is r/w and is where the final
stage should place the produced image.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Stages should be as stateless as possible. Don't provide an easy way out
of that.
Only the dnf stage used stage to save the dnf cache. That's only useful
during development and can be solved by pointing to a local repo mirror.
The output makes it hard to see which stage is currently processed and
how to enter the build container. Also, it doesn't include all relevant
logs.
Instead, stream log output into /tmp/output in the build container. Keep
outputting it to stdout, so that osbuild can collect it in the future.
Introduce `run-stage` script, which sets up the build environment before
running the stage. Run `ldconfig`, `systemd-sysusers`, and
`systemd-tmpfiles` in it.
