Commit graph

1783 commits

Author SHA1 Message Date
Christian Kellner
d6c421faf3 loop: add LoopInfo.is_bound_to helper
Add a helper that will check if the loop devices is backed by
the file identified via the stat(2) result, i.e. the inode on
the correspoding device.
Add a correspoding test for the new helper.
2021-08-11 20:59:51 +02:00
Christian Kellner
61d5f15420 loop: add autoclear propert to LoopInfo
Small convenience property to check if the autoclear flag is set
in the `lo_flags` member of the `LoopInfo`.
Also add a corresponding test for it.
2021-08-11 20:59:51 +02:00
Christian Kellner
62082733e9 loop: don't leak dir_fd for LoopControl
If `dir_fd` is not passed into the constructor of LoopControl,
"/dev" will be opened, but it was not closed and thus would
leak the fd.
2021-08-11 20:59:51 +02:00
Christian Kellner
73f24c68a2 loop: add get_status method
Implement a `Loop.get_status` method, to get the properties of the
loop device, corresponding to LOOP_GET_STATUS64, and counterpart
to the existing `Loop.set_status` method. Use the new `get_status`
call in the `set_status` call, replacing the existing code that
does the same thing.
Add a basic test for the `get_status` method. Also fix an actual
leak, where the loop device was closed but the fd was not cleared
inside the test.
2021-08-11 20:59:51 +02:00
Christian Kellner
2c64c65608 test/assembler: use new loop_for_fd helper
Use the new `loop_for_fd` helper that got added to `LoopControl`,
which is basically a version of the existing code.
2021-08-11 20:59:51 +02:00
Christian Kellner
6293da5874 devices/loopback: use new loop_for_fd helper
Use the new `LoopControl.loop_for_fd` helper, which is basically
a modified version of the existing code in `make_loop`.
2021-08-11 20:59:51 +02:00
Christian Kellner
d5ded743bd test/mod: add checks for loop module
Add some basic checks for the loop module, including the new
`loop_for_fd` helper method.
2021-08-11 20:59:51 +02:00
Christian Kellner
3c953ae49d loop: helper to get a loop and bind it
Add a new helper, `loop_for_fd` that will get (or create) an
unbounded loop device, bind it to an fd and then set its
status. Since this is racy and can fail the method does these
steps in a retry-loop.
2021-08-11 20:59:51 +02:00
Christian Kellner
099cfbcea1 remoteloop: close loop device controller
When cleaning up the `RemoteLoop` API instance, close the loop
controller in addition to closing all the open loop devices.
2021-08-11 20:59:51 +02:00
Christian Kellner
3d08e84ceb devices/loopback: close loop device controller
In the `close` method of the device also close the loop controller.
2021-08-11 20:59:51 +02:00
Christian Kellner
82ecc530a1 loop: ability to close the loop device controller
Add a `close` method to the loop controller class `LoopControl` since
it actually opens a file descriptor, which should be closed once the
loop controller is no longer needed.
Assert that the controlling file descriptor is open for all methods
that require this.
2021-08-11 20:59:51 +02:00
Christian Kellner
234997eeb3 stages/grub2.inst: fix prefix for dos layouts
When the partition layout is `dos` or `mbr`, the correct name for
it in the prefix is `msdos`. The function to convert the option
to the label already existed but was not used. Fix it by actually
using said function.

Reported-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-10 20:19:06 +02:00
Christian Kellner
569697e9a9 stages: add org.osbuild.untar
Add a new stage that can be used to extract a tarball.
2021-08-07 11:50:00 +02:00
Christian Kellner
ff63bb6b51 stages/fstab: fix partabel option
Use the `partlabel` value for the `partlabel` option instead of
the `label` option.
2021-08-07 11:50:00 +02:00
Ondřej Budai
97331fc014 runners: add a runner for RHEL 8.6
Just a symlink to RHEL 8.2, no changes are required.
2021-08-05 22:11:24 +02:00
Ondřej Budai
7fc8ac231d stages/grub2.inst: add location to required properties
The location property is required, otherwise the stage will fail due to
KeyError at line 261:

location = options["location"]

This commit adds the property to the list of required ones.
2021-08-05 09:58:23 +02:00
Christian Kellner
c8427c392e devices/loopback:
Getting unbound loop devices is racy, so we do it in a retry loop:
in case of a recoverable error, like when the loop device signals
it is busy, we close the it and try another one. Indeed the code,
closed the loop device but did in fact not open a new one and we
would therefore see file descriptor cannot be a negative integer
errors when trying in `lo.set_fd(fd)` since `lo` is in fact closed
now and thus indeed '-1'.
Open a new loop device at the beginning of the retry-loop to fix
this issue.
2021-07-30 11:12:05 +02:00
Christian Kellner
0c9a54386b test/data: check for nm.conn's path property
Add a check for the new path property of `org.osbuild.nm.conn`.
2021-07-28 22:01:54 +01:00
Christian Kellner
9977c4e1f9 stages/nm.conn: allow creating files anywhere
Introduce a new `path` property that can be used instead of
`filename` to create files anywhere in the file system.
2021-07-28 22:01:54 +01:00
Christian Kellner
bfcc92a92e stages/nm.conn: set 0o600 permission for files
NetworkManager will refuse to load connection files that have the
to wide permsisions set. Restrict the files we create to 0o600.
2021-07-28 22:01:54 +01:00
Christian Kellner
6f7382b885 stages: add org.osbuild.nm.conf
New stage to create NetworkManager configuration files. Currently
only a subset of settings are supported. Include a simple stage
test case for it.
2021-07-28 22:01:54 +01:00
Tomas Hozza
152b14a0d6 stages/cloud-init: configuration file must contain at least one option
Add back a schema constrain, that at least one configuration must be
specified for the configuration file.

This has been forgotten/missed as part of
https://github.com/osbuild/osbuild/pull/739.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-27 10:40:04 +02:00
Christian Kellner
35de3093a7 30
This completes the development of osbuild version 30.
2021-07-22 21:15:37 +02:00
Christian Kellner
d8b5581e1c NEWS: update for release 30 2021-07-22 21:15:37 +02:00
Ondřej Budai
124cda90e2 mockbuild: add rhel 9 2021-07-22 17:58:41 +02:00
Christian Kellner
b4c571ad87 devices/loopback: add some info message
Show the file descriptor that was opened for the file passed to
the device. Recently, in CI, we have seen errors opening the
loop device with `fd` being `-1` and this ensures that at least
the file itself could be opened.
2021-07-22 15:23:39 +02:00
Christian Kellner
7f97f2c12e test/stages: new test for the tar stage
Include a simple test for the `tar` stage that does basic checks,
i.e. that a tarball was successfully created, but also checks
that we do not create entries with a `./` prefix.
2021-07-22 15:23:39 +02:00
Christian Kellner
873defb330 stages/tar: option to not include the root node
When building the tar archive, the command that is used normally
includes the root node as `./` and also leads to all files having 
a "./" prefix. On the other hand, the oci stage as well as the 
old ostree.commit assembler, with the tarball option, would
enumerate the contents instead of passing `.`, thus not including
the rood node and also avoiding the `./` prefix. 
Add a new option `root-node` that controls whether the rood node
is included or node. 

mkdir test
touch test/file

tar -c -C test file | tar tv
-rw-r--r-- root/root         0 2021-07-22 10:39 file

tar -c -C test . | tar tv
drwxr-xr-x root/root         0 2021-07-22 10:39 ./
-rw-r--r-- root/root         0 2021-07-22 10:39 ./file
2021-07-22 15:23:39 +02:00
Christian Kellner
e1df62fdc8 stages: add org.osbuild.nm.conn
Add a new stage to configure NetworkManger system connections.
Currently only ethernet connections are supported with a limited
set of options.
2021-07-22 15:18:33 +02:00
Christian Kellner
e0db66f624 stages/fstab: support device nodes and partlabel
For the fs spec field, support traditional device nodes as well
as partition labels. Adjust the test accordingly.
2021-07-22 12:05:02 +02:00
Christian Kellner
7fde451ecd stages: extract systemd.unit from systemd
Create a new `org.osbuild.systemd.unit` stage with the code for
create systemd drop-ins extracted from `org.osbuild.systemd`.
In contrast to the old code the new stage can only handle one
drop-in file. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top level comment.
The file is called `org.osbuild.systemd.unit` since in the future
it could also support creating unit files (not just drop-ins).
2021-07-22 01:05:29 +02:00
Christian Kellner
9a786ce4fc stages/logind: write only one drop-in file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top level comment.
2021-07-22 01:05:29 +02:00
Christian Kellner
79c6e65976 stages/logind: configuration_files → config
Rename the `configuration_files` key to `config` key.
2021-07-22 01:05:29 +02:00
Christian Kellner
9f1d97a8b7 stages/modprobe: write only one config file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top level comment.
2021-07-22 01:05:29 +02:00
Christian Kellner
0190c991ae stages/dracut: write only one config file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top-level comment.
2021-07-22 01:05:29 +02:00
Christian Kellner
a1703dc298 stages/cloud-init: write only one config file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify the directory, `/etc` vs `/usr` via a new
top level `directory` key.
2021-07-22 01:05:29 +02:00
Christian Kellner
0becf66454 stages/dracut.conf: configuration_files → config
Rename the `configuration_files` key to `config` key.
2021-07-22 01:05:29 +02:00
Christian Kellner
f259b4e5ca stages/modprobe: configuration_files → config
Rename the `configuration_files` key to `config` key.
2021-07-22 01:05:29 +02:00
Christian Kellner
9e259ae0bc stages/cloud-init: configuration_files → config
Rename the `configuration_files` key to `config` key.
2021-07-22 01:05:29 +02:00
Diaa Sami
8be74157ae test: fix wrongly named class
The test class for the v2 tests was wrongly named `TestFormatV1`.
2021-07-21 16:46:09 +02:00
Martin Sehnoutka
54d7128d65 util/rhsm: Implement a fallback to the previous behaviour
The problem is that some deployments might not have the redhat.repo
file, yet they might have the key and certificate to access Red Hat CDN.
If that was the case, the new approach would cause a regression compared
to the previous behavior.

This patch uses the previous method if the redhat.repo file is not
found or does not contain any matching URL.
2021-07-21 14:38:18 +02:00
Christian Kellner
0cab954d84 test/fmt_v2: more tests for mounts
Add tests for the duplicate mount, missing device detection.
2021-07-21 13:28:22 +02:00
Christian Kellner
3965291895 test/fmt_v2: include devices and mount
Include devices and mounts in the basic pipeline to check that
parsing and describing them works as expected.
2021-07-21 13:28:22 +02:00
Christian Kellner
a1df2d5d9d mounts: add org.osbuild.noop
Add a noop mount that is usefull for testing.
2021-07-21 13:28:22 +02:00
Christian Kellner
4d73760dcd formats/v2: fix mount description for source
The mount has a `source` not `device`. It is just called device
internally because for now it is always a device.
2021-07-21 13:28:22 +02:00
Christian Kellner
ae1296e33a formats/v2: mounts are arrays
The order of entries in a dictionary is not specified by the JSON
standard and hard to control when marshalling dictionaries in Go.
Since the order of mounts is important and the wrong order leads
to wrong mount trees change the `mounts` field to an array. This
breaks existing manifests but after careful deliberation it was
concluded that the original schema with mounts as dictionaries
is not something we want to support. Apologies to everyone.

Adjust the schema of the copy and zipl stage accordingly.
2021-07-21 13:28:22 +02:00
Christian Kellner
50627b713c schema/v2: make options for devices optional
Some devices might not need any options.
2021-07-21 13:28:22 +02:00
Christian Kellner
31943072c7 stages/noop: allow mounts and devices
Allow the noop stage to have mounts and devices, which is useful
for testing.
2021-07-21 13:28:22 +02:00
Diaa Sami
02ceb02d2a osbuild: Add validation for source references
Validate source references while loading manifests so that a bad
reference would result in a meaningful error message instead of a
hard-to-understand Python exception.
2021-07-20 20:22:27 +02:00
Christian Kellner
9e599fca17 tools/mpp: expand basurl after dep-solving
We create the urls for the packages after dep-solving. During the
big refactoring (802f4010) the code that would expand the base url
during that step got lost. Re-introduce that so that local repos
work correctly again.
Also make sure the path is absolute before calling `as_uri` since
it is not guranteed that the result will be a `file:///` uri.

Reported-by: Stephen Smoogen <smooge@smoogespace.com>
2021-07-20 13:23:58 +02:00