Add testing on pythong 3.14 and keep testing on python 3.13 for the
current stable Fedora version (42 at this time).
Use osbuild-ci:latest-202506112350 for tests.yml
On Python 3.6 we're getting a *different* error message from other
versions in CI. Let's accept both versions.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
An error message has changed in skopeo [1] (or one of it's underlying
libraries). This new version is now in our CI so let's fix our tests.
[1]: f423f01d1b
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Python 3.14 removed the long deprecated `ast.Str` [1]. Rewrite to use
`ast.Constant` conditionally for any version that isn't 3.6.
Since 3.6 is our lowest supported version we don't check for below.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
For the terminal_input, terminal_output, and serial variables in the
GRUB_CFG_TEMPLATE, the keys were not set when those variables were not
defined in the options. This caused the template variables to show up
in the final file itself, which could break the config. For example,
the following line was being generated for one of our images:
serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1${terminal_input}${terminal_output}
Setting the keys to an empty string when they're not defined solves the
issue.
Since this is a stage behaviour change, the new behaviour is toggled by
an option that maintains backwards compatibility when not set.
This introduces a new convention of adding a `compat_version` number to
the stage options to control the behaviour. The value defaults to 1,
which is the old (broken) behaviour. Manifest generators should always
set it to 2 to get the correct behaviour.
In Python 3.14 configparser started validating keys. We use hacky bits
to write duplicate keys which means we have delimiters in our keys which
is now no longer allowed.
Let's explicitly turn off key validation for now as suggested by
mhroncok in [1].
[1]: https://github.com/osbuild/osbuild/issues/2109#issuecomment-2964585958
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
In Python 3.14 configparser started validating keys. We use hacky bits
to write duplicate keys which means we have delimiters in our keys which
is now no longer allowed.
Let's explicitly turn off key validation for now as suggested by
mhroncok in [1].
[1]: https://github.com/osbuild/osbuild/issues/2109#issuecomment-2964585958
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Add the ability to exclude files and directories from the erofs
image. This uses the mkfs.erofs --exclude-regex feature, so simple
matches or regexes can be used.
Add the ability to exclude files and directories from the squashfs
image. This uses the mksquashfs -regex -e FILES... feature, so simple
matches or regexes can be used.
This also adds a new test for squashfs, based on the existing test for
erofs.
This commit adds a small unit test that the inline source
works as expected by creating some test_data and then
checking that it ends up in the cache.
Note that this is also already tested in the tests in
osbuild/test/run/test_sources.py but there its a lot more
indirect and in the spirit of tests-as-documentation having
an explicit test here seems useful (its also quicker to
run and easier to discover).
Small followup for https://github.com/osbuild/osbuild/pull/2090
where this was discussed originally.
The `org.osbuild.tar` stage only supports auto compression which is
based on the filename. For several (newer) artifacts such as Vagrant
(`.box`) and WSL (`.wsl`) we want to explicitly give the compression
algorithm.
I've chosen a (few) commonly used compression algorithms. If others are
needed they are one-line followups away.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
In some (ununderstood) cases the combination of `--no-single-branch` and
`--depth=1` leads to the revision we want to check out not being
available.
Achilleas suggested to change the command to this instead.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
RPM 6.0 (landed in F43, current rawhide), no longer uses short key ID
for the 'gpg-pubkey' pseudo-package, but to prevent any collision, it
uses full key ID.
This means that we can't consistently just compare the pipeline rpm
stage metadata from the test data with what we get from building the
test manifest. The reason is that we need to keep running the unit test
in upstream and downstream CI pipelines for OSes, which may ship RPM
6.0, but also older RPM version.
Extend the test case to do a special case check for any 'gpg-pubkey'
package in the metadata if the package version length differs between
the test data and data got from building the manifest.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a new %post option to the kickstart stage that supports adding
multiple post blocks to a kickstart file, with all the options supported
by the directive.
Rename the 'success' case to 'base64_success' as a preparation for
adding test case for other encodings.
The test case was originally embedding the tools/inline-source.py
script. Since the script has been updated, let's update the embedded
date in the test case as well.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the script to support specifying the data encoding. Keep
'base64' as the default encoding.
Add support for 'lzma+base64' encoding.
Also use the 'base64' module, instead of 'binascii' module for base64
encoding. This is consistent with what the actual source implementation
uses.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the inline source to support lzma compressed and base64 encoded
'data'. This will allow us to reduce the potential manifest size when
embedding big files. The aim is specifically at eventually embedding
SBOMs of the image.
An example single SBOM can be a JSON file with size of about 1.9 MiB.
The lzma+base64 combination reduces the 'data' to embed to "only" around
250 KiB.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This commit adds an error message if no repositories are
defined in the dnfjson query. We had the issue in
https://github.com/osbuild/bootc-image-builder/issues/922
that in a RHEL bootc-container no repositories are defined.
Here the error is quite confusing, as it complains about
error marking packages which is technically correct but
hides the root of the problem.
With this detect we can construct a more useful error
message in the higher layers.
Drop `module_platform_id` as it is now optional and none of
our tests is using it (i.e. has any observable difference if
missing).
Once we start using it we need to add it (and maybe a
"with_platform_id" as parameter on top so that both with/without
platform_id is tested).
The PLATFORM_ID got retired from fedora-43 [0] and it
seems like it was always kinda optional. So lets make
it optional for real to avoid failing to build fedora-43
images.
[0] https://fedoraproject.org/wiki/Changes/Drop_PLATFORM_ID
DNF apparently doesn't check the expiration time on the cache unless you
call this first. It may help improve issues with the cache being out of
sync.
Internally dnf checks the mtime of the main repomd.xml file when using
baseurl, and it checks the hash values when using metalink so this
should not result in any noticeable extra slowdowns when things are
fresh.
This works around a file conflict on /usr/bin/typer between
python3-typer-cli and erlang-dialyzer until the problem can be fixed in
python-typer and/or erlang. See RHBZ#2359557, RHBZ#2359567.
After the file conflict is resolved, the python3-typer-slim dependency
will still be more precise and minimal than python3-typer, so no further
change will be required.
By depending on typer-slim[standard], we still get the nice-to-have
optional dependencies that plain typer would bring in (currently, rich
and shellingham), but without the generic typer command-line tool.
The old function has been deprecated by dnf5, use load_repos directly
and only load the available repos (the ones osbuild has setup), not the
system repos.
Fixes#2080
The `generic.ins` template uses:
images/kernel.img
images/initrd.img
However, kernel and initrd are currently stored as:
images/pxeboot/vmlinuz
images/pxeboot/initrd.img
This change ensures correct paths are used.
