Python 3.14 removed the long deprecated `ast.Str` [1]. Rewrite to use
`ast.Constant` conditionally for any version that isn't 3.6.
Since 3.6 is our lowest supported version we don't check for below.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
This commit adds an error message if no repositories are
defined in the dnfjson query. We had the issue in
https://github.com/osbuild/bootc-image-builder/issues/922
that in a RHEL bootc-container no repositories are defined.
Here the error is quite confusing, as it complains about
error marking packages which is technically correct but
hides the root of the problem.
With this detect we can construct a more useful error
message in the higher layers.
The PLATFORM_ID got retired from fedora-43 [0] and it
seems like it was always kinda optional. So lets make
it optional for real to avoid failing to build fedora-43
images.
[0] https://fedoraproject.org/wiki/Changes/Drop_PLATFORM_ID
DNF apparently doesn't check the expiration time on the cache unless you
call this first. It may help improve issues with the cache being out of
sync.
Internally dnf checks the mtime of the main repomd.xml file when using
baseurl, and it checks the hash values when using metalink so this
should not result in any noticeable extra slowdowns when things are
fresh.
The old function has been deprecated by dnf5, use load_repos directly
and only load the available repos (the ones osbuild has setup), not the
system repos.
Fixes#2080
This commit adds support for more debug for `qemu-user` options.
When settings:
```
$ sudo IMAGE_BUILDER_EXPERIMENAL=debug-qemu-user bootc-image-builder ...
```
extra debug will be printed. This hopefully helps to track down
the root cause of
https://github.com/podman-desktop/extension-bootc/issues/1475
This commit adds two new helpers:
- util.experimentalflags.get_bool()
- util.experimentalflags.get_string()
similar to what we added in the images library in PR:
https://github.com/osbuild/images/pull/1248
The idea is that we provide experimentalflags for osbuild via
an environment like `OSBUILD_EXPERIMENTAL` and for those we
make no API promises. This will be initially used for better
debug of qemu-user.
Always return License ref IDs as is, if used as package license,
regardless if license_expression package is available. This will prevent
wrapping them again as extracted license info and generating yet another
license ref ID.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Allow passing a custom license index db file for SBOM generation by
specifying it in the solver configuration.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Introduce a new class `SpdxLicenseExpressionCreator`, responsible for
converting license texts extracted from packages, into an SPDX-compliant
license expressions. If the `license_expression` Python package is
available on the system, it is used to determine the license text
extracted from a package is a valid SPDX license expression. If it is,
it's returned as is back to the caller. If it is not, or of the package
is not available on the system, the license text is wrapped in a
`ExtractedLicensingInfo` instance.
The `SpdxLicenseExpressionCreator` object keeps track of all generated
`ExtractedLicensingInfo` instances and de-duplicates them based on the
license text. This means that if two packages use the same
SPDX-non-compliant license text, they will be wrapped by an
`ExtractedLicensingInfo` instance with the same `LicenseRef-` ID.
The reason for fallback when `license_expression` package is not
available is that it is not available on RHEL and CentOS Stream. This
implementation allows us to ship the functionality in RHEL and
optionally enabling it by installing `license_expression` from a 3rd
party repository. In any case, the generated SBOM document will always
contain valid SPDX license expressions.
Extend unit tests to cover the newly added functionality.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
FIXUP: sbom/spdx: use compliant license expressions
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the SPDX v2 model to support referencing extracted licensing
information, which is either not in the SPDX license list or can't be
expressed by the SPDX-compliant license expression.
Cover the new functionality by unit tests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This commit limits the output in the json pipeline to a "reasonable"
length. We ran into issues (e.g. [0]) from a combination of a stage
that produce tons of output (dracut, ~256 kb, see issue#1976) and
the consumer ("images" osbuild/monitor.go) that used a golang scanner
with a max default buffer of 64kb before erroring. So limit it
here.
The stage result from via json is mostly for information and any error
will most likely at the end. Plus consumers can collect the individual
log lines on their own if desired via the "log()" messages that are
stream in "real-time" with the added benefit that e.g. timestamps
can be added to the logs etc.
[0] https://issues.redhat.com/browse/RHEL-77988
This commit fixes a race/threading issue with the way the monitor
works. The osbuild monitor can be called from multiple threads,
e.g. in buildroot.py:run() monitor.log() is called but also
in host.py:_stdout_ready(). This can lead to out-of-order writes
when many messages need to be processed.
We did not notice this so far because we were lucky and also
log was just used for information. But now it is used to transmit
the jsonseq data which means out-of-order communication results
in broken json.
Closes: https://github.com/osbuild/image-builder-cli/issues/110
The NEVRA field accidentally made it into the public API. It shouldn't
be as it is not used downstream and in fact breaks downstream.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Remove the requirement for `:` in the name which would have been
selected by the fronted, instead asking dnf "is this a module?".
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Instead of returning the string contents of the module configuration
file let's return some more structured data so the resulting file can be
written with a stage.
Note that we don't do the same for the fallback file as we are only
provided this as a YAML blob and unparsing it to then immediately
reparse it again is counterproductive.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
This changes the depsolver to expect modules inside the normal
`package-specs` instead of having a separate `module-install-specs`.
Modules passed to a "normal" `dnf install` must start with an `@` and
contain a `:`. This is up to the user to pass correctly.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
The commit implements modularity for the dnf4 dependency solver.
The dependency solver now takes two new keys per transaction:
`module-install-specs` and `module-enable-specs`, each of which is a
list of module specifications.
When modules are used and packages for them are installed a new
dictionary is returned in the response where each key is the name of an
enabled module and the value are the contents of the module
configuration and the module failsafe file that DNF needs in the
resulting system.
These values can be used by consumers of the dependency solver, such as
`images`, to create the appropriate inline sources and copy stages.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
The name 'cache' is more recognisable and conveys the purpose of the
option just as well (if not better) than 'store'.
Also, the option that controls the size of this directory is called
'cache-max-size', so we're already referring to it as a cache.
This commit tweaks build() to be mypy clean without the need to
call assert. This drops the map() and instead we use the existing
dict-like access of the manifest to get the pipeline. In practise
this should not happen but lets be prepared.
Note that a small tweak for the error is needed to make it clear
what is happening.
Thanks to Simon for raising this.
