With the enabling of AppStream some more (perhaps optional) packages are
included.
This also adds a test case which installs a module and verifies that
that module is returned.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
The CentOS Stream 9 repository metadata contains modules; these are
necessary for testing modularity depsolving.
Note that the filelists metadata is kept empty to keep repository size
down.
Co-authored-by: Michael Vogt <michael.vogt@gmail.com>
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
This changes the depsolver to expect modules inside the normal
`package-specs` instead of having a separate `module-install-specs`.
Modules passed to a "normal" `dnf install` must start with an `@` and
contain a `:`. This is up to the user to pass correctly.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
The commit implements modularity for the dnf4 dependency solver.
The dependency solver now takes two new keys per transaction:
`module-install-specs` and `module-enable-specs`, each of which is a
list of module specifications.
When modules are used and packages for them are installed a new
dictionary is returned in the response where each key is the name of an
enabled module and the value are the contents of the module
configuration and the module failsafe file that DNF needs in the
resulting system.
These values can be used by consumers of the dependency solver, such as
`images`, to create the appropriate inline sources and copy stages.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
The name 'cache' is more recognisable and conveys the purpose of the
option just as well (if not better) than 'store'.
Also, the option that controls the size of this directory is called
'cache-max-size', so we're already referring to it as a cache.
This helps to clear a risk concern for
automotive.
Add optional flag to org.osbuild.mkfs.ext4 stage
to enable/disable both lazy_itable_init and
lazy_journal_init extended options. Both mke2fs
options are controled by the same flag, namely
`lazy_init`.
Signed-off-by: Albert Esteve <aesteve@redhat.com>
Add mpp files to create manifests that will embed the container (that
contains /dev/null) in an OS tree and compose an ostree commit from it.
This used to cause the rpm-ostree compose command to fail with:
error: Writing commit: While writing rootfs to mtree: Not a regular file or symlink: null
This commit tweaks build() to be mypy clean without the need to
call assert. This drops the map() and instead we use the existing
dict-like access of the manifest to get the pipeline. In practise
this should not happen but lets be prepared.
Note that a small tweak for the error is needed to make it clear
what is happening.
Thanks to Simon for raising this.
This commit makes the returns of the of {Build,Download}Result.as_dict()
explicit. Ideally this would just be a dataclass and dataclass.asdict()
but because we need to support python3.6 this is not possible today.
This commit adds error reporting from source download errors
to the monitor. It reuses the `BuildResult` for symmetry but
we probably want to refactor this a bit to make source handling
a bit more similar to stages.
In order to avoid having to rely on the output of `osbuild --json`
when using `--progress=JSONSeqMonitor` the monitor needs to include
the `osbuild.pipeline.BuildResult` for each individual stage.
This commit adds those to the montior.
This commit relaxes the "mirror" schema regex from only allow
hashes to the repo ID regex as described in
https://dnf.readthedocs.io/en/latest/conf_ref.html#description
This is needed because in e.g. bootc-image-builder we use
"SetRootDir()" to get the repository information from an existing
bootc image and there the repo_id (that is then used as the mirror_id)
is written in a human friendly way instead of hex.
Thanks to Tomáš Hozza for the suggestions and background info, see
https://github.com/osbuild/osbuild/pull/1974#issuecomment-2579325272
This commit adds support for the `org.osbuild.mtls` sources and
adds a test.
Note that it reuses the `OSBUILD_SOURCES_CURL_` - this may or
may not be desired, it is certainly simpler this way and librepo
is using libcurl as well internally but it feels a bit messy so
maybe we want to rename the env?
This commit fixes the pylint errors and as a drive-by tweaks some naming
in `fetch_all()`. Maybe they became a bit verbose along the way,
idk.
Thanks also to Tomáš Hozza for the dict.setdefault() suggestion.
This commit adds unit a basic test for librepo and also updates
the code to the latest requirements to provide "fetch_all".
It also fixes a bug in the handling of org.osbuild.rhsm secrets.
This commit just ran:
```
find -iname "*.json" -print -exec sh -c 'jq --indent 2 . {} | sponge {}' \;
```
to ensure that the new test json data from librepo is in the format
the linter expects.
Most modules do not support both schema versions. This is masked by
module type code in get_schema() in most cases, but really should not be
tested. This skips running check_moduleinfo if the module doesn't
support the version. eg. org.osbuild.librepo only supports v2.
This helps prevent testing against invalid manifest data. It runs on the
source's manifest data, using the highest schema version parsed for the
source.
These tests test downloading packages using mirrorlist:
* Using a single mirrorlist repo
* Using two mirrorlist repos
* A bad checksum (or file) from the download which is expected to fail
* Two mirrorlist paths with the first one returning a 404, expected to
succeed with the 2nd mirrorlist path.
NOTE: The metalink xml file points to a repomd.xml file which does not
exist. In practice librepo doesn't download this, it only uses the url
to construct the mirror urls, so it is not present.
These tests test downloading packages using metalink:
* Using a single metalink repo
* Using two metalink repos
* A bad checksum (or file) from the download which is expected to fail
* Two metalink paths with the first one returning a 404, expected to
succeed with the 2nd metalink path.
Using a metalink or mirrorlist along with the package paths and
checksums allows them to be reliably downloaded even when mirrors are
not all in sync. It will retry with a new mirror until it succeeds, or
has tried all of the mirrors.
This commit adds support to run the tests on Debian/Ubuntu systems.
Here the binary is called `grub-mkimage` instead of `grub2-mkimage`,
in addition the "grub-pc-bin" package must be installed there.
Don't include the "location" offset, and use just a path for the prefix
section to set the path of the grub2 modules on the iso. eg.
{
"filename": "eltorito.img",
"platform": "i386-pc",
"core": {
"type": "mkimage",
"partlabel": "gpt",
"filesystem": "iso9660"
},
"prefix": {
"path": "/boot/grub2/"
}
}
The mount ID must be unique. So far, we were using the device as the ID
for the mount because that was unique to each mount. With btrfs
subvolumes however, the device and partition are the same for all, so we
need another way to differentiate.
Btrfs volumes typically only contain subvolumes instead of (parts of)
the OS tree directly. In our images in particular, this is always the
case. When searching for root to find /etc/fstab, search through the
subvolumes on a btrfs volume for the file and return the path to the
root subvolume.
Co-authored-by: Michael Vogt <michael.vogt@gmail.com>
Always set partition=None for the kwargs of the Mount() constructor.
The previous code was added for backwards compatibility with older
versions of the Mount() constructor that didn't include the 'partition
argument. It's safe to remove now because:
1. It's been long enough that we wont run osbuild-image-info with an old
version of osbuild.
2. The tool is packaged with osbuild so there is no version drift and no
compatibility issues.
When the fstab file isn't found, the root_tree will never be set after
being initialised to "" and an exception is raised "The root filesystem
tree is not mounted". It's a lot clearer if the failure happens closer
to the root cause, which is that fstab wasn't found and there are no
fstab entries to iterate through and find the root filesystem.
When iterating partitions to mount, skip any with filesystem type
"swap". This is done in two places:
1. When mounting partitions to find /etc/fstab.
2. When mounting partitions and volumes to analyse the tree.
When iterating through partitions, store the fstype along with the other
information. This will be useful for identifying btrfs partitions,
which we will need to scan for subvolumes, and for identifying swap
partitions, so we can avoid trying to mount them.
