Both file embedding and depsolves start by iterating over all stages,
and we want to add another similar one for container installs, so
break out the iteration over the containers so that it is done in
one place only.
This adds a stage called org.osbuild.skopeo that installs docker and
oci archive files into the container storage of the tree being
constructed.
The source can either be a file from another pipeline, for example one
created with the existing org.osbuild.oci-archive stage, or it can
be using the new org.osbuild.skopeo source and org.osbuild.containers
input, which will download an image from a registry and install that.
There is an optional option in the install stage that lets you
configure a custom storage location, which allows the use of the
additionalimagestores option in the container storage.conf
to use a read-only image stores (instead of /var/lib/container).
Note: skopeo fails to start if /etc/containers/policy.json is
not available, so we bind mount it from the build tree to the
buildroot if available.
The client side does meta.get("source-epoch", default), but for
this to work we need to have the key unset if not specified,
but currently we set it to None.
Also, make sure the check for "not None" is explicit, because
we do consider a value of `0` to be a valid source-epoch.
This tries to make the various tar-balls produced by the stage more
likely to be identical in separate runs. We do this by sorting
the names and removing some unnecessary metadata for the files.
The most important thing to get right is the layer tarball, because
that is what defines the container id. We sort the names to avoid
random differences, and drop ctimes and atimes because these are
generally just set to the something near the current build time which
is not useful to encode in the container image. This is as opposed
to the mtime which generally comes from e.g. the rpms that where
installed in the pipeline.
For the actual archive tarball we can standardize metadata even more,
because none of the metadata are used when consuming the archive.
ioctl contants are platform dependent. It should be the same on
x86, aarch64 and s390x but it is indeed different on ppc64le.
This lead to the call to `ioctl_blockdev_flushbuf` actually
raising an exception of `OSError: [Errno 22] Invalid argument`.
The constant was calculated with a little python snippet that
in theory could also go directly into the code, but for now
the simpler condition in this patch is enough.
The snippet is a port of the defines from the Linux kernel,
specifically /usr/include/asm-generic/ioctl.h.
class IOConstants:
"""IO Commands for Linux"""
if platform.machine() == "ppc64le":
NRBITS = 8
TYPEBITS = 8
SIZEBITS = 13
DIR_NONE = 1
else:
NRBITS = 8
TYPEBITS = 8
SIZEBITS = 14
DIR_NONE = 0
NRSHIFT = 0
TYPESHIFT = NRSHIFT+NRBITS
SIZESHIFT = TYPESHIFT+TYPEBITS
DIRSHIFT = SIZESHIFT+SIZEBITS
@classmethod
def make(cls, directory, iotype, nr, size):
return ((directory << cls.DIRSHIFT) |
(iotype << cls.TYPESHIFT) |
(nr << cls.NRSHIFT) |
(size << cls.SIZESHIFT))
@classmethod
def make_dir_none(cls, iotype, nr):
return cls.make(cls.DIR_NONE, iotype, nr, 0)
This is used to get the value for `BLKFLSBUF` taken from the
include `/usr/include/linux/fs.h`:
#define BLKFLSBUF _IO(0x12,97) /* flush buffer cache */
The value is then obtained via:
print("0x%x" % IOConstants.make_dir_none(0x12,97))
0x20001261
Currently we always write the kernel command line to the `grubenv`
file, if only to include the root device. Starting with Fedora 33
and thus RHEL 9, the kernel command line included statically in
the BLS snippets and the grubenv `kernelopts` variable not used.
Instead one of the {/usr/lib,/etc}/kernel/cmdline files is read
and the parameters in them used during the creation of the BLS
snippets.
Therefore we add a new `write_cmdline` option that, if set to
FALSE, will prevent us from writing the kernel command line.
The sysconfig stage currently does not produce expected results when
used multiple times within the same pipeline. Specifically, the stage
always truncates respective configuration files for properties `kernel`
and `network`, if if these are not set in the stage options. Due to this
reason, the outcome of the image builds may depend on the order of
multiple occurrences of the sysconfig stage.
The following two pipeline snippets would produce different
configuration files content:
Configuration files are truncated:
```
{
"type": "org.osbuild.sysconfig",
"options": {
"kernel": {
"update_default": true,
"default_kernel": "kernel"
},
"network": {
"networking": true,
"no_zero_conf": true
}
}
},
{
"type": "org.osbuild.sysconfig",
"options": {
"network-scripts": {
"ifcfg": {
"eth0": {
"bootproto": "dhcp",
"device": "eth0",
"ipv6init": false,
"onboot": true,
"peerdns": true,
"type": "Ethernet",
"userctl": true
}
}
}
}
},
```
No configuration files are truncated:
```
{
"type": "org.osbuild.sysconfig",
"options": {
"network-scripts": {
"ifcfg": {
"eth0": {
"bootproto": "dhcp",
"device": "eth0",
"ipv6init": false,
"onboot": true,
"peerdns": true,
"type": "Ethernet",
"userctl": true
}
}
}
}
},
{
"type": "org.osbuild.sysconfig",
"options": {
"kernel": {
"update_default": true,
"default_kernel": "kernel"
},
"network": {
"networking": true,
"no_zero_conf": true
}
}
},
```
Change the stage to not touch respective configuration files if the
`kernel` and `network` properties are not set in the stage options.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The treesum of a filesystem tree is the content hash of all its
files, its directory structure and file metadata.
By storing trees by their treesum we avoid storing duplicates of
identical trees, at the cost of computing the hashes for every
commit to the store.
This has limited benefit as the likelihood of two trees being
identical is slim, in particular when we already have the ability
to cache based on pipeline/stage ID (i.e., we can avoid rebuilding
trees if the pipelines that built them were the same).
Drop the concept of a treesum entirely, even though I very much
liked the idea in theory...
Signed-off-by: Tom Gundersen <teg@jklm.no>
Pacman is the default package manager for Arch Linux and derivates, the
pacman.conf stage generate a valid pacman.conf configuration file.
Co-Authored-By: Jelle van der Waa <jvanderwaa@redhat.com>
This introduces a new dependency resolver to osbuild-mpp for Arch Linux
which uses the pacman package manager. The used solver is determined by
the `solver` field in the `mpp-depsolve` object inside the manifest
file, if it does not exists it falls back to the DepSolver for dnf/rpm.
Co-Authored-By: Jelle van der Waa <jvanderwaa@redhat.com>
Add a new stage `org.osbuild.dnf-automatic.config` for configuring DNF
Automatic.
The stage changes persistent DNF Automatic configuration. Currently, only
a subset of options can be set:
- 'commands' section
- apply_updates
- upgrade_type
Fix#908
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new stage `org.osbuild.yum.repos` for creating YUM / DNF `.repo`
files in `/etc/yum.repos.d`. All repo-specific options are supported but
only a subset of options which can be set for a repo as well as in the
[main] section are supported.
Add unit test for the new stage.
Fix#907
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Set the container environment variable to indicate to programs
inside the build root that they are indeed running inside a
container (see also https://systemd.io/CONTAINER_INTERFACE/).
