Use the new partiton layout support in MPP. NB: start data as
well as the size of the last partition have been omitted now,
since `sfdisk` will figure those out for us.
It is now possible to define a partition layout via `mpp-define-image`.
The defined layout is actually written to a temporary sparse file and
read back via `sfdisk`, so that all partition data like `size` and
`start` include actual padding and such. The `image` variable will be
defined with `size` and `layout` keys, the latter containing the
partition layout data. It can be accessed via the "String expansion"
mechanism.
Catch the BrokenPipeError exception when sending a reply. This will
happen when the other side closes their side of the connection/pipe
so in that case we just break out of the serve loop.
Now that arguments are transmitted via a mapped, i.e. bind-mounted,
file instead of using the jsoncomm RPC mechanism, all the methods
related to the latter can be removed from API.
Instead of using the jsoncomm API to transmit stagge arguments,
write them out to a file that is then mapped into the container.
The `api.arguments` function is re-written just read that file
from within the container.
On rare occasions, when trying to mount a file system via the
loopback device that was created in the stage before the mount
fails with `wrong fs type, bad option, bad superblock on ...`.
It is not yet fully clear why this appears since in theory the
kernel should handle the sequence of operations we do, but it
might be down to the caching or bugs in the underlying file-
system.
To mitigate with potential caching issue we therefore now sync
the file that is backing the loopback device after the device
has been closed. In order to not have to re-open the file we
keep the file descriptor around as long as the loopback device
is open.
This should, in theory, not be necessary because the bubblewrap
process and its children should be stopped already and umount
should just block until it is finished. But, if the store is on
a filesystem, like the one used by docker machine, unmounting
frequently produces errors like:
`umount: .../tmp9nlyzwdu-writer: target is busy.`
Syncing the filesystem before that seems to help in some cases
and it surely does not hurt.
Previous versions of mpp would already set the arch and basearch
substitution, which would work for dep-solving itself, but not
properly re-write the resulting URLs which means that the manifest
was broken. Fix this by properly replacing the substitutions in
the URL. Also support official 'releasever' substitution.
Instead of passing dictionaries around that are inconvenient to
use in code and even more in the `mpp-format-*` directives, use
a simple class to represent package information. Use that in
the `pkginfo` dict that can be accessed via `mpp-format-*`. Use
the `evra` property instead of string manipulation in the
`fedora-boot.mpp.json` and `-ostree-bootiso.mpp.json` manifest.
This is needed for the pylint to work correctly with the new
cloud-init stage.
While there, add some more utilities useful for debugging. Also
add glibc-langpack-en needed for `en_GB`.
This uses size computations to simplify the partition size/offsets
and the depsolv results to get the kernel version.
This makes no changes to the resulting json files.
This lets a mpp manifest define some variables and then use
these variables inside python f-strings to expand things in a
flexible way. This allows a single value such as `rootfs_size` to
be expanded in various places, including using computations to
e.g. define the partition offsets/sizes.
Move all the global code into a `main` function and call that. This
fixes a lot of pylint warnings where variable names were re-used
from the main, and thus global, context.
Create a DepSolver class that carries the global state such as dirs
and subscription information, as well as local state, like the
repositories and basedir. The latter can be reset so the class can
easily be re-used for all dep-solve sections.
This avoids having any global state.
Sort the checksum: urls dictionary via the url, so that no matter
where the urls came from (import or any dep-solve section), the
checksum: url dict is the same for the same set of urls.
Add support for additional paths that are searched when trying to
load a manifest. Currently only the path of the manifest that has
the include is searched. With this changed additional directories
will be included after that, in the order they were given ton the
command line.
Both v1 and v2 code paths used the same logic to add the resolved
dependencies to the `source_urls` dict, which is already stored
in the base class. Move the logic into the base class too.
Rewrite image pre-processor to single tool so that it is easier to use.
Now also supports `ignore-weak-deps` when dep-solving and supports
relative paths for local files.
Also create a symlink to the osbuild package, so that the tools can be
run from the source checkout and have access to the osbuild package.
