Commit graph

16 commits

Author SHA1 Message Date
Pierre-Yves Chibon
65b98448c6 Add support for defining variables from other variables or basic expression
Using this we can now define variables in the variable section using
basic expression that are evaluated as f-string.
For example, you can use the syntax:

```
"mpp-vars": {
   "rootfs_uuid": {"mpp-format-string": "{__import__('uuid').uuid1()}"},
   "bootfs_uuid": "156f0420-627b-4151-ae6f-fda298097515"
},
```

This will automatically call uuid.uuid1() for rootfs_uuid, thus allowing
to dynamically set the uuid for the rootfs variable.
This variable being able to be overridden via the -D argument of the
osbuild-mpp tool.

In addition, you can also define variable based on variables defined
above, for example:

```
"mpp-vars": {
   "rootfs_size": 4294967296,
   "homefs_size": {"mpp-format-string": "{rootfs_size}"}
},
```

For context, we're in particular interested in this for the uuid
generation, as we have had issues with xfs failing to mount a filesystem
if the uuid is already mounted. In particular, if we built e.g. a
raspberry pi image using osbuild and then we use that raspberry pi for building
the same manifest (typically a later version of it) the osbuild fails in the
loopback mount case. Being able to easily support reproducible fixed-uuid
images, as well as ones with a real unique uuid makes a lot of sense.

This commit was created with a lot of inputs from Alexander Larsson.

Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
2021-09-24 13:32:51 +02:00
Pierre-Yves Chibon
2d4bc41cfc Fix the assert as .sort() returns None
Since `.sort()` returns None, we were checking that None == None
which is not what we aimed to do.

Quick reproducer:
```
>>> assert [1,2,3].sort() == [2,3,4].sort()
>>> assert sorted([1,2,3]) == sorted([2,3,4])
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    assert sorted([1,2,3]) == sorted([2,3,4])
AssertionError
```

Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
2021-09-21 16:09:47 +02:00
Christian Kellner
9e599fca17 tools/mpp: expand basurl after dep-solving
We create the urls for the packages after dep-solving. During the
big refactoring (802f4010) the code that would expand the base url
during that step got lost. Re-introduce that so that local repos
work correctly again.
Also make sure the path is absolute before calling `as_uri` since
it is not guranteed that the result will be a `file:///` uri.

Reported-by: Stephen Smoogen <smooge@smoogespace.com>
2021-07-20 13:23:58 +02:00
Christian Kellner
c4db24c481 tools/mpp: substitute vars in mpp blocks
Allow the manifest variables, defined via mpp-vars, to be used from
within the mpp blocks. For this template strings are used, where
variables are marked via `$`. We cannot use the `mpp-format` logic
easily there, since that is processed after other mpp directives
have been processed.
As a result remove the built-in substitution from support from dnf
dep-solving, since we had to post-process the resulting urls with
variable substitution afterwards. Now that is covered with this
more generic mechanism.
2021-07-14 23:51:34 +02:00
Christian Kellner
4a00895ba5 tools/mpp: sort import
Makes David happy, which makes us happy.
2021-07-14 23:51:34 +02:00
Christian Kellner
fd1b688f78 tools/mpp: single mpp node retrieval method
Introduce a helper method that will return the mpp node of a given
name or `None` if it does not exist. Delete it from the parent
dictionary if it did.

Reorder the typing imports as well to be sorted below the non-
`from` imports.
2021-07-14 23:51:34 +02:00
Christian Kellner
fdb0faca86 tools/mpp: extract vars initialization
Move the initialization for manifest vars out of `__init__`.
2021-07-14 23:51:34 +02:00
Christian Kellner
368d0a5c18 tools/mpp: support multiple image definitions
Support more than one image definition by introducing a `id` attribute
which can be used to specify how the variable for the definition is
called.
2021-07-09 19:50:38 +01:00
Christian Kellner
042a3424e4 tools/mpp: support skip_if_unavailable
Add `skip_if_unavailable` as an supported attribute
for the repository configuration.
2021-07-09 19:50:38 +01:00
Christian Kellner
d39e3a239e tools/mpp: support for defining partitions
It is now possible to define a partition layout via `mpp-define-image`.
The defined layout is actually written to a temporary sparse file and
read back via `sfdisk`, so that all partition data like `size` and
`start` include actual padding and such. The `image` variable will be
defined with `size` and `layout` keys, the latter containing the
partition layout data. It can be accessed via the "String expansion"
mechanism.
2021-07-08 22:47:45 +02:00
Christian Kellner
1dc845f331 tools/mpp: properly support substitutions
Previous versions of mpp would already set the arch and basearch
substitution, which would work for dep-solving itself, but not
properly re-write the resulting URLs which means that the manifest
was broken. Fix this by properly replacing the substitutions in
the URL. Also support official 'releasever' substitution.
2021-07-07 15:10:00 +02:00
Christian Kellner
7c9035e48a tools/mpp: set module_platform_id again
This, together with the "ignore-weak-deps" option got lost in the
refactoring. Add it back.
2021-07-07 15:10:00 +02:00
Christian Kellner
d47f735112 tools/mpp: create PkgInfo class for package info
Instead of passing dictionaries around that are inconvenient to
use in code and even more in the `mpp-format-*` directives, use
a simple class to represent package information. Use that in
the `pkginfo` dict that can be accessed via `mpp-format-*`. Use
the `evra` property instead of string manipulation in the
`fedora-boot.mpp.json` and `-ostree-bootiso.mpp.json` manifest.
2021-07-02 11:28:43 +02:00
Christian Kellner
8a27de0d10 tools/osbuild-mpp: include nevra in package list
Include the nevra in the list of dep-solved packages. Use that for the
package info. Adapt the boot manifest accordingly.
2021-07-02 11:28:43 +02:00
Christian Kellner
ff76b6fade tools/osbuild-mpp: small whitespace fixes
Since David is on leave, I have to do them. Also Move the pylint
comment above the line, not next to it.
2021-07-02 11:28:43 +02:00
Christian Kellner
44fec6cd91 tools/mpp: rename mpp.py to osbuild-mpp
Prepare the optional installation of the script via a more unique
and langauge independent name.
2021-07-02 11:28:43 +02:00
Renamed from tools/mpp.py (Browse further)