Implement a phase for the `imageBuilderBuild` task that is provided by
the `koji-image-builder` plugin, which schedules tasks to run with
`image-builder`.
This change is part of an accepted change proposal [1] for Fedora to use
`koji-image-builder` to build (some of) its variants.
[1]: https://fedoraproject.org/wiki/Changes/KojiLocalImageBuilder
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
This option can be set for a particular image or globally for all
kiwibuild images (with individual overload).
Fixes: https://pagure.io/pungi/issue/1833
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When the compose is configured to include any container image, it just
followed the provided URL. This is not particularly reproducible. If the
image spec contains a tag, it may point to different images at different
time.
This commit adds a step to validating the configuration that will query
the registry and replace the tag with a digest.
This makes it more reproducible, and also fixes a problem where changing
container image would not stop ISO reuse. There's still a chance of
non-container file changing and not forcing the reuse, but that is not
very common.
JIRA: RHELCMP-14381
JIRA: RHELCMP-14465
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The default base name is probably fine in most cases, but there
are some where we might want to tweak it. We already allow this
for other phases (e.g. the livemedia phase).
Also, we should include the arch in the image filename. Not doing
this doesn't blow up the compose as, while they have identical
filenames, the images for different arches are in different paths,
but it's confusing for people who actually download and use the
images.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The image metadata construction code allows for subvariant to be
set in the image config dict, but checks.py doesn't expect it;
fix that. Also, when a subvariant is set, use it in the image
name template rather than the variant; otherwise you can't
build more than one subvariant in any variant (they will have
identical names, which isn't alllowed).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It is not possible to reliably detect what the type for an image should
be in the metadata. This commit adds an option for user to explicitly
provide it.
It can only be configured on the specific image, not globally.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The version follows the same rules as versioning for live media etc.
That means it's always going to be set. The precedence goes like this:
* image specific option
* `kiwibuild_version`
* `global_version`
* `release_version` or `<release_version>_<label_milestone>`.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This was only usable in live_images phase that doesn't exist anymore,
and wasn't used much in the first place.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This phase was used to create live images with livecd-creator
and 32-bit ARM images with appliance-creator. We also remove
get_create_image_cmd from the Koji wrapper as it was only used
for this phase, remove associated tests, and remove related
configuration settings and documentation.
Fixes: https://pagure.io/pungi/issue/1753
Merges: https://pagure.io/pungi/pull-request/1774
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Neal wanted this to work - he tried using global_description_scm
and global_description_path in the initial PR - but it wasn't
wired up to work. This should make it possible to set
`kiwibuild_description_scm` and `kiwibuild_description_path`.
It also technically lets you set `global_` for both, since the
`get_config` implementation is very generic, but it doesn't add
it to the checks, so you'd still get an "unrecognized config
option" warning, I think. It seems appropriate to encourage
setting this as a phase-level option rather than a global one
since it seems quite specific to the kiwibuild phase.
Merges: https://pagure.io/pungi/pull-request/1737
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The osbuild Koji plugin supports passing customizations for an image
build. This is also supported in the Koji CLI plugin. Some teams want to
pass image customizations for images built as part of Pungi composes.
Extend the osbuild phase to support passing customizations in the Pungi
configuration.
Merges: https://pagure.io/pungi/pull-request/1733
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This patch connects the phase into the main script, and adds other
modifications:
* The archive is now stored in the images/ subdirectory in the compose.
* Documentation is updated to correctly mention that variant repos are
not available.
* Configuration for path and name of the final archive is dropped. There
are reasonable defaults for this and there's no point in having users
configure it.
* The extra message for the archive is no longer sent.
* The pungi-make-ostree utility is no longer required in the buildroot.
The pungi-make-ostree utility doesn't do any significant work. It
modifies configuration files (which can happen on the compose host), and
it starts other processes.
This patch changes the ostree-container phase to no longer need the
script in the buildroot. Instead, the utility is called on the compose
host to do the config manipulation and output the needed commands. Those
are then passed into the runroot task.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
```
pungi/checks.py:575:17: F601 [*] Dictionary key literal `"type"` repeated
pungi/phases/pkgset/pkgsets.py:617:12: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:241:16: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:244:16: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:370:16: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:374:20: E721 Do not compare types, use `isinstance()`
```
Signed-off-by: Timothée Ravier <tim@siosm.fr>
Allow the manifest type used to be specified in the pungi configuration
instead of always selecting the manifest type based on the koji output.
Signed-off-by: Simon de Vlieger <cmdr@supakeen.com>
With this patch, Pungi can be configured with a local directory to be
used as a cache for RPMs, and it will download packages from Koji over
HTTP instead of reading them from filesystem directly.
The files from the cache can then be hardlink as usual.
There is locking in place to avoid different composes running at the
same time to step on each other.
This is now supported for RPMs only, be it real builds or scratch
builds.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch adds an additional field `options` to scm_dict, which can be
used to provide additional information to the backends.
It implements a single new option for GitWrapper. This option allows
setting a custom git credentials wrapper. This can be useful if Pungi
needs to get files from a git repository that requires authentication.
The helper can be as simple as this (assuming the username is already
provided in the url):
#!/bin/sh
echo password=i-am-secret
The helper would need to be referenced by an absolute path from the
pungi configuration, or prefixed with ! to have git interpret it as a
shell script and look it up in PATH.
See https://git-scm.com/docs/gitcredentials for more details.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
JIRA: RHELCMP-11808
Copying ExcludeArch/ExclusiveArch from source rpm to noarch is an easy
option to block shipping that particular noarch package from a certain
architecture. However, there is no way to bypass it, and it is rather
confusing and not discoverable.
An alternative way to remove an unwanted package is to use the good old
`filter_packages`, which has enough granularity to remove pretty much
anything from anywhere. The only downside is that it requires a change
in configuration, so it can't be done by a packager directly from a spec
file.
When we decide to break backwards compatibility, this option should be
removed and the entire ExcludeArch/ExclusiveArch inheritance removed
completely.
JIRA: ENGCMP-2606
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The `koji-osbuild` plugin supports additional formats for the `repo`
property since v4 [1]. Specifically, a repo can be specified as a
dictionary with `baseurl` key and `package_sets` list containing
specific package set names, that the repository should be used for.
Extend the configuration schema to reflect the plugin change.
Extend the documentation to cover the new repository format.
Extend an existing unit test to specify additional repository using the
added format.
[1] https://github.com/osbuild/koji-osbuild/pull/82
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The `koji-osbuild` Hub schema has been relaxed a bit in the latest
release (v11). Adjust the schema in Pungi to reflect changes in
`koji-osbuild`.
For more information on the changes in `koji-osbuild`, see:
https://github.com/osbuild/koji-osbuild/pull/108
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Since version 9, the `koji-osbuild` plugin supports specifying upload
options as part of a Koji build. This enables one to upload the built
image directly to the cloud environment as part of the image build in
Koji.
Extend the configuration schema with `upload_options`.
Extend the documentation and describe valid `upload_options` values.
Add a unit test testing a scenario when `upload_options` are specified.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Modify the osbuild configuration schema to accept only an array with a
single value as the `image_types`, in addition to a single string. The
single string was supported by the schema also before, but this fact was
not mentioned in the documentation, nor it was supported by the
`koji-osbuild` plugin of version lower than `9`.
Update the documentation accordingly.
Add unit test for invalid configuration containing more than one image
type.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
In order to start building Fedora IoT images with osbuild, we need to be able
to pass ostree options from pungi to the koji's osbuildImage task.
This commit adds support for it via new configuration options: ostree_url,
ostree_url and ostree_parent.
A test was added to cover these new options and they are were also added
into the documentation.
JIRA: COMPOSER-1702
Merges: https://pagure.io/pungi/pull-request/1624
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Fedora Rawhide (to be 37) packages jsonschema 4.9.0 at the moment, so we
can no longer get by with limiting the requirements. This patch makes
the validation work with both old and new version.
Fixes: rhbz#2113607
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When nothing in configuration or the image itself changed, let's just
copy the older one.
JIRA: RHELCMP-5969
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch allows Pungi to reuse ISO image created in previous compose
if a list of assumptions proves to hold:
* If image is bootable, buildinstall phase must have been reused too.
* Compose configuration must have not changed (except for a few
whitelisted options).
* Volume ID of the ISO much not have changed.
* No RPM on the ISO must have changed.
The ISO also contains other files. Changes in extra files and product ID
certificates should be visible in configuration (the SHA will differ).
Similarly any repodata configuration would be reflected in
configuration.
JIRA: RHELCMP-5969
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
