With the new images library changes to move the pipeline
roles out of the image types into the manifest we need
to tweak osbuild-composer to use the new way of getting
the payload and build pipelines.
This commit implements that new method based on the
manifest instead of the image type. See images pr#1766
for the rational of the change.
This commit fixes the issue that --extra-repo are not added
correctly to the repo registry when doing cross arch building.
The issue is that they get added early at the reporegistry
level and because we don't know yet what arch will be used
it was initially only added to the host arch. This was
fine before we supported cross-arch building.
But now that we do support cross-arch building this is
of course an issue and produces subtle bugs. This commit
is the "easy" fix, the extra repos are just added to for
all architectures. This may not be right but because they
are optional its the reponsibility of the user to provide
the right ones. Eventually we should probably be smarter
here and move extra repo support into manifestgen but
that requires changes in images and an images release so
this is good enough for now.
Thanks to Simon de Vlieger for reporting this.
Drop using the "images" library blueprint types and use the ones
from github.com/osbuild/blueprint/pkg/blueprint instead.
This also moves to the images library to v0.172.0 and the blueprints
library to v1.12.0 as this is required for this to work.
The current MockCmd.Calls() is a bit ambiguous so follow the example
of pytest here and rename to the cleaner CallArgsList() that
captures more closely what the helper is actually doing.
While looking over the code I noticed that the `Restore()` helper
is not needed (and arguably wrong as it does not reset PATH).
We already use `t.TempDir()` and `t.Setenv()` as part of the
command setup so manually cleanup is not neccessary (and is today
even incomplete). So YAGNI and we can remove Restore().
Include the commit of the build (if available in the build environment),
also include the version of `images` in the build; if available.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Make sure that images built by ibcli have correct rhsm.Facts set for
distributions that set it (this is handled by the distro definition, not
by the caller of the manifest generator).
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The '*-simplified-installer' images require the installation device
to be specified as a BP customization to serialize their Manifest.
The 'describe' command was not setting those, which resulted in errors
when describing such image types.
Set a dummy installation device BP customization for image types with
their name ending with '-simplified-installer', to enable describing
of such images.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
OSTree images require additional arguments to serialize their Manifest.
The 'describe' command was not setting those, which resulted in errors
when describing such image types.
Set the required minimum OSTree arguments to dummy values to enable
describing of such images.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add an unit test to verify that the `describe` command can describe all
image types. I've recently ran into issue when describing ostree-based
and `*-simplified-installer` image types.
Add new dependecy for working with YAML - sigs.k8s.io/yaml.
This is what we will probably converge to, since our currently used YAML
library is unmaintained.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This is another convenience feature for the `image-builder upload`
command: when we do not know the target architecture try to
guess it from the filename. This is not perfect but it will
help a lot of users and should be fine until the day we go
and inspect the image.
When using `image-builder upload --to=aws` we do not know
the bootmode. Ideally we would introspect the image for
the boot mode but that is not trivial right now.
So for now just default to platform.BOOT_HYBRID which
translated to `uefi-preferred` in the AWS API calls
which should offer the widest compatbility and should
fix the issue that aarch64 does not boot currently
when uploaded via ibcli.
This commit passes the boot mode to AWS when doing a combined
build/upload. Here we know what boot mode to use and we can
pass it easily.
This also adds a "XXX" to think about how to handle what to
do when uploading a pre-existing image where we do not know
the boot mode.
This commit fixes the cross-arch uploading in the most simple
case by reading the `--arch` when `image-builder build --upload`
is used. Note that this is not a complete fix as it will not
take boot mode into account nor will it (by default) DTRT when
`image-builder upload` is used on a previously build images
for a different architecture.
Thanks to `FrostyX` for reporting the issue.
Image types were renamed [1] in images for the Fedora distribution.
While aliases were left behind for compatibility those are not used
when directly requesting manifests it seems.
Let's rename them to their canonical names.
[1]: 65194a4bf
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
There was previously a bug in the Insights client configuration stage
which omitted a key [1]. A testcase hardcoded this invalid format. Let's
update it to the expected value.
[1]: 389b629ce2
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Fedora 40 was removed from `images` as it is EOL, we can no longer use
it in our test cases. Bump to Fedora 42.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
This commit adds error handling for the `f.Close()` errors when
we write the SBOM. Errors on close for RW fds are rare but we
should handle them so we return the result of `f.Close()` now
when returning in sbomWriter(). We still keep the `defer f.Close()`
to ensure we do not leak file descriptors when e.g. `io.Copy()`
fails. In the "happy" case f is closed without an error and
then the defer f.Close() runs and will error with "ErrClosed"
but we can ignore that.
An alternative implementaiton might be:
```golang
func sbomWriter(outputDir, filename string, content io.Reader) (err error) {
...
f, err := os.Create(p)
if err != nil {
return err
}
defer func() { err = errors.Join(err, f.Close()) }()
...
return nil
}
```
no super strong opinion here.
Thanks to Flo for finding this issues!
Add a unit test for testing that providing only the required command
line arguments produces expected output.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This new flag allows to add a file with registration data. This
is meant to eventually hold all sort of registrations like
ansible or satelite but initially only contains the redhat
subscription. Currently only JSON is supported.
It looks like:
```json:
{
"redhat": {
"subscription": {
"activation_key": "ak_123",
"organization": "org_123",
"server_url": "server_url_123",
"base_url": "base_url_123",
"insights": true,
"rhc": true,
"proxy": "proxy_123"
}
}
}
```
This is not part of the blueprint (today) because its more
ephemeral than the things we usually put into the blueprint.
This allows us to build images that are immediately registered. It
also keeps our options open in the future. If we move to a new
blueprint format where we support multiple blueprints and also
ephemeral data like this the "registrations" flag just becomes an
alias for "--blueprint".
This commit fixes the issue that a test blueprint was not converted
from json to TOML. This was not caught in CI apparently because
our test container misses createrepo_c.
Adding the disk customization to the blueprint test helps to
ensure that strict TOML checking works as expected. There was
a (brief) regression because manifest generation with disk
toml was not tested.
The `completion` command doesn't need to be quite so discoverable. Let's
have only actual subcommands listed and mention this in (future)
documentation instead.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Other commands such as `manifest`, and `build` auto detect the
distribution if none is given. `describe` is the odd one out that
requires `--distro`. Let's also autoselect there.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Rename the `list-images` command to `list`. We don't have `-image(s)` in
our other subcommands so this is for consistency.
We keep a `list-images` alias behind for compatibility reasons.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
When running `image-builder` warnings are emitted during manifest
generation. Depending on the definitions or customizations packages
can be excluded from groups which leads to:
```
No match for group package "dracut-config-rescue"
```
Interspersing with normal output. Let's pass along another byte buffer
for the warnings from manifest generation to be written into.
Note that this also needs plumbing [1] to land in `images` first.
[1]: https://github.com/osbuild/images/pull/1384
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Change the progress imports to refer to `pkg/progress` inside this
repository as it was imported from `bootc-image-builder`.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Allow users to define the seed that's used for manifest generation.
Regenerating an image with a given seed leads to the same manifest
(provided depsolving does the same).
The seed is normally mostly used to generate random filesystem UUIDs.
This will need a bunch of documentation in a follow up since the use
cases are meant to be advanced but it can really speed up multi-type and
rebuilds of the same image.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
This commit tweaks the handling of the `--output-name` option so
that is a name with the same extension as the image is passed that
is just silently ignored. Its a common issue that first time
users run:
```console
$ image-builder build --output-name foo.qcow2 qcow2
```
which currently leads to a foo.qcow2.qcow2. With this commit
the expected "foo.qcow2" will appear.
This commit adds the content of the output directory when a
build is finished. This is a convenient feature to make it easier
for the users.
Thanks to Simon for suggesting this!
This simplifies the way that ibcli determines if the manifest
needs a bootstrap stage or not. The logic is simple:
- when doing `image-builder manifest` we never include one
- when doing `image-builder build --arch` we add one if the
target arch and host arch differ
Instead of passing a callback this is now just a simple bool
option. Thanks to Achilleas for suggesting this.
Co-Authored-by: Achilleas Koutsou <achilleas@koutsou.net>
A small test for a basename that includes dotted parts (as used by for
example Fedora's buildsystem).
Also some code changes to make this test pass, it seems that SBOM base
was determined based on some other filename which is now no longer
relevant?
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
This commit skips the arch checks if the experimental "bootstrap"
option is used. The main use-case of this option is to bootstrap
a foreign architecture so just assume that and skip arch checks
when set.
This allows to write:
```
$ IMAGE_BUILDER_EXPERIMENTAL=bootstrap=ghcr.io/mvo5/fedora-buildroot:41 \
./image-builder build --arch=riscv64 minimal-raw --distro=fedora-41
```
and do a riscv64 cross arch build.
