This commit fixes a silly mistake from PR#810. The issue is that
in #810 we started to collect the osbuild stdout/stderr so that
we can show crashes from osbuild or other unexpected output.
However when a stage fails this is reported via the json progress
and not directly on stdout/stderr - this was missed when #810
was done.
This commit does a short term fix by collecting the buildlog again
and showing it in the error and also updates the test to be more
realistic. However we really need a test that actually tests
the real behavior, ideally a real osbuild run with a stage error
so that we can be sure we capture this (criticial!) functionality.
This commit tweaks an issue that potentially an incorrect status
from osbuild would fail the build with a bad error message and
without us getting the full buildlog.
This commit adds catpure of os.Std{out,err} when running osbuild so
that we capture the error log and can display it as part of
an error from osbuild, e.g. when osbuild itself crashes.
This gives us more accurate error reporting if anything fails
during the osbuild building.
This commit makes the previously internal `progress` package an
external API. The is the result of the discussion in images
PR#1150 where we decided that the progress module is not a great
fit for the "images" library.
We want to share this code between bootc-image-builder and
image-builder-cli now. In the future we will also want to
use it in the `worker-executor` in `osbuild-composer` to
parse the stream data from `osbuild`.
We plan to merge bootc-image-builder and image-builder-cli
medium term so importing code from bootc-image-buider in
image-builder cli is not that stange.
When we (longer-term) use this code the `worker-executor`
we will need to think about this again and maybe put it
back into images. However this commit unblocks us without
making anything worse.
Moves the files imported from `bootc-image-builder` to the appropriate
path under `pkg/` so they can be imported reverse. Also fix up the
import paths that are in these files to refer to their new locations.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Pulling container images before building would break in the case of
authenticated images on podman machine, since the auth file lives on the
host and not podman machine and won't know about it.
This commit deprecates the `--local` flag and warns users when it is
passed to the CLI so that this won't break things for anyone who might
already be using the flag. This change means that the user will have to
ensure that the container is pulled to the local container store before
initiating the build.
Co-authored-by: Ondřej Budai <obudai@redhat.com>
This commit checks early if cross architecture building support via
`qemu-user-static` (or similar tooling) is missing and errors in
a more user friendly way.
Note that there is no integration test right now because testing
this for real requires mutating the very global state of
`echo 0 > /proc/sys/fs/binfmt_misc/qemu-aarch64`
which would make the test non-parallelizable and even risks
failing other cross-arch tests running on the same host (because
binfmt-misc is not namespaced (yet)).
The getContainerSize() was not using some of the modern go helpers.
So let's use `exec.Command().Output()` and introduce a new
`util.OutputErr()` helper that will be able to also show stderr to
the user if the Output() call returns an error.
I ran into the `--local` option not working at least in my
setup with the current container image (which for some reason
hasn't been updated, and predates the latest tip commit
which talks about btrfs)
On this current test system (MacOS + podman 5 + default podman-machine)
things do work with the tip commit.
However...I don't quite understand the need to *both* try to
run `systemd-detect-virt` *and* parse `/proc/self/mountinfo`.
(BTW, the logic for `insideContainer` was really confusing me because
it only returned `true` if we were *not* in a container...I wonder
if that was really intentional?)
Anyways, I think the goal here is just a friendly direct error
message if it doesn't look like the mount is there, which we can
do by just checking for the file path. If it isn't mounted
from the host then nothing will be there.
Signed-off-by: Colin Walters <walters@verbum.org>
When the container storage is checked it may come from a btrfs
subvolume. In /proc/self/mountinfo this looks like:
```
1364 1345 0:33 /root/var/lib/containers/storage /var/lib/containers/storage rw,relatime - btrfs /dev/mapper/luks-1356123c-c7b9-1bd7-a8bf-e13d4cbb5500 rw,seclabel,compress=zstd:1,ssd,discard=async,space_cache,subvolid=257,subvol=/root
```
as a quick fix just compare the suffix of the root instead of the
full path.
Thanks for Ondrej!
Ensure to error early when the user is not passing the required
```
-v /var/lib/containers/storage:/var/lib/containers/storage
```
when doing a local build.
Ensure that the disk size of a container is taken into account
when the image is generated. The current heuristic is that we
just double the container size.
The test will not build an image just generate a manifest and
check that the image file is generated with the expected size.
The way osbuild works is to synthesize a filesystem tree in the
store, then copy it to the disk. This ensures the label for the
store is `root_t` which ends up being the labeling for
the "infrastructure" bits in the `/ostree` repository in the
target root.
This in turn is blocking a lot of things.
Closes: https://github.com/osbuild/bootc-image-builder/issues/149
This ensures that the new `partscan` feature in osbuild works. By
default the containers only have a static snapshot of /dev on a
tmpfs. This means that anything later added by losetup will be
missing inside the container.
It also means that https://github.com/osbuild/osbuild/pull/1468
can be reverted.
In most cases e.g. we do want to show stdout/stderr, and
it's handy to have a debug log when we're running
a subprocess.
While we're here, switch to just forking `cp` in
the setup code.
Signed-off-by: Colin Walters <walters@verbum.org>
Allow users to define the seed that's used for manifest generation.
Regenerating an image with a given seed leads to the same manifest
(provided depsolving does the same).
The seed is normally mostly used to generate random filesystem UUIDs.
This will need a bunch of documentation in a follow up since the use
cases are meant to be advanced but it can really speed up multi-type and
rebuilds of the same image.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
This commit tweaks the handling of the `--output-name` option so
that is a name with the same extension as the image is passed that
is just silently ignored. Its a common issue that first time
users run:
```console
$ image-builder build --output-name foo.qcow2 qcow2
```
which currently leads to a foo.qcow2.qcow2. With this commit
the expected "foo.qcow2" will appear.
This commit adds the content of the output directory when a
build is finished. This is a convenient feature to make it easier
for the users.
Thanks to Simon for suggesting this!
This simplifies the way that ibcli determines if the manifest
needs a bootstrap stage or not. The logic is simple:
- when doing `image-builder manifest` we never include one
- when doing `image-builder build --arch` we add one if the
target arch and host arch differ
Instead of passing a callback this is now just a simple bool
option. Thanks to Achilleas for suggesting this.
Co-Authored-by: Achilleas Koutsou <achilleas@koutsou.net>
Add strict checking for toml keys in blueprints. This allows us
to error early if there are unknown keys in a toml blueprint and
helps our users by spotting e.g. typos early.
This is similar to
https://github.com/osbuild/bootc-image-builder/pull/549
(thanks Ondrej!).
This release includes a fix for the issue that custom unmarshaled
toml was marked in the metadata as "Undecoded()" [0].
With that we can enable strict checking for blueprint configuration.
[0] BurntSushi/toml#425
This commit uses the new `build --arch=` support to perform
a cross arch build for aarch64, ppc64le, riscv64, s390x on
fedora. We could do also centos-9 each arch is already 20min.
A small test for a basename that includes dotted parts (as used by for
example Fedora's buildsystem).
Also some code changes to make this test pass, it seems that SBOM base
was determined based on some other filename which is now no longer
relevant?
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
This commit skips the arch checks if the experimental "bootstrap"
option is used. The main use-case of this option is to bootstrap
a foreign architecture so just assume that and skip arch checks
when set.
This allows to write:
```
$ IMAGE_BUILDER_EXPERIMENTAL=bootstrap=ghcr.io/mvo5/fedora-buildroot:41 \
./image-builder build --arch=riscv64 minimal-raw --distro=fedora-41
```
and do a riscv64 cross arch build.
Let's ship our package in epel-9 and epel-10 as well as all the
supported Fedora branches. I've manually created, built, and created
updates for these branches already.
I'm not adding automation to the epel-10.0 branch in this PR as we need
to probably consider that more carefully. However, I *have* submitted
version 13 for epel-10.0 manually.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Bump to latest `images` to make Fedora Minimal work with all the new
bits and bobs. Manual PR because the other one was being weird.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
