Previously, the Cloud API endpoint `/v1/compose/{id}` return value's
`image_status.status` for a running worker job was "running", which didn't
comply with the Cloud API specification. Equivalents allowed by the API
specification are "building", "uploading" or "registering".
As a result, the Image Builder API also does not comply in this regard
to its specification, because it currently just copies the status value
string returned by osbuild-composer.
Define the `image_status.status` as a reusable type in the Cloud API
specification. This forces openapi to generate an explicit type for it,
which can be then explicitly used in the code, instead of plain strings.
Return "building", instead of "running" for running compose.
Modify api integration test to check for all valid `image_status.status`
values for a compose.
Add News entry explaining this change.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
A result from manifest v2 contains logs from pipelines. The individual
pipelines are stored as an object, thus they have no order. Well, at least
in Go, because it doesn't guarantee one when parsing maps, see:
https://github.com/golang/go/issues/27179
Unfortunately, this makes the Result.fromV2 method return unpredictable
results because the pipeline results are processed in basically a random
order.
This caused the TestUnmarshalV2Failure test (result_test.go:124) to randomly
fail because it expects the ordering to be stable. I decided to fix this by
ordering the pipeline results by their name. When this fix is applied, the
output from Result.fromV2 is well-defined.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Detecting if a result is in the v1 format using DisallowUnknownFields was
clever. However, the format of v1 result changed in the worst way at some
point in the past: A field (id) got added. Therefore, this was an unknown
field to osbuild-composer and it fall back to decoding the input as a v2
result which led to empty logs.
This commit reimplements the result schema check: The new format always
have a top-level type field, whereas the old format doesn't have it. The
new implementation uses this fact to distinguish between v1 and v2 format
of a result.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
When a stage is successful in a manifest v2, the success field is omitted from
the result. In other words, the default value of the success field is true
which is against the default value of boolean in Go. This commit implements
a workaround.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The previous version constructed multiple temporary variables and then
create job result from them. This was needed because we had multiple
upload targets but now that we have only one, this is only fragile
version of what can be done in a simplified way.
This PR removes the temporary variables and assigns errors and success
states right after the upload or build has finished.
Multiple upload targets are not supported by osbuild-composer any more.
Dropping support for this in worker therefore doesn't change anything
from the user's perspective, but it allows us to simplify the code a
bit.
Replace calls to "continue" with "return nil" because the job finished
correctly even though it failed to perform the task. But the failure was
reported to osbuild-composer for further processing so there is no need
to duplicate and report the same error in worker process
Drop support for LocalTarget, this has not been used in a long time,
and we don't really need to stay compatible across many releases
(just as long as we don't get problems with having to deploy in
lock-step), at least not yet.
Also drop support for KojiTarget, this has been replaced by the
osbuild-koji job type.
The previous implementation exited before reporting back to the worker
API in few branches. This left the compose status in RUNNING state even
though the worker did not work of the job any more. Refactoring the
API call into the `deref` part makes sure it gets called every time.
This commit only moves bits of the code around so that the status gets
back to osbuild-composer, but it still doesn't contain any useful
information in case osbuild fails etc. This will be introduced in
subsequent commits.
Run tests with user-specified ostree ref.
The ref needs to be specified in the container build as well (without a
URL) to be used for the commit that will be served to the installer.
The same variable/ref is also used to retrieve the $UPGRADE_HASH for the
final OS test.
Bug fix for changes introduced in #1244.
The new image types, rhel-edge-container and rhel-edge-installer, would
ignore the user-supplied ostree ref and use the default everywhere.
The default should only be used when a ref is not specified, which the
weldr API takes care of before calling the Manifest() method.
User home directories don't survive the rpm-ostree stage. They are
converted to systemd-tmpfiles via rpm-ostree post-process, but the
contents are left behind, so any keys we add to the authorized_keys file
will be gone.
This stage sets up a first-boot service that writes the user's public
key to the file in the home directory during the first system boot.
In the Anaconda pipeline, the kickstart stage should fetch the commit
we're embedding. It was mistakenly trying to fetch from the URL used to
build the image instead.
New image type that generates a Boot ISO. The ISO contains a RHEL Edge
commit and an installer. On Boot, it sets up a new RHEL Edge system
with the commit.
The RHEL Edge commit (ostree commit) is downloaded during build from a
URL that should be supplied with the compose request. The commit's hash
and URL need to be added to the Sources list in the Manifest.
Unlike other types, the new image type defines its own "build" package
set that is added to the distro and arch build package lists.
We need to add the URL to the manifest as an ostree source repo so that
osbuild can pull the commit to embed it in the boot ISO for the new
rhel-edge-installer image type.
- org.osbuild.anaconda
Configures Anaconda. For now, only enabling kickstart modules is
supported.
- org.osbuild.buildstamp
Creates a buildstamp file, which is required by Anaconda.
- org.osbuild.kickstart
Creates a kickstart file.
- org.osbuild.lorax-script
Uses lorax template helpers to execute a template.
- org.osbuild.bootiso
Prepares a bootable file system tree suitable for writing on an ISO
file system
- org.osbuild.discinfo
Creates a .discinfo file, used by the Anaconda installer.
- org.osbuild.xorrisofs
Uses the `xorrisofs` command line utility to an ISO.
- org.osbuild.implantisomd5
Uses the `implantisomd5` command to implant MD5 checksums into an
ISO.
The org.osbuild.oci-archive stage now supports an arbitrary number of
layers on top of the Base layer. The keys for these layers follow the
pattern "layer.N" (N = 1, 2, 3, ...).
We use a custom marshaller and unmarshaller for the
OCIArchiveStageInputs to handle this. The unmarshaller also validates
the layer keys to match the pattern in the schema.
Recently added stages org.osbuild.sysconfig and
org.osbuild.kernel-cmdline were missing from the Manifest unmarshal
method causing it to fail when trying to unmarshal manifests that
contained them.
When shelling out for a CLI test the error returned from the Start()
command prints the exit code which is not very informative. Capturing
and printing stderr is a lot more useful.
osbuild requires the export flag otherwise it wont produce an artifact.
For the older manifest format (v1), the export value is always
"assembler". For v2 manifests, it is the name of the last pipeline.
If an unknown version number is read the script now fails. This should
help catch manifest changes that may affect test case generation in the
future.
Log output from osbuild has a very different format when using the
new schema. The osbuild1.Result object now supports unmarshalling the
new format and adapting it to the old format.
The most important field to set is the Success field to signal whether
the build succeeded.
Secondarily, it also copies over the output from each stage in order to
provide build job log output through the weldr API.
Since the new format contains multiple pipelines with multiple stages
each, the stages are flattened to fit the old format. A unique name for
each stage is created by prepending the name of the pipeline to its
index in the pipeline and its type.
