Update the progress line only when another line was received, which in
this case means a job has started or finished.
No need to keep reprinting the progress.
This abstracts away the manifest instantiation. The idea is that we define one
of these image kind types to represent a group of image types that are
sufficiently similar. Each image kind will have a struct with with all the
properties that can be customised for the image and a function to turn that into
an actual manifest. This is similar to how distro/fedora/manifest.go and
cmd/osbuild-playground works today, and aspires to move these closer together
and to eventually make the distro definitions simpler.
For now cmd/osbuild-playground is moved over to using the new abstraction.
This is meant to encapsulate the tweaks we do to the OS tree
orthogonally to anything else. For now it still contains some
configuration that only sometimes applies, but this should
continue being reworked until all the fields in this struct
always apply to any artefact that is using it.
At the same time, stop instantiating with default values, as the
empty values should work. This is not a functional change as the
caller always sets these now.
The AWS and Azure RHUI images are produced as compressed archives, which
can be uploaded to Koji, but they can't be uploaded to the cloud
provider in this format. To support cloud upload for these types of
images, we need to decompress them before the upload.
Add a workaround for AWS and AzureImage targets to check if the image
has `.xz` suffix and if yes, decompress it before uploading to cloud.
This workaround is needed until image definitions will support and use
multiple exports per image to allow using different export per upload
target.
Enhance the `koji-finalize` job implementation to be able to cope with
multiple upload targets being specified for an `OSBuildJob`.
Implement a convenience method `OSBuildJobResult.TargetResultsByName()`
for filtering the target results attached to the job result by their
name. Cover the method with an unit test. And lastly use this method in
the `koji-finalize` job to find the appropriate Koji upload target
results.
This is a preparation for enabling cloud uploads for Koji composes.
Enhance the `koji-finalize` job implementation to use deferred function
to ensure that the job status is always reported back to the composer.
In addition, if the `JobError` is set, also fail the Koji job.
Previously, composer and Koji were not updated in some corner cases when
the job would fail.
Add a new `containers` section that can be used to request the
embedding of containers into images. The only requirement is
the source property to specify where to fetch the container from.
This suppports specifying the digest of the container or the tag.
In case none is given it defaults to the `latest` tag. The `Name`
field can be used to optionally specify a name to use inside the
image.
NB: currently no tools or apis support container resolution yet.
This follows in the next commits.
This is the first step to support embedding container images. Here
we add the `containers []container.Spec` argument to supply images
with resolved container specifications. For now all distros will
return an error in case a container is actually supplied since none
of them currently support embedding containers. NB: also no apis or
tools will actually resolve containers.
Instead of keeping an extra field in `Client`, we just use the
existing `sysCtx.DockerAuthConfig` structure. When the context
is later copied during the upload operation the credentials
will be copied as well. It also saves us from syncing the
credentials if we directly use said `sysCtx` for operations.
Instead of having an extra field, `TlsVerify`, on the `Client` and
then later setting the corresponding `SystemContext` options, use
the existing `SystemContext` field of `Client`. The corresponding
field is a tri-state: unset, true, false, which is represented as
a pointer to boolean in the `Client`'s new getter and setter. This
also inverts the boolean logic from verify TLS to skip TLS which
aligns very well with the corresponding fields in the upload target
struct.
In addition we properly capitalize some existing variables.
Koji API removed by the previous commit was the last user of osbuild-koji job.
Let's remove it since nothing uses it. This also removes all of the
compatibility code in Cloud API, see concerns below:
Compatibility concerns:
- the internal deployment was moved to a completely different composer
instance, thus there are no old jobs
- Fedora deployment is still unused in prod, thus we don't care about keeping
backward compatibility of the old jobs
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
We no longer use it, let's remove it. If you are wondering what to use instead,
use Cloud API. It supports everything that Koji API supported and more.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This introduces an expiry date (default: 14 days from insert date) and
adjust the service-maintenance script to delete jobs that are older than
the expiration date.
We have three kinds of operating system trees, until we unify them to one,
hide them behind one interface. Use this to read the architecture from the
Tree rather than pass it in as a string to parent pipelines.
Also, make the filename parameter optional in a few places, there should be no
reason to set this rather than introspect it (except for backwards
compatibility).
Lastly, add another playground example sample to build a raw image.
For now all it does is represent the name of the runner and what requirements
it has of the build pipeline.
Move some package definitions from the runner package set to where it belongs.
These objects describes the hardware an image runs on. Including
- architecture
- bootloader
- required firmware
Use the platform abstraction to move firmware packages out of the package set
definitions.
The kernel name is optional and can be set later.
The bootloader we skip entirely. Instead, set the architecture, which now becomes
mandatory. Use it to deduce the bootloader, and in the future other pipelines can read
this property from the OS Pipeline, rather than having it passed in.
These should both default to being disabled, so move them away from the constructor.
Rename grubLegacy to BIOSPlatform and document that setting it enables BIOS support.
The OSTree parameters can be set after initialisation. We should only require parameters
to be set at initialisation time if we have no good defaults. In the case of OSTree the
default is to not enable OSTree support.
This is meant for rapid prototyping of single image types and for
osbuild development, as an alternative to osbuild-mpp. The same
primitives are used as in the image definitions, but without any
policy or inheritance applied.
The user is expected to only edit `playground.go` and then run
the tool to produce osbuild manifests.
The package sets for an image can depend on the blueprint, and
by the same logic there is no reason it should not be able to
depend on the image options.
This is so far a non-functional change, but makes a follow-up
commit simpler (though still without actually depending on
the image options to compute the package sets).
With 8.3 support dropped, the rhel86 package defines all the supported
RHEL 8 versions except 8.4.
RHEL 8.4 will be merged into the rhel8 package soon.
Drop the fallback to the `assembler` export if no is specified in the
job and return Job Error in this case.
Remove the constraint to support only a single osbuild export. The job
is now able to use multiple osbuild exports and each target may use a
different one.
The osbuild export is specific to the upload target and different
targets may require using a different export. While osbuild-composer
still does not support multiple exports for osbuild jobs, this prepares
the ground for such support in the future.
The backward compatibility with older implementations of the composer
and workers is kept on the JSON (Un)mashaling level, where the JSON
message is always a super-set of the old and new way of providing the
exports to osbuild job.
Weldr API already does not rely on this code and nothing else uses it.
Since the code has been used only on-premise, where we expect the
composer and workers to be always of the same version, there is no need
to keep backward compatibility in the worker.
The uploading of artifacts back to the worker server for the on-premise
(Weldr) use case was signaled to the worker by setting the `ImageName`
in the `OSBuildJob` definition. The code also relies on the osbuild
exports being specified in the `OSBuildJob`, instead of in the target
(this is not implemented yet).
Prepare the ground for moving osbuild export definition from
`OSBuildJob` to `Target` by introducing an explicit `Worker Server"
upload target. This target will signal to the worker that it should
upload the image back to the worker server. The new target is not yet
used by any API implementation.
Extend the worker osbuild job implementation to handle the new upload
target.
The filename of the image as produced by osbuild for a given export is
currently set in each target options type in the `Filename` struct
member. However, the value is not really specific to any target type,
but to the specific export used for the target. For this reason move the
value form target type options to the `Target` struct inside a new
struct `OsbuildArtifact` under the name`ExportFilename`.
The backward compatibility with older implementations of the composer
and workers is kept on the JSON (Un)mashaling level, where the JSON
object is always a super-set of the old and new way of providing the
export filename in the Target.
