Move the filesystem customization tests into their own file.
Additionally add tests for unmarshalling filesystem customizations
from TOML, since we have added a new `minsize` tag to ensure
consistency with the json tag.
The new tests check the validation of the TOML input and ensures
that either one or both of the `minsize` and `size` inputs are
set. If both are set, the input is checked to ensure that both
match.
Due to an oversight, the toml and json tags for the `MinSize`
field had different keywords. This commit fixes this by creating
a `minsize` toml tag and ensuring backwards compatability by
checking the old `size` tag.
If both `minsize` & `size` are set in the toml block, the
custom unmarshal function validates the input for inconsistencies.
Forgot the #1 rule of map iteration. Don't depend on the order.
This uses a slice instead, which should be just as good for testing the
loop variable alias behavior.
The SourceConfig pointer may be a loop variable that gets reused. This
results in unexpected behavior when the value pointed to is overwritten
by the loop calling this function.
Includes a test to make sure this is fixed.
So, DO NOT point to unsafe variables. Make a new pointer using
common.ToPtr where it is passed by value and returns a pointer to that
new value.
NOTE: This is NOT caught by golangci-lint. There may be other places
where this happens, but I have gone through the potential looking code
in osbuild-composer and images and not found any (other than a couple
places already noted with G601 tags as not a problem).
These functions are leftover from the `osbuild/images` split and are not
used by any code in osbuild composer. Instead, the version in
`osbuild/images` is used by distro definitions.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This function is no longer used by any code. Instead, its copy in the
`osbuild/images` repository is used by distro definitions to validate
the customization.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This copy of the package in osbuild-composer is no longer used by any
code. Instead, the copy which is in the `osbuild/images` repository (and
module) is the implementation used by distro definitions.
Delete this code to reduce potential confusion and code duplication.
Also delete code in `internal/blueprint` package, which uses the
pathpolicy package. This code is no longer used, but instead the version
from `osbuild/images` is being used.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
When uploading and distributing a commit, wait for any async tasks to
finish before returning. There are two tasks that can block this
function:
- Creating a distribution: this only happens when a new repository is
created.
- Import commit: this will always happen in this function.
Helper function that performs the whole upload and distribute procedure.
Two more helper functions are added for retrieving the href for a
repository based on its name, and for retrieving the base URL for a
repository's distribution.
Define the task state enum based on the available values defined in the
API.
Add a helper function that returns true if a task is running or waiting
and ignores errors.
Function for importing a commit artifact (that's already been uploaded)
into a given repository. Note that the "repo" argument to the
NewOstreeImportAll() function refers to the name of the repository
directory inside the archive, which for the commits we produce in
osbuild is always "repo".
There is a lot of repeated checks for bp.Customization != nil, this
simplifies that by creating an empty blueprint.Customizations at the
top, and checking to see if it is still empty at the bottom and setting
it back to nil.
Includes a new test for calling with an empty (not nil)
v2.Customizations set on the request.
The function is a leftover from the image definitions split and it is
not used. Moreover, the `images` copy of it is being reimplemented by
[1]. It is better to remove this copy to prevent any unintended use of
it or confusion.
[1] https://github.com/osbuild/images/pull/195
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Include the osbuild version used to build the image, in the image extra
metadata attached to the Koji build and image output.
Extend `koji.sh` to verify that the version is set and of the expected
value.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This will make it easier to reproduce image builds using the same
tooling which was used by the worker.
This change will enable to include the osbuild version in Koji build
metadata.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the extra metadata attached to each manifest output, also to the
build extra metadata under `osbuild_manifest` property. The value is a
map of all manifest metadata with keys being the filename of each
manifest.
This will expose the information about osbuild-composer and
osbuild/images versions used to produce the manifest in the Koji
buildinfo, including the Web UI.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Include the osbuild-composer and its dependencies versions in the extra
metadata associated with the Manifest output when importing it to Koji.
This will make it possible to pin-point the exact version combination
which was used to generate the osbuild manifest used to built the image
imported to Koji.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Copy the Manifest info data from the Manifest job result to the Koji
target result, so that this information can be then imported to Koji
build metadata by the koji-finalize job.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Include the osbuild/images module version in the Manifest job result.
The module has direct impact on image definitions and the content of
produced manifest, therefore including this information in the Manifest
job result is very helpful for various purposes (debugging,
traceability).
This will enable to embed this information in the Koji build metadata.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the Manifest job result structure to hold information about
osbuild-composer version, which produced the manifest. This will be
useful for other job types which depend on it and can then push this
information further as needed.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the osbuild information used to produce the image, in the image
extra metadata under the `osbuild_artifact` property.
This information will get included in the image / build extra metadata
and make it explicit how to reproduce the image build using osbuild and
the attached manifest.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add the information about osbuid artifact to the target result.
Specifically the name of the osbuild pipeline which was exported for the
specific target, and the filename of the exported file.
This will later enable embedding this information in Koji build metadata
to make it easy to reproduce the image build using the attached
manifest.
Add any non-Koji upload target results attached to an OSBuild result, to
the image extra metadata. This will make it easy to locate any image
from Koji uploaded to cloud, in the target cloud environment.
The rationale behind including only non-Koji target results is that one
can find it only in Koji, so there is no added value in including the
Koji target results at all.
Extend the `koji.sh` to check the target results in image metadata when
testing Koji scenario with cloud upload.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Import osbuild manifest and build log to the Koji build as outputs. Also
note the respective filenames in the image output extra metadata.
Note that the osbuild manifest is imported as a log file for now. Koji
has very limited set of output types defined and I still need to
determine the best way to use a custom output type in Koji instances (as
other content generators do).
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The `KojiTargetResultOptions` previously contained information only
about the uploaded image file. And even then, some information, such as
the filename, were scattered in other structures such as
`KojiFinalizeJob` struct.
Since the plan is to start uploading also osbuild manifest and osbuild
build log to Koji, we need to extend the result options structure to
hold more information and also make it specific to which file is the
information related.
Rework the `KojiTargetResultOptions` to contain information about:
- the built image
- build log
- osbuild manifest
Information about each file contains:
- filename
- checksum type
- file checksum
- file size
For now, only the built image information is set and consumed by the
worker.
Add custom JSON (un)marshaler for `KojiTargetResultOptions` to handle
backward compatibility when old version of worker or composer server
interact with each other. Cover them with unit tests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
