This handles corrupt job json files by skipping them. They still exist,
and errors are logged, but the system keeps working.
If one or more of the json files in /var/lib/osbuild-composer/jobs/
becomes corrupt they can stop the osbuild-composer service from
starting, or stop commands like 'composer-cli compose status' from
working because they quit on the first error and miss any job that
aren't broken.
Add the new upload_statuses under the image_status in the result of the
ComposeStatus object. The first status is also included in the old
top-level 'upload_status' property for backwards compatibility.
Tests are updated to match the new results.
Test some valid and invalid combinations for the GetTargets() upload
target selection.
Includes tests with and without the upload options for the default
target.
Read the upload target types and options in the UploadTargets array of
the ImageRequest and initialise the Target array. If the top-level
(old) UploadOptions are also specified, prepend them to the array using
the image type's default target type.
Each upload target type is checked against a support map for
compatibility.
Add an array of targets in the imageRequest and return an array from
ImageRequest.GetTargets() (renamed from GetTarget()). Currently, the
function still only returns one target, the default for the image type
with the top level upload options.
Separate the target selection in GetTarget() into two steps. First
determine the default target name for the image type and then use the
name to initialise the target object. This is a bit more work (and
double switching) but will be needed to support selecting targets
externally.
Separate the handling of each individual target type into its own
function called by GetTarget()'s case switch. This makes the function
more readable and the target object creation reusable.
Added an empty line after each creation of irTarget to make it easier to
visually distinguish the cases that fall through.
Add an upload_targets field to the image request. This lets the API
caller specify multiple upload targets and upload options to be used.
If the upload target type does not match the upload options, the request
is invalid.
For backwards compatibility, the upload targets field is optional. If
it is not specified, the default upload target and upload options for
the image type are assumed, which is the same as the old behaviour.
Adding an explicit selection to the request makes it possible to support
multiple upload targets for the same image type. We plan to support
ostree commits being uploaded to both aws.s3 and pulp.
To report on the multiple upload requests, we add an upload_statuses
field to the ImageStatus response.
Move the filesystem customization tests into their own file.
Additionally add tests for unmarshalling filesystem customizations
from TOML, since we have added a new `minsize` tag to ensure
consistency with the json tag.
The new tests check the validation of the TOML input and ensures
that either one or both of the `minsize` and `size` inputs are
set. If both are set, the input is checked to ensure that both
match.
Due to an oversight, the toml and json tags for the `MinSize`
field had different keywords. This commit fixes this by creating
a `minsize` toml tag and ensuring backwards compatability by
checking the old `size` tag.
If both `minsize` & `size` are set in the toml block, the
custom unmarshal function validates the input for inconsistencies.
Forgot the #1 rule of map iteration. Don't depend on the order.
This uses a slice instead, which should be just as good for testing the
loop variable alias behavior.
The SourceConfig pointer may be a loop variable that gets reused. This
results in unexpected behavior when the value pointed to is overwritten
by the loop calling this function.
Includes a test to make sure this is fixed.
So, DO NOT point to unsafe variables. Make a new pointer using
common.ToPtr where it is passed by value and returns a pointer to that
new value.
NOTE: This is NOT caught by golangci-lint. There may be other places
where this happens, but I have gone through the potential looking code
in osbuild-composer and images and not found any (other than a couple
places already noted with G601 tags as not a problem).
These functions are leftover from the `osbuild/images` split and are not
used by any code in osbuild composer. Instead, the version in
`osbuild/images` is used by distro definitions.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This function is no longer used by any code. Instead, its copy in the
`osbuild/images` repository is used by distro definitions to validate
the customization.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This copy of the package in osbuild-composer is no longer used by any
code. Instead, the copy which is in the `osbuild/images` repository (and
module) is the implementation used by distro definitions.
Delete this code to reduce potential confusion and code duplication.
Also delete code in `internal/blueprint` package, which uses the
pathpolicy package. This code is no longer used, but instead the version
from `osbuild/images` is being used.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
When uploading and distributing a commit, wait for any async tasks to
finish before returning. There are two tasks that can block this
function:
- Creating a distribution: this only happens when a new repository is
created.
- Import commit: this will always happen in this function.
Helper function that performs the whole upload and distribute procedure.
Two more helper functions are added for retrieving the href for a
repository based on its name, and for retrieving the base URL for a
repository's distribution.
Define the task state enum based on the available values defined in the
API.
Add a helper function that returns true if a task is running or waiting
and ignores errors.
Function for importing a commit artifact (that's already been uploaded)
into a given repository. Note that the "repo" argument to the
NewOstreeImportAll() function refers to the name of the repository
directory inside the archive, which for the commits we produce in
osbuild is always "repo".
There is a lot of repeated checks for bp.Customization != nil, this
simplifies that by creating an empty blueprint.Customizations at the
top, and checking to see if it is still empty at the bottom and setting
it back to nil.
Includes a new test for calling with an empty (not nil)
v2.Customizations set on the request.
The function is a leftover from the image definitions split and it is
not used. Moreover, the `images` copy of it is being reimplemented by
[1]. It is better to remove this copy to prevent any unintended use of
it or confusion.
[1] https://github.com/osbuild/images/pull/195
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
