In the OS pipeline, the parent configuration was used to detect if the
pipeline's setup was meant for an ostree commit or not. Also, the
pipeline used a new type to specify the ostree parameters.
- Use the ostree.CommitSpec for the parent configuration.
- Add a new attribute, OSTreeRef, that defines the ref for the ostree
commit being built. An empty string indicates that the tree is not
for an ostree commit.
Additionally, in the ImageKind configurations for the ostree archive and
container, separate the ostree ref from the parent spec, make the parent
spec optional (pointer) and the ostree ref mandatory, by requiring it in
the constructor of the ImageKind.
Instead of using the ostree.RequestParams in the OSTReeImageOptions,
define a new struct specific to ImageOptions for the ostree parameters.
This is almost identical to the new ostree.CommitSpec but the meaning of
the parameters changes based on image type and it would not be clear if
the CommitSpec was used in all cases. For example, the parameters of
the new OSTreeImageOptions do not always refer to the same commit. The
URL and Checksum may point to a parent commit to be pulled in to base
the new commit on, while the Ref refers to the new commit that will be
built (which may have a different ref from the parent).
The ostree.ResolveParams() function now returns two strings, the
resolved ref, which is replaced by the defaultRef if it's not specified
in the request, and the resolved parent checksum if a URL is specified.
The URL does not need to be returned since it's always the same as the
one specified in the request.
The function has been rewritten to make the logic more clear.
The docstring for the function has been rewritten to cover all use cases
and error conditions.
The CommitSource was used to specify the source URL and checksum of a
commit for use in manifest sources. Renaming to CommitSpec and adding a
Ref parameter generalises the type so that we can use it to specify
commits in various situations. This is building towards separating when
ostree parameters are used for fetching a commit, fetching a parent
commit, and building one.
The CommitSpec is (very roughly) analogous to the rpmmd.PackageSpec.
With an empty or missing version number the commit message would not
include the version (which is set to 0.0.0 by calling Initialize). This
adds a call to Initialize() in the API code before constructing the
commit message. It also moves the check for non-empty blueprint name
into the Initialize call where it belongs.
This fixes a bug where tagging a blueprint would overwrite the blueprint
in the blueprintChanges storage with an older version of the blueprint.
It also adds checks to make sure the blueprint is present in the
blueprintsCommits and blueprintsChanges maps before trying to reference
them -- just in case something in the store gets out of sync.
TagBlueprint has a bug, it replaces the tagged blueprint with the last
untagged blueprint. The test wasn't testing for this, so correct that
before fixing the actual bug.
Don't pass blueprint Users and Groups options all the way down to the
osbuild stage bindings. Instead, convert them to the internal
users.User and users.Group structs.
Ideally we would do this even higher up in the code path, before
reaching the distro, but this is the first step towards that.
- Use newer granular stages to build the bootiso tree rather than the
deprecated bootiso.mono stage.
- Adjust the ISOTree struct properties: add the ones needed by the new
stages and remove any properties that have been moved to the new
pipelines.
- Remove stage-specific input types when they are org.osbuild.tree input
types.
- Use PipelineTreeInputs when stage requires a single tree input
reference with an arbitrary key.
- For Stages that require a specific key with a tree input, make the key
part of the NewXStage() function and only allow specifying the name of
the pipeline from which to copy the tree as part of the function
arguments.
Convenience function for creating a map with a single input pointing to
a pipeline's tree with a given key.
Different stages use different keys in the map (often "tree").
Functions will be added for each stage to create a map with the
appropriate key when necessary.
The references field in org.osbuild.tree inputs currently supports one
of three forms, all of which are functionally equivalent:
- Array of one string
- Array of one object with key "id" and string value
- Single object with no properties (only key/name)
We use the first form which is the simplest.
The string should refer to a pipeline by name (as name:<pipelinename>),
which means the input refer to the final tree of the named pipeline.
Change partition tables on edgeBase images to use
'LVM partitioning'. We need to ensure that LVM
stages are done before LUKS stages (e.g. remove-key)
or the pipelines will break (we cannot open a device
when its password has changed).
Add relevant tests on device_test.go plus a new
test partition table on common_test.go
Remove the 'fedora' prefix from the canonical name for fedora-iot image
types. Make the previous names aliases.
This has little functional change since we're simply swapping the
canonical name with an existing alias.
A small test is added to cover this new option. Also, a check for AWS region
in the URL was removed. The public URL doesn't actually contain it and it's
imho useless - S3 buckets are always tied to a certain region.
The elaborate grep command parsing a path from a URI was switched to being
a small python script. The python script can actually handle an URI without
a query (which was always present in the pre-signed URI, but it's not in the
ordinary one)
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
If the object is marked as public, its direct download URL will be returned
instead of the presigned one.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
By setting the object's ACL to "public-read", anyone can download the object
even without authenticating with AWS.
The osbuild-upload-generic-s3 command got a new -public argument that
uses this new feature.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Using new() to create a new struct assigns an empty struct to the
variable, meaning it can never be tested for nil. This means this code
would never detect a missing kernel package.
This satisfies the linter complaint about potential Slowloris attack
where headers are read slowly in an attempt to DoS the server.
The uses of ListenAndServe are only for testing purposes and are not run
in the production server so ignore the lint errors in
osbuild-mock-openid-provider.
This is used to cache the results of dump and search requests for 60s.
Once the timeout has passed the request is repeated and the timeout
reset. The timeout is *not* reset on every cache hit which prevents, for
example, a request every 59 seconds from keeping the cache from
updating.
When the existing CleanCache() function is called to check the on-disk
metadata cache it will also delete any expired entries from the
resultCache in order to keep it from eventually consuming all memory.
Extract the non-RHUI specific package set and image configuration into a
common definitions, which will be used by both image types.
Redefine the package sets and default image configuration used by both
image types to inherit from a common definition.
Regenerate image manifests for RHEL-8 / c8s `vhd` and `azure-rhui`
images.
There is no change in the resulting manifest for the `azure-rhui` image
type. However there are substantial changes to the `vhd` image
definition, which is now almost identical to the `azure-rhui` image
type, to provide consistent experience regardless if using RHUI or not.
The default partition table used by the `vhd` image type has been kept
as it was before, since there is yet no consensus on what size to
standardize for both image types.
Extract the non-RHUI specific package set and image configuration into a
common definitions, which will be used by both image types.
Redefine the package sets and default image configuration used by both
image types to inherit from a common definition.
Regenerate image manifests for RHEL-9 / c9s `vhd` and `azure-rhui`
images.
There is no change in the resulting manifest for the `azure-rhui` image
type. However there are substantial changes to the `vhd` image
definition, which is now almost identical to the `azure-rhui` image
type, to provide consistent experience regardless if using RHUI or not.
The default partition table used by the `vhd` image type has been kept
as it was before, since there is yet no consensus on what size to
standardize for both image types.
Move all code related to Azure / VHD images to a separate file,
similarly as it is done in rhel7 distro. This approach makes it easier
to find all the code related to a specific image type family.
Move all code related to Azure / VHD images to a separate file,
similarly as it is done in rhel7 distro. This approach makes it easier
to find all the code related to a specific image type family.
Don't redefine the storage unit multiples in each distro, but use the
constants defined in the `common` package. This will make it easier to
split related image type definitions into separate files.
