Change partition tables on edgeBase images to use
'LVM partitioning'. We need to ensure that LVM
stages are done before LUKS stages (e.g. remove-key)
or the pipelines will break (we cannot open a device
when its password has changed).
Add relevant tests on device_test.go plus a new
test partition table on common_test.go
Remove the 'fedora' prefix from the canonical name for fedora-iot image
types. Make the previous names aliases.
This has little functional change since we're simply swapping the
canonical name with an existing alias.
A small test is added to cover this new option. Also, a check for AWS region
in the URL was removed. The public URL doesn't actually contain it and it's
imho useless - S3 buckets are always tied to a certain region.
The elaborate grep command parsing a path from a URI was switched to being
a small python script. The python script can actually handle an URI without
a query (which was always present in the pre-signed URI, but it's not in the
ordinary one)
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
If the object is marked as public, its direct download URL will be returned
instead of the presigned one.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
By setting the object's ACL to "public-read", anyone can download the object
even without authenticating with AWS.
The osbuild-upload-generic-s3 command got a new -public argument that
uses this new feature.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Using new() to create a new struct assigns an empty struct to the
variable, meaning it can never be tested for nil. This means this code
would never detect a missing kernel package.
This satisfies the linter complaint about potential Slowloris attack
where headers are read slowly in an attempt to DoS the server.
The uses of ListenAndServe are only for testing purposes and are not run
in the production server so ignore the lint errors in
osbuild-mock-openid-provider.
This is used to cache the results of dump and search requests for 60s.
Once the timeout has passed the request is repeated and the timeout
reset. The timeout is *not* reset on every cache hit which prevents, for
example, a request every 59 seconds from keeping the cache from
updating.
When the existing CleanCache() function is called to check the on-disk
metadata cache it will also delete any expired entries from the
resultCache in order to keep it from eventually consuming all memory.
Extract the non-RHUI specific package set and image configuration into a
common definitions, which will be used by both image types.
Redefine the package sets and default image configuration used by both
image types to inherit from a common definition.
Regenerate image manifests for RHEL-8 / c8s `vhd` and `azure-rhui`
images.
There is no change in the resulting manifest for the `azure-rhui` image
type. However there are substantial changes to the `vhd` image
definition, which is now almost identical to the `azure-rhui` image
type, to provide consistent experience regardless if using RHUI or not.
The default partition table used by the `vhd` image type has been kept
as it was before, since there is yet no consensus on what size to
standardize for both image types.
Extract the non-RHUI specific package set and image configuration into a
common definitions, which will be used by both image types.
Redefine the package sets and default image configuration used by both
image types to inherit from a common definition.
Regenerate image manifests for RHEL-9 / c9s `vhd` and `azure-rhui`
images.
There is no change in the resulting manifest for the `azure-rhui` image
type. However there are substantial changes to the `vhd` image
definition, which is now almost identical to the `azure-rhui` image
type, to provide consistent experience regardless if using RHUI or not.
The default partition table used by the `vhd` image type has been kept
as it was before, since there is yet no consensus on what size to
standardize for both image types.
Move all code related to Azure / VHD images to a separate file,
similarly as it is done in rhel7 distro. This approach makes it easier
to find all the code related to a specific image type family.
Move all code related to Azure / VHD images to a separate file,
similarly as it is done in rhel7 distro. This approach makes it easier
to find all the code related to a specific image type family.
Don't redefine the storage unit multiples in each distro, but use the
constants defined in the `common` package. This will make it easier to
split related image type definitions into separate files.
Generators for creating UsersStage and GroupsStage from the new internal
types.
These are almost identical to the existing stage option creation
functions. Those will be removed once every use is replaced with the
new generators.
New types that represent a User or Group for internal representation.
These are meant to be used as intermediate representations between API
or Blueprint user customizations and the final OSBuild stage options.
These are based on the official Fedora IoT Raw Image but are added to
the base OSTreeDeployment manifest. For now, let's add them to all
ostree deployments that will use these new types and we can extract the
values to make them configurable when we need to.
A new struct in ostree can be used to define configuration options for
the ostree remote of an image. So far remotes were always set up with
the remote URL used to pull the commit. Now we support setting a
different remote with extra configuration options.
This is used by the fedora-iot-raw-image to set up the remote
configuration of the final image, separately from the source of the
commit.
Test manifests updated.
Stages that modify files in the ostree deployment need the sysroot
bindmount for any changes to take effect.
Adding the mount to the keymap and locale stages since they need to
modify files in /etc.
New mount type added: org.osbuild.ostree.deployment.
Adding this to a stage will setup all needed bind mounts so that a given
`tree` will look like an active OSTree deployment, very much as OSTree
does during early boot.
This is often necessary when making changes to files in /etc for an
ostree image.
Adding support for config options to OSTreeDeployment that are required
by the IoT raw image:
- Kernel command line options
- Keyboard layout
- Locale
Test manifests updated.
Include the platform packages when getting the build packages for the
RawOSTreeImage.
rpm-ostree is explicitly added for this image type.
dracut-config-generic and efibootmgr are temporarily added here, but we
should define a platform that includes them instead (some cleanup
required in general).
The map is meant to maintain compatibility with changes in image type
names. Most entries however map to themselves. It's redundant to keep
updating this map with every new image type name and map it to itself.
Instead, check the map for a compat string and if one doesn't exist,
return the original.
Conversely, when the inverse lookup is performed, first check if the
compat string is a valid image type name and only iterate the map if
it's not.
Now we don't need to update the map every time we introduce a new image
type, unless we need to map an old name to a new one.
Add the new image type to the list in each architecture and update
tests.
Ignore ostree raw images in Kernel count test in distro_test_common:
Edge and IoT raw images don't need a kernel specified in their OS
pipeline. The kernel (and the OS in general, including all packages)
come from the commit that is pulled and deployed in the image.
This test passes on RHEL (for edge-raw-image types) because the
blueprint defaults to returning the main kernel, but this isn't
necessary and is likely to change in the near future.
Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
