Instead of hard coding a padding of 100 sectors for all layouts, i.e.
MBR and GTP, adjust the needed space depending on the layout: for MBR
we don't need to reserve any space at all since it does not have a
secondary header. For GTP we reserve 33 sectors, as indicated in the
UEFI specific, which allows for the header itself and up to 128 entries.
To not modify the layout of already released distributions, like RHEL
8.4 and 8.5, a new member called `ExtraPadding` is added to `Partition
Table` and then used in the corresponding layouts to preserve the
existing padding of 100.
The method assert.GreaterOrEqual(t, a, b) asserts that a >= b,
but the arguments were (expected, actual), which is the wrong
way around since we want to assure that the actual size is at
least the expected size, i.e. actual >= expected and thus the
argument should be (actual, expected).
The partition table size (pt.Size) is in sectors while our checks and
customizations are in bytes, so we need to convert when comparing.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
Apply the minimum size specified in the file system customizations
also to existing partitions. Specifically, this now allows to set
a minimum size also for the root partition.
Allow the sector size to be specified on the partition table level by
introducing a new `SectorSize` field. Modify the conversion helpers
to use that new field with a fallback to default sector size in case
the field is `0`.
Add simple helper methods that convert between bytes and sectors. This
is a method of `PartitionTable`, since the sector size can in theory
be per partition table and this prepares for that case.
Modify the signature of `CreatePartitionTable` so that it is
possible to return errors from the function. This is not yet
used, but will be in the near future. Change all call sites
accordingly: in most cases we can just bubble up the error.
Instead of generating the UUIDs directly when new partitions are
created and separately for the boot and root partition, use the
new `PartitionTable.GenerateUUIDs` method to generate all UUIDs
that are missing in one go. Since this changes the order in
which the uuids are generated the test manifests UUIDs changed
and needed to be updated:
I used to following patch to get the updated manifests:
--- a/internal/distro/distro_test_common/distro_test_common.go
+++ b/internal/distro/distro_test_common/distro_test_common.go
@@ -105,6 +105,12 @@ func TestDistro_Manifest(t *testing.T, pipelinePath string, prefix string, regis
require.NoError(t, err)
diff := cmp.Diff(expected, actual)
+ if diff != "" {
+ tt.Manifest = got
+ data, _ := json.MarshalIndent(tt, "", " ")
+ path := filepath.Join("/tmp", filepath.Base(fileName))
+ _ = ioutil.WriteFile(path, data, 0644)
+ }
require.Emptyf(t, diff, "Distro: %s\nArch: %s\nImage type: %s\nTest case file: %s\n", d.Name(), arch.Name(), imageType.Name(), fileName)
}
})
And the following fish snippet to update the existing ones, using the
jq and sponge utilities:
for file in /tmp/rhel_85-*.json
set filename (basename $file)
jq -s '.[0].manifest = .[1].manifest | .[0]' test/data/manifests/$filename /tmp/$filename | sponge test/data/manifests/$filename
end
Add a new helper to `PartitionTable` that generates all necessary
uuids for uuid fields that are empty but should not. These are
the file system uuids and, in case of a GPT layout, partition ids.
Pass the `basePartitionTable` argument of `CreatePartitionTable`.
Now that we clone the partition table at the beginning of the
method there is no need to pass a copy of the partition table.
Simple wrapper around FindFilesystemForMountpoint that will
return a boolean if a filesystem with the given mountpoint
is defined in the partition table.
The partition table is modified `CreatePartitionTable`, which is
not a problem for the table itself since it is currently passed
by value. However, all shallow copies share the same file system
pointers and `CreatePartitionTable` will modify those as well.
As there is a data race where two concurrent thread modify the
content of the Fileystem object at the same time before writing
the uuids out to the manifest via the stage options and thus the
resulting manifest would be broken.
Therefore we use the new `Clone` methods to make a dee@ copy of
the `PartitionTable` object in the `CreatePartitionTable` method;
This will allow us to pass the `basePartitionTable` object by val
and also return the resulting `PartitionTable` as a pointer.
The `PartitionTable` object contains an array of `Partitions`, which
is shared among all shallow copies of `PartitionTable` instances, as
when it is used via a value receiver in method calls. The individual
objects are thus not fully isolated since modifications of array
elements, or in some cases modifications to the array itself, are
shared and visible between all shallow `PartitionTable` copies. Thus
use a pointer receiver for all methods of `PartitionTable` to make
it explicit that modifications will affect other objects.
What this function is actually doing is to create a filesystem,
together with a new partition; for LVM that filesystem could
also be created on a new logical volume though.
The previous commit made sure that `PartitionTable.RootPartition`
is indeed returning the pointer partition object inside the array
of the `PartitionTable`. Thus changes to the returned object will
now directly affect the object and thus there is no need to call
`PartitionTable.updateRootPartition` anymore.
When we iterate over the partitions to find the root partition,
we are getting the partition by value so even if we return a
pointer to that then, it is actually a pointer duplicated object.
Return the pointer to the actual partition object instead.
In various places we are interested in the root and boot file-
systems. Currently those were accessed via by retrieving the
partition that contain them and the accessing the filesystem
member. Add accessors to `PartitionTable` that directly return
the needed filesystem. This will help if the file system is
stored inside a container like LVM or LUKS instead of directly
on a partition.
The dump function will be used to search packages and as we are implementing third party repositories, adding the repo_id of the package will allow us to identify the package repository used.
Defaults according to https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config:
Defaults to a chain of credential providers to search for credentials in
environment variables, shared credential file, and EC2 Instance Roles.
If nothing is specified fall back to whatever instance role.
The longer running ostree tests (all except ostree.sh) benefit from
running on large runners because of their long runtime. Changing all to
large runners. Should not be a problem with our increased Openstack
quota.
The customizations struct contains different kinds of customizations,
such as users and filesystems. If both filesystems and users are
configured, the current code overwrites the user configuration.
Change this by unconditionally allocating the customizations struct
when creating the empty blueprint, and ammending it for each
customization we find.
This means that the customizations will be non-nil, also when it is
empty, which was not the case in the past. This should not change
the behavior of the code.
We (mistakenly) don't enforce a minimum size for /var,
so setting it to 1024 (1kB) causes the image build to fail.
CI does not expose this in a helpful way at the moment,
so this is a bit tricky to debug.
Also skip customizations for the AWS.S3 upload type. Not all the
image types with this upload type support filesystem customizations
and that's as expected. We could make a more fine-grained test in
the future, but testing with a coulpe of targets should be
sufficient.
