Use a single GenCopyFSTreeOptions() function from osbuild2 instead of
implementing it multiple times in distros.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Use a single GenMkfsStages() function from osbuild2 instead of
implementing it multiple times in distros.
Co-Authored-By: Christian Kellner <christian@kellner.me>
All disk.Entitity types now implement Clone() which should return a
deep copy of the same object. Add the Clone() method to the entity
interface. The return type is Entity, but callers can assume it's
safe to convert back to the original type.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Structs to represent a Btrfs file system with subvolumes.
Btrfs implements the following interfaces:
- Entity
- Container
- VolumeContainer
BtrfsSubvolume implements the following interfaces:
- Entity
- Sizeable
- Mountable
Co-Authored-By: Christian Kellner <christian@kellner.me>
Types to represent LVM2 setups with logical volumes being contained
in volume groups.
LVMVolumeGroup implements the following disk interfaces:
- Entity
- Container
- VolumeContainer
LVMLogicalVolume implements the following disk interfaces:
- Entity
- Container
- Sizeable
Co-Authored-By: Christian Kellner <christian@kellner.me>
Representation for the encrypted storage container of the same name.
Implements the following disk interfaces:
- Entity
- Container
Co-Authored-By: Christian Kellner <christian@kellner.me>
In our model a partition table has a size to represent the disk size.
Also each individual partition has a size, so they both implement the
Sizeable interface.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
A `PartitionTable` is a container for one or more instances of
`Partition`, which itself might contain a `Filesystem`.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Implement the base type of the new entity system, i.e. `Entity`,
for all relevant components:
- PartitionTable
- Partition
- Filesystem
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
Add a generic entity data model that can be used to navigate and operate
on any specific disk image layout. Since the latter allows for arbitrary
nesting of different container types, such as LUKS2, LVM2, and with file
systems that can have sub-volumes, the entity model is very generic.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
Split up disk.go into smaller more manageable pieces: finish
by moving `Partition` to its own file.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Split up disk.go into smaller more managable pieces: continue
with moving `Filesystem` to its own file.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Split up disk.go into smaller more managable pieces: start with
extracting `PartitionTable` and all its methods to a new file.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Use bytes internally everywhere and convert to sectors only when writing
the options for the stages.
Changed the AlignUp() method to not do the alignment if the input is
already aligned. This changes the behaviour when the size is 0, but
that's not a realistic use case. Updated unit tests to match.
Manifests are unaffected.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Instead of just reserving a fixed amount of sectors for the
partition header and footer, calculate the size of the
header and footer dynamically when the layout is GPT.
Return an error if the maximum numbers of partitions has been
reached and thus creating further partitions would result in
errors.
Currently we limit MBR partition types to 4 as we dont support
logical partitions and GPT layouts to 128. According to the
UEFI specificatio (2.8) a minimum of 16384 bytes are reserved
for the partition entries array. Each entry is 128 bytes wide
thus resulting in at least 128 entries; we choose this to be
the maximum as well for now.
This is needed so that we can do different things depending on the
given layout; this will be used in tests for now only. Only GPT
allows for arbitrary number of partitions and once we assert this
in code we will need to adjust the tests accordingly.
NB: This method might be removed again in the future, once generic
LVM support is added everywhere and the ability to differentiate
between MBR and GPT layouts is not needed anymore.
Do not rely on `distro.imageOptions` having any size information,
i.e. `Size` being `0`. Instead use `imageType.Size()` and the
information in the blueprint customization to calculate the size.
This makes the individual distro definitions idenpendent of the
API entry points that currently calculate the size, e.g.:
internal/cloudapi/v1/v1.go:PostCompose line 184
internal/cloudapi/v2/v2.go:PostCompose line 197
internal/kojiapi/server.go:PostCompose line 135
internal/weldr/api.go:composeHandler line 2289
Align partitions during their re-layout. Specifically, we align the
first partition which removes the need for the hard coded magic
number of 2048, which represents the proper alignment for a grain
size of 1MB given a sector size of 512 (our setup).
NB: Since all our partitions should sizes that are themselves
aligned, i.e. evenly dividable by the default grain (1MB) this
will not actually change any manifest.
Add a utility method that can be used to align data to the next "grain"
which is currently defined as 1 MiB -- the default used by sfdisk.
Add corresponding tests.
Instead of hard coding a padding of 100 sectors for all layouts, i.e.
MBR and GTP, adjust the needed space depending on the layout: for MBR
we don't need to reserve any space at all since it does not have a
secondary header. For GTP we reserve 33 sectors, as indicated in the
UEFI specific, which allows for the header itself and up to 128 entries.
To not modify the layout of already released distributions, like RHEL
8.4 and 8.5, a new member called `ExtraPadding` is added to `Partition
Table` and then used in the corresponding layouts to preserve the
existing padding of 100.
The method assert.GreaterOrEqual(t, a, b) asserts that a >= b,
but the arguments were (expected, actual), which is the wrong
way around since we want to assure that the actual size is at
least the expected size, i.e. actual >= expected and thus the
argument should be (actual, expected).
The partition table size (pt.Size) is in sectors while our checks and
customizations are in bytes, so we need to convert when comparing.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
Apply the minimum size specified in the file system customizations
also to existing partitions. Specifically, this now allows to set
a minimum size also for the root partition.
Allow the sector size to be specified on the partition table level by
introducing a new `SectorSize` field. Modify the conversion helpers
to use that new field with a fallback to default sector size in case
the field is `0`.
Add simple helper methods that convert between bytes and sectors. This
is a method of `PartitionTable`, since the sector size can in theory
be per partition table and this prepares for that case.
Modify the signature of `CreatePartitionTable` so that it is
possible to return errors from the function. This is not yet
used, but will be in the near future. Change all call sites
accordingly: in most cases we can just bubble up the error.
Instead of generating the UUIDs directly when new partitions are
created and separately for the boot and root partition, use the
new `PartitionTable.GenerateUUIDs` method to generate all UUIDs
that are missing in one go. Since this changes the order in
which the uuids are generated the test manifests UUIDs changed
and needed to be updated:
I used to following patch to get the updated manifests:
--- a/internal/distro/distro_test_common/distro_test_common.go
+++ b/internal/distro/distro_test_common/distro_test_common.go
@@ -105,6 +105,12 @@ func TestDistro_Manifest(t *testing.T, pipelinePath string, prefix string, regis
require.NoError(t, err)
diff := cmp.Diff(expected, actual)
+ if diff != "" {
+ tt.Manifest = got
+ data, _ := json.MarshalIndent(tt, "", " ")
+ path := filepath.Join("/tmp", filepath.Base(fileName))
+ _ = ioutil.WriteFile(path, data, 0644)
+ }
require.Emptyf(t, diff, "Distro: %s\nArch: %s\nImage type: %s\nTest case file: %s\n", d.Name(), arch.Name(), imageType.Name(), fileName)
}
})
And the following fish snippet to update the existing ones, using the
jq and sponge utilities:
for file in /tmp/rhel_85-*.json
set filename (basename $file)
jq -s '.[0].manifest = .[1].manifest | .[0]' test/data/manifests/$filename /tmp/$filename | sponge test/data/manifests/$filename
end
Add a new helper to `PartitionTable` that generates all necessary
uuids for uuid fields that are empty but should not. These are
the file system uuids and, in case of a GPT layout, partition ids.
Pass the `basePartitionTable` argument of `CreatePartitionTable`.
Now that we clone the partition table at the beginning of the
method there is no need to pass a copy of the partition table.
Simple wrapper around FindFilesystemForMountpoint that will
return a boolean if a filesystem with the given mountpoint
is defined in the partition table.
The partition table is modified `CreatePartitionTable`, which is
not a problem for the table itself since it is currently passed
by value. However, all shallow copies share the same file system
pointers and `CreatePartitionTable` will modify those as well.
As there is a data race where two concurrent thread modify the
content of the Fileystem object at the same time before writing
the uuids out to the manifest via the stage options and thus the
resulting manifest would be broken.
Therefore we use the new `Clone` methods to make a dee@ copy of
the `PartitionTable` object in the `CreatePartitionTable` method;
This will allow us to pass the `basePartitionTable` object by val
and also return the resulting `PartitionTable` as a pointer.
