Currently there is a race condition that occurs between the
dbjobqueue enqueue and dequeue functions. Both queries make
use of the postgres `now()` timestamp function which returns
the timestamp of when the transaction started and not when
the statement is executed. The result of this is a timestamp
for a job's `started_at` field to be earlier than its
`queued_at` field causing a constraint violation.
Since the dequeue query will never be executed before the
enqueue query, changing the postgres timestamp function to
`statement_timestamp()` resolves this issue.
The Fedora IoT Raw image sets the default remote URL to the official
Fedora IoT source at build time. We need to replace it in our tests
with our local test repo.
3013889727
changed the behaviour of --boot uefi. Now, it doesn't pass the loader options
but uses firmware="efi" that uses libvirt's ability to auto-detect the correct
loader.
Therefore, for distributions that ship the new virt-install, we have to change
the --boot argument. --boot uefi should now be able to automatically set the
uefi loader correctly without passing any extra loader options. The only issue
is that CS9 has issues with secure boot and edge, so we need to disable
secure boot using a new (and arcane) syntax (the old loader arguments no
longer work):
firmware.feature0.name=secure-boot,firmware.feature0.enabled=no
(https://libvirt.org/formatdomain.html#bios-bootloader)
See the CS9 bug for secure boot issues on edge:
https://bugzilla.redhat.com/show_bug.cgi?id=2108646
Also, since --os-type is deprecated in libvirt >= 4, let's remove it.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The image_type variable in the playbook is used for the name of the
deployment, which is 'redhat' by default for RHEL and 'fedora-iot' for
Fedora.
This affects the deployment path checks, for example:
{{ device_name }}[/ostree/deploy/{{ image_type }}/var]
In Fedora 37, the message for reaching the boot-complete.target changed
from
Reached target Boot Completion Check
to
Reached target boot-complete.target - Boot Completion Check
which is making the string matching test fail.
Instead of looking for message strings, check that the target's status
is good.
Instead of defining users in the ostree commits, create them during the
deployment and creation of the raw image.
Defining users in commits locks them into the base system: they can't be
removed and their password can't be changed without creating an upgrade
commit and they will always be part of the initial system.
Also, in Fedora 37, password authentication for users in commits is not
working.
Generators for creating UsersStage and GroupsStage from the new internal
types.
These are almost identical to the existing stage option creation
functions. Those will be removed once every use is replaced with the
new generators.
New types that represent a User or Group for internal representation.
These are meant to be used as intermediate representations between API
or Blueprint user customizations and the final OSBuild stage options.
Instead of keeping the loop device of the base image and then opening
each partition as a loop device, remove the original loop device of the
base image and then create a loop device for each partition from the
file itself using the partition offsets.
The open_image() function is renamed to convert_image() and now only
handles converting qcow2 files to raw files if necessary.
The loop_open() context is done in analyse_image() instead, so that the
base loop device can be closed without removing the converted image.
This fixes the following issue with LVM partitions:
When the same lvm partition UUID is on two devices (e.g., /dev/loop0p4
and /dev/loop1), the 'vgchange -ay' command fails with the following
error:
Cannot activate LVs in VG rootvg while PVs appear on duplicate
devices.
This happens when we open the LVM partition as a separate loop device,
which we do for all partitions that we want to inspect.
NB: It's possible to restrict the vgchange command to a specific device
with --devices, but this isn't available in older versions of lvm2 (it
was introduced in 2.03.11).
Since we're sharing functions between test scripts, move greenprint(),
the most rewritten function in the history of the project, to
shared_lib.sh and source it everywhere.
- Handle the array responses from the new weldr-client (>= 35.6).
- Move the `get_build_info` function to shared_libs.sh to source and
reuse in multiple places.
The newest weldr-client (35.6) changed its responses to always be
arrays/lists of json objects. The tests have been updated to parse this
structure as well.
The current workflow for parsing responses from the weldr-client is as
follows:
- If weldr-client is installed on the system:
- Try to parse the newest structure version: array of objects, each
with a body field.
- If that fails, initialise the first element of the array and parse
the response into it.
- If the weldr-client is not installed, initialise the array with one
element and parse the response into the body field of the first
element of the array.
These are based on the official Fedora IoT Raw Image but are added to
the base OSTreeDeployment manifest. For now, let's add them to all
ostree deployments that will use these new types and we can extract the
values to make them configurable when we need to.
A new struct in ostree can be used to define configuration options for
the ostree remote of an image. So far remotes were always set up with
the remote URL used to pull the commit. Now we support setting a
different remote with extra configuration options.
This is used by the fedora-iot-raw-image to set up the remote
configuration of the final image, separately from the source of the
commit.
Test manifests updated.
Stages that modify files in the ostree deployment need the sysroot
bindmount for any changes to take effect.
Adding the mount to the keymap and locale stages since they need to
modify files in /etc.
New mount type added: org.osbuild.ostree.deployment.
Adding this to a stage will setup all needed bind mounts so that a given
`tree` will look like an active OSTree deployment, very much as OSTree
does during early boot.
This is often necessary when making changes to files in /etc for an
ostree image.
Adding support for config options to OSTreeDeployment that are required
by the IoT raw image:
- Kernel command line options
- Keyboard layout
- Locale
Test manifests updated.
Include the platform packages when getting the build packages for the
RawOSTreeImage.
rpm-ostree is explicitly added for this image type.
dracut-config-generic and efibootmgr are temporarily added here, but we
should define a platform that includes them instead (some cleanup
required in general).
The map is meant to maintain compatibility with changes in image type
names. Most entries however map to themselves. It's redundant to keep
updating this map with every new image type name and map it to itself.
Instead, check the map for a compat string and if one doesn't exist,
return the original.
Conversely, when the inverse lookup is performed, first check if the
compat string is a valid image type name and only iterate the map if
it's not.
Now we don't need to update the map every time we introduce a new image
type, unless we need to map an old name to a new one.
Add the new image type to the list in each architecture and update
tests.
Ignore ostree raw images in Kernel count test in distro_test_common:
Edge and IoT raw images don't need a kernel specified in their OS
pipeline. The kernel (and the OS in general, including all packages)
come from the commit that is pulled and deployed in the image.
This test passes on RHEL (for edge-raw-image types) because the
blueprint defaults to returning the main kernel, but this isn't
necessary and is likely to change in the near future.
Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
