This is used to cache the results of dump and search requests for 60s.
Once the timeout has passed the request is repeated and the timeout
reset. The timeout is *not* reset on every cache hit which prevents, for
example, a request every 59 seconds from keeping the cache from
updating.
When the existing CleanCache() function is called to check the on-disk
metadata cache it will also delete any expired entries from the
resultCache in order to keep it from eventually consuming all memory.
Extract the non-RHUI specific package set and image configuration into a
common definitions, which will be used by both image types.
Redefine the package sets and default image configuration used by both
image types to inherit from a common definition.
Regenerate image manifests for RHEL-8 / c8s `vhd` and `azure-rhui`
images.
There is no change in the resulting manifest for the `azure-rhui` image
type. However there are substantial changes to the `vhd` image
definition, which is now almost identical to the `azure-rhui` image
type, to provide consistent experience regardless if using RHUI or not.
The default partition table used by the `vhd` image type has been kept
as it was before, since there is yet no consensus on what size to
standardize for both image types.
Extract the non-RHUI specific package set and image configuration into a
common definitions, which will be used by both image types.
Redefine the package sets and default image configuration used by both
image types to inherit from a common definition.
Regenerate image manifests for RHEL-9 / c9s `vhd` and `azure-rhui`
images.
There is no change in the resulting manifest for the `azure-rhui` image
type. However there are substantial changes to the `vhd` image
definition, which is now almost identical to the `azure-rhui` image
type, to provide consistent experience regardless if using RHUI or not.
The default partition table used by the `vhd` image type has been kept
as it was before, since there is yet no consensus on what size to
standardize for both image types.
There was a missing `overrides` property in the `azure-rhui` request
definition, which resulted in:
```
$ ./tools/test-case-generators/generate-test-cases --distro rhel-90
--arch x86_64 --image-types azure-rhui --store /dev/null --output
/home/thozza/devel/osbuild-composer/test/data/manifests
--keep-image-info
Traceback (most recent call last):
File "/home/thozza/devel/osbuild-composer/./tools/test-case-generators/generate-test-cases", line 176, in <module>
main(args.distro, args.arch, args.image_types, args.keep_image_info, args.store, args.output)
File "/home/thozza/devel/osbuild-composer/./tools/test-case-generators/generate-test-cases", line 153, in main
if distro in test_case_request["overrides"]:
KeyError: 'overrides'
```
Move all code related to Azure / VHD images to a separate file,
similarly as it is done in rhel7 distro. This approach makes it easier
to find all the code related to a specific image type family.
Move all code related to Azure / VHD images to a separate file,
similarly as it is done in rhel7 distro. This approach makes it easier
to find all the code related to a specific image type family.
Don't redefine the storage unit multiples in each distro, but use the
constants defined in the `common` package. This will make it easier to
split related image type definitions into separate files.
Currently there is a race condition that occurs between the
dbjobqueue enqueue and dequeue functions. Both queries make
use of the postgres `now()` timestamp function which returns
the timestamp of when the transaction started and not when
the statement is executed. The result of this is a timestamp
for a job's `started_at` field to be earlier than its
`queued_at` field causing a constraint violation.
Since the dequeue query will never be executed before the
enqueue query, changing the postgres timestamp function to
`statement_timestamp()` resolves this issue.
The Fedora IoT Raw image sets the default remote URL to the official
Fedora IoT source at build time. We need to replace it in our tests
with our local test repo.
3013889727
changed the behaviour of --boot uefi. Now, it doesn't pass the loader options
but uses firmware="efi" that uses libvirt's ability to auto-detect the correct
loader.
Therefore, for distributions that ship the new virt-install, we have to change
the --boot argument. --boot uefi should now be able to automatically set the
uefi loader correctly without passing any extra loader options. The only issue
is that CS9 has issues with secure boot and edge, so we need to disable
secure boot using a new (and arcane) syntax (the old loader arguments no
longer work):
firmware.feature0.name=secure-boot,firmware.feature0.enabled=no
(https://libvirt.org/formatdomain.html#bios-bootloader)
See the CS9 bug for secure boot issues on edge:
https://bugzilla.redhat.com/show_bug.cgi?id=2108646
Also, since --os-type is deprecated in libvirt >= 4, let's remove it.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The image_type variable in the playbook is used for the name of the
deployment, which is 'redhat' by default for RHEL and 'fedora-iot' for
Fedora.
This affects the deployment path checks, for example:
{{ device_name }}[/ostree/deploy/{{ image_type }}/var]
In Fedora 37, the message for reaching the boot-complete.target changed
from
Reached target Boot Completion Check
to
Reached target boot-complete.target - Boot Completion Check
which is making the string matching test fail.
Instead of looking for message strings, check that the target's status
is good.
Instead of defining users in the ostree commits, create them during the
deployment and creation of the raw image.
Defining users in commits locks them into the base system: they can't be
removed and their password can't be changed without creating an upgrade
commit and they will always be part of the initial system.
Also, in Fedora 37, password authentication for users in commits is not
working.
Generators for creating UsersStage and GroupsStage from the new internal
types.
These are almost identical to the existing stage option creation
functions. Those will be removed once every use is replaced with the
new generators.
New types that represent a User or Group for internal representation.
These are meant to be used as intermediate representations between API
or Blueprint user customizations and the final OSBuild stage options.
Instead of keeping the loop device of the base image and then opening
each partition as a loop device, remove the original loop device of the
base image and then create a loop device for each partition from the
file itself using the partition offsets.
The open_image() function is renamed to convert_image() and now only
handles converting qcow2 files to raw files if necessary.
The loop_open() context is done in analyse_image() instead, so that the
base loop device can be closed without removing the converted image.
This fixes the following issue with LVM partitions:
When the same lvm partition UUID is on two devices (e.g., /dev/loop0p4
and /dev/loop1), the 'vgchange -ay' command fails with the following
error:
Cannot activate LVs in VG rootvg while PVs appear on duplicate
devices.
This happens when we open the LVM partition as a separate loop device,
which we do for all partitions that we want to inspect.
NB: It's possible to restrict the vgchange command to a specific device
with --devices, but this isn't available in older versions of lvm2 (it
was introduced in 2.03.11).
Since we're sharing functions between test scripts, move greenprint(),
the most rewritten function in the history of the project, to
shared_lib.sh and source it everywhere.
- Handle the array responses from the new weldr-client (>= 35.6).
- Move the `get_build_info` function to shared_libs.sh to source and
reuse in multiple places.
The newest weldr-client (35.6) changed its responses to always be
arrays/lists of json objects. The tests have been updated to parse this
structure as well.
The current workflow for parsing responses from the weldr-client is as
follows:
- If weldr-client is installed on the system:
- Try to parse the newest structure version: array of objects, each
with a body field.
- If that fails, initialise the first element of the array and parse
the response into it.
- If the weldr-client is not installed, initialise the array with one
element and parse the response into the body field of the first
element of the array.
These are based on the official Fedora IoT Raw Image but are added to
the base OSTreeDeployment manifest. For now, let's add them to all
ostree deployments that will use these new types and we can extract the
values to make them configurable when we need to.
A new struct in ostree can be used to define configuration options for
the ostree remote of an image. So far remotes were always set up with
the remote URL used to pull the commit. Now we support setting a
different remote with extra configuration options.
This is used by the fedora-iot-raw-image to set up the remote
configuration of the final image, separately from the source of the
commit.
Test manifests updated.
Stages that modify files in the ostree deployment need the sysroot
bindmount for any changes to take effect.
Adding the mount to the keymap and locale stages since they need to
modify files in /etc.
