A small test is added to cover this new option. Also, a check for AWS region
in the URL was removed. The public URL doesn't actually contain it and it's
imho useless - S3 buckets are always tied to a certain region.
The elaborate grep command parsing a path from a URI was switched to being
a small python script. The python script can actually handle an URI without
a query (which was always present in the pre-signed URI, but it's not in the
ordinary one)
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
If the object is marked as public, its direct download URL will be returned
instead of the presigned one.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
By setting the object's ACL to "public-read", anyone can download the object
even without authenticating with AWS.
The osbuild-upload-generic-s3 command got a new -public argument that
uses this new feature.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Add a common function to check for all environment variables needed to
test VSphere image. Call the function from all S3 implementations of
`checkEnv()` if testing the VSphere image type.
Move the installation of the VSphere client out of the
`api/common/aws.sh` `installClient()` function into a dedicated
`installClientVSphere()` function in `api/common/vsphere.sh`.
Rename the `installClient()` in `api/common/aws.sh` to
`installAWSClient()`. Introduce `installClient()` in both,
`api/aws.sh` and `api/aws.s3.sh`, which calls `installAWSClient()`.
Call `installClientVSphere()` conditionally when testing VMDK image type
in `api/aws.s3.sh` and `api/generic.s3.sh`.
The `verifyEdgeCommit()` and `verifyDisk()` functions are already
defined in the `test/cases/api/common/s3.sh` file sourced by
`test/cases/api/aws.s3.sh`.
shared_lib.sh now contains some more helper functions and the
verification script uses them. Copy it over so that we can source it.
Also install all other dependencies before the actual upgrade.
Using new() to create a new struct assigns an empty struct to the
variable, meaning it can never be tested for nil. This means this code
would never detect a missing kernel package.
This satisfies the linter complaint about potential Slowloris attack
where headers are read slowly in an attempt to DoS the server.
The uses of ListenAndServe are only for testing purposes and are not run
in the production server so ignore the lint errors in
osbuild-mock-openid-provider.
This is used to cache the results of dump and search requests for 60s.
Once the timeout has passed the request is repeated and the timeout
reset. The timeout is *not* reset on every cache hit which prevents, for
example, a request every 59 seconds from keeping the cache from
updating.
When the existing CleanCache() function is called to check the on-disk
metadata cache it will also delete any expired entries from the
resultCache in order to keep it from eventually consuming all memory.
Extract the non-RHUI specific package set and image configuration into a
common definitions, which will be used by both image types.
Redefine the package sets and default image configuration used by both
image types to inherit from a common definition.
Regenerate image manifests for RHEL-8 / c8s `vhd` and `azure-rhui`
images.
There is no change in the resulting manifest for the `azure-rhui` image
type. However there are substantial changes to the `vhd` image
definition, which is now almost identical to the `azure-rhui` image
type, to provide consistent experience regardless if using RHUI or not.
The default partition table used by the `vhd` image type has been kept
as it was before, since there is yet no consensus on what size to
standardize for both image types.
Extract the non-RHUI specific package set and image configuration into a
common definitions, which will be used by both image types.
Redefine the package sets and default image configuration used by both
image types to inherit from a common definition.
Regenerate image manifests for RHEL-9 / c9s `vhd` and `azure-rhui`
images.
There is no change in the resulting manifest for the `azure-rhui` image
type. However there are substantial changes to the `vhd` image
definition, which is now almost identical to the `azure-rhui` image
type, to provide consistent experience regardless if using RHUI or not.
The default partition table used by the `vhd` image type has been kept
as it was before, since there is yet no consensus on what size to
standardize for both image types.
There was a missing `overrides` property in the `azure-rhui` request
definition, which resulted in:
```
$ ./tools/test-case-generators/generate-test-cases --distro rhel-90
--arch x86_64 --image-types azure-rhui --store /dev/null --output
/home/thozza/devel/osbuild-composer/test/data/manifests
--keep-image-info
Traceback (most recent call last):
File "/home/thozza/devel/osbuild-composer/./tools/test-case-generators/generate-test-cases", line 176, in <module>
main(args.distro, args.arch, args.image_types, args.keep_image_info, args.store, args.output)
File "/home/thozza/devel/osbuild-composer/./tools/test-case-generators/generate-test-cases", line 153, in main
if distro in test_case_request["overrides"]:
KeyError: 'overrides'
```
Move all code related to Azure / VHD images to a separate file,
similarly as it is done in rhel7 distro. This approach makes it easier
to find all the code related to a specific image type family.
Move all code related to Azure / VHD images to a separate file,
similarly as it is done in rhel7 distro. This approach makes it easier
to find all the code related to a specific image type family.
Don't redefine the storage unit multiples in each distro, but use the
constants defined in the `common` package. This will make it easier to
split related image type definitions into separate files.
Currently there is a race condition that occurs between the
dbjobqueue enqueue and dequeue functions. Both queries make
use of the postgres `now()` timestamp function which returns
the timestamp of when the transaction started and not when
the statement is executed. The result of this is a timestamp
for a job's `started_at` field to be earlier than its
`queued_at` field causing a constraint violation.
Since the dequeue query will never be executed before the
enqueue query, changing the postgres timestamp function to
`statement_timestamp()` resolves this issue.
The Fedora IoT Raw image sets the default remote URL to the official
Fedora IoT source at build time. We need to replace it in our tests
with our local test repo.
3013889727
changed the behaviour of --boot uefi. Now, it doesn't pass the loader options
but uses firmware="efi" that uses libvirt's ability to auto-detect the correct
loader.
Therefore, for distributions that ship the new virt-install, we have to change
the --boot argument. --boot uefi should now be able to automatically set the
uefi loader correctly without passing any extra loader options. The only issue
is that CS9 has issues with secure boot and edge, so we need to disable
secure boot using a new (and arcane) syntax (the old loader arguments no
longer work):
firmware.feature0.name=secure-boot,firmware.feature0.enabled=no
(https://libvirt.org/formatdomain.html#bios-bootloader)
See the CS9 bug for secure boot issues on edge:
https://bugzilla.redhat.com/show_bug.cgi?id=2108646
Also, since --os-type is deprecated in libvirt >= 4, let's remove it.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The image_type variable in the playbook is used for the name of the
deployment, which is 'redhat' by default for RHEL and 'fedora-iot' for
Fedora.
This affects the deployment path checks, for example:
{{ device_name }}[/ostree/deploy/{{ image_type }}/var]
