jobimpl-osbuild
---------------
Add GenericS3Creds to struct
Add method to create AWS with Endpoint for Generic S3 (with its own credentials file)
Move uploading to S3 and result handling to a separate method (along with the special VMDK handling)
adjust the AWS S3 case to the new method
Implement a new case for uploading to a generic S3 service
awscloud
--------
Add wrapper methods for endpoint support
Set the endpoint to the AWS session
Set s3ForcePathStyle to true if endpoint was set
Target
------
Define a new target type for the GenericS3Target and Options
Handle unmarshaling of the target options and result for the Generic S3
Weldr
-----
Add support for only uploading to AWS S3
Define new structures for AWS S3 and Generic S3 (based on AWS S3)
Handle unmarshaling of the providers settings' upload settings
main
----
Add a section in the main config for the Generic S3 service for credentials
If provided pass the credentials file name to the osbuild job implementation
Upload Utility
--------------
Add upload-generic-s3 utility
Makefile
------
Do not fail if the bin directory already exists
Tests
-----
Add test cases for both AWS and a generic S3 server
Add a generic s3_test.sh file for both test cases and add it to the tests RPM spec
Adjust the libvirt test case script to support already created images
GitLabCI - Extend the libvirt test case to include the two new tests
Followup from, f34380d5b5 and
3a1765a5a8, copied to the rest of the RHEL
distro definitions.
For now, these customizations have no effect on the manifest.
The new `with-users` variants of the edge-installer test cases include
the user customizations in the blueprint, but the manifests are
(currently) the same as the corresponding base cases.
RHEL 9.0 will ship a brand new auxiliary key. Let's use it everywhere in our
RHEL 9 stuff. Taken from current RHEL 9.0's redhat-release package.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
We should honour `pkg.CheckGPG` when creating the file inputs for the
rpm stage. This was lost in the transition from v1 to v2 manifests.
Regenerate image test manifests.
Co-authored-by: Tomas Hozza <thozza@redhat.com>
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The nightly compose (currently) only has a 'latest' directory for 8.7.
Switching to the development composes which have 'latest' for 8.4
onwards.
Enable -x in script for easier troubleshooting.
The decision logic which jobs to run is quite confusing but that's how we
roll for now:
Jenkins builds RHEL images only on main
Schutzbot builds RHEL images only in PRs
Schutzbot builds Fedora images on both PRs and on main
To achieve this, the commit re-enables running Packer on main on Schutzbot.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Arch was easy.
For passing the repository distribution and osbuild_commit (it can be
different for each distro), I decided to go in the way of ansible
inventory directories. It adds a bit of structure but I think it's
the most clean solution.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Most test scripts don't have any documentation regarding it's purpose,
although it can be guessed by the code. There's value in adding this
small comment.
[skip-ci]
When encountering an LVM2 layout, activate all its logical volumes
so that they can be mounted.
NB: we need to pass "norecovery" to the mount options because LVM
does not setup the device mapper tables read-only even though the
underlying loopback device is and then xfs will try to write to
its journal and the kernel will panic. Attempts to reload the DM
tables as readonly didn't work.
NB: this will not work if we are trying to inspect an image that
has a volume group name that is also present on the host. We
could open the image file read-write and modify its vg name, but
that would mean modifying the image file and thus we would need
to copy it first.
Pass `-c /dev/null` to `blkid` to force it not to use its cache.
When iterating over partitions, only record the ones that have a file-
system and save them in a filesystem to device map. Then use that for
mounting. This also prepares the way for LVM and LUKS where there is
not a 1:1 mapping between partition and filesystem.
Image info currently cannot handle LVM and will abort with something
like:
mount: /tmp/tmpzwlch91r: unknown filesystem type 'LVM2_member'.
Detect LVM setup and just exit for now.
Add support for building images for the Azure marketplace: add a
new image type "azure-rhui" that can be used to build images
tailored to the Azure marketplace.
Add two sample manifests for 8.5 and 8.6, but note that even the
8.5 is using the 8.6 distro definitions. Also no image-info is
included since `image-info` cannot (yet) handle LVM setups and
the azure marketplace images use the LVM setup.
When we are running inside a container we generally wont be booted
with systemd and thus systemctl will fail. Fall back to check for
the dnf json socket by checking the path exists and bail otherwise.
Adds ostree params to the request to generate a test manifest for
edge-installers and necessary customisations in the blueprint for the
edge-simplified installer. The manifest is not buildable but works for
checking for changes in the pipeline and packages for the installers.
Add rpmrepo repositories for testing.
Symlink centos-9 to centos-stream-9.
Add rpmrepo snapshot repositories to test and
test-case-generator repos.
Remove unused repositories from cs9 test repositories.
test/data/repositories:
new snapshots for RHEL 8.6 (20220201) and 9.0 (20220208)
Switching RHEL 8.6 and RHEL 9.0 to the latest rpmrepo snapshots:
We need updated packages for greenboot, used by edge image types, so
we technically only need to change the architectures that are
supported by edge, but let's update all of them for consistency.
test/data/repositories:
Remove rhel-90 symlink to beta
Keep the rhel-90-beta.json repository definitions, but make rhel-90.json
a file that contains the previously named rhel-90-ga.json definitions.
Extend the information gathered by `image-info`. This is needed to
properly inspect GCE images.
Enhancements:
- read all ssh client configuration files
- read all sshd configuration files
- read all YUM / DNF repos
- read DNF Automatic configuration
- fix reading of DNF configuration
Signed-off-by: Tomas Hozza <thozza@redhat.com>
This is needed to satisfy RPM dependencies of GCP guest tools, which
will be installed on the `gce` image type.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add F34 and F35 image test cases for all remaining image types, which
were previously not tested. With this PR, image test cases are now
generated for all image types on all architectures as supported by the
Fedora distro definition.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
`json_query` requires python3-jmespath which, while available in the
repos, it can sometimes cause issues when the ansible interpreter is
different from the system interpreter.
The `json_query` is only used in a handful of locations that can easily
be served by `jq`, which we use in other places already.
