Collect partition tables / boot loaders / kernels together and
make them conditional on the system being bootable.
As a side-effect, we no longer install the grub2 modules in ostree commits.
- Fixed shellcheck errors
- Moved checkEnv from common to individual tests
- Fixed package install section in spec file:
Globs which include a directory fail on el-like distros.
- Use gcloud cli to ssh
- (re)Introduce generic s3 tests
Each cloud now has its own file that's sourced on-demand by the main api.sh
script. The main goal of this commit is to reduce the amount of clutter in
api.sh. I, personally, find 1300 lines of bash overwhelming and I think that
this is a reasonable beginning to start cleaning things up.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
No longer name the packageSetChains after the package set, but
keep them named after the pipelines. This should be a
non-functional change as dnf-json does not care about what the
chains are called, only that the names are unique.
Add a plain `rhel-8` alias as the default distribution name and version
for the `rhel8` package. The `rhel-86` distro is still available via
the NewRHEL86() constructor. These two distributions are identical.
Repositories
------------
The rhel-8 repositories (repositories/rhel-8.json) are now set to the
CDN repositories with no minor version:
https://cdn.redhat.com/content/dist/rhel8/8/...
The rhel-8 test repositories (test/data/repositories/rhel-8.json) were
already set to the plain `8` repositories. The Google repos have been
added.
The test case generator repositories used for `rhel-8` are the rpmrepo
snapshots as for rhel-86.
The rhel8 package represented RHEL 8.3, which is EOL.
The current rhel86 package will be renamed to rhel8 and be responsible
for building all RHEL 8 minor versions.
This was a stop-gap until the actual rhel 9 distro was created. It
is in a sad state, quite broken and shout not be used by anybody.
Put it out of its misery.
Define the distribution strings for RHEL 8.5 in distro/rhel86 and add
constructors. Remove the old 8.5 from the distro registry and use the
new constructors.
Composer can now build RHEL 8.5 image-installer on aarch64, which wasn't
supported before.
RHEL 8.5 manifests have changed to minimise the differences from 8.6.
Some changes are fixes made in 8.6 but never backported to 8.5 because
of our (older) policy of not changing definitions after the release of a
distro.
Other changes are non-functional (e.g., stage or package order).
See the list below for the source of each change.
Manifest changes:
- Stage order changed for org.osbuild.systemd-logind and
org.osbuild.rhsm.
- org.osbuild.grub2 options: config.default = "saved"
Reverted 111cd8871f
- Partition sizes: RHEL 8.5 had extra arbitrarily sized padding for the
header. Now all partitions are sized to fit headers exactly.
Original change at b7abef54e8.
- SELinux set to permissive in Anaconda. This was changed in RHEL 8.6
and 9.0 but never backported to 8.5.
See a7fbe916b7.
- Installer isolevel set to 3. Like above, this was changed in
8.6 and 9.0.
Original change at d8d161480e.
- Specify a remote for edge deployments.
Original change at b18b4e80a0.
Added utility function for comparing RHEL version strings.
Conditions added:
- greenboot subpackages were changed between RHEL 8.5 and RHEL 8.6.
- fido client packages aren't available in RHEL prior to 8.6.
- the ec2 SAP image type is not supported in RHEL prior to 8.6.
- the edge-simplified-installer and edge-raw-image image types are not
supported in RHEL prior to 8.6.
- They were previously supported in 8.5 without FDO support, but now
it's dropped from 8.5 completely.
The script isn't supposed to fail when the manifests differ.
Initialise err to 0 and assign it the exit code of the diff call if it
returns with an error.
Add a container image type that is based on the existing fedora
container image. There is a delta in terms of the configuration
because osbuild does not yet provide all the neccessary means,
but the package set is already very close.
The script runs the gen-manifests command first on the PR head and then
on the merge-base with the PR's base branch (typically 'main') and
checks for any differences. It creates a review comment on the PR on
GitHub if any changes are detected.
The message is posted as a simple COMMENT type review to inform the
author and reviewers that changes exist.
The script doesn't fail if there's a diff. CI shouldn't fail if changes
are detected since they can be intentional. The job fails if something
goes wrong with the script execution (manifest generation, comment
posting, etc).
The script exits immediately if not run from a PR.
The gen-manifests run is silenced with `> /dev/null`. In the future,
this should be handled by flags to the command itself to control the
output format noisiness.
The gen-manifests command is run 50 workers. Testing with 100 seemed to
make the execution stall, likely because of the resources on the worker.
We can experiment with this value more in the future.
`osbuild-mock-openid-provider`'s `/token` endpoint expects URL-encoded
values in the POST request body. Use the same values as those that would
be used by the worker when refreshing a token.
The `amdgpu` module causes issues on certain GPU-enabled instances
on Azure and it must not be loaded by default.
Modules are sorted alphabetically.
Signed-off-by: Major Hayden <major@redhat.com>
Co-Authored-By: Christian Kellner <christian@redhat.com>
In commit 5c1530e we disabled `skx_edac` and `intel_cstate` but
after further consultation with Prarit Bhargava it was agreed that
for RHEL 9 we should indeed allow them.
Use case
--------
If Endpoint is not set and Region is - upload to AWS S3
If both the Endpoint and Region are set - upload the Generic S3 via Weldr API
If neither the Endpoint and Region are set - upload the Generic S3 via Composer API (use configuration)
jobimpl-osbuild
---------------
Add configuration fields for Generic S3 upload
Support S3 upload requests coming from Weldr or Composer API to either AWS or Generic S3
Weldr API for Generic S3 requires that all connection parameters but the credentials be passed in the API call
Composer API for Generic S3 requires that all conneciton parameters are taken from the configuration
Adjust to the consolidation in Target and UploadOptions
Target and UploadOptions
------------------------
Add the fields that were specific to the Generic S3 structures to the AWS S3 one
Remove the structures for Generic S3 and always use the AWS S3 ones
Worker Main
-----------
Add Endpoint, Region, Bucket, CABundle and SkipSSLVerification to the configuration structure
Pass the values to the Server
Weldr API
---------
Keep the generic.s3 provider name to maintain the API, but unmarshel into awsS3UploadSettings
tests - api.sh
--------------
Allow the caller to specifiy either AWS or Generic S3 upload targets for specific image types
Implement the pieces required for testing upload to a Generic S3 service
In some cases generalize the AWS S3 functions for reuse
GitLab CI
---------
Add test case for api.sh tests with edge-commit and generic S3
The script would test if the test case generation script when the script
would run normally if the osbuild-dnf-json.service was stopped.
This is no longer necessary.
This value is set in the worker config. In future it might also be
passed through the api to upload into target accounts, but it should
never be set in composer.
Those images are forced to be 64GiB in size but mostly consist of zeros.
This makes them hard to handle, e.g. uploading to brew takes a forever.
The vhdPipelines is converted to a function returning the pipelinesFunc
and it has a single argument `compress` that will add the compression
pipeline bits if `true`. Will return exactly the old pipeline in case
of `false`.
Those images are forced to be 64GiB in size but mostly consist of zeros.
This makes them hard to handle, e.g. uploading to brew takes a forever.
The vhdPipelines is converted to a function returning the pipelinesFunc
and it has a single argument `compress` that will add the compression
pipeline bits if `true`. Will return exactly the old pipeline in case
of `false`.
