We want to be able to safely gather any artifacts without worrying about
any possible secrets leaking. Every artifacts that we want to upload
will now have to be placed in /tmp/artifacts which will then be uploaded
to S3 by the executor and link to the artifacts will be provided in the
logs. Only people with access to our AWS account can see them.
With the more fine-grained build package set, different images will use
different build pipelines and each of them will be smaller.
We don't currently cache build pipelines so there is no downside to this. Even
when we start caching the difference between having one build pipeline per
image and one shared one is minimal at scale. This will still benefit users
doing one-off builds on-prem.
Most importantly, this tracks things correctly, making pipelines more
composable without having to maintain a global list of dependencies.
We stopped testing on RHEL 8.4 because it wasn't changing, but now it
will be (or might) since it lives inside the common rhel8 package.
Testing the distro ensures we don't break it. RHEL 8.4 is still
supported as EUS.
We will soon change the distro definition to specifically build 8.4 EUS.
Pin osbuild version for RHEL 8.4.
Change the ostree test to support 8.4 (and not 8.5).
Manifest diffs can sometimes get large and putting them in the log makes
life harder for everyone.
Save them in a single file in the job artifacts instead.
Update the comment left by Schutzbot on the PR to mention the artifacts.
Converted by loading them through the manifest parser in osbuild and
formatting them through v2 before dumping.
These are not the "real" manifests that osbuild-composer would generate,
but they will make it a tiny bit easier to compare and detect changes in
the distro definition when it's moved to the common rhel8 package.
Always include the tools for all the filesystem types in the partition table. There may be
usecases for having additional ones, for instance if the partition table is not known, but
this gives us a minimal baseline.
This includes dosfstools in images that have a vfat partition but did not include the tools.
This is a partial revert of 006c5b26, where kernel settings and bootloaders were only
installed on bootable systems.
However, ostree-based systems need the ability to pick up kernels and bootloaders from
the commit to install on the instance, so make this conditional on being bootable or an
ostree commit. This is probably an indication that we need a different abstraction.
Currently an image type could override the boot loader in the architecture, but we should
not allow an image type to select a bootloader type the architecture does not support. So
only in case the architecture supports hybrid boot do we allow the image type to select
one of the two types.
This enables UEFI on AMIs for aarch64.
Collect partition tables / boot loaders / kernels together and
make them conditional on the system being bootable.
As a side-effect, we no longer install the grub2 modules in ostree commits.
- Fixed shellcheck errors
- Moved checkEnv from common to individual tests
- Fixed package install section in spec file:
Globs which include a directory fail on el-like distros.
- Use gcloud cli to ssh
- (re)Introduce generic s3 tests
Each cloud now has its own file that's sourced on-demand by the main api.sh
script. The main goal of this commit is to reduce the amount of clutter in
api.sh. I, personally, find 1300 lines of bash overwhelming and I think that
this is a reasonable beginning to start cleaning things up.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
No longer name the packageSetChains after the package set, but
keep them named after the pipelines. This should be a
non-functional change as dnf-json does not care about what the
chains are called, only that the names are unique.
Add a plain `rhel-8` alias as the default distribution name and version
for the `rhel8` package. The `rhel-86` distro is still available via
the NewRHEL86() constructor. These two distributions are identical.
Repositories
------------
The rhel-8 repositories (repositories/rhel-8.json) are now set to the
CDN repositories with no minor version:
https://cdn.redhat.com/content/dist/rhel8/8/...
The rhel-8 test repositories (test/data/repositories/rhel-8.json) were
already set to the plain `8` repositories. The Google repos have been
added.
The test case generator repositories used for `rhel-8` are the rpmrepo
snapshots as for rhel-86.
The rhel8 package represented RHEL 8.3, which is EOL.
The current rhel86 package will be renamed to rhel8 and be responsible
for building all RHEL 8 minor versions.
This was a stop-gap until the actual rhel 9 distro was created. It
is in a sad state, quite broken and shout not be used by anybody.
Put it out of its misery.
Define the distribution strings for RHEL 8.5 in distro/rhel86 and add
constructors. Remove the old 8.5 from the distro registry and use the
new constructors.
Composer can now build RHEL 8.5 image-installer on aarch64, which wasn't
supported before.
RHEL 8.5 manifests have changed to minimise the differences from 8.6.
Some changes are fixes made in 8.6 but never backported to 8.5 because
of our (older) policy of not changing definitions after the release of a
distro.
Other changes are non-functional (e.g., stage or package order).
See the list below for the source of each change.
Manifest changes:
- Stage order changed for org.osbuild.systemd-logind and
org.osbuild.rhsm.
- org.osbuild.grub2 options: config.default = "saved"
Reverted 111cd8871f
- Partition sizes: RHEL 8.5 had extra arbitrarily sized padding for the
header. Now all partitions are sized to fit headers exactly.
Original change at b7abef54e8.
- SELinux set to permissive in Anaconda. This was changed in RHEL 8.6
and 9.0 but never backported to 8.5.
See a7fbe916b7.
- Installer isolevel set to 3. Like above, this was changed in
8.6 and 9.0.
Original change at d8d161480e.
- Specify a remote for edge deployments.
Original change at b18b4e80a0.
Added utility function for comparing RHEL version strings.
Conditions added:
- greenboot subpackages were changed between RHEL 8.5 and RHEL 8.6.
- fido client packages aren't available in RHEL prior to 8.6.
- the ec2 SAP image type is not supported in RHEL prior to 8.6.
- the edge-simplified-installer and edge-raw-image image types are not
supported in RHEL prior to 8.6.
- They were previously supported in 8.5 without FDO support, but now
it's dropped from 8.5 completely.
The script isn't supposed to fail when the manifests differ.
Initialise err to 0 and assign it the exit code of the diff call if it
returns with an error.
Add a container image type that is based on the existing fedora
container image. There is a delta in terms of the configuration
because osbuild does not yet provide all the neccessary means,
but the package set is already very close.
The script runs the gen-manifests command first on the PR head and then
on the merge-base with the PR's base branch (typically 'main') and
checks for any differences. It creates a review comment on the PR on
GitHub if any changes are detected.
The message is posted as a simple COMMENT type review to inform the
author and reviewers that changes exist.
The script doesn't fail if there's a diff. CI shouldn't fail if changes
are detected since they can be intentional. The job fails if something
goes wrong with the script execution (manifest generation, comment
posting, etc).
The script exits immediately if not run from a PR.
The gen-manifests run is silenced with `> /dev/null`. In the future,
this should be handled by flags to the command itself to control the
output format noisiness.
The gen-manifests command is run 50 workers. Testing with 100 seemed to
make the execution stall, likely because of the resources on the worker.
We can experiment with this value more in the future.
