We are on a quest to reduce clutter on our Slack channels. Thus,
I decided to simplify the daily CI notifications:
- the link to the edge pipelines got removed, it's now in bookmarks
- several words were removed to make the message shorter
- the link to the pipeline is now a hyperlink
- the whole message should be a one liner
- less text is now bold
I've also simplified the format in which we send the message. I think
that the block format used before makes redundant line-breaks.
Unfortunately, the mentions need to be done using user IDs instead
of user names. If you ever need to find them, go to the user's profile,
click on the three dots and select "Copy member ID".
The current path sometimes launches two instances, which is problematic
because the rest of the secure instance code expects exactly one
instance. A security group could be attached to both instances, and
would block the worker from launching any more SIs, as it tries to
delete the old security group first, which is still held by one of the
surplus SIs which didn't get terminated.
Only retry if:
- on "UnfulfillableCapacity" or "InsufficientInstanceCapacity" error codes;
- there wasn't an instance launched anyway.
If either of these checks fail, do not try to launch another one, and
just fail the job.
Now and then there are leftover secure instances, probably when worker
instances get terminated during builds, this is possible in ASGs. 2
hours as a cutoff should be enough, since the build times out after 60
minutes, and fetching the output archive after 30 minutes, so that
leaves 30 minutes for booting and connection.
Imagine this scenario: the packer job is ran, an AMI gets created.
We configure our deployment to use this AMI. Then, someone retries the
packer job. Since we have force_deregister=true, this will not only
create a new AMI, but also remove the old one (because it has the same
name). Thus, our deployment will get broken, because the source AMI
no longer exists. This means that the ASG cannot replace any broken
instances, and the secure instance feature gets absolutely broken
because it cannot spawn new secure instances (they "inherit" the AMI
ID from their parents).
Let's remove force_deregister=true, so the AMI never gets replaced.
This might cause some pipelines to start failing because they are
rerunning the packer job for same commit (the GA pipeline currently).
Let's fix those then, rerunning the packer job is just confusing.
If this causes some unexpected issues, we can always resort to using
unique AMI names (by appending a timestamp to their name), but having
multiple AMIs with different names, but same tags will cause our
terraform configuration to be reapplied everytime there's a rerun,
which is also not great.
The errors returned by create fleet are not entirely clear. It seems it
also returns `InsufficientInstanceCapacity` in addition to
`UnfulfillableCapacity`. Let's just retry three times regardless of the
create fleet error, that way there's no need to chase error codes which
aren't clearly defined.
In case the on demand option failed as well, retry one more time across
availability zones. This significantly increases the pool of available
instances, but increases network related costs, as transferring data
between AZs is not free.
There were errors using the latest oc 4.17 version:
error: failed to read input object (not a Template?): unable to decode
"templates/openshift/composer.yml": no kind "Template" is registered for
version "v1" in scheme "k8s.io/kubectl/pkg/scheme/scheme.go:28"
When generating x509 v3 certs we need to explicitely set "CA:TRUE"
otherwise they're not trusted to be used. Also start running the tests
on RHEL-9.5 and RHEL-10.0
In the Service scenario, we should be testing that osbuild-composer
works fine, without any repository configurations being present on the
filesystem.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The `reporegistry.New()` has been enhanced to return an error, in case
there were no repositories loaded. This was to fix the situation in many
unit tests, which were previously not loading any repositories and
silently not running any tests.
This however broke our SaaS deployment, where we actually do not
configure any repositories on the filesystem. As a result,
osbuild-composer started to fail on it.
Workaround this situation in osbuild-composer by reverting to the old
behavior by loading the repo configs separately and then using the
loaded repos (which could be empty map) to initialize the RepoRegistry.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This is needed for GPG key import to work on RHEL-9, because the key
uses SHA-1. This results in the following error when importing the key
during the build of the build container:
"Signature not supported. Hash algorithm SHA1 not available."
We do not need the AUX key in our RHEL-8 repo definitions used for
testing.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the API unit test for Koji composes, to verify that the newly
added /sboms endpoint works correctly.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the unit test for regular (non-Koji) composes, to verify that
the newly added /sboms endpoint works correctly.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a new /sboms API endpoint, for getting SBOM documents for a given
compose ID. The endpoint returns an array of SBOM documents for each
image built as part of the compose. For each image, there is an SBOM
document for each osbuild pipeline, which installs RPM packages. This is
usually one 'buildroot' and one 'image' pipeline.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the `manifestJobResultsFromJobDeps()` function to also return the
manifest `JobInfo`. This will be useful to inspect the job dependencies
and eliminate the need to add a specialized function for getting only
the `JobInfo`.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
None of our worker is running RHEL-8 any more. There's no value in
testing the Koji scenario on RHEL-8, RHEL-9 is fully sufficient.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
We have been testing builds of RHEL-9 on RHEL-8 for the Koji use case.
However, all of our workers are now running the latest GA RHEL-9
version. Therefore we should flip the test and test building of RHEL-8
on RHEL-9.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Adjust the test case to cope with the SPDX SBOM documents uploaded to
the Koji. Also explicitly check that there is the expected number of
SBOM documents uploaded as the image build output.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the Koji target handling in the osbuild job, to also upload SBOM
documents attached to the related depsolve job result.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
If the Koji target result contains information about any uploaded SBOM
documents, import them to Koji as part of the finalize task.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
For Koji composes, all files are uploaded to Koji as part of the osbuild
job (specifically as part of handling the Koji target). So in order to
be able to upload SBOM documents to Koji as part of Koji compose, the
osbuild job needs to to be able to access the depsolve job result, which
contains the SBOM documents. For this, the osbuild job must depend on
the depsolve job.
For Koji composes, make sure that osbuild job depends on the depsolve
job and set the DepsolveDynArgsIdx.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
