Commit graph

6883 commits

Author SHA1 Message Date
Lukas Zapletal
f41c764ca7 build(deps): bump deps specifically CVE-2024-45338 2025-01-09 16:30:20 +01:00
schutzbot
bdc755f71b Post release version bump
[skip ci]
2025-01-08 08:13:07 +00:00
Lukas Zapletal
2aaaaf05b3 ci: do not perform linting on vendor/ 2025-01-07 11:57:51 +01:00
Achilleas Koutsou
8b0a1d1714 schutzbot: update Achilleas' ssh keys 2025-01-02 08:03:39 +01:00
Tomáš Hozza
8c99478a69 upgrade_verify.sh: update el10 repo URLs to non-Beta
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-02 07:33:05 +01:00
Simon Steinbeiss
e90010f153 github/actions: Enable /jira-epic slash commands
This change allows for using the command to create Jira Tasks under a given
Epic both in a pull request comment or in the pull request description.

To trigger the action, you can simply add a comment to a given pull request with
the following content:
/jira-epic ISSUE-1234

This will trigger the action and a bot will create a Jira Task under the
Epic ISSUE-1234. Once this is successful, it will update the pull request
title and description to contain a link to the newly created Jira ticket,
which also means that the two will be linked.
Alternatively, you can also add the command to the pull request description
(if you e.g. want to create the Task at PR creation time).

Note: This is currently only enabled for the HMS project.
2024-12-13 11:27:52 +01:00
Florian Schüller
6fdc6d6a55 go.mod: upgrade golang.org/x/crypto
Upgrade golang.org/x/crypto due to
https://github.com/osbuild/osbuild-composer/security/dependabot/43
2024-12-12 16:03:32 +01:00
schutzbot
f6d320886d Post release version bump
[skip ci]
2024-12-11 08:18:23 +00:00
Florian Schüller
153bcadf2d osbuild-service-maintenance/aws: merge errors
Collect and merge errors, instead of
nesting errors.
In this case we want to continue execution if only one
cleanup fails.
2024-12-10 11:43:51 +01:00
Florian Schüller
87ef1527fc osbuild-service-maintenance/aws: avoid error on empty list
Passing an empty list to `TerminateInstances` causes an
error message, which is not necessary, as there is
nothing to terminate.
2024-12-10 11:43:51 +01:00
Florian Schüller
24e256c225 osbuild-service-maintenance: add test for allTerminated
HMS-3632
2024-12-10 11:43:51 +01:00
Florian Schüller
65b7ee65b2 osbuild-service-maintenance: implement removal of launch templates
Launch templates of instances that are terminated should be removed.
HMS-3632
2024-12-10 11:43:51 +01:00
Florian Schüller
a96ea533c0 osbuild-service-maintenance: implement removal of security groups
Security groups of instances that are terminated should be removed.
HMS-3632
2024-12-10 11:43:51 +01:00
Florian Schüller
7ebe266d3c osbuild-service-maintenance: implement removal on invalid parent
Add a safeguard to ensure secure instances without valid
parent instances are terminated, as they are unnecessary to retain.
Typically, the parent does not exist if the secure instance is
older than 2 hours, but this check provides additional validation.
HMS-3632
2024-12-10 11:43:51 +01:00
Florian Schüller
a7119a4d0f osbuild-service-maintenance/aws: support aws credential file
Support running the maintenance locally with a valid
`~/aws/credentials` file. HMS-3632
2024-12-10 11:43:51 +01:00
Tomáš Hozza
580366d1f3 osbuild-dnf-json-tests: don't set OSTree options for non-OSTree images
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-12-09 09:46:54 +01:00
Tomáš Hozza
1f590aa232 Weldr/ComposeRequest: OSTree options nil if not set
Previously, the `OSTree` property in the Weldr API `ComposeRequest`
struct was not a pointer to the `ostree.ImageOptions` type. As a result,
it was initialized to an empty struct, even if not set in the client API
call.

As a result, the `OSTree` property in the `distro.ImageOptions` was
always not `nil`, when initializing the osbuild manifest. However, after
a change in `osbuild/images` [0], providing OSTree options for
non-OSTree image types is no longer considered valid. This caused a
failure to submit a new compose for any non-OSTree image type.

Change the `OSTree` property in Weldr `ComposeRequest` to be a pointer
and mark it as optional.

[0] https://github.com/osbuild/images/pull/1071

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-12-09 09:46:54 +01:00
Tomáš Hozza
fdff0a903e Update osbuild/images to v0.105.0
This brings a lot of new stuff, among others:
 - Internal image types for RHEL-10 (COMPOSER-2357)
 - Support for Swap partitions in DiskCustomizations (COMPOSER-2400)
 - Support for new partitioning customizations for CS and RHEL
 - Fix for RHEL 8 and 9 AMI boot mode (RHEL-69628)

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-12-09 09:46:54 +01:00
dependabot[bot]
73f3aa22a2 build(deps): bump codecov/codecov-action from 4 to 5
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-09 09:46:42 +01:00
Tom Koscielniak
1193187e0a tools/tests: Update rhel10 compose url
Update a rhel 10 compose url to point to nightly instead of public beta.
Fix for a failing rhel 10 nightly pipeline.
2024-12-06 12:06:40 +01:00
Ondřej Budai
3561202acc github: prevent script injections via PR branch names
Prior this commit, ${{ github.event.workflow_run.head_branch }} got
expanded in the bash script. A malicious actor could inject
an arbitrary shell script. Since this action has access to a token
with write rights the malicious actor can easily steal this token.

This commit moves the expansion into an env block where such an
injection cannot happen. This is the preferred way according to the
github docs:
https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-an-intermediate-environment-variable
2024-12-05 18:13:17 +01:00
Tom Koscielniak
f5a5705b7e schutzbot/tests: bump rhel10 to nightly, update tf sha and osbuild deps
Bump RHEL 10 from beta to nightly by updating terraform SHA and osbuild dependencies to start testing RHEL 10 nightly and to meet the CTC schedule.
2024-12-05 14:33:12 +01:00
schutzbot
ee41e3dce6 schutzfile: Update snapshots to 20241203 2024-12-04 10:33:09 +01:00
Sanne Raymaekers
f6feb7675b cloud/awscloud: use any instance create fleet returns
Even in case of errors, as long as create fleet returns an instance,
attempt to use it.

In some cases AWS returns `InsufficientInstanceCapacity` but still
creates an instance:
```
msg="Won't retry CreateFleet with OnDemand instance, retry: false, errors: InsufficientInstanceCapacity: There is no Spot capacity available that matches your request.; Already launched instance ([i-...]), aborting create fleet"
msg="doCreateFleetRetry: returning retry: false, msg: [InsufficientInstanceCapacity: There is no Spot capacity available that matches your request. Already launched instance ([i-...]), aborting create fleet]"
msg="doCreateFleetRetry: cancelling retry, instance already exists: [i-...]"
msg="doCreateFleetRetry: setting retry to true"
msg="Checking to retry fleet create on error InsufficientInstanceCapacity (msg: There is no Spot capacity available that matches your request.)"
```
2024-12-03 14:00:12 +01:00
Lukas Zapletal
4b55bc2825 cloudapi: carry ostree MTLS secret over 2024-12-03 13:59:45 +01:00
Sanne Raymaekers
779053d910 cloud/awscloud: give secure instances a name
That way you can just enter the parent instance id into the search bar
and get both the worker and its executor.
2024-12-03 11:56:52 +01:00
Jakub Rusz
f68fcff400 CI: remove jrusz from notifications
Removing myself from the slack notifications as I'm no longer a QE
member.
2024-12-02 14:34:51 +01:00
Sanne Raymaekers
38b799f162 cloud/awscloud: exclude really old instance types
RHEL 10 (nightly) builds fail on stage with "Fatal glibc error: CPU does
not support x86-64-v3", this is most likely due to very old instance
types not supporting a specific instruction set.
2024-11-29 15:42:27 +01:00
Lukas Zapletal
e7a7cda3bc cmd: extra env logging for osbuild worker 2024-11-29 11:57:11 +01:00
Sanne Raymaekers
1372be3f6e test/cases/api/azure: actually verify the HyperV Generation 2024-11-29 10:37:24 +01:00
Lukas Zapletal
6310115629 deps: update images to 0.102 2024-11-27 13:03:29 +01:00
schutzbot
2e1afdc829 Post release version bump
[skip ci]
2024-11-27 08:17:34 +00:00
Ondřej Budai
64ff0e3dad awscloud: add very verbose logging to createFleet creation
We still see this error sometimes:

Unable to start secure instance: Unable to create fleet: InsufficientInstanceCapacity: There is no Spot capacity available that matches your request

This is awkward because the message mentions that there is no spot
capacity, even though the current code should retry on
InsufficientInstanceCapacity. I also confirmed this by searching for
the retries log messages: there are none in the logs.

We need a bigger hammer. Let's log everything that happens in the
createFleet method in order to have better understanding why the
retry logic isn't triggered. We should probably move most of the newly
added logs to the debug level, but let's delay that until we have
more insight into what's happening.
2024-11-26 16:12:09 +01:00
Sanne Raymaekers
54ffc08814 awscloud/secure-instance: pass on fleet information on error
By surfacing the output even in case of an error, the fleet ID and
instance ID can be extracted if present. Thus the instance can be
terminated before its dependencies are deleted.
2024-11-26 12:52:12 +01:00
Sanne Raymaekers
7a166cd356 awscloud/secure-instance: log error code comparisons
We're seeing some behaviour where create fleet is not retried and
subsequently the SI cleanup fails due to the security group already
being tied to an existing instance. There is no error that an instance
was launched anyway.
2024-11-26 12:52:12 +01:00
Sanne Raymaekers
7f2766793d templates/composer: bump rhel-9 distro alias
RHEL 9.5 is now GA.
2024-11-26 11:25:33 +01:00
Michael Vogt
bc7b8355bf worker: report cashes directly to logrus
This is a bit of an RFC commit, I noticed that when we discussed
a crash from the worker we looked at individual message from
syslog/journald for the stacktrace deatils. I was wondering if
having a more direct crash report would be more useful? We can
of course also add more logrus features to flag those with tags
like "crash" or something (I did not do that in this PR, I don't
know much about the operational side, sorry).
2024-11-25 12:02:05 +01:00
Sanne Raymaekers
27e9e22639 Schutzfile: update osbuild commit
Newer osbuild uses dnf 4 on Fedora 41, as we've seen Fedora 41 in
combination with dnf 5 (and SBOMs) failing.
2024-11-22 16:39:52 +01:00
Florian Schüller
446e8448e3 awscloud/secure-instance: retry for 10 minutes
retry for 10 x 60sec. and don't log retries twice
2024-11-22 12:19:32 +01:00
Florian Schüller
4ec8894244 awscloud/secure-instance: retry on error in terminated waiter
terminated waiter sometimes responded
with "waiter state transitioned to Failure"
where we want to retry waiting for the termination
2024-11-22 12:19:32 +01:00
Sanne Raymaekers
3a6a8813a5 test/cases/api/azure: use v2 HyperV generation 2024-11-21 11:22:20 +01:00
Sanne Raymaekers
8fd36225be cloudapi/v2: support HyperV generation in Azure upload options 2024-11-21 11:22:20 +01:00
Sanne Raymaekers
f672610509 cmd/osbuild-worker: specify hyper v gen for azure images 2024-11-21 11:22:20 +01:00
Sanne Raymaekers
fb3e1b0701 internal/upload/azure: support different hyper v generations
When registering an image, users should be able to choose their hyper V
gen, as gen1 is quite outdated by now.
2024-11-21 11:22:20 +01:00
Sanne Raymaekers
d2f50a4224 internal/target: add Azure image HyperV generation 2024-11-21 11:22:20 +01:00
Yi He
867ff9c06e ci: change to rhel 9.6 2024-11-21 08:14:22 +01:00
Tom Koscielniak
d8295ea2ea Test with rhel-9.6 nightly 2024-11-21 08:14:22 +01:00
Lukas Zapletal
ff2660ef00 deps: update images to 0.99 2024-11-20 12:24:49 +01:00
Florian Schüller
b5c71cd7e2 awscloud/secure-instance: enrich logging with secure instance id
we'll log as direct URL to the console for easier tracing
2024-11-19 17:26:23 +01:00
Florian Schüller
992f876da0 cloudapi/v2/server: rephrase error message 2024-11-19 13:55:38 +01:00