Commit graph

15 commits

Author SHA1 Message Date
Sanne Raymaekers
2a621521a8 osbuildexecutor/aws.ec2: set hostname of executor via cloud-init
This way much more of the journal will be captured under the new
hostname.
2024-06-25 10:58:10 +02:00
Sanne Raymaekers
ae4467ab0d internal/awscloud: retry CreateFleet
When receiving the "UnfillableCapacity" error from CreateFleet, retry
the request with an OnDemand instance.
2024-06-24 12:50:37 +02:00
Sanne Raymaekers
2e31ea50aa cloud/awscloud: use instance requirements when creating secure instance 2024-06-14 10:59:58 +02:00
Sanne Raymaekers
314ed4b527 cloud/awscloud: allow internet access on secure instance again
The executor is timing out and there are no logs. This will require some
further work. Remove the restriction for now.
2024-03-20 14:58:25 +01:00
Sanne Raymaekers
79b5b736e9 cloud/awscloud: restrict network egress for secure instance
The security instance should no longer have any internet access.
2024-03-19 17:07:30 +01:00
Tomáš Hozza
e7743f17ec Worker: allow configuring executor CloudWatch group
We need the ability to use different CloudWatch group for the
osbuild-executor on Fedora workers in staging and production
environment.

Extend the worker confguration to allow configuring the CloudWatch group
name used by the osbuild-executor. Extend the secure instance code to
instruct cloud-init via user data to create /tmp/cloud_init_vars file
with the CloudWatch group name in the osbuild-executor instance, to make
it possible for the executor to configure its logging differently based
on the value.

Cover new changes by unit tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-08 13:13:44 +01:00
Sanne Raymaekers
040eec4089 osbuild-worker: allow adding key to aws.ec2 executor
This is useful during testing to set up the executor machine.
2024-03-01 19:20:51 +01:00
Sanne Raymaekers
5025ec31d3 cloud/awscloud: describe security groups using filters
Using the group names option only works for the default VPC, the workers
are not running in the default VPC. For non-default VPCs filters should
be used.
2024-02-20 15:23:52 +01:00
Sanne Raymaekers
7fce482baa cloud/awscloud: create secure instance in the same subnet
This reduces network costs as transferring data between AZs is not free.
2024-02-16 15:21:20 +01:00
Sanne Raymaekers
ee6b198b0a cloud/awscloud: remove restricting egress rule from SG
The machine still needs to be able to fetch sources, so just keep the
default 0.0.0.0/0 rule.
2024-02-15 14:23:18 +01:00
Sanne Raymaekers
8e6717fa1b cloud/awscloud: take instance type from host
InstanceRequirements is very flakey, the create fleet request fails
almost consistently with the same error.

To continue with testing use a fixed instance type for now. As a
followup we can expand the instance type selection logic or figure out
what was wrong with the InstanceRequirements.
2024-02-14 18:15:25 +01:00
Sanne Raymaekers
8a1d66a0bd cloud/awscloud: max 4 overrides are allowed when creating a fleet
```
InvalidParameterValue: Your request contains more than the maximum allowed number of InstanceRequirements (4)
```
2024-02-14 15:24:42 +01:00
Sanne Raymaekers
7fd150b938 cloud/awscloud: specify subnets when creating secure instance
For non-default VPCs, AWS needs the subnets it can launch the instance
in, otherwise it will try to launch the instance in the default VPC,
even if the supplied security groups are attached to a non-default VPC.

Furthermore there can only be 1 subnet specified per availability zone,
so query the subnets in the VPC of the host (as the instance needs to be
launched in the same network), and pick 1 of the VPC's subnets per AZ.
2024-02-14 13:45:52 +01:00
Sanne Raymaekers
a2fb1bfc61 cloud/awscloud: add userdata to secure instance
This way the `worker-initialization.service` knows to spin up the
builder instead of the worker.
2024-02-14 09:54:11 +01:00
Sanne Raymaekers
3db88960c2 cloud/awscloud: add ability to run a secure instance to awscloud
This instance can only contact the host, and requires this host to be
running on AWS itself with the appropriate IAM role.
2024-02-14 09:54:11 +01:00