Commit graph

19 commits

Author SHA1 Message Date
Ondřej Budai
d9cd9f228f worker: log target errors
This should make debugging of target uploads so much easier.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-03-08 18:22:27 +01:00
Tom Gundersen
b062d1070d worker/jobimpl-osbuild/azure: set storage account
This was lost in a refactor.
2021-03-06 15:40:48 +00:00
Ondřej Budai
2e39d629a9 worker: add azure image upload target
This commit adds and implements org.osbuild.azure.image target.

Let's talk about the already implemented org.osbuild.azure target firstly:
The purpose of this target is to authenticate using the Azure Storage
credentials and upload the image file as a Page Blob. Page Blob is basically
an object in storage and it cannot be directly used to launch a VM. To achieve
that, you need to define an actual Azure Image with the Page Blob attached.

For the cloud API, we would like to create an actual Azure Image that is
immediately available for new VMs. The new target accomplishes it.
To achieve this, it must use a different authentication method: Azure OAuth.
The other important difference is that currently, the credentials are stored
on the worker and not in target options. This should lead to better security
because we don't send the credentials over network. In the future, we would
like to have credential-less setup using workers in Azure with the right
IAM policies applied but this requires more investigation and is not
implemented in this commit.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-03-06 15:40:48 +00:00
Ondřej Budai
4f66ab5d7c upload/azure: rename Image to PageBlob
The UploadImage method doesn't actually create an image. It creates a Page
Blob. Blob is something like S3 object but in the Azure terminology. Page
Blob means that's optimized for random access and it's the only blob type
that can be used to create images.

This commit cleans up the terminology so it's less confusing.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-03-06 15:40:48 +00:00
Ondřej Budai
478f69e092 upload/azure: move UploadImage under a new StorageClient struct
We will soon introduce new methods to the storage client.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-03-06 15:40:48 +00:00
Tomas Hozza
319400c280 cloudapi: implement GCP target results and include UploadOptions
Return GCP-specific target results form the worker, similar as it is
done for AWS.

Extend Cloud API to allow GCP-specific upload Options.

Modify Cloud API to return UploadOptions as part of the UploadStatus.

Modify Cloud API integration test to check returned upload Options and
upload Type.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-03-04 15:44:04 +01:00
Chloe Kaubisch
f091af55d8 cloudapi: add targetresults
Add the TargetResult struct to OSBuildJobResult. Include the 'options'
interface on TargetResult to contain target-specific information,
for example amiID and region from AWS. Expose 'options' on a status
call as an UploadStatus field. Add logic to support AWS within this
format, which can be used as a template for other targets.
2021-03-02 13:22:11 +01:00
Tomas Hozza
94d399f010 cloudapi: Add support for GCP as upload target
Add support for GCP as an upload target to the internal API.

Extend the cloudapi to allow GCP as an upload target in the compose
request. Regenerate the cloudapi go code. Added GCP-specific upload
result component in the API definition, similar to AWS. It is not yet
used, but it will be once returning a target-specific result from
worker is supported.

Add support for GCP upload target to the worker job implementation.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-25 18:44:21 +00:00
Tom Gundersen
f0b7cc0973 worker: keep output directory in /var/cache
Let's keep this on the same filesystem as the osbuild store, and
in particular stay away from /var/tmp and its scary semantics.

We are not aware of any issues caused by /var/tmp, but getting
rid of it means we don't have to think about that when debugging,
if nothing else.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Jozef Mikovic
5a9d2fe8b1 test: Add integration test for VMWare upload 2021-02-16 19:06:01 +00:00
Jozef Mikovic
0597ac48a7 upload/vmware: document uploadImage function 2021-02-16 19:06:01 +00:00
Jozef Mikovic
d497bc3b83 osbuild-worker: ensure that uploaded vmdk image has the user specified name
VMDK image has default name 'disk.vmdk' and there is no option to change the name when uploading to vSphere,
so I'm using symlink so that uploaded image has the name user specified instead of the default one.
2021-02-16 19:06:01 +00:00
Jozef Mikovic
08cfc08805 weldr: add vmware upload target to api
Extend Weldr API to allow user to specify option to upload built image to VMWare,
makes use of previously define upload target.
2021-02-16 19:06:01 +00:00
Jozef Mikovic
1a81489ef1 osbuild-worker: add target for upload to vmware
New upload target for VMWare, similar to the ones for AWS and Azure,
allowing users to set credentials for their vSphere instance.
Commit also includes function that performs the actual upload.
2021-02-16 19:06:01 +00:00
Chloe Kaubisch
899d78f7e1
cloudapi: expose upload status
Expose a more detailed job status result - specifically, include upload status
alongside image status. Expand openapi.yml accordingly and add an UploadStatus
field to the OSBuildJobResult struct. At the moment, only represent the
"success" and "failure" states of UploadStatus - to differentiate between
"pending" and "running" would involve significant design decisions and should be
addressed in a separate commit.
2021-02-05 12:34:28 +01:00
Ondřej Budai
4548923a09 upload/aws: fix architecture for aarch64 images
Previously, composer wrongly set x86_64 architecture even for aarch64 images.
This commit fixes it.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2020-12-01 08:27:44 +01:00
Sanne Raymaekers
22c9f6af61 cloudapi: Share an ec2 snapshot/ami with an account 2020-11-26 13:08:18 +00:00
Lars Karlitski
6b6cd7ca9f worker: introduce JobImplementation interface
Introduce JobImplementation and turn the current RunJob() into
OSBuildJobImpl. Make main() select a job impl based on job type.

This is in preparation to add additional impls.
2020-11-09 14:17:19 +01:00
Lars Karlitski
3bc642e4aa worker: split osbuild job implementation into separate file
No code was changed.
2020-11-09 14:17:19 +01:00