Commit graph

18 commits

Author SHA1 Message Date
Ondřej Budai
1e2ba4da64 upload/azure: use cheaper storage accounts
Previously, we used RAGRS which means that all our data was always replicated
to at least two regions for increased safety. This is cool but expensive, this PR
switches the API to use LRS that just uses one region.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-17 17:51:23 +02:00
Ondřej Budai
385648223d spec: drop hacks for Fedora 32
There are not needed anymore, yay!

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-05 11:16:08 +02:00
Ondřej Budai
2e39d629a9 worker: add azure image upload target
This commit adds and implements org.osbuild.azure.image target.

Let's talk about the already implemented org.osbuild.azure target firstly:
The purpose of this target is to authenticate using the Azure Storage
credentials and upload the image file as a Page Blob. Page Blob is basically
an object in storage and it cannot be directly used to launch a VM. To achieve
that, you need to define an actual Azure Image with the Page Blob attached.

For the cloud API, we would like to create an actual Azure Image that is
immediately available for new VMs. The new target accomplishes it.
To achieve this, it must use a different authentication method: Azure OAuth.
The other important difference is that currently, the credentials are stored
on the worker and not in target options. This should lead to better security
because we don't send the credentials over network. In the future, we would
like to have credential-less setup using workers in Azure with the right
IAM policies applied but this requires more investigation and is not
implemented in this commit.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-03-06 15:40:48 +00:00
Ondřej Budai
4b031a4692 upload/azure: rename azure.go to azurestorage.go
This file contains a client for Azure Storage API. As we soon introduce the
client for Azure API, we need a distinction here.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-03-06 15:40:48 +00:00
Ondřej Budai
4f66ab5d7c upload/azure: rename Image to PageBlob
The UploadImage method doesn't actually create an image. It creates a Page
Blob. Blob is something like S3 object but in the Azure terminology. Page
Blob means that's optimized for random access and it's the only blob type
that can be used to create images.

This commit cleans up the terminology so it's less confusing.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-03-06 15:40:48 +00:00
Ondřej Budai
478f69e092 upload/azure: move UploadImage under a new StorageClient struct
We will soon introduce new methods to the storage client.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-03-06 15:40:48 +00:00
Ondřej Budai
f67ca8b616 azure: return an early error if unaligned
If the image size isn't aligned to 512 bytes, the Azure API returns very hard
to understand error message. Let's do this check ourselves early so we can
return a sane error.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-03-06 15:40:48 +00:00
Ondřej Budai
1b05192298 upload/azure: use the new azure/azblob API on Fedora 33+ & RHEL
Fedora 33 and rawhide got an updated version of the azblob library. Sadly, it
introduced a non-compatible API change. This commit does the same thing as
a67baf5a did for kolo/xmlrpc:

We now have two wrappers around the affected part of the API. Fedora 32 uses
the wrapper around the old API, whereas Fedora 33 and 34 (and RHEL with its
vendored deps) use the wrapper around the new API. The switch is implemented
using go build flags and spec file magic.

See a67baf5a for more thoughts.

Also, there's v0.11.1-0.20201209121048-6df5d9af221d in go.mod, why?

The maintainers of azblob probably tagged a wrong commit with v0.12.0 which
breaks go. The long v0.11.1-.* version is basically the proper v0.12.0 commit.
See https://github.com/Azure/azure-storage-blob-go/issues/236 for more
information.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-01-06 16:31:28 +01:00
Ondřej Budai
8d1c187f3c upload/azure: delete the unused test
This test is not run anywhere because it was surpassed by image tests with
azure boot type which perform more than just uploading a randomly generated
file to Azure. Let's delete dead code.
2020-08-25 10:41:10 +02:00
Ondřej Budai
c711b9149e remove travis mentions in codebase
Travis is no longer used, let's ditch these.
2020-08-25 10:41:10 +02:00
Ondřej Budai
6513263a14 upload/azure: force .vhd extension
Azure cannot create an image from a storage blob without .vhd extension.
This commit ensures that image always has the right extension.
2020-04-29 18:15:13 +02:00
Ondřej Budai
1d52dfcc2b upload/azure: fix typos 2020-04-29 18:15:13 +02:00
Ondřej Budai
5415be465f upload/azure: remove an unneeded custom error type
The errorString was not exported and used anywhere else, this commit replaces
it with a standard method of creating simple errors.
2020-04-29 18:15:13 +02:00
Ondřej Budai
e52a35d84e upload/azure: add a context to errors
Debugging azure upload fails is currently pretty hard, this commit adds
a context to all errors Azure upload can emit.
2020-04-29 18:15:13 +02:00
Ondřej Budai
d7cbc22da4 lint: fix unhandled errors 2020-03-02 14:28:55 +01:00
Ondřej Budai
1676cf58da lint: fix gosimple/S1004 errors 2020-03-02 14:28:55 +01:00
Martin Sehnoutka
364ea62f59 Introduce Azure upload test
The test is very simple, it first upload a file with random content and
then downloads it back and compare hashes.
2019-11-29 12:19:40 +01:00
Martin Sehnoutka
1135e9fe01 Refactor Azure upload code to a separate package
as part of this, move also the AWS upload code to the same directory
2019-11-27 12:14:07 +01:00