Prior this commit we only had support for username/password authentication
in the koji integration. This wasn't particularly useful because this
auth type isn't used in any production instance.
This commit adds the support for GSSAPI/Kerberos authentication.
The implementation uses kerby library which is very lightweight wrapper
around C gssapi library.
Also, the koji unit test and the run-koji-container script were modified
so the GSSAPI auth is fully tested.
In the near future, we will need to communicate with Koji using HTTPS.
This will surely bring the need for ignoring bad certificates/providing
our own self-signed ones. Thus, this commit prepares the Koji integration
by adding a way to accept a custom http transport which can be used to
customize the TLS settings.
Previously, Koji instance could be both logged-in and not logged-in.
This change disallows it: Now, the Koji instance is created by calling
koji.Login, so it must be always logged-in. This change should lead to more
robust code.
run-koji-container has now two actions: start and stop:
- ./run-koji-container.sh start
- ./run-koji-container.sh stop
The start action starts all containers. When it exits, all containers are
started and running in the background. To stop and removethem, use the stop
action.
This change is needed so we're able to easily use this script also in the CI
environment.
The setup should be container engine agnostic. This changes allows this script
to be run on systems which prefer docker over podman (e.g. Github Actions).
- inside RunJob() there is a deferred function which will remove
the entire temporary directory in which images are created, including
the streamOptimized file
- inside testBootUsingVMware(), which wants to use this function,
there is already a deferred function which removes the converted
image
This test is not run anywhere because it was surpassed by image tests with
azure boot type which perform more than just uploading a randomly generated
file to Azure. Let's delete dead code.
Applying a `Name` tag to the snapshot makes the name of the image appear
in the snapshot listing in EC2's console. It also makes it easier to
remove a snapshot after deregistering the AMI.
Signed-off-by: Major Hayden <major@redhat.com>
The AWS API always uses pointers to strings everywhere, rather than
plain strings. The log messages got this wrong and printed the pointers
rather than dereferencing them.
Don't require users to do a memory dump to get at the AMIs, but instead
print the actual strings. Also change to %s from %v to let the linter
warn us in the future. %s accepts only strings, whereas %v prints
whatever you throw at it.
Signed-off-by: Tom Gundersen <teg@jklm.no>
When ImportSnapshot fails, it goes into the deleted state. However, the code
didn't consider this state as a failure. This commit fixes that.
Fixes#706
The AWS upload process takes time and has multiple steps that can fail
or take an extended time to run.
Add some basic logging to various parts of the AWS upload process to
assist with troubleshooting in CI and production.
Signed-off-by: Major Hayden <major@redhat.com>
Specifying the image format in the `DiskContainers` part of the snapshot
import process sometimes causes AWS to fail the import due to formatting
issues. However, AWS seems to do a decent job at determining the image
format on its own without specifying the format explicitly.
Signed-off-by: Major Hayden <major@redhat.com>
Importing an image into an EBS snapshot can sometimes take a long time.
The customer experience for a timeout is not great.
Allow the import to continue running for as long as needed without
timing out. This allows a customer to monitor the process of their
import until it has a success or failure status.
Fixesosbuild/osbuild-composer#632.
Signed-off-by: Major Hayden <major@redhat.com>
This does not yet actually upload the image, and it only supports empty
images. You need to place a an empty file named <filename>, with a valid
extension (e.g., .qcow2) in /mnt/koji/work/<directory>/.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Prior this commit the ami image type produced raw.xz images. This was bad for
two reasons:
- The upload was broken because AWS doesn't support tar.xz format
- XZ compression is terribly slow
This commit changes the format to vhdx, which is supported by AWS and also
quite quick. See https://github.com/osbuild/osbuild-composer/issues/257
why vhdx was chosen.
Fixes#257
The old aws unit test is not very useful now, because there's the new
integration test which covers a lot more. Therefore, this commit
removes it.
Fixes#512
us-east-1 seems to very slow these days, some imports can take up to
15 minutes. This commit raises the number of attempts before we give up.
Previously, we did 40 attempts, each delayed by 15 seconds, making the total
timeout equal to 10 minutes. Now we do 80 attempts with the same delay,
making the total timeout 20 minutes.
After the snapshot is imported to EC2, there's no point in having the s3
object anymore. If the s3 object is not deleted, the user will be charged
for it periodically, while the object doesn't provide any value.
