Prior this commit we only had support for username/password authentication
in the koji integration. This wasn't particularly useful because this
auth type isn't used in any production instance.
This commit adds the support for GSSAPI/Kerberos authentication.
The implementation uses kerby library which is very lightweight wrapper
around C gssapi library.
Also, the koji unit test and the run-koji-container script were modified
so the GSSAPI auth is fully tested.
In the near future, we will need to communicate with Koji using HTTPS.
This will surely bring the need for ignoring bad certificates/providing
our own self-signed ones. Thus, this commit prepares the Koji integration
by adding a way to accept a custom http transport which can be used to
customize the TLS settings.
Previously, Koji instance could be both logged-in and not logged-in.
This change disallows it: Now, the Koji instance is created by calling
koji.Login, so it must be always logged-in. This change should lead to more
robust code.
Services in Github Actions are cool but have some drawbacks:
1) We want to be able to use the container setup locally, therefore there's
run-koji-container script which does exactly the same setup as it's defined
in Github Actions. We don't want duplicities though.
2) In the near future, we will need more complicated setup - generating
certificates before a container is started. This is not possible with
the current Github Actions capabilities.
This commit removes the container setup from Github Actions and just reuses
the run-koji-container script in the GH Actions environment. This way we
have only one setup which is also more flexible.
run-koji-container has now two actions: start and stop:
- ./run-koji-container.sh start
- ./run-koji-container.sh stop
The start action starts all containers. When it exits, all containers are
started and running in the background. To stop and removethem, use the stop
action.
This change is needed so we're able to easily use this script also in the CI
environment.
The setup should be container engine agnostic. This changes allows this script
to be run on systems which prefer docker over podman (e.g. Github Actions).
- inside RunJob() there is a deferred function which will remove
the entire temporary directory in which images are created, including
the streamOptimized file
- inside testBootUsingVMware(), which wants to use this function,
there is already a deferred function which removes the converted
image
osbuild support returning metadata about each of the stages/assembler
runs. Parse the results from the rpm stage, which contains the header
fields from the installed RPMs, in particular the MD5 sum of the RPMs in
question. This information is needed to be passed as metadata to koji
when uploading images.
Signed-off-by: Tom Gundersen <teg@jklm.no>
In the same way `osbuild.Manifest` is the input to the osbuild API,
`osbuild.Result` is the output. Move it to the `osbuild` package where
it belongs.
This is not a functional change.
Signed-off-by: Tom Gundersen <teg@jklm.no>
#932 bumped the minimal Go version to 1.13. Prepare-source script had to be
rerun because of some changes in vendoring.
Unfortunately, while this PR was waiting to merged, #797 also touching
vendoring was merged. This "commit race condition" led to a failing check
on master branch because of bad vendoring.
This commit reruns the prepare-source script to fix the vendoring once again.
All our downstream platforms now support Go 1.13:
RHEL 8.2: golang-1.13.4
Fedora 31: golang-1.13.14
There's no reason anymore to stay on 1.12, therefore this commit bumps
the minimal required Go version to 1.13
This test is not run anywhere because it was surpassed by image tests with
azure boot type which perform more than just uploading a randomly generated
file to Azure. Let's delete dead code.
The tests are no longer run on Travis, therefore we don't need the special
setup to run them there.
This change should also fix#929 that is probably caused due to osbuild
executed in a weird way.
Fixes#929
Applying a `Name` tag to the snapshot makes the name of the image appear
in the snapshot listing in EC2's console. It also makes it easier to
remove a snapshot after deregistering the AMI.
Signed-off-by: Major Hayden <major@redhat.com>
Although the obuild-remote-worker@.service unit can be started, it can't
be enabled at boot time since the `Install` section is missing in the
unit file.
Add a small `[Install]` section with the same `WantedBy` as
osbuild-composr.service.
Fixes#924.
Signed-off-by: Major Hayden <major@redhat.com>
For reasons unknown, golangci-lint's default 1m0s timeout is *slightly*
too short for CI runs occasionally. Extend it to 5 minutes to ensure the
job always has enough time to run.
Signed-off-by: Major Hayden <major@redhat.com>
acf91a4 enabled fastestmirror but also calls `base.init_plugins()` to
initialize dnf plugins. This is not necessary and not what we want
conceptually.
Not necessary, because `fastestmirror` is a dnf built-in (it was a
plugin during yum-times [1]). The same patch sets the `fastestmirror`
option as well. Thus, this patch does not revert functionality.
Not what we want, because we're using dnf more as a library, explicitly
passing all options. Plugins depend on additional host configuration,
which we'd like to avoid pulling in. In particular, the
subscription-manager plugin tries reading certificates in `/etc/pki`,
which are not readable by the `osbuild-composer` user. This leads to
these errors in the journal:
[ERROR] dnf-json:54297:MainThread @logutil.py:194 -
[Errno 13] Permission denied: '/var/log/rhsm/rhsm.log' -
Further logging output will be written to stderr
[ERROR] dnf-json:54297:MainThread @identity.py:156 -
Reload of consumer identity cert /etc/pki/consumer/cert.pem
raised an exception with msg:
[Errno 13] Permission denied: '/etc/pki/consumer/key.pem'
These errors are not fatal, but could confuse people when inspecting
logs to find unrelated problems. This patch makes them disappear.
[1] https://fedoraproject.org/wiki/Yum_to_DNF_Cheatsheet
The vmdk images are meant to be imported into VMWare vCenter, not to be
booted directly.
Our image-info tests verifies that this work, they should be converted
into integration tests similar to the aws.sh script in the future.
Signed-off-by: Tom Gundersen <teg@jklm.no>
vCenter requires images to be uploaded as vmdk StreamOptimized. Lorax
always produced images on this format, so we should make sure to do the
same for our VMWare images.
Allow LocalTarget to request the images produced by osbuild be converted
to be streamOptimized before saving in composer, and hook the weldr API
up to enable this option for vmdk images.
Ideally this should simply be an option in osbuild, but that would
require some more work, which we will not manage in time for RHEL8.3.
Therefore do this minimal fix.
Note that that means the images produced by our manifests (including in
our image-test test cases) are not on the format that the weldr API
returns, so the tests we run on them would also, for now, need to
convert before uploading to vCenter.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This is not shipped in RHEL, so use the library directly to query the IP
address. This is a massive hack, but let us revisit this after the next
release.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We will automatically detect whether to use GA or Beta content, so we
can drop the override as 8.2 is GA so will get GA content.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We need the same RPMs to work equally well on a host running a beta
release (pulling beta content) as on a machine running GA (pulling GA
content). Detect this at run-time and point at the right repository.
Testing this is a bit hairy as we are building 8.3 images, but obviously
there is currently no 8.3 content at the GA URLs.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Change the translation from our internal structs to the structs used for
weldr serialization to drop account details. These must obviously be
passed in to configure an upload, but exposing them in the logs may be
surprising.
There is no notion of user accounts in the weldr API, and the state
should not be considered private. However, this is likely to take people
by surprise, so let us guard the secrets entrusted to us.
Fixes#907.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Improve the message returned by osbuild-composer when a user asks for
logs of a compose that is still running.
Signed-off-by: Major Hayden <major@redhat.com>
