Support for adding containers in non-ostree images. The reason we
don't support OSTree artefacts just yet is that the default storage
location for container is `/var/lib/containers/storage`. But for
OSTree images all content in `/var` is discarded, since that is
deployment specific data. We therefore need to store the containers
somewhere else, e.g. `/usr/share/containers/storage`, but then also
need to configure the system to find containers in that location.
osbuild only recently gained the corresponding stage to do so and
thus this will be done in a follow up.
Add a new `containers` section that can be used to request the
embedding of containers into images. The only requirement is
the source property to specify where to fetch the container from.
This suppports specifying the digest of the container or the tag.
In case none is given it defaults to the `latest` tag. The `Name`
field can be used to optionally specify a name to use inside the
image.
NB: currently no tools or apis support container resolution yet.
This follows in the next commits.
This is the first step to support embedding container images. Here
we add the `containers []container.Spec` argument to supply images
with resolved container specifications. For now all distros will
return an error in case a container is actually supplied since none
of them currently support embedding containers. NB: also no apis or
tools will actually resolve containers.
Add bindings for the `org.osbuild.skopeo` that can be used to copy
container images, accessed via the `org.osbuild.containers` input,
into images.
The constructor is designed with ease of use in mind and takes
the needed container inputs and the storage path option, i.e.
where to store the container in the images.
Add bindings for `org.osbuild.conainer` inputs which can be used to
supply containers to stages. Currently only fetching containers via
sources is supported.
Add a new class `container.Resolver` which can be used to resolve
multiple container images to their respective ids in parallel.
It should make it easy for all existing tools and api endpoints
to adpot container resultion.
Create a small only mock container registry to test `Client`.
Currently the registry is read-only and thus cannot be used
for upload tests but it can and will be used for container
resolution checks.
Add a new `Resolve` method to `Client` that will resolve its `Target`
to the corresponding manifest digest id and its corresponding iamge
identifier. The former can be used in the URL to fetch a specific
image from the registry via `<name>@<digest>` and the latter uniquely
identifies a container image via the hash of its configuration object.
This should stay the same across pulls and is also the id returned via
`podman pull` and `podman images`.
Since (most) container images are OS and architecture specific a tag
often points to a manifest list that contains all available options.
Therefore the resolve operation needs to choose the correct arch for
image. A new pair of getters `Set{Architecture,Variant}Choice` lets
the user control which architecture/variant is selected during the
resolution process.
Ensure that the `Client.AuthFilePath` points to a sane location,
which here means that the location is either accessible by the
current user or does not exist. This is because any other error
opening the auth file with lead to a overall failure when trying
to access container registries, even if the target resources is
public.
The reason we have to set it ourselves is that by default the
containers library looks in a sub-path of `XDG_RUNTIME_DIR` and if
that variable is not set it falls-back to `/run/containers/<uid>`.
Since `XDG_RUNTIME_DIR` is indeed not set for the composer process
started via systemd, it will fall-back, but it does not have access
to `/run/containers` and finding the authorization info for any
request will fail with "permission denied".
Add a setter so that we can set the `Client.AuthFilePath` to a
different location than the default one.
Instead of keeping an extra field in `Client`, we just use the
existing `sysCtx.DockerAuthConfig` structure. When the context
is later copied during the upload operation the credentials
will be copied as well. It also saves us from syncing the
credentials if we directly use said `sysCtx` for operations.
Instead of having an extra field, `TlsVerify`, on the `Client` and
then later setting the corresponding `SystemContext` options, use
the existing `SystemContext` field of `Client`. The corresponding
field is a tri-state: unset, true, false, which is represented as
a pointer to boolean in the `Client`'s new getter and setter. This
also inverts the boolean logic from verify TLS to skip TLS which
aligns very well with the corresponding fields in the upload target
struct.
In addition we properly capitalize some existing variables.
This prepares the usage of the `internal/container` from composer
directly, as opposed to the existing use in the worker. Said pkg
uses the `containers/image/v5`, which uses `proglottis/gpgme` and
the latter needs the gpgme C library. We therefore install it and
its dependencies.
The go package `proglottis/gpgme` a dependency of `containers/image/v5`
package uses `libgpgme`. In the near future `internal/container`, which
depends on `containers/image/v5`, will be used directly in composer and
thus we need to install the `gpgme` devel package and its build deps.
This test is compiling `gen-manifests` via `go run` and thus needs
to pick up build requirements for the source. Instead of manually
installing the go toolchain use the `dnf build-dep` command on the
spec file so we pick up current and future build dependencies.
Koji API removed by the previous commit was the last user of osbuild-koji job.
Let's remove it since nothing uses it. This also removes all of the
compatibility code in Cloud API, see concerns below:
Compatibility concerns:
- the internal deployment was moved to a completely different composer
instance, thus there are no old jobs
- Fedora deployment is still unused in prod, thus we don't care about keeping
backward compatibility of the old jobs
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
We no longer use it, let's remove it. If you are wondering what to use instead,
use Cloud API. It supports everything that Koji API supported and more.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This test tested two things:
1) Invalid route - this is already covered by TestUnknownRoute
2) Invalid UUID in the compose status route - this is now covered by
TestComposeStatusInvalidUUID
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Fedora 34 is EOL, let's remove all traces of it, including:
- distro definition
- repositories (and test one)
- test manifests
- special package set rules
- hacks from the spec file
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
It was failing on:
vendor/github.com/proglottis/gpgme/data.go:4:11: fatal error: gpgme.h: No such file or directory
Let's install this package before running the check, I verified that this
fix works locally.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
We want to be able to safely gather any artifacts without worrying about
any possible secrets leaking. Every artifacts that we want to upload
will now have to be placed in /tmp/artifacts which will then be uploaded
to S3 by the executor and link to the artifacts will be provided in the
logs. Only people with access to our AWS account can see them.
If the password is set to "" it will get hashed, allowing access to the
account in some circumstances. Console and ssh login don't appear to
work in practice, but su to the account from another user account is
possible.
This sets the empty password to nil which makes sure that it ends up as
a locked account.
This commit changes blueprint behavior to always store the hash of the
password for the 'customizations.user' accounts. Note that missing or
blank passwords are not hashed and should be dealt with at a lower
layer.
Resolves: rhbz#2107358
The CentOS Linux 8 packages have been removed from the mirrors.
CentOS 8 is replaced by CentOS Stream 8. [0]
Keep the centos-8.json symlinked to centos-stream-8.json because
composer's host distro detection picks up CS8 as centos-8.
[0] https://www.centos.org/news-and-events/convert-to-stream-8/
Additional packages are required to build the
docker worker. This fix updates the builder
container to install the required libraries
and then create the worker binary.
