Add a new class `container.Resolver` which can be used to resolve
multiple container images to their respective ids in parallel.
It should make it easy for all existing tools and api endpoints
to adpot container resultion.
Create a small only mock container registry to test `Client`.
Currently the registry is read-only and thus cannot be used
for upload tests but it can and will be used for container
resolution checks.
Add a new `Resolve` method to `Client` that will resolve its `Target`
to the corresponding manifest digest id and its corresponding iamge
identifier. The former can be used in the URL to fetch a specific
image from the registry via `<name>@<digest>` and the latter uniquely
identifies a container image via the hash of its configuration object.
This should stay the same across pulls and is also the id returned via
`podman pull` and `podman images`.
Since (most) container images are OS and architecture specific a tag
often points to a manifest list that contains all available options.
Therefore the resolve operation needs to choose the correct arch for
image. A new pair of getters `Set{Architecture,Variant}Choice` lets
the user control which architecture/variant is selected during the
resolution process.
Ensure that the `Client.AuthFilePath` points to a sane location,
which here means that the location is either accessible by the
current user or does not exist. This is because any other error
opening the auth file with lead to a overall failure when trying
to access container registries, even if the target resources is
public.
The reason we have to set it ourselves is that by default the
containers library looks in a sub-path of `XDG_RUNTIME_DIR` and if
that variable is not set it falls-back to `/run/containers/<uid>`.
Since `XDG_RUNTIME_DIR` is indeed not set for the composer process
started via systemd, it will fall-back, but it does not have access
to `/run/containers` and finding the authorization info for any
request will fail with "permission denied".
Add a setter so that we can set the `Client.AuthFilePath` to a
different location than the default one.
Instead of keeping an extra field in `Client`, we just use the
existing `sysCtx.DockerAuthConfig` structure. When the context
is later copied during the upload operation the credentials
will be copied as well. It also saves us from syncing the
credentials if we directly use said `sysCtx` for operations.
Instead of having an extra field, `TlsVerify`, on the `Client` and
then later setting the corresponding `SystemContext` options, use
the existing `SystemContext` field of `Client`. The corresponding
field is a tri-state: unset, true, false, which is represented as
a pointer to boolean in the `Client`'s new getter and setter. This
also inverts the boolean logic from verify TLS to skip TLS which
aligns very well with the corresponding fields in the upload target
struct.
In addition we properly capitalize some existing variables.
Koji API removed by the previous commit was the last user of osbuild-koji job.
Let's remove it since nothing uses it. This also removes all of the
compatibility code in Cloud API, see concerns below:
Compatibility concerns:
- the internal deployment was moved to a completely different composer
instance, thus there are no old jobs
- Fedora deployment is still unused in prod, thus we don't care about keeping
backward compatibility of the old jobs
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
We no longer use it, let's remove it. If you are wondering what to use instead,
use Cloud API. It supports everything that Koji API supported and more.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This test tested two things:
1) Invalid route - this is already covered by TestUnknownRoute
2) Invalid UUID in the compose status route - this is now covered by
TestComposeStatusInvalidUUID
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Fedora 34 is EOL, let's remove all traces of it, including:
- distro definition
- repositories (and test one)
- test manifests
- special package set rules
- hacks from the spec file
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
If the password is set to "" it will get hashed, allowing access to the
account in some circumstances. Console and ssh login don't appear to
work in practice, but su to the account from another user account is
possible.
This sets the empty password to nil which makes sure that it ends up as
a locked account.
This commit changes blueprint behavior to always store the hash of the
password for the 'customizations.user' accounts. Note that missing or
blank passwords are not hashed and should be dealt with at a lower
layer.
Resolves: rhbz#2107358
Make the v1StageResult.Metadata a simple json.RawMessage and perform the
RawMessage to StageMetadata conversion in the convertStage functions.
This lets us get rid of the custom v1StageResult Unmarshaller and the
v1RawStageResult, and makes the whole conversion process easier to
trace.
Types, parsing functions and helpers copied from osbuild1 to
osbuild2/v1result.go.
The metadata handling is simplified: osbuild1 stage metadata for the RPM
and ostree-commit stages is identical to the osbuild2 counterparts.
The test_distro Manifest, which is used in tests across multiple
packages, was using the old structure. Updated to the v2 structure and
adapted all tests.
This introduces an expiry date (default: 14 days from insert date) and
adjust the service-maintenance script to delete jobs that are older than
the expiration date.
We have three kinds of operating system trees, until we unify them to one,
hide them behind one interface. Use this to read the architecture from the
Tree rather than pass it in as a string to parent pipelines.
Also, make the filename parameter optional in a few places, there should be no
reason to set this rather than introspect it (except for backwards
compatibility).
Lastly, add another playground example sample to build a raw image.
For now all it does is represent the name of the runner and what requirements
it has of the build pipeline.
Move some package definitions from the runner package set to where it belongs.
This should have no practical effect, but ldconfig is used from
runners, so it is strictly speaking a requirement.
At the same time document the remaining TODO's in the build
pipeline.
The build pipeline requires the selinux packages only if we are going to be
labelling the files in the target OS. Otherwise, skip it.
manifest/build: pull in selinux-targeted unconditionally
This is unconditionally used by the build pipeline itself, until we make that
conditional, it needs to be installed.
edge-raw and edge-simplified-installer: only on 8.6+
ec2 and ec2-ha: available on all RHEL 8
ec2-sap: available on 8.4 and 8.6+ (no 8.5)
The ec2-sap image requires ansible, which in 8.4 is called `ansible` and
was replaced by `ansible-core` in 8.6.
This represents how our systems should integrate into their environment, typically using
some sort of agent, or commonly cloud-init.
In the future we could imagine this representing network configuration or any other kind
of configuration necessary to reach the environment as well.
For now EC2 and Azure is supported, and stub environments are
added to show the idea, but these are not implemented/used
yet.
The workload encapsulates what the user wants to run on top of the image. Everything
else we do abstracts away the OS, the hardware, the environment, and what is left is what
matters: the workload.
For now only the `Custom` payload is implemented which requires the user to name the
packages they want installed, the repositories to pull them from and what systemd
services to enable.
A few other stub workloads are added to show the idea, but these are not used.
The ideal is for the workload to have only the minimal number of configuration options.
Always include the tools for all the filesystem types in the partition table. There may be
usecases for having additional ones, for instance if the partition table is not known, but
this gives us a minimal baseline.
This includes dosfstools in images that have a vfat partition but did not include the tools.
Any package that is added by the pipeline definition should be in the base package set
and the user package set should only be for packages explicitly added by the user.
Any combination of implicitly added packages should depsolve, or it is a bug. However,
user provided packages can have conflicts which must be handled gracefully.
This change is not breaking, as that would be a bug (per the above) and it makes our
behaviour more predictable as any conflicts are caused by explicitly added packages.
Note that this changes the logic from the kernel package being depsolved twice to only
being depsolved in the base package set.
Pass PackageSets when initialising the Manifest, and read the chains back out.
This also fixes a bug where all repos were always used, rather than filtering per
package set.
Finally, this moves the 'chrony' inclusion from distro.go to the OSPipeline where
it belongs. In doing so the logic is changed slightly, where chrony is now
installed if NTP servers are configured (regardless of source), whereas in the
past it was included if the timezone was set in the blueprint (which made no sense).
If the kernel name is set, then the packaegSpecs must include the kernel package, ensure
this by including the kernel in the package list.
We currently include the kernel both in the userPackages and the base packages.
Including it only in the user packages does not work as the base could end up pulling it
in as well. However, it would be semantically more correct to include it only in the base
set, so if possible we should do that and drop it from the user packages (in a follow-up).
If the partition table includes logical volumes, the lvm2 package should be installed on
the target system.
Drop the corresponding logic from fedora/distro.go.
These objects describes the hardware an image runs on. Including
- architecture
- bootloader
- required firmware
Use the platform abstraction to move firmware packages out of the package set
definitions.
