The names of the pipelines that make up a Manifest for a job are
attached to the job data that is stored in the queue. The pipelines are
separated into Build and Payload.
This information is useful for identifying the build pipeline results
and metadata and for the order of the pipelines as they appeared in the
manifest.
Moved and adapted tests from osbuild1 to osbuild2.
Moved test data from osbuild1 to osbuild2.
Added conversion tests for v1 to v2.
Added full v2 result raw data from successful build.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Each image type now implements BuildPipelines(), which returns a list of
pipeline names that set up the build environment, and
PayloadPipelines(), which returns a list of pipeline names that create
the OS image (all non-build pipeline names).
Older distros that produce v1 manifests should call the distro Fallback
functions to return the common defaults.
A Fallback function for the Exports() method is also added and called by
older distros.
All image types that produce v2 manifests (distros after RHEL 8.4)
should include the information in the image type definition and should
not rely on fallbacks for default values.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Convert osbuild1.Result{} to osbuild2.Result{}.
For the Metadata objects, it assumes they are directly convertible: the
stage metadata structs have the same members.
The old conversion code from v2 to v1 is removed and the equivalent
conversion logic is moved to osbuild2:
- version detection based on stub
- custom unmarshaller that calls conversion function if v1 result is
detected
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Implementing writer for osbuild2 Result type.
Since Go maps don't have stable ordering, sorting by the pipeline name
provides stable output.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Reading stage metadata using osbuild's v2 result format.
For RPM stages we only want the core (OS) RPMs (not the build root
RPMs). Skip the build pipeline by name, but this should be handled
better since names are arbitrary.
Using type switch to convert metadata types instead of relying on the
type string of the stage result.
The rpmmd helper function isn't used anymore since that requires two
conversion passes (osbuild.StageMetadata -> rpmmd.RPM ->
cloudapi.PackageMetadata).
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Function renamed to better fit the argument element type
(StageMetadata).
Argument is a map to fit the pipeline metadata in the result object.
Signature function is made public to be reused in the cloud API
conversion.
Metadata test raw value updated to v2 result format.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
We will soon need to dequeue a job using its ID. This commit adds ability
to do that to the Jobqueue interface. As always, the fsjobqueue implementation
is slightly naive but it should fine for the usecases that it's designed for.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Previously, we deleted empty channels when a job was dequeued. This is
completely wrong because there still might be some clients waiting for
a job. This commit removes the cleanup and adds a regression test.
Note that this has the potential to leak memory if we ever use a lot of
job types. Currently, we have just handful of them, so this is fine.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Previously, we only needed the log directory to be writeable.
In newer versions of nginx, it also needs to create directories in
/var/lib/nginx, so we make that directory writeable and traversable as
well.
osbuild2 was imported twice, once with its own name and once aliased as
plain osbuild.
Use only the plain osbuild alias.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
- Start of RHEL 9.0 GA definition
- Initial distro name: rhel-90-ga
- rhel-90 alias remains for rhel-90-beta for now
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Test case like this was completely missing. A similar one is present in
the Weldr package, but this one is specific to testing blueprints and
thus easier to understand and extend.
Add a special case for the root user to the work-around for ssh
keys in OSTree commits. As a little refresher: OSTree does not
support having any content in home directories; we therefore
include a first-boot stage in the commit that will create the
ssh keys on first boot. However, until now we did not special
case the root user, which has a separate root directory (/root,
as a symlink to /var/roothome). This patch fixes this.
Workers now depsolve in parallel to image builds, so we can
again move depsolivng to the workers. This will help us deal
with increases in traffic as we currently only have one
depsolve handler per pod. It would also avoid any issues with
composer running out of disk space due to dnf metadata caches.
This reverts commit c65b1e9b26.
EC2 RHEL images keep the RHSM DNF plugins enabled, but this shouldn't be
added on CentOS since it doesn't have RHSM.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
A package set that only gets added for some distros.
Now that the rhel86 subpackage defines both RHEL 8.6 and CentOS 8, we
need to be able to separate out packages that are only available on one
of the two distros. Currently, this only includes "insights-client"
which is not available on CentOS.
The packages are added to the OS package sets conditionally based on the
distro name.
Removed old alias from RHEL 8.4.
To make aliasing distributions simpler, all distro-specific strings were
added to the local distribution type.
These were previously global constants of the distribution package and
alias alternatives were used conditionally. Now the two distributions
are predefined in global map.
CentOS exclusions:
- s390x arch
- ec2 image types: rhui client is not available
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
