Replace Job() and JobStatus() with typesafe versions, and introduce JobType()
for the rare instances where we don't know the type up front.
Additionally, catch a few more error cases:
- if OSBuildResult is nil, then we failed to invoke osbuild
- make sure the same JobResult handling is done for osbuild-koji, as for osbuild
This only extends the API, the backend can still only deal with composes of a single build.
I aimed to keep the API practically backwards compatible, i.e., no current consumer of it should notice the change. I hope I didn't mess that up.
fixup: image statuses
In addition to individual image status, have an
overall status that captures success or failure
of the compose as a whole.
This is not as fine grained, and only distinguishes
between "pending", "failure" and "success".
This captures other jobs than the image builds, which
is relevant for the koji composes, which consists also
of koji-init and koji-finalize, in addition to the build
jobs.
For now upload requests are required if and only if we are not
using koji. When using the koji integration the produced artifacts
are uploaded to koji only. In the future we may want to support
also uploading to the cloud providers.
Extend the compose endpoints to have minimal koji support.
This is intended to replace the current koji API so that it
can be consumed through api.openshift.com.
We may need to use several SSO providers, so extend our
configuration to allow that.
Based on PoC from Sanne:
```
package main
import (
"net/http"
"log"
"github.com/openshift-online/ocm-sdk-go/authentication"
"github.com/openshift-online/ocm-sdk-go/logging"
)
type H struct{}
func (h *H) ServeHTTP(w http.ResponseWriter, r *http.Request) {
log.Println("HURRAY")
}
func main() {
logBuilder := logging.NewGoLoggerBuilder()
logger, err := logBuilder.Build()
if err != nil {
panic(err)
}
aH, err := authentication.NewHandler().
KeysURL("https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/certs").
KeysURL("https://identity.api.openshift.com/auth/realms/rhoas/protocol/openid-connect/certs").
Logger(logger).Next(&H{}).Build()
if err != nil {
panic(err)
}
log.Fatal(http.ListenAndServe(":8080", aH))
}
```
It should be `1048576` (exactly 512 MiB), like it is for all other
distributions. It somehow got mingled in when the distribution was
forked off from 8.5/9.0 beta (1048676 to 1048576 strongly suggests
a sed command was involved, so we blame that).
When we compute the overall status of a koji compose, the individual
build jobs are checked. Currently, a job is considered a failure, if
a build job has output (`OSBuildOutput`) and the output's `Success`
field is `false`. But `OSBuildOutput` will be `nil` when osbuild
crashed or refused the manifest input. Therefore the job status is
a failure if `OSBuildOutput` is `nil`, since if osbuild was run,
and the run was successful we must have a non-`OSBuildOutput` field.
Add a special cases for the root user to the work-around for ssh keys in
OSTree commits.
See 93e54cd872 for the original,
equivalent change in RHEL 8.6.
The unification of the partition table also introduced uuids and
types in uuid form for partition tables in dos layout, sill used
on PPC64LE and s390x. The org.osbuild.sfdisk stage did work with
that but produced a `/boot` partition with the wrong type, which
grub2 refused to read from and thus prevented boot. Fix this by
removing uuids from the dos partition tables.
Reported-by: Jakub Rusz <jrusz@redhat.com>
Since the workers will use structured error messages
going forward, it is necessary to maintain backwards
compatability for there errors in composer. Tests have
been added to the various apis to ensure that each api
checks for both kinds of errors, old and new.
Implement the structured errors as defined by the worker client.
Every error for each of the job types now returns a structured
error with a reason and a specific error code. This will make
it possible to differentiate between 4xx errors and 5xx errors.
This commit refactors the way errors are implemented in the workers,
but maintains backwards compatability in composer by checking for
both kinds of errors.
Define worker errors to give more structured
error messages. The error api is:
id: VALIDATION_ERROR_NUMBER, reason: STRING, details: { issues: [{...}, {...}] }
The api was agreed upon with osbuild so that,
in future, osbuild errors will share the same
structure
Port all of the pipeline refactoring done to RHEL-90 to RHEL-86. Both
distros now use the same approach.
Regenerate all RHEL-8.6 and CentOS 8 image test cases.
[1] https://git.centos.org/centos/kickstarts/tree/master
Signed-off-by: Tomas Hozza <thozza@redhat.com>
RHSM configuration is now applied conditionally only on RHEL. The same
applies to the customization to subscribe the system on first boot.
The reason is that the CentOS `@core` package group does not contain
`subscription-manager`. Thus it is not installed on CentOS Stream by
default and also CentOS 8 image definitions don't apply any changes
to the RHSM configuration [1].
In addition, make sure to not install any subscription-manager
packages on CentOS Stream images.
Regenerate all CentOS 8 image test cases.
[1] https://git.centos.org/centos/kickstarts/tree/master
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Enhance the `osPipeline` to add necessary stages to the returned
pipeline, in case the image is RPM OSTree based. As a result, delete the
`ostreeTreePipeline` and replace its uses by `osPipeline`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Make the `osPipeline` self-contained in the sense, that no stages are
added to the returned pipeline outside of the function and the returned
pipeline is usable as returned.
Modify the `osPipeline` to add Kernel Cmdline, FSTab and bootloader
config stages to the pipeline if a valid partition table was passed to
the function. As the last one, the SELinux stage is appended to the
returned pipeline.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the EC2 SAP image specific configuration from `ec2SapPipelines`
to the EC2 SAP default image configurations data structure. As a
result, remove the `ec2SapPipelines` and `rhelEc2SapPipelines` entirely
and use `rhelEc2Pipelines` for all RHEL EC2 images.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the x86_64 specific configuration from `ec2X86_64BaseTreePipeline`
to x86_64-specific image configurations for EC2 / AMI images. As a
result, remove the `ec2X86_64BaseTreePipeline` entirely and replace it
with `osPipeline`.
Regenerate image test cases. While there are changed in the manifests,
the actual image configuration didn't change at all and thus the
`image-info` report was not changed.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move all hard-coded image configuration from the `ec2BaseTreePipeline`
function to the `ImageConfig` structure and update the respective EC2
images default configuration structure.
Update `osPipeline` and `ostreeTreePipeline` to handle all of the new
configuration values from `ImageConfig`.
Completely remove the `ec2BaseTreePipeline` and replace it with
`osPipeline`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the RHSM configuration settings to `ImageConfig` structure and use
when handling subscriptions in `osPipeline`, `ec2BaseTreePipeline` and
`ostreeTreePipeline` functions.
Regenerate image test cases. While there are changed in the manifests,
the actual image configuration didn't change at all and thus the
`image-info` report was not changed.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Introduce a new data structure `ImageConfig` holding the default OS
configuration applied when building an image. The structure can be used
to hold the default image configuration on the distribution level with
possible overrides defined on the image-type level.
As a starting point, move hard-coded default values and configuration
common for `osPipeline`, `ec2BaseTreePipeline` and `ostreeTreePipeline`
to the distribution and image-type default image configuration. This is
preparing the ground for merging all of these three pipeline functions
into `osPipeline`, which will produce the appropriate OS pipeline based
on the image-type configuration and the fact if it is rpmOstree or not.
Regenerate affected EC2 and AMI manifests. There is however no change in
the resulting image configuration and image-info report.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Bring the Tar stage implementation on par with the current osbuild
schema. Specifically add the 'format' and 'root-node' options to the
stage options structure.
Add stage options validation along with appropriate unit tests.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `Kernel` and `Network` members of the sysconfig stage options
structure were previously not declared as pointers. As a result, they
always appeared in the resulting JSON object, even though they were
empty. Use pointers to ensure that the members are omitted from the
resulting JSON object, if they were not defined.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add support for the new `org.osbuild.dnf-automatic.config` stage for
configuring DNF Automatic.
Add appropriate new unit tests for the stage implementation and modify
necessary existing unit tests.
Related to https://github.com/osbuild/osbuild/pull/936
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add support for the new `org.osbuild.yum.repos` stage for creating DNF /
YUM repository configuration files.
Add appropriate new unit tests for the stage implementation and modify
necessary existing unit tests.
Related to https://github.com/osbuild/osbuild/pull/932
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add support for the `PermitRootLogin` option in the `sshd.config` stage.
Valid values can be of type `bool` or `string`. Due to this reason, a
custom interface type is defined and a custom `UnmarshalJSON()` method is
defined for the `SshdConfigConfig` structure.
Modify unit tests to test the newly added option and test
(un)marhsalling of valid values of both types.
Related to https://github.com/osbuild/osbuild/pull/917
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The change between the 32s bucket and the 64s bucket is too drastic
for measuring the duration of depsolve jobs. At present, 90% of the
depsolve jobs have a duration inbetween 32s and 64s, making the 32s
bucket too sensitive and the 64s bucket not sensitive enough.
The service is started via systemd activation sockets.
The service serves http POST requests, the same json as before is
expected as the body of the request, and the same json as before is sent
as the response of the request.
Add a separate /boot partition to the default partition table used on
RHEL-9.0. The size is set to 500 MB, which is the value used by RHEL EC2
images. This change is needed to unify the default partitioning scheme
used by all RHEL-9.0 images [1].
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2022805
Signed-off-by: Tomas Hozza <thozza@redhat.com>
