This was added in osbuild: https://github.com/osbuild/osbuild/pull/875
Introduce the same option in composer and make it optional by specifying
it as a pointer to bool value. It would work the same even if it was
there every time, but as it should be an edge case, don't use it
everywhere. Also osbuild doesn't require it to be present, so it seems
like the right thing to do.
The `compat-sap-c++-9` package no longer exists on RHEL-9. It has been
removed from the RHEL-9.0 Beta EC2-SAP image's package set, but then got
readded as part of adding the RHEL-9.0 distro and merging package sets
code with RHEL-8.6.
This issue have been found by the test case which tests manifests from
mage test cases, including the package set depsolving.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The commit adding distro-specific package set to RHEL-8.6 [1] introduced
a bug in the `image-installer` image type due to a typo. The
`insights-client` package have been removed from the
`bareMetalPackageSet` and moved to `distroSpecificPackageSet`. However
the `distroBuildPackageSet` got appended to `bareMetalPackageSet`,
instead of `distroSpecificPackageSet`, which caused `insights-client` to
be removed from the image's package set.
This issue have been found by the test case which tests manifests from
image test cases, including the package set depsolving.
[1] ed0cb5ea24#
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Generated image test case manifests for all supported distros, arches and
image-types are being tested as part of distro unit tests. However due
to time constrains, the unit test does not depsolve the image's default
package sets and thus does not check if they changed in the internal
osbuild-composer's representation, compared to the generated image test
case.
Extend the `TestDistro_Manifest()` function used by the unit test to
allow depsolving image's package sets.
Introduce a new test case binary `osbuild-composer-manifest-tests`
allowing to check the manifests generated by composer for all supported
combinations of images against generated manifests, including depsolving
image's default package sets.
Introduce a new CI test case `manifest_tests.sh` executing the
`osbuild-composer-manifest-tests` binary and testing all existing image
test cases. Run it in CI on RHEL-9 runner.
Modify SPEC file to ship the newly added test case.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
We actually need 2 * 16 connections at minimum (one worker waits for two
jobs). Let's bump the maximum connection count even moar.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
By default, pgxpool.Pool has 4 connections (or number of cpus if higher).
Currently, we have 3 replicas, that means max 3*4=12 DB connections.
The dequeue operation is actually blocking - when a worker is waiting for
a job, one connection is blocked. My theory is that with 16 workers, we just
don't have enough connections that causes all sorts of weird slowdowns.
This commit bumps the number of connection from one replica to 10, therefore
we should be at 30 connections in total.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The stage now allows for customizations specific to YUM or DNF. So far
it is just an alias to the same definition, meaning that composer can
use exactly the same structures for both.
Ref: https://github.com/osbuild/osbuild/pull/876
Before dereferencing the method receiver in Write(), check if the object
is nil and return early.
Fixes#2002
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
The CDN repositories require client side SSL certificates, it would have
to be specified in httpd configuration which is technically possible but
in practise it is more troubles than benefits.
It is similar case for the RH internal repositories, there is a new
issue every time we enable the test with them.
Stop wasting dev&qe time and disable the test for anything but rpmrepo
snapshots. It is better to have it running at least for few distros than
to have it disabled globally.
Also display logs for the HTTP proxy to see if there were any errors.
Rework the stage options data validation to be done in constructor
methods, instead of when being marshalled to JSON.
Add validation of values passed to constructor methods for modprobe
command structures.
Add validation of the configuration filename based on stage schema.
Related to issue #1785.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add support for the 'install' modprobe command in the modprobe osbuild
stage implementation.
Extend unit tests to verify marshalling the stage options into JSON.
Related to https://github.com/osbuild/osbuild/pull/867.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Use newly added RHUI-4 repo snapshots for all RHEL-9 EC2* image test
cases. This includes RHEL-9.0 and RHEL-9.0-Beta images. The removal
of installed repos on EC2-HA and EC2-SAP images is expected, as RHUI
client RPMs for these variants are empty for 9.0 Beta. This should
change for GA once there are updated RHUI clients RPMs available.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Running the job in this case is basically undefined, so let's just skip it
in order to not break anything.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Mistakenly removed in 4577ac0717. Composer
itself does the authentication, not the gateway, therefore we do need
the auth exclude.
Added a comment to explain why it's attached to the api socket and not a
separate listener.
Two new tests, one for OSBuild and one for Koji jobs. Both follow the
same flow:
- Enqueue a job that doesn't specify PipelineNames (oldJob)
- Enqueue a job that does specify PipelineNames (newJob)
- Read the job data for the oldJob and check that the default
PipelineNames were added
- Read the job data for the newJob and check that it's unchanged
- Finish oldJob and add results without specifying PipelineNames
- Finish newJob and add results with PipelineNames
- Read the oldJob result and check that the default PipelineNames were
added
- Read the newJob result and check that it's unchanged
This is meant to test several scenarios that can occur when upgrading
the service:
1. The existing jobqueue has old jobs in it that were queued before the
PipelineNames were part of the data structure. The worker should be
able to read these and add the fallback data.
2. New jobs are added while old jobs still exist in the queue and the
worker can read both types.
3. The existing jobqueue has old finished jobs in it that were finished
and had results written before the PipelineNames were part of the
result data structure. The worker should be able to read these and
add the fallback data.
4. New jobs are finished and results are written while old jobs still
exist in the queue and the worker can read both result types.
When worker reading data into the job and result types, check if the
PipelineNames are populated and, if not, add the fallback values from
distro.
This makes it simpler to work with job queues that contain old data
before the introduction of the PipelineNames. In any situation where the
job or result data are read, the reader can assume that the
PipelineNames are non-nil and that if they belong to an old job, they
have the fallback names.
This assumption goes hand-in-hand with the change in v2 format for
osbuild results, since old jobs that don't have PipelineNames set *must*
contain results in the old format for the names to be valid.
