This pulls out the OS pipeline, without changing the parameters. The dependency
between the OS pipeline and build pipeline, is now explicit, rather than by name.
RPM Spec
--------
Remove all Go dependecies
Add Start and End marker comments for bundling information
Add '-k' to goprep to preserve the vendor directory
tools
-----
Add script to update the RPM spec file to generate the indication lines
based on vendor/modules.txt
Packit
------
Run the new script as a post-upstream-clone hook
Makefile
--------
Run the new script on the generated spec file before generating the RPM
mockbuild.sh
------------
Run the new script before creating the RPM
We only need one runner and it should use the internal network for
access to all repositories.
Set a rule so it doesn't run on 'main' (makes no sense).
Set git depth to 500:
We need a long history in order to find the merge-base between the PR
and 'main'. It's unclear whether there's a straightforward way to find
the depth of the PR to limit the clone depth accurately. 500 should be
enough for any PR (I'd hate to see a PR that makes this statement
false).
The script runs the gen-manifests command first on the PR head and then
on the merge-base with the PR's base branch (typically 'main') and
checks for any differences. It creates a review comment on the PR on
GitHub if any changes are detected.
The message is posted as a simple COMMENT type review to inform the
author and reviewers that changes exist.
The script doesn't fail if there's a diff. CI shouldn't fail if changes
are detected since they can be intentional. The job fails if something
goes wrong with the script execution (manifest generation, comment
posting, etc).
The script exits immediately if not run from a PR.
The gen-manifests run is silenced with `> /dev/null`. In the future,
this should be handled by flags to the command itself to control the
output format noisiness.
The gen-manifests command is run 50 workers. Testing with 100 seemed to
make the execution stall, likely because of the resources on the worker.
We can experiment with this value more in the future.
Add a new option to `GenImagePrepareStages`, which is used by all
modern pipelines for partitioning, to optionally use the `sgdisk`
partitioning tool via `org.osbuild.sgdisk`.
THe `rpmmd.RepoConfig` configuration supports setting "package sets"
for each repository, which allows the associate the individual repos
to specific package sets. Add a new `package_set` option to the
repo configuration of the compose request so that this feature can
be used.
The worker server API handler `UploadJobArtifact()` was previously
silently discarding artifacts uploaded by the worker, if the server was
configured to not accept artifacts.
Change the behavior to return HTTP error "Bad Request" (`400`) to the
worker, in case it tries to upload artifact to the server, but the
server is configured to not accept any artifacts.
Add a new unit test testing the new behavior and adjust existing unit
tests, which were relying on the artifact being previously silently
discarded.
When the Koji target support was added to the osbuild job, based on the
osbuild-koji job, the meaning of target option values got messed up.
The side effect of the issue is that when Koji composes are
submitted via Cloud API the resulting image is currently always uploaded
back to the worker server.
`OsBuildKoji` job
-----------------
- `OSBuildKojiJob.ImageName` is set to the filename of the image as
exported by osbuild.
- `OSBuildKojiJob.KojiFilename` is set to the desired filename which
should be used when uploading the image to Koji.
`OsBuild` job + `KojiTargetOptions` before
------------------------------------------
- `OSBuildJob.ImageName` is set to the filename of the image as exported
by osbuild. This is done only by the Cloud API code for Koji composes.
Cloud API does not set this for regular composes and any other target.
The variable is set in common case only by Weldr API code with the
same meaning and it is used by the `OsBuild` job implementation as an
indication that the image should be uploaded back to the worker server.
- `Target.ImageName` is not set at all. Other targets use it for the
desired filename which should be used when uploading the image to the
target environment.
- `KojiTargetOptions.Filename` is set to the desired filename which
should be used when uploading the image to Koji. All other target
types use `Filename` variable in their options for the filename of the
image as exported by osbuild.
`OsBuild` job + `KojiTargetOptions` after
-----------------------------------------
- `OSBuildJob.ImageName` is still set to the filename of the image as
exported by osbuild. This is kept for a backward compatibility of new
composer with older workers.
- `Target.ImageName` is set to the desired filename which should be used
when uploading the image to Koji.
- `KojiTargetOptions.Filename` is set to the filename of the image as
exported by osbuild.
This change is backward incompatible, meaning that old worker won't be
able to handle Koji compose requests submitted via Cloud API using a new
composer and also a new worker won't be able to handle Koji compose
requests submitted by a new composer. This is intentional, because after
discussion with Ondrej Budai, the Cloud API Koji integration is
currently not used anywhere in production.
After a depsolve, each package inherits the `IgnoreSSL` value from its
repository configuration.
This information is not yet used. It will be used to expose this
information to osbuild's org.osbuild.curl stage.
The test data is updated to match the new behaviour:
The test repository config specifies `IgnoreSSL=true` and the packages
in the response inherit the value.
The internal repository configuration (RepoConfig) supports IgnoreSSL
which, when set to `true`, will run a depsolve job with the dnf repo
parameter `sslverify` set to `false`.
The serialisable repo object (repository) did not support reading this,
so it was impossible to set in global repo configs (from
/usr/share/osbuild-composer/repositories and
/etc/osbuild-composer/repositories).
It was, however, possible to set it through the weldr API when adding a
new source.
Makes curl skip the verification step for secure connections and proceed
without checking.
The default (empty) value is 'false'.
osbuild counterpart: c8073b5836
The BaseSolver is an object which gets constructed when the worker
starts, and the subscriptions attached to it expire after about 3
days. By refreshing the subscriptions each time a new Solver is created,
valid subscriptions are used.
