The dump function will be used to search packages and as we are implementing third party repositories, adding the repo_id of the package will allow us to identify the package repository used.
Defaults according to https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config:
Defaults to a chain of credential providers to search for credentials in
environment variables, shared credential file, and EC2 Instance Roles.
If nothing is specified fall back to whatever instance role.
The longer running ostree tests (all except ostree.sh) benefit from
running on large runners because of their long runtime. Changing all to
large runners. Should not be a problem with our increased Openstack
quota.
The customizations struct contains different kinds of customizations,
such as users and filesystems. If both filesystems and users are
configured, the current code overwrites the user configuration.
Change this by unconditionally allocating the customizations struct
when creating the empty blueprint, and ammending it for each
customization we find.
This means that the customizations will be non-nil, also when it is
empty, which was not the case in the past. This should not change
the behavior of the code.
We (mistakenly) don't enforce a minimum size for /var,
so setting it to 1024 (1kB) causes the image build to fail.
CI does not expose this in a helpful way at the moment,
so this is a bit tricky to debug.
Also skip customizations for the AWS.S3 upload type. Not all the
image types with this upload type support filesystem customizations
and that's as expected. We could make a more fine-grained test in
the future, but testing with a coulpe of targets should be
sufficient.
- name mangling from centos to centos-stream was enabled also for
centos 9
- CS9 mock configs from mock-core-configs-36.3-1.fc35.noarch were added
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Tests need to be on the internal network to fetch the cert for epel for
EL9 (see provision.sh).
Use the large runner for all ostree-based tests since the regular one
seems to be running out of space.
Add rpmrepo repositories for testing.
Symlink centos-9 to centos-stream-9.
Add rpmrepo snapshot repositories to test and
test-case-generator repos.
Remove unused repositories from cs9 test repositories.
NOTE that the tests against the rpmmd fixure for running and waiting are
a bit of a kludge. It is impossible to mock up a running or waiting that
will allow an actual cancel, so the tests check for the internal server
error generated by using the fixure data.
Getting this error means that the API code did it's job and tried to
cancel the compose so that should be a good compromise.
Fix the cancel API to allow a waiting compose to be canceled.
This also fixes the cancel return code to be 400, the lorax-composer
behavior was a bug, and using 400 allows composer-cli to properly
display the error.
Move the `ostreePullStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewOstreePullStageInputs()`.
Delete `stage_inputs.go` from all distro definitions.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `bootISOMonoStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewBootISOMonoStagePipelineTreeInputs()`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `rpmStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewRpmStageSourceFilesInputs()`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move `OSBuildMetadataToRPMs()` and `PackageMetadataToSignature()`
functions from the `rpmmd` package to `osbuild2` package to prevent
import cycles while de-duplicating `rpmStageInputs()` function from
`stage_inputs.go` of distro definitions.
Rename `PackageMetadataToSignature()` to
`RPMPackageMetadataToSignature()`, since it takes specifically
`RPMPackageMetadata` type as an argument.
Adjust affected parts of code (unit tests, cloudapi, worker).
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `qemuStageInputs()` function duplicated in most
distro definitions to the `osbuild2` package as
`NewQemuStagePipelineFilesInputs()`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `xorrisofsStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewXorrisofsStagePipelineTreeInputs()`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `copyPipelineTreeInputs()` function duplicated in many
distro definitions to the `osbuild2` package as
`NewCopyStagePipelineTreeInputs()`.
This will prevent creating another copy of the code in rhel-84 for
the `gce` image.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `kernelVerStr()` function duplicated in many
distro definitions to the `rpmmd` package as
`GetVerStrFromPackageSpecListPanic()`.
I could not come up with a better name, sorry.
This will prevent creating another copy of the code in rhel-84 for
the `gce` image.
This change initially exposed a bug in the original implementation of
`kernelVerStr()`. Since on the first line, we allocate an empty structure
into `kernelPkg` variable, it can never be `nil` and the function never
panicked even if there was no `kernel` package in the PackageSpec list.
Fix all unit tests to provide valid arguments when calling `Manifest()`
method of image types.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
kernelVerStr fixup
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `kernelCmdlineStageOptions()` function duplicated in many
distro definitions to the `osbuild2` package as
`NewKernelCmdlineStageOptions()`.
This will prevent creating another copy of the code in rhel-84 for the
`gce` image.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
This should save us some money. 10 GB is the size of the underlying
RHEL 8.5 AMI so this should be the minimum.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Previous implementation of fsjobqueue is amazing but it has its drawbacks:
- dequeueing can be done only based on a job type
- it's limited to 100 jobs per a job type
As we soon want to be able to dequeue also by another criteria (job channel),
we need to refactor the queue.
The new implementation is more naive but also more flexible. It basically
works like the dbjobqueue - dequeueing goroutines listen for newly added
jobs. When that happens, a signal is sent to all of them and they all inspect
all pending jobs and dequeue ones that match their needs. Ones that don't find
a suitable job, are waiting for the next signal.
This is certainly slower implementation as every time a new job is added into
the queue, all dequeueing goroutines will have to iterate over all
pending jobs. I think that's fine because fsjobqueue is not recommended
to use for composer instances with heavy load.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
By default, the API will fetch only 30 open PRs. This is enough currently
as we have 35 open ones. Bump the limit to the maximum. 100 should be enough
for some time. 🤞
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
