The partition table is modified `CreatePartitionTable`, which is
not a problem for the table itself since it is currently passed
by value. However, all shallow copies share the same file system
pointers and `CreatePartitionTable` will modify those as well.
As there is a data race where two concurrent thread modify the
content of the Fileystem object at the same time before writing
the uuids out to the manifest via the stage options and thus the
resulting manifest would be broken.
Therefore we use the new `Clone` methods to make a dee@ copy of
the `PartitionTable` object in the `CreatePartitionTable` method;
This will allow us to pass the `basePartitionTable` object by val
and also return the resulting `PartitionTable` as a pointer.
The `PartitionTable` object contains an array of `Partitions`, which
is shared among all shallow copies of `PartitionTable` instances, as
when it is used via a value receiver in method calls. The individual
objects are thus not fully isolated since modifications of array
elements, or in some cases modifications to the array itself, are
shared and visible between all shallow `PartitionTable` copies. Thus
use a pointer receiver for all methods of `PartitionTable` to make
it explicit that modifications will affect other objects.
What this function is actually doing is to create a filesystem,
together with a new partition; for LVM that filesystem could
also be created on a new logical volume though.
The previous commit made sure that `PartitionTable.RootPartition`
is indeed returning the pointer partition object inside the array
of the `PartitionTable`. Thus changes to the returned object will
now directly affect the object and thus there is no need to call
`PartitionTable.updateRootPartition` anymore.
When we iterate over the partitions to find the root partition,
we are getting the partition by value so even if we return a
pointer to that then, it is actually a pointer duplicated object.
Return the pointer to the actual partition object instead.
In various places we are interested in the root and boot file-
systems. Currently those were accessed via by retrieving the
partition that contain them and the accessing the filesystem
member. Add accessors to `PartitionTable` that directly return
the needed filesystem. This will help if the file system is
stored inside a container like LVM or LUKS instead of directly
on a partition.
Defaults according to https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config:
Defaults to a chain of credential providers to search for credentials in
environment variables, shared credential file, and EC2 Instance Roles.
If nothing is specified fall back to whatever instance role.
The customizations struct contains different kinds of customizations,
such as users and filesystems. If both filesystems and users are
configured, the current code overwrites the user configuration.
Change this by unconditionally allocating the customizations struct
when creating the empty blueprint, and ammending it for each
customization we find.
This means that the customizations will be non-nil, also when it is
empty, which was not the case in the past. This should not change
the behavior of the code.
NOTE that the tests against the rpmmd fixure for running and waiting are
a bit of a kludge. It is impossible to mock up a running or waiting that
will allow an actual cancel, so the tests check for the internal server
error generated by using the fixure data.
Getting this error means that the API code did it's job and tried to
cancel the compose so that should be a good compromise.
Fix the cancel API to allow a waiting compose to be canceled.
This also fixes the cancel return code to be 400, the lorax-composer
behavior was a bug, and using 400 allows composer-cli to properly
display the error.
Move the `ostreePullStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewOstreePullStageInputs()`.
Delete `stage_inputs.go` from all distro definitions.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `bootISOMonoStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewBootISOMonoStagePipelineTreeInputs()`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `rpmStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewRpmStageSourceFilesInputs()`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move `OSBuildMetadataToRPMs()` and `PackageMetadataToSignature()`
functions from the `rpmmd` package to `osbuild2` package to prevent
import cycles while de-duplicating `rpmStageInputs()` function from
`stage_inputs.go` of distro definitions.
Rename `PackageMetadataToSignature()` to
`RPMPackageMetadataToSignature()`, since it takes specifically
`RPMPackageMetadata` type as an argument.
Adjust affected parts of code (unit tests, cloudapi, worker).
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `qemuStageInputs()` function duplicated in most
distro definitions to the `osbuild2` package as
`NewQemuStagePipelineFilesInputs()`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `xorrisofsStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewXorrisofsStagePipelineTreeInputs()`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `copyPipelineTreeInputs()` function duplicated in many
distro definitions to the `osbuild2` package as
`NewCopyStagePipelineTreeInputs()`.
This will prevent creating another copy of the code in rhel-84 for
the `gce` image.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `kernelVerStr()` function duplicated in many
distro definitions to the `rpmmd` package as
`GetVerStrFromPackageSpecListPanic()`.
I could not come up with a better name, sorry.
This will prevent creating another copy of the code in rhel-84 for
the `gce` image.
This change initially exposed a bug in the original implementation of
`kernelVerStr()`. Since on the first line, we allocate an empty structure
into `kernelPkg` variable, it can never be `nil` and the function never
panicked even if there was no `kernel` package in the PackageSpec list.
Fix all unit tests to provide valid arguments when calling `Manifest()`
method of image types.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
kernelVerStr fixup
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `kernelCmdlineStageOptions()` function duplicated in many
distro definitions to the `osbuild2` package as
`NewKernelCmdlineStageOptions()`.
This will prevent creating another copy of the code in rhel-84 for the
`gce` image.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previous implementation of fsjobqueue is amazing but it has its drawbacks:
- dequeueing can be done only based on a job type
- it's limited to 100 jobs per a job type
As we soon want to be able to dequeue also by another criteria (job channel),
we need to refactor the queue.
The new implementation is more naive but also more flexible. It basically
works like the dbjobqueue - dequeueing goroutines listen for newly added
jobs. When that happens, a signal is sent to all of them and they all inspect
all pending jobs and dequeue ones that match their needs. Ones that don't find
a suitable job, are waiting for the next signal.
This is certainly slower implementation as every time a new job is added into
the queue, all dequeueing goroutines will have to iterate over all
pending jobs. I think that's fine because fsjobqueue is not recommended
to use for composer instances with heavy load.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
In the greenboot 0.13 release we updated packaging due to the increase
in new tests and it not making sense to have packaging so granular.
Signed-off-by: Peter Robinson <pbrobinson@gmail.com>
When the server is restarted the blueprint changes, which are only held
in memory, are lost. This checks for missing changes and returns an
error.
The test is also adjusted for the new error.
Related: rhbz#1922845
The blueprint name should never be empty, as it can cause other problems
like with the blueprints list results. Return an error if one is pushed
to the store, either as a blueprint commit or as a blueprint workspace.
Also adjusts the new test for the new error.
Related: rhbz#1922845
There is a problem with blueprint changes, once the server is restarted
the previous changes are all lost because they are not serialized to
disk.
This adds test fixture support so that new tests can be added before
fixing the problem. It adds store.FixtureOldChanges with blueprints
changes and empty blueprints.
Related: rhbz#1922845
- Any repository without package_sets is added to the general `Repos`
field of the DepsolveJob, just like before.
- Repositories with package_sets are added to the `PackageSetsRepos`
map, indexed by the package set names.
- Repositories defined in the customizations as `PayloadRepositories`
are considered to be associated only with the `PayloadPackageSets`
names from the image type and are added to the `PackageSetsRepos`
under the payload sets.
The repository collection and conversion of repository structs (from
Repository to RepoConfig) has been moved to a separate function.
Tags is an array of strings that associates repositories with package
sets. A repository tagged with a package set name will be used only for
the named package sets.
We have two fields, `Repos` and `PackageSets`. Renaming
`PackageSetsRepositories` to `PackageSetsRepos` for consistency.
The struct is for internal use only so the rename has no impact as long
as the serialised name is the same (json tag).
Also it's shorter.
Added docstring to the struct that explains the arguments in the same
way as they are described for the `depsolve()` function.
Changing the name of the argument in the internal `depsolve()` function
for the same reasons.
When deploying an ostree commit, specify a remote, currently hard-
coded to `rhel-edge`, so that updates work automatically, if they
are served from the same location as the initial commit is pulled
from.
NB: now that the remote is specified in the raw image, remove the
corresponding bits form the tests.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
