The latest version of osbuild/blueprint changed some BP properties
to be 'omitempty', thus the expected API responses need to be adjusted
to account for this change.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the ${SUBJ} customization in Cloud API. Weldr API inherited it
automatically with the update of `osbuild/blueprint` to v1.6.0.
Adjust relevant test cases and unit tests to use this new customization.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Update the weldr API to work with the new depsolve API.
Update tests to match (adding repo_id).
Co-authored-by: Achilleas Koutsou <achilleas@koutsou.net>
This updates composer to use the updated API in images around the
seed handling for manifests, see images PR#1107 for details.
Note that this has no semantic changes yet. We could now simplfy
some things because images will auto-seed but that is for a followup.
Add support for RHSM customizations, which currently allow various
aspects of RHSM. Specifically enabling / disabling DNF plugins shipped
by subscription-manager package and setting a subset of options in the
rhsm.conf.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add support for RPM customizations, which currently allow to import RPM
GPG key from a file installed in the image. This is e.g. done for the
Azure RHUI image type.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
"rand.Seed has been deprecated since Go 1.20 and an alternative has been
available since Go 1.0: As of Go 1.20 there is no reason to call Seed
with a random value. Programs that call Seed with a known value to get a
specific sequence of results should use New(NewSource(seed)) to obtain a
local random generator."
Multiple blueprint fixes:
- Extend the blueprint service customizations to accept services to be
masked.
- The `storage-path` and `container-transport` fields were removed in
imagees 41.0 in order to simplify the way local storage containers are
handled.
Update the osbuild/images to the version which introduces "dot notation"
for distro release versions.
- Replace all uses of distroregistry by distrofactory.
- Delete local version of reporegistry and use the one from the
osbuild/images.
- Weldr: unify `createWeldrAPI()` and `createWeldrAPI2()` into a single
`createTestWeldrAPI()` function`.
- store/fixture: rework fixtures to allow overriding the host distro
name and host architecture name. A cleanup function to restore the
host distro and arch names is always part of the fixture struct.
- Delete `distro_mock` package, since it is no longer used.
- Bump the required version of osbuild to 98, because the OSCAP
customization is using the 'compress_results' stage option, which is
not available in older versions of osbuild.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This handles corrupt job json files by skipping them. They still exist,
and errors are logged, but the system keeps working.
If one or more of the json files in /var/lib/osbuild-composer/jobs/
becomes corrupt they can stop the osbuild-composer service from
starting, or stop commands like 'composer-cli compose status' from
working because they quit on the first error and miss any job that
aren't broken.
Commit 312d87c6d5fab7ffd085a303e27b8db41111c86e adds validation of the
checksums to the ImageType.Manifest call, so it requires a valid looking
checksum, otherwise it will fail with a 'ManifestCreationFailed' error
when running the unit tests.
Remove all the internal package that are now in the
github.com/osbuild/images package and vendor it.
A new function in internal/blueprint/ converts from an osbuild-composer
blueprint to an images blueprint. This is necessary for keeping the
blueprint implementation in both packages. In the future, the images
package will change the blueprint (and most likely rename it) and it
will only be part of the osbuild-composer internals and interface. The
Convert() function will be responsible for converting the blueprint into
the new configuration object.
Add a checksum as a hash of URL + Ref.
Use the parent ref instead of the image ref when it's set. This makes
the test distro always behave like ostree commit and container types
(image types that can use an ostree parent) and not raw image or
installers (that use ostree commits as a payload).
Modify the weldr API test with the expected error message.
Do not expose the content of the manifest statically and instead rely on
the public methods to retrieve source specifications dynamically.
Since the methods require iterating through the pipelines to collect
source specifications, we should avoid calling the function multiple
times when we can reuse the returned values.
The new test_distro's manifest produces a slightly different empty
manifest when serialized even without content. Cloud API and Koji tests
have been adapted to match.
Weldr tests have been updated in several ways:
- The test_distro content resolver is used to resolve manifest content
before serializing.
- The test scenarios in TestCompose have been named for easier
troubleshooting (easier to identify a failing test by name).
- Manifests that work with the secondary ostree repo (the "other") use
the appropriate URL and ref and create a secondary "other" serialized
manifest.
The weldr API's test flag for resolving ostree commits does not produce
the same, fixed hash every time but instead computes a sha256 from the
URL + ref, like we do in the test manifests.
Return manifest.Manifest from the Manifest() function without
serializing. The caller then has to call the manifest.Serialize()
function using the depsolved packages.
This moves towards changing the order of actions required to generate a
manifest. With this change, the manifest creation and depsolving can be
done independently, but this still requires instantiating the manifest
object twice (InstantiateManifest() is called in PackageSets() and
Manifest()), which we don't want to have to do.
When a blueprint containing package name globs is frozen it was failing
because it could not find the string in the dependency list. This fixes
it by replacing the package glob with all of the matching packages from
the dependency list.
This removes the setPkgEVRA function and adds a new expandBlueprintGlobs
function that handles the package name glob expansion, and replacement of
the version globs with the dependency EVRA.
Also includes testing for the new function.
Explicitly specify the AMI boot mode in AWS upload target in Weldr API
compose handler. The value is determined based on image type's boot
mode.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Convert some of the fields in the `RepoConfig` struct
to pointers. Since `RepoConfig` will be used to convert
custom repositories to an array of `osbuild.YumRepository`,
we need to ensure that fields that are not set explicitly
are not saved to the `/etc/yum.repos.d` repository files.
Update the internal RepoConfig object to
accept a slice of baseurls rather than a
single field. This change was needed to
align RepoConfig with the dnf spec [1].
Additionally, this change adds custom json
marshal and unmarshal functions to ensure
backwards compatibility with older workers.
Add json tags to the internal rpmmd config
since this is serialized in dnfjson.
Add unit tests to check the serialization
is okay.
[1] See dnf.config
This adds a new field `Warnings` to the `ComposeReply`
struct, allowing to send back any warnings (e.g. deprecation
notices) generated during the `checkOptions` step of the
manifest initialization.
See also https://github.com/osbuild/weldr-client/pull/99 which
handles the weldr-client side of things.
Signed-off-by: Irene Diez <idiez@redhat.com>
ioutil has been deprecated since go 1.16, this fixes all of the
deprecated functions we are using:
ioutil.ReadFile -> os.ReadFile
ioutil.ReadAll -> io.ReadAll
ioutil.WriteFile -> os.WriteFile
ioutil.TempFile -> os.CreateTemp
ioutil.TempDir -> os.MkdirTemp
All of the above are a simple name change, the function arguments and
results are exactly the same as before.
ioutil.ReadDir -> os.ReadDir
now returns a os.DirEntry but the IsDir and Name functions work the
same. The difference is that the FileInfo must be retrieved with the
Info() function which can also return an error.
These were identified by running:
golangci-lint run --build-tags=integration ./...
After introducing Go 1.18 to a project, it's required by law to convert at
least one method to a generic one.
Everyone hates IntToPtr, StringToPtr, BoolToPtr and Uint64ToPtr, so let's
convert them to the ultimate generic ToPtr one.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This is the first step to support embedding container images. Here
we add the `containers []container.Spec` argument to supply images
with resolved container specifications. For now all distros will
return an error in case a container is actually supplied since none
of them currently support embedding containers. NB: also no apis or
tools will actually resolve containers.
The test_distro Manifest, which is used in tests across multiple
packages, was using the old structure. Updated to the v2 structure and
adapted all tests.
The osbuild export is specific to the upload target and different
targets may require using a different export. While osbuild-composer
still does not support multiple exports for osbuild jobs, this prepares
the ground for such support in the future.
The backward compatibility with older implementations of the composer
and workers is kept on the JSON (Un)mashaling level, where the JSON
message is always a super-set of the old and new way of providing the
exports to osbuild job.
