Bump the `rhel-9` and `rhel-10` distro aliases to the next
in-development releases, because we don't plan to land any new releases
in 9.6 or 10.0.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This commit updates osbuild-composer for the new API in images
for the `reporegistry.New()`. The main incompatible change is
that the `/repositories` part is not longer automatically added
inside the library so we need to add it on the call-site.
This needs https://github.com/osbuild/images/pull/1179
This commit changes the spec file so that most of the repositories
are taken from the `images` library. See images PR#1112 for details.
Note that we still need the -no-auth-keys and the centos-stream
symlinks.
The alias in the built-in config is meant for on-prem and needs to be
set to the current in-development minor RHEL version.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Newer versions of the go compiler (1.24 in this case) fail when running
go test during a mock rebuild of the srpm created by 'make srpm' on
Fedora 42.
Even though we currently don't support go1.24, fix these so they don't
become an issue when we do.
osbuild/images added an error type that's returned when the reporegistry
loader doesn't find any repository configurations to load [1]. This
lets callers decide whether to stop or continue execution based on
whether repository configurations are required.
A new top-level configuration option is added for osbuild-composer that
makes it possible to start the service without having static rpm
repositories configured. This is useful in certain (SaaS) modes where
build requests specify their own repository configurations.
The `reporegistry.New()` has been enhanced to return an error, in case
there were no repositories loaded. This was to fix the situation in many
unit tests, which were previously not loading any repositories and
silently not running any tests.
This however broke our SaaS deployment, where we actually do not
configure any repositories on the filesystem. As a result,
osbuild-composer started to fail on it.
Workaround this situation in osbuild-composer by reverting to the old
behavior by loading the repo configs separately and then using the
loaded repos (which could be empty map) to initialize the RepoRegistry.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Composer can load configuration values defined as map from ENV.
Previously, when loading the configuration from ENV, the whole map would
get overridden, not just values defined in the ENV. This is however not
intended and not consistent with how loading configuration from file
works.
Adjust the configuration loading from ENV and adjust the unit test
accordingly.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
New releases are landing in 9.5 already, so the on-prem version should
reflect that. The service can and does override this using a
configuration.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add new configuration option `distro_aliases`, which is a map of
strings, allowing to specify distro name alias for supported
distributions.
Define aliases for RHEL major versions without the minor version
specified.
For now, the distro aliases map is not used by any API
implementation and it is ignored.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Previously, the old dnfjson cache dirs for unsupported distributions
were deleted in the osbuild-composer binary on startup. This is no
longer possible, since the supported distros are determined by loading
available repositories. Loading repositories happens in the Weldr API
constructor. Move the cleanup code there.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Update the osbuild/images to the version which introduces "dot notation"
for distro release versions.
- Replace all uses of distroregistry by distrofactory.
- Delete local version of reporegistry and use the one from the
osbuild/images.
- Weldr: unify `createWeldrAPI()` and `createWeldrAPI2()` into a single
`createTestWeldrAPI()` function`.
- store/fixture: rework fixtures to allow overriding the host distro
name and host architecture name. A cleanup function to restore the
host distro and arch names is always part of the fixture struct.
- Delete `distro_mock` package, since it is no longer used.
- Bump the required version of osbuild to 98, because the OSCAP
customization is using the 'compress_results' stage option, which is
not available in older versions of osbuild.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a local socket for communicating with the cloudapi. It is started by
osbuild-composer.socket and is located at /run/cloudapi/api.socket
cloudapi requests can be passed to it using curl like this:
curl -k --unix-socket /run/cloudapi/api.socket --header 'Content-Type: application/json' \
--data request.json http://localhost/api/image-builder-composer/v2/compose
A simple request.json looks like this:
{
"distribution": "fedora-38",
"image_request":
{
"architecture": "x86_64",
"image_type": "guest-image",
"upload_options": {},
"repositories": [
{
"name": "fedora",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=fedora-38&arch=x86_64",
"check_gpg": false
},
{
"name": "updates",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=updates-released-f38&arch=x86_64",
"check_gpg": false
}
]
}
}
UBI and the oldest support Fedora (37) now all have go 1.19, so we are
cleared to switch.
gofmt now reformats comments in certain cases, so that explains the formatting
changes in this commit.
See https://go.dev/doc/go1.19#go-doc
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Remove all the internal package that are now in the
github.com/osbuild/images package and vendor it.
A new function in internal/blueprint/ converts from an osbuild-composer
blueprint to an images blueprint. This is necessary for keeping the
blueprint implementation in both packages. In the future, the images
package will change the blueprint (and most likely rename it) and it
will only be part of the osbuild-composer internals and interface. The
Convert() function will be responsible for converting the blueprint into
the new configuration object.
This adds a function, CleanupOldCacheDirs, that checks the dirs under
/var/cache/osbuild-composer/rpmmd/ and removes files and directories
that don't match the current list of supported distros.
This will clean up the cache from old releases as the are retired, and
will also cleanup the old top level cache directory structure after an
upgrade.
NOTE: This function does not return errors, any real problems it
encounters will also be caught by the cache initialization code and
handled there.
ioutil has been deprecated since go 1.16, this fixes all of the
deprecated functions we are using:
ioutil.ReadFile -> os.ReadFile
ioutil.ReadAll -> io.ReadAll
ioutil.WriteFile -> os.WriteFile
ioutil.TempFile -> os.CreateTemp
ioutil.TempDir -> os.MkdirTemp
All of the above are a simple name change, the function arguments and
results are exactly the same as before.
ioutil.ReadDir -> os.ReadDir
now returns a os.DirEntry but the IsDir and Name functions work the
same. The difference is that the FileInfo must be retrieved with the
Info() function which can also return an error.
These were identified by running:
golangci-lint run --build-tags=integration ./...
This satisfies the linter complaint about potential Slowloris attack
where headers are read slowly in an attempt to DoS the server.
The uses of ListenAndServe are only for testing purposes and are not run
in the production server so ignore the lint errors in
osbuild-mock-openid-provider.
For each of the supported distros start a goroutine to depsolve
'filesystem' which will preload the metadata making subsequent responses
faster.
This is safe to do without limits because we only supposed a limited
number of distros, and without additional locking because this is the
the same as hitting the API with multiple depsolve requests at the same
time.
We no longer use it, let's remove it. If you are wondering what to use instead,
use Cloud API. It supports everything that Koji API supported and more.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
All calls to rpmmd.Depsolve() are now replaced with the equivalent call
to solver.Depsolve() (or dnfjson.Depsolve() for one-off calls).
Attached an unconfigured dnfjson.BaseSolver to all APIs and server
configurations where rpmmd.RPMMD used to be. This BaseSolver instance
loads the repository credentials from the system and carries the cache
directory, much like the RPMMD field used to do. The BaseSolver is used
to create an initialised (configured) solver with the platform variables
(module platform ID, release ver, and arch) before running a Depsolve()
or FetchMetadata() using the NewWithConfig() method.
The FillDependencies() call in the modulesInfoHandler() of the weldr API
has been replaced by a direct call to the Depsolve() function. This
rpmmd function was only used here. Replacing the rpmmd.Depsolve() call
in rpmmd.FillDependencies() with dnfjson.Depsolve() would have created
an import cycle. The FillDependencies() function could have been moved
to dnfjson, but since it's only used in one place, moving the one-line
function body into the caller is ok.
For testing:
The mock-dnf-json is compiled to a temporary directory during test
initialisation and used for each Depsolve() or FetchMetadata() call.
The weldr API tests now use the mock dnfjson. Each rpmmd_mock.Fixture
now also has a dnfjson_mock.ResponseGenerator.
All API calls in the tests use the proper functions from dnfjson and
only the dnf-json script is mocked. Because of this, some of the
expected results in responses_test had to be changed to match correct
behaviour:
- The "builds" array of each package in the result of a module or
project list is now sorted by version number (ascending) because we
sort the package list in the result of dnfjson by NVR.
- 'check_gpg: true' is added to the expected response of the depsolve
test. The repository configs in the test weldr API specify 'CheckGPG:
True', but the mock responses returned it as false, so the expected
result didn't need to include it. Since now we're using the actual
dnfjson code to convert the mock response to the internal structure,
the repository settings are correctly used to set flag to true for
each package associated with that repository.
- The word "occurred" was mistyped as "occured" in rpmmd and is now
fixed in dnfjson.
