During development of a new distro, we need to test composer against
nightly or beta repositories, but we cannot ship composer itself
with the nightly repository information hardcoded in. At the same
time, we want to distinguish between the system repositories of the
host and the repositories we use to generate images (the host may not
use the same distro/version/architecture as the target, and it may
include custom repositories that the target should not).
We therefore ship per distro repository information that can be
overriden (typically in testing) by dropping files in /etc.
For now use the latest nightlies for RHEL-8.2, we may want to
replace these with the official mirrors for GA eventually.
Signed-off-by: Tom Gundersen <teg@jklm.no>
dnf-json relies on dnf's ability to cache repository metadata. This is
important, because the API calls it quite often to serve requests for
package lists and depsolves.
However, osbuild's dnf stage always fetches new metadata, because it
doesn't have access to the host's cache. Since metadata is valid for
some time, even after a repository changed, the checksum we put in
the pipeline might be old.
Force a new metadata download when producing the pipeline. This is still
not perfect, but greatly reduces the probability of putting stale
metadata into the pipeline.
Instead of having a static repository checksum, set it dynamically from
the metadata that osbuild-composer last saw. This is implemented in
dnf-json, which returns the checksums for each repository on every call.
This enables the use of repositories that change over time, such as
fedora-updates. Note that the osbuild pipeline will break when such a
repository changes. This is intentional: pipelines have to be
reproducible.
We should always specify a checksum when describing a repository to pull
content from. For now, fedora-30 duplicated the checksum. The idea is
that we can have it in one place at some point.
These endpoint are similar in many ways, therefore just one commit. Their
functionality is basically same as in lorax except for error messages and
weird edge cases when handling trailing slashes.
closes#64, closes#65
Add function to convert between a user source and a repo which can be
passed to dnf-json. This is neccessary because user-defined sources have
a slightly different format than dnf repos.
We want to test API methods which calls dnf. Unfortunately, calling dnf
is expensive operation - it requires network access and downloading
a lot of (meta)data. This commit changes the rpmmd implementation
so that it can be mocked.
When dnf-json dumps the packages from the repos passed to it, it does
not sort the packages. In order to properly list and search the
packages, the package list is now sorted before being returned by the
FetchPackageList function.
Make each command accept a `repos` key containing repository
descriptions.
Make weldr API pass the repository like this. Nothing should change,
because the repos were the same (Fedora 30).
log.Fatalf not only writes into a log, but also exits the process.
Instead, we want the caller to handle the error.
Also, the logged message is imho wrong.
osbuild-composer now uses socket activation instead of hardcoded paths
in the code. osbuild-worker is an http client therefore it uses only
service unit. osbuild-worker must be started after the socket is
created. osbuild-composer service requires osbuild-worker to run, because without
it no jobs can be started.
osbuild-composer is executed as a regular user (newly created
_osbuild-composer user) as opposed to the worker which must run as root
in order to execute osbuild itself
Keep main.go files under cmd/ and internal libraries under internal/.
This will allow us to add more exutables under cmd/ (whereas only one
was possible when main.go was kept in the root).
Signed-off-by: Tom Gundersen <teg@jklm.no>
