For now, this simply wraps Pipeline and Sources, and retruns the
resulting manifest object. In the future, Pipeline and Sources
may be dropped from the interface.
Signed-off-by: Tom Gundersen <teg@jklm.no>
A manifest is simply a struct containing a sources and a pipeline
object. We want to store and transfer pielines always with their
sources, and will use the manifest for this.
When serialized, a manifest can be the input to `osbuild`, just
like a bare pipeline can be. This means there will be no need
to pass in sources separately on the commandline.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We were verifying two things: if the passed distroArg exists in the
distribution mapping in common/types.go and if the it is an actually
registered distro. Since you cannot have distros registered that don't
correspond to a type, the first test is unnecessary.
Merge the two tests by moving the (much better) error message down into
the second test. This makes DistributionExists redundant, because
Registry.GetDistro() checks this implicitly.
Also, move ListDistributions() to the Registry object, because we want
to show distributions that are actually registered.
Add a test which checks that Registry.List() works and that all included
distributions register correctly.
This changes it to an int pointer so that the JSON will output null.
This means it needs to be checked for nil or for 0 in go.
0 is not a valid revision in the WELDR response, they always start at 1
and increment for each new revision tag so either way is a valid way
to indicate it isn't set.
These tests build on the client functions to run integration tests on a
running API server.
It uses the reflect module to examine the methods attached to the
checkBlueprintsV0 struct and run the ones with names that start with
'Check', also checking the type signature of the functions and failing
the test if any of them don't match.
This will make it easier to add more checks without needing to add
boilerplate call/registration of the functions in the top level runner.
Just add the new function with the right name and signature and it will
be run when checkBlueprintsV0.Run() is called.
Checks for other API routes should be added to their own modules. There
will be some duplication of the Run function in each, but I think that
it will help keep things more manageable by separating them instead of
putting them all into a single giant Run() call.
Currently the responses are all embedded in the weldr API functions.
They need to be usable by the client helper functions so I'm copying
them here and giving them names.
A later commit will go through the API can refactor it to use these
instead of the embedded ones.
This package will contain functions for communicating with the API that
can be used in both the integration tests as well as in a future cmdline
tool similar to composer-cli
When possible the client functions will return the same structures used
by weldr/api.go which have been exported in weldr/json.go
Return errors from all distro's New() functions instead of logging and
returning nil. Also, return errors instead of panicking from
NewRegistry() and NewDefaultRegistry().
These packages (and their tests) shouldn't access the distro package,
because that's cyclic.
Also, these packages should only test the objects they expose.
WithSingleDistro() doesn't follow go's naming convention for creating
objects (New*). Rename it to NewRegistry() and rename the old
NewRegistry() to NewDefaultRegistry().
The idea is that NewRegistry() can be used to create full Registry
objects from outside the package. NewDefaultRegistry() is a convenience
function that creates a Registry with all known distros.
rpmmd looked at the CACHE_DIRECTORY environment variable to set a path
for the dnf repository cache. Aside from being a smelly thing to do
from a library, this breaks osbuild-pipeline and osbuild-dnf-json-tests,
which don't run as systemd services and thus don't have CACHE_DIRECTORY
set.
Explicitly pass the cache directory to rpmmd. Keep using a path based on
CACHE_DIRECTORY for osbuild-composer. Use the user's `.cache` directory
for osbuild-pipeline and a temporary directory for the tests.
Images can be built for rhel 8.1. The pipeline generation and distro
tests are based off of the rhel 8.2 ones. Repository information as been
added for rhel 8.1. The repo urls are internal ones and will only work
if the user is on the Red Hat vpn.
If no packages are included in a blueprint, the slice remains `nil`,
which translates to `null` in json. Always initialize the slice by
pointing it to an empty array.
This can happen when CacheDirectory= is missing from the service file.
That's unlikely to happen, but it's hard to figure out what caused the
failure when it does. Be explicit and panic.
This changes osbuild-composer's behavior to match lorax-composer when
encountering invalid versions. Instead of leaving them as-is it will
return a BlueprintError explaining the problem. eg.
"errors": [
{
"id": "BlueprintsError",
"msg": "Invalid 'version', must use Semantic Versioning: is not in dotted-tri format"
}
]
This is enforced on new blueprints (including the workspace). If a
previously stored blueprint has an invalid version and a new one is
pushed it will use the new version number instead of trying to bump the
invalid one.
This also moves the version bump logic into blueprint instead of store,
and adds an Initialize function that will make sure that the blueprint
has sane default values for any missing fields.
This includes tests for the Initialize and BumpVersion functions.
This adds returning errors from the store PushBlueprint* functions, and
adds handling of the errors to the API code in preparation for new code
to check the blueprint before saving it.
The LoadRepositories function interates over a list of paths and expects
to find a distro configuration in one of them. The case when no path
with valid configuration is found was not handled. This patch introduces
the check.
Without passing in a cachedir, dnf would create a random one for every
invocation. This meant that caches were never reused, nor cleaned up
properly.
Let systemd create a cache directory for us in /var/cache/ and use
that via the environment variable systemd sets for us.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This is needed for unit tests, because it wasn't possible to mock the
rpmmd module before. This also requires that the checksum is moved to
the compose request and evaluated in the endpoint handler instead of
push compose. I think it makes sense to have the checksum in the compose
request directly.
Also a "module platform ID" is required now, but we don't have the
"global" distribution any more, so this patch introduces mapping from a
distribution to the module platform ID.
the name was misleading because the function could do more than just
download package list. In PushComposeRequest it is also used to fetch
checksums for the repositories, therefore I decided to rename it to
reflect this usage.
The change also requires customizations in the error handling, as some
errors are now handled automatically by the custom unmarshaler.
Include a note about HTTP return types.
There was a bug in the previous implementation which used to pass the
argument as a value but that does not work because we need to change the
value of it. The new implementation uses pass by reference.
Create a test to cover this scenario.
If the Epoch is > 0 the it should be added to the front of the version,
separated by a colon.
Also include a depsolve package with a non-zero Epoch and adjust the
tests accordingly.
Without making a deep copy of the blueprint the changes made to the
package and module versions will persist in memory, causing it to lose
the package and module version globs.
This can be seen by executing a freeze request and then a depsolve. The
blueprint included in the depsolve had the version globs replaced by the
frozen EVRA values.
The Blueprint struct is complex, deep, and full of references. This
means that any changes to it in memory will persist. Sometimes you need
an actual copy of it, so this adds DeepCopy which uses the json.Marshal
and Unmarshal functions to create a deep copy with no references to the
original.
This is not very efficient, but the alternative is adding Copy functions
to all the member structs and then calling them to build the copy.
This adds the modules to the list of package specs to be depsolved. It
includes a new function to build the version glob package string, as
well as tests for the new function and for depsolving with modules in
the blueprint.
This adds support for the modules field. It moves the version
replacement into a separate function, setPkgEVRA, and adds tests for the
new function as well as for blueprints with packages in both the
packages and modules lists.
The dependencies are not sorted, so depending on what order they were
returned in the freeze route would or would not return the correct
results (exhibited by the version being the original glob instead of the
EVRA).
This also fixes the tests so that the depsolve results are slightly
unsorted by adding a dep-package3 to the start of the list.
The current `NewRegistry` implementation allows for nil values in the
map, but this leads to subtle bugs when using the registry. This patch
enforces non-nil values by introducing additional checks before we
insert the value into the map.
The change unfortunately breaks a lot of tests and therefore it is
necessary to create additional mock: distro.
The new mock is used instead of the previous "real" implementation,
which used to contain nil values.
Images can be built for fedora 32. The pipeline generation and distro
tests are based off of the fedora 30 ones. Repository information has
also been added for the fedora 32 repos.
Images can be built for fedora 31. The pipeline generation and distro
tests are based off of the fedora 30 ones. Repository information has
also been added for the fedora 31 repos.
