When gsl with version * was specified in the blueprint,
composer depsolved both x86_64 and i686 version of gsl.
This test case should prevent this from happening.
gsl is used because it has x86_64 and i686 versions on both RHEL and Fedora.
Also, gsl-devel package exists, which is not dependant on gsl and shouldn't
be depsolved.
blueprint.GetPackages() method was used to depsolve blueprints prior the
dnf-to-rpm switch. However, it got dropped during the switch. This commit
makes weldr use it again.
The nice effect of this change is that we can drop getPkgNameGlob function and
have only one function for getting package name-versions from a blueprint.
Also, blueprint.GetPackages() works better with * version. Previously, we had
issues with composer depsolving bash of version * to both x86_64 and i686
versions of bash package. GetPackages() converts the package to name-version
of just bash, which dnf-json correctly depsolves to just one architecture. On
the contrary, the previous method converted bash to name-version bash-*.*.*,
which confused dnf-json.
Note that conversion to bash-* is also wrong because this will cause dnf-json
to install all packages with prefix "bash-*".
This makes it easier to use the test binary with the `-run` argument.
Instead of the full path:
-test.run TestImages//usr/share/tests/osbuild-composer/cases/rhel_8.2-x86_64-openstack-boot.json
this only requires the actual name:
-test.run TestImages/rhel_8.2-x86_64-openstack-boot.json
When edd7b37ea added `--output-directory` to the invocation of osbuild,
it also removed `--store`.
This was a mistake: osbuild's default store is `.osbuild`, which is not
what we want. Restore the old behavior of passing a temporary directory,
but use the same for each test run.
Treating stdout and stderr separately makes it hard to match what
happened when. It's also easy to miss when `-v` is passed to the test
binary.
Print the output to stdout when osbuild fails, because the test
framework we're using does not print errors if they're too large.
Also, don't special-case exec.ExitError. Output might be useful in any
case.
Now that mock builds for RHEL 8.3 look good, add tests for RHEL 8.3
nightly builds. The tests are wrapped in `catchError()` which allows us
to fail the stage without failing the whole pipeline just because RHEL
8.3 is having a bad day.
Signed-off-by: Major Hayden <major@redhat.com>
RHEL doesn't have python3 in PATH by default. Instead it has /usr/libexec/
platform-python. When building the RPM, RHEL correctly mangles dnf-json's
shebang to the platform python. This commit switches osbuild-composer
to use this shebang because it should always respect target platform's
default python binary.
Fixes#745
Previously, all the osbuild-composer tools must be run from a directory with
dnf-json. This was often confusing, especially with the dnf-json-tests. This
commit changes the path to be absolute, so this is no longer an issue.
RPMMD had hardcoded path to dnf-json helper. This required all executables
using RPMMD to be run in the directory where dnf-json was located. This commit
makes RPMMD take the path to dnf-json as an argument. This allows its
consumers to specify whichever path they want.
Not a functional change
Ansible configuration can be managed by an `ansible.cfg` ini file, but
setting up that file and ensuring it's always in the same place is
difficult.
Configure Ansible via environment variables instead.
Signed-off-by: Major Hayden <major@redhat.com>
Add an end-to-end qcow2 test that follows a customer's steps with
`composer-cli`. The image is booted with libvirt to allow the best
virtualization options to be chosen by libvirt. It also uses libvirt's
default network.
Signed-off-by: Major Hayden <major@redhat.com>
Support analyzing ostree repositories or directory that contain an
ostree commit as created by osbuild. Will return the mode of the
repository along with the references and their commits. For the
first references, the commit is resolved and checked-out to a temp
directory. This directory in turn is then analyzed via the existing
append_filesystem function. The latter has gained some small ostree
specific tweaks.
NB: for the ostree checkout /usr/etc/ is bind mounted to /etc in
order to make append_filesystem happy. The rpm verification step
is NOT run, because that is not really compatible with ostree.
Move the code of the current main into is own method and call that
from main. This prepares support for analyzing other types than
images. Additionally, add argument parsing via argparse to get a
help texts.
qcow2, unlike ami, is guaranteed to exist on all the architectures we
support.
Also, now that we preserve the cache, make two composes, rather than
just one. This verifies that nothing breaks horribly just because the
cache not being cold. The fact that we have a cache at all should make
this tolerably fast.
This should fix#693.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The `osbuild-image-tests` code does a good job testing image builds for
AWS, but it doesn't use the same code paths that a customer would when
using `composer-cli`.
Add a test that builds a compose and uploads the image using
osbuild-composer and its worker. Also, build an instance in AWS based on
the image we imported and check to see if the smoke test file is
present.
Signed-off-by: Major Hayden <major@redhat.com>
Allow a test case to be generated for a specific image type, rather than
for all at once.
This is useful when adding additional image types, rather than
regenerating the existing ones.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This is a very minimal image type, which is not expected to do anything
with an empty blueprint. It is documented as a way to install RHEL for
Satellite.
This fixes#720.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Schutzbot currently runs all the same tests an all arches/distro
combinations. This will not work as we introduce image types only on
some distro/arches.
In the future we should make the image-test binary more clever, so
Schutzbot won't have to tell it which cases to run at all. For now,
simply don't fail if the specified test-case does not exist.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This is not a functional change, just helps to make the output more
readable to indicate which sub-command is being tested.
Signed-off-by: Tom Gundersen <teg@jklm.no>
most invocations of composer-cli do not need to parse JSON so use
the canonical invocation instead. I've left existing functions
which assert on the returned results intact.
'quiet' parameter has been removed b/c it was never used
Until osbuild-14, the images were unconditionally kept in the cache,
meaning the cache could grow very large. Now only the downloaded RPMs
are saved, which greatly limits how big it can grow.
Having the RPMs cached should speed up all but the first image build a
lot, so we should take advantage of that by not flushing the cache
between each build.
The cache is still flushed when the worker is stopped / restarted.
This moves the cache from /var/tmp/osbulid-worker* to
/var/cache/osbulid-worker/osbulid-worker-*. This means that each worker
gets a dedicated cache, in case there are several on one machine. In the
future we may want to combine them and only ever have one cache, but for
that we need improvements in parallel access and cache-cleanup.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The input options of qemu assembler do not require filesystem for every
partition. This patch makes it optional and omits the object in case it
is empty.
osbuild schema definition:
162797f206/assemblers/org.osbuild.qemu (L63)
Partition type, as defined in osbuild qemu assembler, is of type string
because it is either UUID or identifier. See here for details:
162797f206/assemblers/org.osbuild.qemu (L88)
Change type of the "Type" struct member to match the definition in
osbuild. This patch is needed for ppc64le support.
This is needed for lorax parity. When multiple blueprints are being frozen in
toml mode, the API returns an error. This is the same behaviour as in the
/blueprints/info route.
Fixes#667
Commit 8dd4554491 introduced a bug because
the output from osbuild is not captured. The problem is that osbuild
itself writes JSON document to STDOUT so two different JSON documents
are written to STDOUT: osbuild output and generated test case. The
generate-test-cases script then fails because it cannot json.loads() the
test case.
This patch simply supresses the output which fixes the issue.
