The old aws unit test is not very useful now, because there's the new
integration test which covers a lot more. Therefore, this commit
removes it.
Fixes#512
Speed up the boot tests by allowing qemu to use all of the available
CPUs on the system. Our CI VMs have at least 2 CPUs and this shortens
the time required for a boot test.
Signed-off-by: Major Hayden <major@redhat.com>
This package does not contain an actual queue, but a server and client
implementation for osbuild's worker API. Name it accordingly.
The queue is in package `store` right now, but about to be split off.
This rename makes the `jobqueue` name free for that effort.
Cockpit-composer has moved from the weldr github organization to the
osbuild organization and its github url has changed. The url in the
man-page source is updated.
The test generation script we use outputs the test cases prefixed with
"fedora_$RELEASE" so I renamed all the current test cases to follow this
convention and also changed the travis job names so that it can run both
x86_64 test cases and aarch64 test cases.
We need to make partition uuids stable to be able to run image-info
tests on images using gpt. For example all aarch64 images use gpt.
Also change the type of the (part) UUID to string because that's what
the other UUIDs use and it is easier to work with.
The osbuild-image-tests don't currently support boot test for any
alternative architecture because the qemu-system-x86_64 command is
hardcoded. This patch introduces a branch specific to aarch64, but
without a KVM support as I was unable to make it run in Beaker, which is
currently the only offering we have with ARM machines. As a workaround
the boot tests will be skipped if kvm kernel module is not found and only
image-info tests will run.
Also adds new types to weldr/json.go to support them.
ComposeEntry had to be duplicated instead of used as-is because it
enforces image type strings that do not match what the API uses (the API
types are all lower case, the internal names are capitalized).
The UUID list should only include UUIDs that are successfully deleted.
If there was an error it should only be listed in the errors list. This
matches the behavior of lorax-composer.
Yes. This goes against my desire not to change code to accommodate
tests. But there is no other good way to test compose results without
long running, and possibly fragile, composes. And this matches lorax's
behavior.
The change adds support for the ?test=1|2 query parameter to the compose
POST, and a new store function - PushTestCompose that handles creating
the fake compose results.
Passing ?test=1 will create a failed compose. Passing ?test=2 will
create a successful compose, but one without any files.
The purpose of these is to be able to test the compose result API
responses like compose/failed, etc.
We currently declare compatibility with golang 1.12 so we should also
use it in our CI. Once we agree that we would benefit from some features
in golang 1.13 or 1.14 we can bump it back.
us-east-1 seems to very slow these days, some imports can take up to
15 minutes. This commit raises the number of attempts before we give up.
Previously, we did 40 attempts, each delayed by 15 seconds, making the total
timeout equal to 10 minutes. Now we do 80 attempts with the same delay,
making the total timeout 20 minutes.
The source of issue here is that our release cycles are synchronized and
if we want to use the latest released osbuild we need to wait for the
bodhi process. It makes sense that we don't want to use latest osbuild
from git master but we should be confident enough in the version we
submit to bodhi.
Using the version from updates-testing therefore fixes the issue with
synchronization and avoids using unreleased versions.
The partuuid is a uuid of a specific GPT partition as opposed to the ptuuid
which is associated with the partition table and filesystem uuid which
is associated with a filesystem on a partition.
The support for partuuid was introduced in osbuild 11, see this PR for
reference: https://github.com/osbuild/osbuild/pull/306
We were using dnf's default of 48h, but that does not work for
updates repositories, as they depend on an expiration time of 6h.
Allow the metadata_expire value to be configured per repository.
If the value is unset, then never expire the metadata. Set the
value to 6h for all the fedora testing repos.
This fixes issue #476.
Signed-off-by: Tom Gundersen <teg@jklm.no>
There is no point in having the grub2 stage in pipelines for image types
that are not bootable. The current version is probably a result of
previous refactoring where the member variable was named `IncludeFSTab`.
Moving the grub2 stage into the conditional branch should also fix test
generation on aarch64.
Finally it is necessary to regenerate test cases for non-bootable image
types.
If you don't set the time it ends up being the default Go time which is
1/1/1 and when you convert that using UnixNano() you get a nice big
negative number (-6795364578871345152) which then eventually shows up in
the queue, finished, and failed output as 'Fri Aug 30 17:47:39 1754'
and since the Time Travel feature is not yet complete this is
impossible.
The fix is to set it when the job is started.
If a user creates a compose with a size of 0, the default image size for
the image type should be used. Also, certain image types have
requirements for the image size. In order to ensure that the proper image
size is stored in the compose object, the compose's ImageBuild object
uses ImageType.Size() to get the correct image size for the image type.
The new crypt tests cannot be run on macOS. Making them conditional for
non-macOS platforms fixes running unit tests in the internal directory
on macOS.
weldr's store is quite complex and handled serialization itself. Move
that part out into a separate package `jsondb`.
This package is more generic than the store needs: it can write an
arbitrary amount of JSON documents while the store only needs one:
state.json. This is in preparation for future work, which introduces a
queue package that builds on top of `jsondb`.
Codecov's patch status measures lines covered in a given pull request,
which fails when moving code around.
Adding code with no coverage still fails the codecov/project status,
which fails if coverage has gone down.
In PR #400:
https://github.com/osbuild/osbuild-composer/pull/400
we changed the boot type of AWS images from qemu-extract to aws, which
resulted in a bug in the generate-test-case script. It tried to attach
xz archive using qemu-nbd which works just fine from qemu-nbd
perspective but sfdisk and blkid of course failed, because the xz
archive is not a disk image.
This patch makes sure we extract the image before attaching it to
/dev/nbdX.
The newest osbuild is needed for support for aarch64. The aarch64 images
are using GPT which requires stable partuuid which was included in the
latest osbuild release (11). This will be used to produce stable
image-info tests.
Even though `dnf.exceptions.RepoError` is documented as the base error,
`dnf.exceptions.Error` is actually the base error (and also documented
as such).
For one, this allows us to use `require` instead of `assert` and
`continue`, which was awkward to read. That works because it's ok to
fail a subtest: the remaining subtests are executed after it.
Also, this shows which test was executed, making debugging easier:
=== RUN TestCrossArchDepsolve
=== RUN TestCrossArchDepsolve/fedora-30
=== RUN TestCrossArchDepsolve/fedora-30/x86_64
=== RUN TestCrossArchDepsolve/fedora-30/x86_64/ami
...
You can now also run those sub tests in isolation:
osbuild-dnf-json-tests -test.run TestCrossArchDepsolve/fedora-30/x86_64/ami
Lastly, it enables running those tests in parallel (not part of this
patch) by calling `t.Parallel()`.
This function prints the formatted unexpected error message, instead of
the error struct in golang syntax.
It also allows to remove the error in the assertion message.
The API is always advertising a content type of application/json, but
not sending JSON on errors.
Change it to send simple JSON objects like this:
{ "message": "something went wrong" }
This can be extended to include more structured information in the
future.
Also return an (for now) empty JSON object from `addJobHandler()`. It
returned nothing before, which is invalid JSON.
Stop testing for the actual error strings in `api_test.go`. These are
meant for humans and can change. Only check what a client could
meaningfully check for, which is only the HTTP status code right now.
Don't panic when the error message cannot be written to the connection.
Ignore it, because there's nothing we can do in this case. The standard
library has the same behavior.
