Rather than use the round-robin mirror, use a fixe stable one.
We should not over-use this, but it seems the round-robin one is
simply too unstable for our tests.
Eventually, we want to use metalink here instead, but we need to
rework the file source first, so we can do that in a way that
does not make the tests dependent on the time/place the test
case was generated (currently the URL to a specific mirror ends
up pinned in the test-case).
Signed-off-by: Tom Gundersen <teg@jklm.no>
Require repository informaiton to be passed as input, rather than
read from the current directory.
Reading from the repository informaiton meant to be used by weldr
has several drawbacks.
- it makes it impractical to use the tool outside a git checkout
- it makes it awkward to adapt the repositories to different use
cases
- it means that the shipped repositories cannot be extended with
update repos, as the same repos are used for testing, and that
would render our tests non-reproducible.
Overall, we are moving towards making repositories something the
caller must always pass in, rather than something that composer
maintains. For the weldr API we need to keep working as before,
but for new APIs we are avoiding that.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This means that the unit tests no longer need to load the
repositories from the git repo, and in a follow-up, osbuild-composer
won't need to either.
By splitting the repositories used for testing from the system
repositories available through the weldr API we are able to extend
the system repositories without affecting the reproducibility of
the tests.
Signed-off-by: Tom Gundersen <teg@jklm.no>
systemctl list-unit-files doesn't produce machine readable output.
parse_unit_files isn't very good at reading it and can produce duplicate
records.
This commit fixes it by deduplicating and sorting the units. This is a bit
hacky solution, but should work just alright. In the future we might
dump list-unit-files and do the job ourself, but let's not recreate the
systemd logic for now.
This script generates json test cases for each supported output format.
It requires an architecure, distro, osbuild store, and output directory
as input. There is also a json object which maps from output format to a
test case request which includes the compose request and boot type.
The script uses these test case requests to call the generate-test-case
script and then outputs the resulting json test cases to files in the
specified output directory.
The generate-test-cases no longer maps from output format to boot
type. Instead, the boot data is passed in through the standard input
alongside the composer-request object.
The script generate-test-cases only creates a single test case per run.
Therefore, the name should be singular. Also, this script has been moved
to a directory which will contain all test case generators.
This includes the packgase and build-packages used by each pipeline.
For now, this information is not used anywhere, but when we move
from dnf to rpm-based pipelines, this is what will be used instead
of the repo metadata checksum.
Signed-off-by: Tom Gundersen <teg@jklm.no>
For tarballs, this is currently not supported, so no point in generating
the images in the first place. This will still be done during testing to
boot-test them.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The test case json files will increase in complexity with the move from
dnf to json. They quantity of them will also continue to grow as new
distros, architectures, boot methods, image types, and blueprint
customizations become available. The generate-test-cases script
simplifies the process of creating new test cases. It accepts a compose
request and boot method as input and then uses osbuild-pipeline,
osbuild, and image-info to generate the test case.
[tomegun: some clean-ups and allow store to be reused]
A script that runs various go tools (mod tidy, mod vendor, and fmt for
now).
The idea is that it prepares the source to be ready for master. As such,
running it on master shouldn't modify any files. Make sure of that by
adding a test.
If we detect a ESP (via its type UUID) remember it and mount it at
any filesystem that has a /boot/efi (there should in theory be
only one). This is needed so grubenv can be read, which is likely
a link from /boot/grub2/grubenv to ../efi/EFI/$vendor/grubenv.
Additionally this will make rpm verify not report that all the efi
binaries are missing, e.g.:
"missing": [
"/boot/efi/EFI",
"/boot/efi/EFI/BOOT",
"/boot/efi/EFI/BOOT/BOOTIA32.EFI",
"/boot/efi/EFI/BOOT/BOOTX64.EFI",
"/boot/efi/EFI/BOOT/fbia32.efi",
"/boot/efi/EFI/BOOT/fbx64.efi",
"/boot/efi/EFI/fedora",
"/boot/efi/EFI/fedora",
...
These tests (will) test more than just image-info: they'll take a
blueprint, verify that `osbuild-pipeline` generates the correct
pipeline, run osbuild with that pipeline and verify that the resulting
image has the expected image-info output.
This change only includes the latter half (i.e., only moves the already
existing tests).
Also drop python's unittest. It was hard to control output (important
for quickly spotting failures and to make travis happy). This introduces
test/run, which runs all test cases in test/cases or the ones given on
the command line.
When a failure occurs, it prints a diff of the actual and the expected
image info.
The main difference (according to image-info) is an additional package
containing a gpg key which was used to verify packages. The one
generated by lorax-composer doesn't have this, because it doesn't verify
signatures.
Also, lorax generates an image with a separate /boot partition. This is
not yet addressed here, because osbuild doesn't support it yet.
The main difference (according to `rpm -qa`) is an additional package
containing a gpg key which was used to verify packages. The one
generated by lorax-composer doesn't have this, because it doesn't verify
signatures.
The main difference (according to image-info) is an additional package
containing a gpg key which was used to verify packages. The one
generated by lorax-composer doesn't have this, because it doesn't verify
signatures.
Also, lorax generates an image with a separate /boot partition. This is
not yet addressed here, because osbuild doesn't support it yet.
The main difference (according to image-info) is an additional package
containing a gpg key which was used to verify packages. The one
generated by lorax-composer doesn't have this, because it doesn't verify
signatures.
This is the output of disk info ran against the images produced
by the specified pipelines.
Skip the actual test for now, because it is taking too long to run.
Signed-off-by: Tom Gundersen <teg@jklm.no>
A test case is now expressed purely by dropping in a json document in
`tools/test_image_info/pipelines`. It should contain the input compose
(output format and blueprint) as well as the expected pipeline. All the
existing tests are moved over to this format.
This shares the same infrastructure as the image tests, ideally we want
to run the blueprint tests and the image tests against the same pipelines.
For now, test cases are skipped from the blueprint tests if they do not
contain a 'compose' section, and from the image tests if they do not
cotain an 'expected' section. In the future we may want to make both
mandatory.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Add two kinds of tests, with one case each:
1. Run image-info against an osbuild pipeline. Uses osbuild
from a submodule to make an image from the pipeline.
2. Run image-info against an existing image, fetched from the internet.
This shows the changes an image has relative to what its rpm database
thinks is installed. Output is:
"rpm-verify": {
"missing": [ <missing files> ],
"changed": { <map from filename to rpm attribute octet > }
}
Alas, this makes running image-info slower.
For each partition, find out if its the root or boot partition and
gather only the relevant information. Make sure that we don't get
information from /boot twice.
Rough draft of image-info, a tool that extracts high-level information
about an os image. It prints this information in JSON form on stdout.
Run it like this:
$ tools/image-info <image>
It supports all images that qemu-ndb supports.
