The osbuild-composer-rcm package was never finished, not in use and will be replaced by osbulid-composer-koji.
Signed-off-by: Tom Gundersen <teg@jklm.no>
`systemd-tmpfiles` will helpfully delete "old" files in /var/tmp at regular
intervals. The files installed from rpm has the timestamps from when they
were packaged, which causes some to be cleaned up when the timer triggers.
The first timer triggers 15 minutes after boot, so we were sometimes hit
by this when our CI was under load.
Fixes#839 and #862.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Now that we're going about multi-arch testing in a different way, we can
restore some of the old parallelism that we used to have in the pipeline
so it will finish faster.
Signed-off-by: Major Hayden <major@redhat.com>
The aarch64 tests are fairly unstable right now and they need work on
out of space issues (#870) and test cases (#861).
Signed-off-by: Major Hayden <major@redhat.com>
The composer-tests use systemd-nspawn, so make sure the `*-tests` RPM
depends on `systemd-container`.
Right now this dependency is recursively fulfilled by the `osbuild`
RPM. However, this might change if we switch to `bubblewrap` as a
sandbox. Furthermore, we should not fulfill our dependencies
recursively, but list them directly.
Currently one would have to pass a `macos` flag explicitly to build on
OSX, but if we replace `macos` with `darwin` this happens automatically.
Signed-off-by: Tom Gundersen <teg@jklm.no>
By default, go's tar archiver uses USTAR header format. Unfortunately, this
format doesn't support sub-second resolution for ModTime. Go solves this by
*rounding* the time. Sometimes, this creates an archive containing a file
with modtime from the future. When such archive is untarred by GNU tar,
the following message is produced:
tar: bf548dfd-0a90-40e6-bbf2-dcdd82fcbb4e.json: time stamp 2020-07-13
13:34:31 is 0.356223173 s in the future
We have two options here:
1) Use gnu header format that supports sub-second resolution. Unfortunately,
it seems that not all tar archivers support this format (e.g. 7-zip).
2) The other option is to truncate the date (instead of rounding).
I went with option 2.
Also, this commit adds a test to check that the header is not from the future.
Without this fix, the test is actually failing, I verified this manually.
Fixes#854
Add Fedora 32 test cases with the base OS + updates (to work around
aarch64 bugs in the original release). Add updates for Fedora 31 for
consistency.
Fixes#861.
Signed-off-by: Major Hayden <major@redhat.com>
During the last PR that restructured the pipeline, some copy/paste
errors happened and RHEL8.3 lost some tests. Also, some of the log
directories for x86/aarch64 tests overlapped.
Signed-off-by: Major Hayden <major@redhat.com>
Add base aarch64 tests for Fedora 31/32 and RHEL 8 CDN.
Also restructure the pipeline so that a slow distro or slow architecture
won't hold up the faster-moving tests.
This helps a bit more with #834.
Signed-off-by: Major Hayden <major@redhat.com>
In #842, we verified that our tests run on AWS properly and we should
now let Jenkins decide where to run the tests since they work in both
places.
Remove the hardcoded AWS label and allow tests to run where they please.
Signed-off-by: Major Hayden <major@redhat.com>
Bugs are being worked out in #798 and we can disable this test until
those bugs are fixed to avoid false failures in CI.
Signed-off-by: Major Hayden <major@redhat.com>
Set a static network configuration using cloud-init so that we don't
have to wait a long time for DHCP to work. Also, since we know the IP
and it is set earlier, this reduces the delay in testing on systems
without nested virt (like AWS).
Fixes#844.
Signed-off-by: Major Hayden <major@redhat.com>
Check to see if the instance is registered prior to attempting
registration. Also, ensure that registration happens during mock build
and during the deployment.
Signed-off-by: Major Hayden <major@redhat.com>
Make it easier to handle PSI outages or those times when PSI is
overloaded with jobs by allowing base and integration tests to
run on PSI OpenStack and AWS.
Signed-off-by: Major Hayden <major@redhat.com>
Seems to not work anyway:
Timeout (30 minutes) reached. Terminating "sudo ./osbuild-image-tests -test.v test/cases/fedora_32-x86_64-vhd-boot.json"
/home/travis/.travis/functions: line 296: kill: (10660) - Operation not permitted
Add support for partitions that can 'blkid' can not handle, like
the bios boot partition. For such partitions, 'blkid' will fail;
in that case all additional information will just be 'None'.
Convert any image that is not a raw image, e.g. a qcow2, to a raw
image and open the partitions via loop devices. This replaces the
usage of nbd, which was racy and flaky.
Instead on relying on the kernel for the partition parsing, this
is now done manually via loop devices and start + offset taken
from sfdisk. As a result the read_partition function has been
adapted to be called at later time, after the partitions were
opened via loop devices.
Not using nbd also means that the partition table is not scanned
by the kernel anymore and udev is not triggered. As a result the
'PARTUUID' property is not present for dos/mbr partition layouts,
since it is auto-generated by udev/blkid. Relevant blkid files
and functions are:
blkid_partition_gen_uuid(par)
called from probe_dos_pt()
in file libblkid/src/partitions/dos.c line 295
defined in libblkid/src/partitions/partitions.c line 1374
which generates the uuid via snprintf using the format:
'"%.33s-%02x", par->tab->id, par->partno'
Based on https://github.com/karelzak/util-linux at ce8985cc7
NB: the loop device code is imported from osbuild, making this
tool depend on osbuild's private library.
NB: As of the image conversion, more disk space is required to
examine non-raw images.
If a compressed file is detected (xz, bzip2, gzip), uncompress it
before analyzing it. NB: this has to happen after the is_tarball
check, because a tarball would also count as compressed.
Directly iterate over the partition list entries instead if using
indices. Optimize visual flow for the happy case by immediately
continuing if a partition has no file-system.
Read the image information before opening the image. This prepares
for a change where we will use information about the image to
convert it to raw instead of using nbd.
Create a GetImageInfoCommand analogous to GetOsbuildCommand that
will adjust the PYTHONPATH for image-info on travis so that the
osbuild python module is accessible.
The time it takes to depsolve a blueprint varies widely depending on
where the job is running and which mirrors are randomly chosen based on
the data returned in the metalink XML.
Use dnf's fastestmirror plugin to choose the fastest mirror for
downloading metadata. This returns consistent results in PSI + AWS and
every depsolve completed in under 60 seconds after 25 tests in each
cloud.
Fixes#845.
Signed-off-by: Major Hayden <major@redhat.com>
List the image tests that match the distro, distro version, and
architecture before running image tests. That ensures we run all of the
tests that are appropriate for the distro/version/arch combination.
Tests can be added or removed without changing the `run_image_tests.sh`
script.
Fixes#840.
Signed-off-by: Major Hayden <major@redhat.com>
As we embark on a CI journey to other architectures, we need to be
explicit about choosing the right architecture for each CI node.
This is a portion of the work required for #834.
Signed-off-by: Major Hayden <major@redhat.com>
Now that RHEL 8.3 pulls from the beta CDN, let's stop hiding the errors
and allow them to make the pipeline fail.
Signed-off-by: Major Hayden <major@redhat.com>
Some dnf bugs existed in early releases of Fedora 31 + 32 around
repository priorities being ignored (like BZ 1733582. Ensure that we are
running the latest version of dnf before we try to install from the mock
build repo (which has a much higher priority than Fedora's default
repos).
Also, check to see if EPEL is installed already before installing it
again.
Fixes#824. 🥰
Signed-off-by: Major Hayden <major@redhat.com>
Remove the F31/F32 internal repositories since they are causing dnf
download errors from time to time.
The 8.2 repository override should have never been there in the first
place since we're using the CDN.
The 8.3 repository is no longer needed since we can register the RHEL
8.3 instances against the staging CDN and get beta content from there.
Fixes#822.
Signed-off-by: Major Hayden <major@redhat.com>
