Commit graph

1303 commits

Author SHA1 Message Date
Brian C. Lane
eb76b9ec8e weldr: Fix the bad blueprint depsolve response
This changes the response to match lorax-composer's behavior. If any of
the blueprints in the list passed to /blueprints/depsolve/... have an
error that error should be appended to the error list, and the blueprint
included in the blueprints list with an empty dependencies section.

It was returning an error 400 and a single error if it hit any depsolve
problems, skipping any other blueprints and returning the wrong
response.

This also adjusts the tests to account for the change.

Fixes #890
2020-07-28 08:18:56 +02:00
Major Hayden
3d538a6663 Add Lorax TOML workaround
Copy the TOML workaround from the aws script to the qemu script.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
574c4f622c Update test cases
Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
3daa751095 distro/rhel8: increase default image size to 4GB
RHEL 8.3 requires more than 2GB to install on aarch64/ppc64le.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
f2d0f5e6f6 aarch64: Remove updates repo from test cases 2020-07-27 10:27:15 -05:00
Major Hayden
5ef678a57b aarch64: workaround qemu-convert CPU bug
The conversion with `qemu-img convert` often fails on aarch64 systems
with LOTS of CPUs. This is fixed in RHEL 8 for aarch64, but not in
Fedora.

Set the maximum coroutines to 1 to avoid this issue until the bug is
fixed.

Bug: https://bugs.launchpad.net/qemu/+bug/1805256

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
48fdc3d831 test case repos: use RHEL 8.3 Beta 1
Switch to RHEL 8.3 Beta 1 for generating test cases.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
f326760971 Remove nvram when undefining VM on aarch64 2020-07-27 10:27:15 -05:00
Major Hayden
7f78502046 s390x: wait longer for smoke test check 2020-07-27 10:27:15 -05:00
Major Hayden
0d56d284d9 Work around ppc64le virt issues 2020-07-27 10:27:15 -05:00
Major Hayden
248db2be0e 🔐 Add OpenStack credentials for image tests
We want to deliver images directly to OpenStack during the CI tests.
Ensure the credentials are available during those tests and move them to
the proper location to be read by OpenStack tools and SDKs.

This helps with #876.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-22 20:02:12 +03:00
Ondřej Budai
5ab8b59803 18
Release osbuild-composer version 18.
2020-07-22 17:04:23 +02:00
Martin Sehnoutka
3fc03503a5 distro/f32: manually relabel cp in buildroot with install_t
By labeling `cp` with `system_u:object_r:install_exec_t:s0` we allow it
to copy labels unknown to the host.

See also corresponding commit in osbuild:
e80130a830
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
f8f35016d6 osbuild: Add "labels" field into the SELinux stage
The SELinux stage supports labeling specific files with specific labels
like this:
7b0db90c76/stages/org.osbuild.selinux (L37)

This can be useful for preventing some SELinux issues during the image
build. You can find example usage here:
7b0db90c76/test/data/stages/selinux/test_basic.json (L5)

And more reasoning here:
e80130a830
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
636b761b6b distro/f32: add SELinux RPM to build pipeline
This patch adds SELinux stage to build pipeline for f32 just like
we do for RHEL8. This helps with issues like this:
https://github.com/osbuild/osbuild-composer/issues/798
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
12172adf47 test-case-generators: refactor customizations
This patch contains refactoring of the part where we handle
customizations. Previously customizations were considered an "image
type" which is not obvious. Now it is a command line switch.
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
60269abdff distro: print more information in case the test fails
This helps when looking for the issue because the previous input did not
mention which specific qcow2 test case failed.
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
a38ee2e7a7 test-case-generators: fix printing of osbuild log
Writing to a file followed by reading it results in printing b'' unless
we seek to the beginning. Fix it by calling seek.
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
09208c9237 spec: bump osbuild requirement to v18
SELinux labels are only supported since this version.
2020-07-22 10:47:46 +01:00
Major Hayden
5d902260b0 Fix post-build telegram notification
Jenkins needs to know which node to use to send the telegram
notification.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-20 10:00:41 -05:00
chloenayon
ae3fb5e1c5 distro: enable cloud-init services for openstack and qcow2 images for fedora
Previously, cloud-init was installed on qcow and openstack images but was not
enabled, preventing a user from logging in to these images. Cloud-init, cloud-final,
cloud-init-local and cloud-config services are enabled to make cloud-init work
and to maintain consistency with official fedora 31 and fedora 32 images.

The image tests have been updated to reflect these changes.

Fixes #698
2020-07-18 11:57:05 +02:00
Lars Karlitski
f38d55da88 distro/tests: use test case filename as test name
These tests used the image type as test name, which is ambiguous. Use
the file name for the test case instead.
2020-07-18 10:52:11 +01:00
Major Hayden
460369b32b 💣 Notify via telegram on master CI failure
We've come a long way and we need to triage failures that occur during
CI for the master branch. This will help us find problems with CI as
well as find other issues that could show up in a customer environment.

Also, let's send a happy notification when everything goes well. 💚

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-17 13:57:30 -05:00
Tom Gundersen
1280035f8f submodule: bump to the most recent release
Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 19:13:15 +01:00
Tom Gundersen
fbfa191c81 rcm: drop sub-package
The osbuild-composer-rcm package was never finished, not in use and will be replaced by osbulid-composer-koji.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 19:13:15 +01:00
Tom Gundersen
79d27ded25 image-tests: avoid /var/tmp
`systemd-tmpfiles` will helpfully delete "old"  files in /var/tmp at regular
intervals. The files installed from rpm has the timestamps from when they
were packaged, which causes some to be cleaned up when the timer triggers.

The first timer triggers 15 minutes after boot, so we were sometimes hit
by this when our CI was under load.

Fixes #839 and #862.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 19:13:15 +01:00
Major Hayden
842b665659
Improve emoji selections 2020-07-17 11:46:12 -05:00
Major Hayden
20fd6521fb 🏇🏻 Speed up pipeline with parallelism (again)
Now that we're going about multi-arch testing in a different way, we can
restore some of the old parallelism that we used to have in the pipeline
so it will finish faster.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-17 11:43:54 -05:00
Major Hayden
60aafdaaab 📤 Temporarily disable aarch64 tests
The aarch64 tests are fairly unstable right now and they need work on
out of space issues (#870) and test cases (#861).

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-17 09:24:12 -05:00
David Rheinsberg
cdcf212a52 spec: pull in systemd-nspawn
The composer-tests use systemd-nspawn, so make sure the `*-tests` RPM
depends on `systemd-container`.

Right now this dependency is recursively fulfilled by the `osbuild`
RPM. However, this might change if we switch to `bubblewrap` as a
sandbox. Furthermore, we should not fulfill our dependencies
recursively, but list them directly.
2020-07-17 07:42:27 -05:00
Ondřej Budai
2beb4e0a2b remove unneeded fedora 30 repositories
Fedora 30 support was dropped in e66368b0, this file has had no use since
then. This commit removes it.
2020-07-17 09:41:32 +02:00
Tom Gundersen
bca53a596c crypt: add automatic darwin support
Currently one would have to pass a `macos` flag explicitly to build on
OSX, but if we replace `macos` with `darwin` this happens automatically.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 07:48:22 +02:00
Ondřej Budai
813275b6c4 drop Travis CI
All tests were moved to schutzbot, there's no need to run them on Travis.
2020-07-16 12:45:54 -05:00
Ondřej Budai
693b4f0823 schutzbot: make image tests upload vhds to Azure
Support for uploading was already in, it's just needed to pass the right
environment variables.
2020-07-16 14:00:15 +02:00
Ondřej Budai
b4a57225a7 api/weldr: add missing modtime to tar returned by /compose/logs
This file was surely not created on 1970-01-01, this commit sets the ModTime
to the current datetime.
2020-07-15 08:58:25 +01:00
Ondřej Budai
454e471af8 api/weldr: bring tars back from the future
By default, go's tar archiver uses USTAR header format. Unfortunately, this
format doesn't support sub-second resolution for ModTime. Go solves this by
*rounding* the time. Sometimes, this creates an archive containing a file
with modtime from the future. When such archive is untarred by GNU tar,
the following message is produced:

tar: bf548dfd-0a90-40e6-bbf2-dcdd82fcbb4e.json: time stamp 2020-07-13
13:34:31 is 0.356223173 s in the future

We have two options here:

1) Use gnu header format that supports sub-second resolution. Unfortunately,
   it seems that not all tar archivers support this format (e.g. 7-zip).

2) The other option is to truncate the date (instead of rounding).

I went with option 2.

Also, this commit adds a test to check that the header is not from the future.
Without this fix, the test is actually failing, I verified this manually.

Fixes #854
2020-07-15 08:58:25 +01:00
Major Hayden
4b0409df5b Update Fedora 31/32 aarch test cases
Add Fedora 32 test cases with the base OS + updates (to work around
aarch64 bugs in the original release). Add updates for Fedora 31 for
consistency.

Fixes #861.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-15 09:46:06 +02:00
Major Hayden
dd65cdbe03 🖼 Add image tests for aarch64
Test image builds on aarch64 for Fedora 31/32 and RHEL 8 CDN.

More work towards finishing #834.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-15 09:46:06 +02:00
Major Hayden
f2775a39b4 🤦🏻‍♂️ Fix errors from pipeline restructuring
During the last PR that restructured the pipeline, some copy/paste
errors happened and RHEL8.3 lost some tests. Also, some of the log
directories for x86/aarch64 tests overlapped.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-15 09:46:06 +02:00
Major Hayden
45c159e7f8 🦾 Add aarch64 base tests; restructure stages
Add base aarch64 tests for Fedora 31/32 and RHEL 8 CDN.

Also restructure the pipeline so that a slow distro or slow architecture
won't hold up the faster-moving tests.

This helps a bit more with #834.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-14 14:25:40 +01:00
Lars Karlitski
bc1bca1538 generate-test-cases: stop and fail when osbuild fails 2020-07-13 22:17:33 +01:00
Lars Karlitski
c8a918cefa generate-test-cases: fix whitespace 2020-07-13 22:17:33 +01:00
Major Hayden
b01bd49241 🤷🏻‍♂️ Allow tests to run on AWS and PSI
In #842, we verified that our tests run on AWS properly and we should
now let Jenkins decide where to run the tests since they work in both
places.

Remove the hardcoded AWS label and allow tests to run where they please.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-13 15:23:03 -05:00
Major Hayden
c0fc2a7e79 🚫 Temporarily disable F32 iot_commit test
Bugs are being worked out in #798 and we can disable this test until
those bugs are fixed to avoid false failures in CI.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-13 19:54:13 +01:00
Major Hayden
43895f83ca Copy the mock template during mockbuild
Copying the file allows the mock build stage to be run again.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-13 11:19:07 -05:00
Major Hayden
41e255bc8c Make qemu integration tests more reliable
Set a static network configuration using cloud-init so that we don't
have to wait a long time for DHCP to work. Also, since we know the IP
and it is set earlier, this reduces the delay in testing on systems
without nested virt (like AWS).

Fixes #844.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-13 11:19:07 -05:00
Major Hayden
db5fb9908a Do RHN registration in deploy + mockbuild
Check to see if the instance is registered prior to attempting
registration. Also, ensure that registration happens during mock build
and during the deployment.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-13 11:19:07 -05:00
Major Hayden
785bdbae30 Allow base/integration tests to run in PSI + AWS
Make it easier to handle PSI outages or those times when PSI is
overloaded with jobs by allowing base and integration tests to
run on PSI OpenStack and AWS.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-13 11:19:07 -05:00
Christian Kellner
64652d1e3e travis: remove timeout for image test
Seems to not work anyway:

Timeout (30 minutes) reached. Terminating "sudo ./osbuild-image-tests -test.v test/cases/fedora_32-x86_64-vhd-boot.json"
/home/travis/.travis/functions: line 296: kill: (10660) - Operation not permitted
2020-07-13 11:31:06 +02:00
Christian Kellner
aa03f1617a image-info: read sector size, if available
Use the sector size reported by 'sfdisk', if available, otherwise
default to the standard value of '512' (as currently assumed).
2020-07-13 11:31:06 +02:00