Commit graph

1311 commits

Author SHA1 Message Date
Major Hayden
edc574065a 🤸🏻 Bump osbuild to v19
Update the osbuild submodule so that osbuild-composer is tested against
osbuild 19.

Signed-off-by: Major Hayden <major@redhat.com>
2020-08-02 11:13:41 +01:00
Major Hayden
8da3d58213 mockbuild: Remove rsync
Although rsync works just fine, it would reduce our package list if we
just used `cp` to create the latest repo.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-30 18:15:38 +02:00
Major Hayden
fe1777102e 🏠 Keep RHEL 8 image tests internal only
The image tests for RHEL 8 still download packages from a server that is
only available inside the Red Hat network. 😢

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-30 18:15:17 +02:00
Major Hayden
a40f93e046 🦸 RHEL 8.3: Switch to production CDN URL
Use the production CDN URL now that RHEL 8.3 beta content is available.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-30 07:58:08 -05:00
Major Hayden
17075cffa0 📦 mockbuild: Create 'latest' repo for branch
Allow anyone to set up a dnf repository to point to the latest
build for the branch by using a repo file pointing to that build.

This would allow anyone to get the latest set of RPMs for a particular
pull request or branch (especially the master branch.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-30 13:31:01 +02:00
Major Hayden
484bb9a182 🐣 Use RHEL 8.3 beta content from CDN
Remove the nightly repos from the RHEL 8.3 jobs and replace them with
CDN registration for beta content.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-29 13:25:29 -05:00
Major Hayden
61e6e5f500 Remove old testing YAML
Signed-off-by: Major Hayden <major@redhat.com>
2020-07-29 08:44:21 +02:00
Brian C. Lane
7ca9579487 client: Add an integration test for a bad blueprint depsolve
This test makes sure that a bad /blueprints/depsolve/... will return a
list of blueprints and a list of errors, not just a single error 400
response.
2020-07-28 08:18:56 +02:00
Brian C. Lane
eb76b9ec8e weldr: Fix the bad blueprint depsolve response
This changes the response to match lorax-composer's behavior. If any of
the blueprints in the list passed to /blueprints/depsolve/... have an
error that error should be appended to the error list, and the blueprint
included in the blueprints list with an empty dependencies section.

It was returning an error 400 and a single error if it hit any depsolve
problems, skipping any other blueprints and returning the wrong
response.

This also adjusts the tests to account for the change.

Fixes #890
2020-07-28 08:18:56 +02:00
Major Hayden
3d538a6663 Add Lorax TOML workaround
Copy the TOML workaround from the aws script to the qemu script.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
574c4f622c Update test cases
Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
3daa751095 distro/rhel8: increase default image size to 4GB
RHEL 8.3 requires more than 2GB to install on aarch64/ppc64le.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
f2d0f5e6f6 aarch64: Remove updates repo from test cases 2020-07-27 10:27:15 -05:00
Major Hayden
5ef678a57b aarch64: workaround qemu-convert CPU bug
The conversion with `qemu-img convert` often fails on aarch64 systems
with LOTS of CPUs. This is fixed in RHEL 8 for aarch64, but not in
Fedora.

Set the maximum coroutines to 1 to avoid this issue until the bug is
fixed.

Bug: https://bugs.launchpad.net/qemu/+bug/1805256

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
48fdc3d831 test case repos: use RHEL 8.3 Beta 1
Switch to RHEL 8.3 Beta 1 for generating test cases.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
f326760971 Remove nvram when undefining VM on aarch64 2020-07-27 10:27:15 -05:00
Major Hayden
7f78502046 s390x: wait longer for smoke test check 2020-07-27 10:27:15 -05:00
Major Hayden
0d56d284d9 Work around ppc64le virt issues 2020-07-27 10:27:15 -05:00
Major Hayden
248db2be0e 🔐 Add OpenStack credentials for image tests
We want to deliver images directly to OpenStack during the CI tests.
Ensure the credentials are available during those tests and move them to
the proper location to be read by OpenStack tools and SDKs.

This helps with #876.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-22 20:02:12 +03:00
Ondřej Budai
5ab8b59803 18
Release osbuild-composer version 18.
2020-07-22 17:04:23 +02:00
Martin Sehnoutka
3fc03503a5 distro/f32: manually relabel cp in buildroot with install_t
By labeling `cp` with `system_u:object_r:install_exec_t:s0` we allow it
to copy labels unknown to the host.

See also corresponding commit in osbuild:
e80130a830
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
f8f35016d6 osbuild: Add "labels" field into the SELinux stage
The SELinux stage supports labeling specific files with specific labels
like this:
7b0db90c76/stages/org.osbuild.selinux (L37)

This can be useful for preventing some SELinux issues during the image
build. You can find example usage here:
7b0db90c76/test/data/stages/selinux/test_basic.json (L5)

And more reasoning here:
e80130a830
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
636b761b6b distro/f32: add SELinux RPM to build pipeline
This patch adds SELinux stage to build pipeline for f32 just like
we do for RHEL8. This helps with issues like this:
https://github.com/osbuild/osbuild-composer/issues/798
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
12172adf47 test-case-generators: refactor customizations
This patch contains refactoring of the part where we handle
customizations. Previously customizations were considered an "image
type" which is not obvious. Now it is a command line switch.
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
60269abdff distro: print more information in case the test fails
This helps when looking for the issue because the previous input did not
mention which specific qcow2 test case failed.
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
a38ee2e7a7 test-case-generators: fix printing of osbuild log
Writing to a file followed by reading it results in printing b'' unless
we seek to the beginning. Fix it by calling seek.
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
09208c9237 spec: bump osbuild requirement to v18
SELinux labels are only supported since this version.
2020-07-22 10:47:46 +01:00
Major Hayden
5d902260b0 Fix post-build telegram notification
Jenkins needs to know which node to use to send the telegram
notification.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-20 10:00:41 -05:00
chloenayon
ae3fb5e1c5 distro: enable cloud-init services for openstack and qcow2 images for fedora
Previously, cloud-init was installed on qcow and openstack images but was not
enabled, preventing a user from logging in to these images. Cloud-init, cloud-final,
cloud-init-local and cloud-config services are enabled to make cloud-init work
and to maintain consistency with official fedora 31 and fedora 32 images.

The image tests have been updated to reflect these changes.

Fixes #698
2020-07-18 11:57:05 +02:00
Lars Karlitski
f38d55da88 distro/tests: use test case filename as test name
These tests used the image type as test name, which is ambiguous. Use
the file name for the test case instead.
2020-07-18 10:52:11 +01:00
Major Hayden
460369b32b 💣 Notify via telegram on master CI failure
We've come a long way and we need to triage failures that occur during
CI for the master branch. This will help us find problems with CI as
well as find other issues that could show up in a customer environment.

Also, let's send a happy notification when everything goes well. 💚

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-17 13:57:30 -05:00
Tom Gundersen
1280035f8f submodule: bump to the most recent release
Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 19:13:15 +01:00
Tom Gundersen
fbfa191c81 rcm: drop sub-package
The osbuild-composer-rcm package was never finished, not in use and will be replaced by osbulid-composer-koji.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 19:13:15 +01:00
Tom Gundersen
79d27ded25 image-tests: avoid /var/tmp
`systemd-tmpfiles` will helpfully delete "old"  files in /var/tmp at regular
intervals. The files installed from rpm has the timestamps from when they
were packaged, which causes some to be cleaned up when the timer triggers.

The first timer triggers 15 minutes after boot, so we were sometimes hit
by this when our CI was under load.

Fixes #839 and #862.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 19:13:15 +01:00
Major Hayden
842b665659
Improve emoji selections 2020-07-17 11:46:12 -05:00
Major Hayden
20fd6521fb 🏇🏻 Speed up pipeline with parallelism (again)
Now that we're going about multi-arch testing in a different way, we can
restore some of the old parallelism that we used to have in the pipeline
so it will finish faster.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-17 11:43:54 -05:00
Major Hayden
60aafdaaab 📤 Temporarily disable aarch64 tests
The aarch64 tests are fairly unstable right now and they need work on
out of space issues (#870) and test cases (#861).

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-17 09:24:12 -05:00
David Rheinsberg
cdcf212a52 spec: pull in systemd-nspawn
The composer-tests use systemd-nspawn, so make sure the `*-tests` RPM
depends on `systemd-container`.

Right now this dependency is recursively fulfilled by the `osbuild`
RPM. However, this might change if we switch to `bubblewrap` as a
sandbox. Furthermore, we should not fulfill our dependencies
recursively, but list them directly.
2020-07-17 07:42:27 -05:00
Ondřej Budai
2beb4e0a2b remove unneeded fedora 30 repositories
Fedora 30 support was dropped in e66368b0, this file has had no use since
then. This commit removes it.
2020-07-17 09:41:32 +02:00
Tom Gundersen
bca53a596c crypt: add automatic darwin support
Currently one would have to pass a `macos` flag explicitly to build on
OSX, but if we replace `macos` with `darwin` this happens automatically.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 07:48:22 +02:00
Ondřej Budai
813275b6c4 drop Travis CI
All tests were moved to schutzbot, there's no need to run them on Travis.
2020-07-16 12:45:54 -05:00
Ondřej Budai
693b4f0823 schutzbot: make image tests upload vhds to Azure
Support for uploading was already in, it's just needed to pass the right
environment variables.
2020-07-16 14:00:15 +02:00
Ondřej Budai
b4a57225a7 api/weldr: add missing modtime to tar returned by /compose/logs
This file was surely not created on 1970-01-01, this commit sets the ModTime
to the current datetime.
2020-07-15 08:58:25 +01:00
Ondřej Budai
454e471af8 api/weldr: bring tars back from the future
By default, go's tar archiver uses USTAR header format. Unfortunately, this
format doesn't support sub-second resolution for ModTime. Go solves this by
*rounding* the time. Sometimes, this creates an archive containing a file
with modtime from the future. When such archive is untarred by GNU tar,
the following message is produced:

tar: bf548dfd-0a90-40e6-bbf2-dcdd82fcbb4e.json: time stamp 2020-07-13
13:34:31 is 0.356223173 s in the future

We have two options here:

1) Use gnu header format that supports sub-second resolution. Unfortunately,
   it seems that not all tar archivers support this format (e.g. 7-zip).

2) The other option is to truncate the date (instead of rounding).

I went with option 2.

Also, this commit adds a test to check that the header is not from the future.
Without this fix, the test is actually failing, I verified this manually.

Fixes #854
2020-07-15 08:58:25 +01:00
Major Hayden
4b0409df5b Update Fedora 31/32 aarch test cases
Add Fedora 32 test cases with the base OS + updates (to work around
aarch64 bugs in the original release). Add updates for Fedora 31 for
consistency.

Fixes #861.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-15 09:46:06 +02:00
Major Hayden
dd65cdbe03 🖼 Add image tests for aarch64
Test image builds on aarch64 for Fedora 31/32 and RHEL 8 CDN.

More work towards finishing #834.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-15 09:46:06 +02:00
Major Hayden
f2775a39b4 🤦🏻‍♂️ Fix errors from pipeline restructuring
During the last PR that restructured the pipeline, some copy/paste
errors happened and RHEL8.3 lost some tests. Also, some of the log
directories for x86/aarch64 tests overlapped.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-15 09:46:06 +02:00
Major Hayden
45c159e7f8 🦾 Add aarch64 base tests; restructure stages
Add base aarch64 tests for Fedora 31/32 and RHEL 8 CDN.

Also restructure the pipeline so that a slow distro or slow architecture
won't hold up the faster-moving tests.

This helps a bit more with #834.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-14 14:25:40 +01:00
Lars Karlitski
bc1bca1538 generate-test-cases: stop and fail when osbuild fails 2020-07-13 22:17:33 +01:00
Lars Karlitski
c8a918cefa generate-test-cases: fix whitespace 2020-07-13 22:17:33 +01:00