Commit graph

1326 commits

Author SHA1 Message Date
Jenn Giardino
bf10ed0fd1 readme: Update API diagram 2020-08-08 00:25:06 +02:00
Alexander Todorov
adc384002d Document which ENV variables are needed for testing in OpenStack 2020-08-07 19:58:37 +02:00
Alexander Todorov
b136332c7f Provide OpenStack environment for image tests
in the same way this is done for Azure we read the OS credentials
and provide them as ENV variables for the test

Note: uses `psi-openstack-creds` and not `psi-openstack-clouds-yaml`
which is a yaml file.
2020-08-07 19:58:37 +02:00
Alexander Todorov
0469be7a22 tests: Change boot typo to openstack. Refs #876
this will cause the images to be uploaded and booted in our
OpenStack cluster.
2020-08-07 19:58:37 +02:00
Brian C. Lane
55136323c5 store: Add newSourceConfigsFromV0 and newSourcesV0 tests 2020-08-06 15:25:02 +02:00
Brian C. Lane
542dfc4ec2 store: Add newChangesFromV0 and newChangesV0 tests 2020-08-06 15:25:02 +02:00
Brian C. Lane
79b84fe52f store: Add newWorkspaceFromV0 and newWorkspaceV0 tests 2020-08-06 15:25:02 +02:00
Brian C. Lane
f0e2f01421 store: Add newBlueprintsFromV0 and newVlueprintsV0 tests 2020-08-06 15:25:02 +02:00
Brian C. Lane
45f3901a6a store: Add newCommitsV0 tests 2020-08-06 15:25:02 +02:00
Major Hayden
a93143ea0c mockbuild: Fix path to latest repo
Using `cp` to copy the repo content caused the last part of the path to
be duplicated:

  Current: `master/latest/rhel82_x86_64/rhel82_x86_64/repodata/repomd.xml`
  Desired: `master/latest/rhel82_x86_64/repodata/repomd.xml`

Remove the VERSION_ID/ARCH from the destination to remove the duplicated
path.

Signed-off-by: Major Hayden <major@redhat.com>
2020-08-06 00:15:38 +02:00
Brian C. Lane
be2ce10b75 weldr: Fix out of range index for missing toml blueprint freeze
The API was crashing if the freeze request was called on a non-existent
blueprint. This changes it to return an empty string, matching
lorax-composer's behavior (since the output is toml it shouldn't return
json).
2020-08-05 11:43:35 +02:00
Martin Sehnoutka
55c86afaa6 test cases: fix f32 iot commit inputhash
The inputhash represents pipeline id, which is calculated from the
pipeline dictionary defined in the test case. The definition has been
modified without correction of the inputhash (because we did not run it
in our CI :-) ).

This patch fixed the inputhash to reflect the changes in the pipeline.
2020-08-03 17:46:49 +02:00
Martin Sehnoutka
b6e1e8b4ce schutzbot: don't skip image test for fedora iot commit
The SELinux bug has been fixed so there is no need to skip it.
2020-08-03 17:46:49 +02:00
Brian C. Lane
ec19003b70 store: Fix copy failure when initializing blueprints commit list
copy() will not append to a slice, so if the length is 0 nothing will be
copied. Initialize the slice with the length instead of the capacity.

This fixes a problem with blueprint commits vanishing when the server is
restarted.

This commit also includes a new test in json_test for the
newCommitsFromV0() function.
2020-08-02 11:21:20 +01:00
Major Hayden
093117559e 🦘 Skip checkout on prepare
Checking out the code from git during the prepare step is a waste of
time and disk space since we don't need the git repository cloned there.

Signed-off-by: Major Hayden <major@redhat.com>
2020-08-02 11:17:11 +01:00
Major Hayden
edc574065a 🤸🏻 Bump osbuild to v19
Update the osbuild submodule so that osbuild-composer is tested against
osbuild 19.

Signed-off-by: Major Hayden <major@redhat.com>
2020-08-02 11:13:41 +01:00
Major Hayden
8da3d58213 mockbuild: Remove rsync
Although rsync works just fine, it would reduce our package list if we
just used `cp` to create the latest repo.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-30 18:15:38 +02:00
Major Hayden
fe1777102e 🏠 Keep RHEL 8 image tests internal only
The image tests for RHEL 8 still download packages from a server that is
only available inside the Red Hat network. 😢

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-30 18:15:17 +02:00
Major Hayden
a40f93e046 🦸 RHEL 8.3: Switch to production CDN URL
Use the production CDN URL now that RHEL 8.3 beta content is available.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-30 07:58:08 -05:00
Major Hayden
17075cffa0 📦 mockbuild: Create 'latest' repo for branch
Allow anyone to set up a dnf repository to point to the latest
build for the branch by using a repo file pointing to that build.

This would allow anyone to get the latest set of RPMs for a particular
pull request or branch (especially the master branch.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-30 13:31:01 +02:00
Major Hayden
484bb9a182 🐣 Use RHEL 8.3 beta content from CDN
Remove the nightly repos from the RHEL 8.3 jobs and replace them with
CDN registration for beta content.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-29 13:25:29 -05:00
Major Hayden
61e6e5f500 Remove old testing YAML
Signed-off-by: Major Hayden <major@redhat.com>
2020-07-29 08:44:21 +02:00
Brian C. Lane
7ca9579487 client: Add an integration test for a bad blueprint depsolve
This test makes sure that a bad /blueprints/depsolve/... will return a
list of blueprints and a list of errors, not just a single error 400
response.
2020-07-28 08:18:56 +02:00
Brian C. Lane
eb76b9ec8e weldr: Fix the bad blueprint depsolve response
This changes the response to match lorax-composer's behavior. If any of
the blueprints in the list passed to /blueprints/depsolve/... have an
error that error should be appended to the error list, and the blueprint
included in the blueprints list with an empty dependencies section.

It was returning an error 400 and a single error if it hit any depsolve
problems, skipping any other blueprints and returning the wrong
response.

This also adjusts the tests to account for the change.

Fixes #890
2020-07-28 08:18:56 +02:00
Major Hayden
3d538a6663 Add Lorax TOML workaround
Copy the TOML workaround from the aws script to the qemu script.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
574c4f622c Update test cases
Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
3daa751095 distro/rhel8: increase default image size to 4GB
RHEL 8.3 requires more than 2GB to install on aarch64/ppc64le.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
f2d0f5e6f6 aarch64: Remove updates repo from test cases 2020-07-27 10:27:15 -05:00
Major Hayden
5ef678a57b aarch64: workaround qemu-convert CPU bug
The conversion with `qemu-img convert` often fails on aarch64 systems
with LOTS of CPUs. This is fixed in RHEL 8 for aarch64, but not in
Fedora.

Set the maximum coroutines to 1 to avoid this issue until the bug is
fixed.

Bug: https://bugs.launchpad.net/qemu/+bug/1805256

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
48fdc3d831 test case repos: use RHEL 8.3 Beta 1
Switch to RHEL 8.3 Beta 1 for generating test cases.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Major Hayden
f326760971 Remove nvram when undefining VM on aarch64 2020-07-27 10:27:15 -05:00
Major Hayden
7f78502046 s390x: wait longer for smoke test check 2020-07-27 10:27:15 -05:00
Major Hayden
0d56d284d9 Work around ppc64le virt issues 2020-07-27 10:27:15 -05:00
Major Hayden
248db2be0e 🔐 Add OpenStack credentials for image tests
We want to deliver images directly to OpenStack during the CI tests.
Ensure the credentials are available during those tests and move them to
the proper location to be read by OpenStack tools and SDKs.

This helps with #876.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-22 20:02:12 +03:00
Ondřej Budai
5ab8b59803 18
Release osbuild-composer version 18.
2020-07-22 17:04:23 +02:00
Martin Sehnoutka
3fc03503a5 distro/f32: manually relabel cp in buildroot with install_t
By labeling `cp` with `system_u:object_r:install_exec_t:s0` we allow it
to copy labels unknown to the host.

See also corresponding commit in osbuild:
e80130a830
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
f8f35016d6 osbuild: Add "labels" field into the SELinux stage
The SELinux stage supports labeling specific files with specific labels
like this:
7b0db90c76/stages/org.osbuild.selinux (L37)

This can be useful for preventing some SELinux issues during the image
build. You can find example usage here:
7b0db90c76/test/data/stages/selinux/test_basic.json (L5)

And more reasoning here:
e80130a830
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
636b761b6b distro/f32: add SELinux RPM to build pipeline
This patch adds SELinux stage to build pipeline for f32 just like
we do for RHEL8. This helps with issues like this:
https://github.com/osbuild/osbuild-composer/issues/798
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
12172adf47 test-case-generators: refactor customizations
This patch contains refactoring of the part where we handle
customizations. Previously customizations were considered an "image
type" which is not obvious. Now it is a command line switch.
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
60269abdff distro: print more information in case the test fails
This helps when looking for the issue because the previous input did not
mention which specific qcow2 test case failed.
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
a38ee2e7a7 test-case-generators: fix printing of osbuild log
Writing to a file followed by reading it results in printing b'' unless
we seek to the beginning. Fix it by calling seek.
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
09208c9237 spec: bump osbuild requirement to v18
SELinux labels are only supported since this version.
2020-07-22 10:47:46 +01:00
Major Hayden
5d902260b0 Fix post-build telegram notification
Jenkins needs to know which node to use to send the telegram
notification.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-20 10:00:41 -05:00
chloenayon
ae3fb5e1c5 distro: enable cloud-init services for openstack and qcow2 images for fedora
Previously, cloud-init was installed on qcow and openstack images but was not
enabled, preventing a user from logging in to these images. Cloud-init, cloud-final,
cloud-init-local and cloud-config services are enabled to make cloud-init work
and to maintain consistency with official fedora 31 and fedora 32 images.

The image tests have been updated to reflect these changes.

Fixes #698
2020-07-18 11:57:05 +02:00
Lars Karlitski
f38d55da88 distro/tests: use test case filename as test name
These tests used the image type as test name, which is ambiguous. Use
the file name for the test case instead.
2020-07-18 10:52:11 +01:00
Major Hayden
460369b32b 💣 Notify via telegram on master CI failure
We've come a long way and we need to triage failures that occur during
CI for the master branch. This will help us find problems with CI as
well as find other issues that could show up in a customer environment.

Also, let's send a happy notification when everything goes well. 💚

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-17 13:57:30 -05:00
Tom Gundersen
1280035f8f submodule: bump to the most recent release
Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 19:13:15 +01:00
Tom Gundersen
fbfa191c81 rcm: drop sub-package
The osbuild-composer-rcm package was never finished, not in use and will be replaced by osbulid-composer-koji.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 19:13:15 +01:00
Tom Gundersen
79d27ded25 image-tests: avoid /var/tmp
`systemd-tmpfiles` will helpfully delete "old"  files in /var/tmp at regular
intervals. The files installed from rpm has the timestamps from when they
were packaged, which causes some to be cleaned up when the timer triggers.

The first timer triggers 15 minutes after boot, so we were sometimes hit
by this when our CI was under load.

Fixes #839 and #862.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-07-17 19:13:15 +01:00
Major Hayden
842b665659
Improve emoji selections 2020-07-17 11:46:12 -05:00