Commit graph

1825 commits

Author SHA1 Message Date
Ondřej Budai
449242ebda test/openstack: increase timeout for the instance to become ACTIVE
The openstack boot test often ruins our days with:

Waiting for instance 63ac19be-2e19-44e2-8bef-9770d68a190c to become Active
failed: A timeout occurred

I decided to investigate. It turns out the first boot of an image can take
up to 18 minutes. The subsequent ones are usually much faster (but don't rely
on this fact, I saw 15 minutes there).

This commit bumps the timeout to 30 minutes. This should be plenty of time
for the instance to spin up and get into the ACTIVE state.

Honestly, I'm not very happy with the solution but it should help with the
failing Schutzbot. As a follow up, I will reach to the PSI OpenStack team
and ask them if we could somehow speed up the process (maybe by using another
flavor, ci.m1.medium.ephemeral just might be slow for some reason, I don't
know).

Anyway, this should help us in the short term because I strongly believe that
a slow test is still better than a failing one.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-25 17:27:01 +00:00
Ondřej Budai
f2243151be test: remove podman-plugins workaround for RHEL/CentOS
https://bugzilla.redhat.com/show_bug.cgi?id=1877865 is now fixed and
podman-plugins is now shipped in stable RHEL. One less hack in our codebase!

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-25 12:14:56 +01:00
Ondřej Budai
ce23f97cb9 schutzbot: move koji-osbuild revdep test to Fedora 33
Fedora 32 will be soon EOL, let's prepare for it sooner rather than later.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-23 14:51:04 +01:00
Ondřej Budai
ed428879a2 schutzbot: use downstream osbuild for Fedora and RHEL 8.4
It landed in all these distros. We need to wait a bit for CentOS Stream
though.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-23 14:51:04 +01:00
Aleksandar Todorov
fbbdd2aa26 tests: Use full repo name to ubi8-minimal to workaround bug
/etc/containers/registries.conf.d/rhel-shortnames.conf shipped in
containers-common-1:1.2.2-1.module+el8.4.0+10073+30e5ea69 has a wrong
shortname for ubi8-minimal:

"ubi8-minimal" = "registry.access.redhat.com/repository/ubi8-minimal"

resulting in `name unknown: Repo not found` when trying to pull the image
via its short name.

Related: rhbz#1931785
2021-02-23 12:11:30 +01:00
David Rheinsberg
f794cb757e tools: move to new utility-container builds
Use the new tags of the `osbuild/containers` repository. The
infrastructure was simplified a lot and the new tags are much easier to
handle without any conflicts when building other images.

Note that this change uses the `latest` tag of all images. As an
alternative, we can also switch to the `latest-<date>` tags, which would
use a pinned immutable image tagged by the given date. However, these
tags are burried somewhere deep down in the ./tools/ directory and are
easy to forget upgrading. So for now use `latest` until we have better
synchronization infrastructure in place.

However, if these images ever break and we want to get back stable CI
behavior, we can always switch back to older builds by picking those
immutable tags instead of `latest`.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2021-02-23 09:47:17 +01:00
Tom Gundersen
5c20d45040 docs/news: describe the new saved_entry support
Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 20:01:15 +01:00
Tom Gundersen
9d477e339b test/manifets: update test cases
Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 20:01:15 +01:00
Tom Gundersen
0efc345b2d rhel84/grub2: set saved_entry
Explicitly set the kernel to boot into.

Also change the blueprint/kernenl handling:

Rather than only falling back to the default kernel name for
getting the package list, let GetKernel() always return the
correct result so we can rely on this being consistent.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 20:01:15 +01:00
Tom Gundersen
e4b9453d40 spec: update osbuild dependency
We now depend on osbuild 26.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 20:01:15 +01:00
Tom Gundersen
5f4a752158 schutzfile: pin osbuild dependency
In order to land a PR before osbuild reaches the stable repositories
we pin the verson to test against (which has been pushed out).

This is an exception from our usual procedures as we would otherwise
not be able to land this bug fix for RHEL8.4.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 20:01:15 +01:00
Tom Gundersen
a76f15648c schutzfile: test against osbuild 26 on RHEL 8.3
So we can update our production setups.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 20:01:15 +01:00
Martin Sehnoutka
8ca6b1ea15 28
Release osbuild-composer version 28
2021-02-20 17:56:23 +00:00
Tom Gundersen
f0b7cc0973 worker: keep output directory in /var/cache
Let's keep this on the same filesystem as the osbuild store, and
in particular stay away from /var/tmp and its scary semantics.

We are not aware of any issues caused by /var/tmp, but getting
rid of it means we don't have to think about that when debugging,
if nothing else.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Tom Gundersen
5ae5cdbf9e docs/news: describe addition to the weldr API
The compose endpoint gained a new `url` parameter for building
ostree-based images.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Tom Gundersen
e77483f007 test/ostree: add a second ostree test to test new features
We are gaining new ostree features that overlap to a great deal
with the current ones. We still need to keep the current features
for backwards compatibility, so add another test run that does the
same but using new API.

For now this simply uses the `url` parameter rather than `parent`
to build update commits. Further changes will be made in follow-up
commits.

Use `curl` rather than `composer-cli` as we have a chicken-and-egg
problem where we can't land this feature without tests, but
`composer-cli` can't add support for it without having it first in
`composer`.
2021-02-20 14:53:49 +01:00
Tom Gundersen
687ac7f615 weldr/compose/ostree: introduce URL parameter
For now this is simply used to resolve the parent commit, in case
one is not provided. In the future it will be used by new image
types to actually pull content from.

This extends the weldr API, so that future work does not have to
modify that.

The logic we now implement for the ostree commit image types is:
If the URL is provided, but the parent commit is not. The parent
commit is taken to be the current HEAD of the ostree repo at the
given url, with the given (or default) ref.

This only provides a small optional convenience, but we will
soon introduce image types where the URL of the repository is
required.

This commit still needs testing.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Tom Gundersen
6cb47862df distro: expose default OSTreeRef()
Rather than setting this automagically, expose it to the caller. For
now the only caller we have simply passes it back in, so this is a
noop.

In follow-up commits this will be used to resolve the parent commit.

This is tested by verifying that the generated manifests do not
change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Tom Gundersen
3c74ea584e gitignore: add config and OSX metadata
Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Antonio Murdaca
21319520e5 internal/distro/rhel8: un-exclude subman from edge
We aim at shrinking our deps eventually but we need subman for the time
being. This patch basically un-exclude subman which was introduced by
https://github.com/osbuild/osbuild-composer/pull/893

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2021-02-18 13:08:32 +01:00
Ondřej Budai
7f4aaa8db6 schutzbot: add mockbuild for Fedora 34
It's coming!

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-18 13:07:28 +01:00
Ondřej Budai
c2c4bf39c3 schutzfile: test against osbuild 25 on RHEL 8.3
So we can update our production setups.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-17 20:35:13 +01:00
Aleksandar Todorov
748519a104 Schutzbot: ping on nightly test results 2021-02-17 10:15:35 +01:00
Jozef Mikovic
5a9d2fe8b1 test: Add integration test for VMWare upload 2021-02-16 19:06:01 +00:00
Jozef Mikovic
177b86b3a9 docs/news: describe new upload target 2021-02-16 19:06:01 +00:00
Jozef Mikovic
0597ac48a7 upload/vmware: document uploadImage function 2021-02-16 19:06:01 +00:00
Jozef Mikovic
d497bc3b83 osbuild-worker: ensure that uploaded vmdk image has the user specified name
VMDK image has default name 'disk.vmdk' and there is no option to change the name when uploading to vSphere,
so I'm using symlink so that uploaded image has the name user specified instead of the default one.
2021-02-16 19:06:01 +00:00
Jozef Mikovic
08cfc08805 weldr: add vmware upload target to api
Extend Weldr API to allow user to specify option to upload built image to VMWare,
makes use of previously define upload target.
2021-02-16 19:06:01 +00:00
Jozef Mikovic
1a81489ef1 osbuild-worker: add target for upload to vmware
New upload target for VMWare, similar to the ones for AWS and Azure,
allowing users to set credentials for their vSphere instance.
Commit also includes function that performs the actual upload.
2021-02-16 19:06:01 +00:00
Achilleas Koutsou
d686abfffb docs/news: describe new blueprint kernel option 2021-02-16 13:51:26 +00:00
Achilleas Koutsou
8438078c50 tools: test cases for alt kernel selection
Two new test cases added to format-request-map and test cases are
generated.

1. kernel-rt for RHEL images:
Requires new package repositories for RHEL 8.3 and 8.4.
Creates an OSTree commit with the `kernel-rt` as a customization.

2. kernel-debug for Fedora images:
kernel-rt isn't included in the official fedora repositories.  Using
kernel-debug at least tests the feature with the fedora-iot-commit type.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
1ef1bab5a8 distro/*: test kernels in image types
Test that all defined image types return at least one kernel when given
an empty blueprint and exactly one kernel for ostree-commit types.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
8d9753a4c2 distro/*: remove kernel from image types
The kernel now comes from the blueprint packages even when it's not
specified.  Removing from the base packages of the image types avoids
duplication and allows for alternative kernels to be specified without
also including the default.

The latter is necessary for RHEL for Edge and Fedora IoT images (ostree
commits) that fail to build when multiple kernels are installed.

ImageType tests modified to fix expected package order.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
d0923af70d blueprint: test kernel customization
Edit GetPackages test to expect automatic kernel inclusion.

New test for all combinations of adding a kernel to customizations and
package list.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
b861042032 blueprint: add kernel name customization
Blueprints can now be used to specify a kernel as part of the kernel
customizations.  Specifying a kernel adds it to the package list.

If no known kernel is specified (neither in the customizations nor the
package list), the default "kernel" is included automatically.

If kernels are specified in both the package list and the
customizations, both are added (even if they're duplicates).
2021-02-16 13:51:26 +00:00
Ondřej Budai
a85511c6de schutzbot: fix the pipeline
https://github.com/osbuild/osbuild-composer/pull/1228 was merged with
a failing Schutzbot's pipeline. The failure is caused because `var` apparently
isn't a right Groovy syntax. Let's use the right word `def` instead.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-16 14:19:25 +01:00
Ondřej Budai
2b823b97ad ci: run the nightly test only on the main branch
Currently, all PRs trigger the daily nightly testing. Let's limit that to just
the main branch.

I found this here:
https://stackoverflow.com/questions/39168861/build-periodically-with-a-multi-branch-pipeline-in-jenkins

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-16 11:00:46 +02:00
Ondřej Budai
8760de26de test/api.sh: set the username for centos-8
This is a merge race condition. Sanne added a new variable, I added a new distro.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-15 09:30:37 +00:00
sanne
9861215ebe test: Let the composer-api testcase also boot the image in aws 2021-02-14 16:09:04 +00:00
Ondřej Budai
bf4280d1e2 distro/rhel84: disable s390x and rhel-commit-edge for Centos
s390x isn't supported on Centos.

rhel-commit-edge sounds just wrong for Centos. We can revert this change any
time. The thing is that I wasn't able to find something like CentOS IoT and
we don't want to be in a position of defining a new distribution spin.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
4492448f49 test/distro/rhel84: invert condition
I'm sorry, I need to extend this condition and my brain isn't powerful enough
to reason about complex negative conditions.

Not a functional change.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
5eb402415d distro/rhel84: add centos 8 stream support
The image definition is shared with the latest RHEL 8.y one (8.4 currently).
I expect that we the introduction of 8.5 support, we point the centos 8
distro at it.

The test repositories and manifests use the official CentOS composes. From
what I can tell, they are persistent. This is not guaranteed though, so we
might need to switch to RPMRepo at some point.

The "classic" CentOS 8 should also be buildable but due to the chicken and egg
issue (this commit will get into Centos "8.4" but Centos "8.4" isn't a thing
yet), we cannot test it and therefore it might be broken.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
6feba70758 test/distro/rhel84: add missing image types to TestImageType_Name
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
4d7b36f08b test/distro/rhel84: add test for Arch.ListImageTypes
We didn't have one so I added it.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
ab773975ea ci: do not run the unit tests with verbose on
When unit tests succeed, no one cares about them.

When unit tests fail, it's hard to find which one failed. This commit removes
the verbose flag, so it's easy to spot what failed.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-13 18:33:09 +00:00
Brian C. Lane
728f2d0f81 weldr: Remove underscores from FreezeHandler error
Error message strings are not guaranteed to be stable, but should
be consistent when used in more than one place.
2021-02-13 08:52:52 +01:00
Tomas Hozza
220c77e37f image-tests: Add RHEL-8.4 test cases for all architectures
Add RHEL-8.4 image test case manifests for all architectures. The set of
image types per architecture is based on what is currently available for
RHEL-8.3.

The image test case for RHEL-8.4 tar image on s390x is not added yet
because of Issue #1220.

Fixes #1167

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-12 18:53:18 +01:00
sanne
31eb215699 test: Fix the compose request in the api test case
Customizations should be alongside image_requests, not inside of it.
2021-02-12 18:46:02 +01:00
Achilleas Koutsou
2e98f13c90 docs/news: add ostree + kernel params news entry 2021-02-12 14:25:48 +01:00
Achilleas Koutsou
d8c292ca79 distro/*: test expected failure for ostree
Test for each distro that runs through all architecture - image type
combinations and calls the Manifest() method with a kernel boot option
customization and checks if the ostree image types produce the expected
error.
2021-02-12 14:25:48 +01:00