Commit graph

1853 commits

Author SHA1 Message Date
Aleksandar Todorov
748519a104 Schutzbot: ping on nightly test results 2021-02-17 10:15:35 +01:00
Jozef Mikovic
5a9d2fe8b1 test: Add integration test for VMWare upload 2021-02-16 19:06:01 +00:00
Jozef Mikovic
177b86b3a9 docs/news: describe new upload target 2021-02-16 19:06:01 +00:00
Jozef Mikovic
0597ac48a7 upload/vmware: document uploadImage function 2021-02-16 19:06:01 +00:00
Jozef Mikovic
d497bc3b83 osbuild-worker: ensure that uploaded vmdk image has the user specified name
VMDK image has default name 'disk.vmdk' and there is no option to change the name when uploading to vSphere,
so I'm using symlink so that uploaded image has the name user specified instead of the default one.
2021-02-16 19:06:01 +00:00
Jozef Mikovic
08cfc08805 weldr: add vmware upload target to api
Extend Weldr API to allow user to specify option to upload built image to VMWare,
makes use of previously define upload target.
2021-02-16 19:06:01 +00:00
Jozef Mikovic
1a81489ef1 osbuild-worker: add target for upload to vmware
New upload target for VMWare, similar to the ones for AWS and Azure,
allowing users to set credentials for their vSphere instance.
Commit also includes function that performs the actual upload.
2021-02-16 19:06:01 +00:00
Achilleas Koutsou
d686abfffb docs/news: describe new blueprint kernel option 2021-02-16 13:51:26 +00:00
Achilleas Koutsou
8438078c50 tools: test cases for alt kernel selection
Two new test cases added to format-request-map and test cases are
generated.

1. kernel-rt for RHEL images:
Requires new package repositories for RHEL 8.3 and 8.4.
Creates an OSTree commit with the `kernel-rt` as a customization.

2. kernel-debug for Fedora images:
kernel-rt isn't included in the official fedora repositories.  Using
kernel-debug at least tests the feature with the fedora-iot-commit type.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
1ef1bab5a8 distro/*: test kernels in image types
Test that all defined image types return at least one kernel when given
an empty blueprint and exactly one kernel for ostree-commit types.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
8d9753a4c2 distro/*: remove kernel from image types
The kernel now comes from the blueprint packages even when it's not
specified.  Removing from the base packages of the image types avoids
duplication and allows for alternative kernels to be specified without
also including the default.

The latter is necessary for RHEL for Edge and Fedora IoT images (ostree
commits) that fail to build when multiple kernels are installed.

ImageType tests modified to fix expected package order.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
d0923af70d blueprint: test kernel customization
Edit GetPackages test to expect automatic kernel inclusion.

New test for all combinations of adding a kernel to customizations and
package list.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
b861042032 blueprint: add kernel name customization
Blueprints can now be used to specify a kernel as part of the kernel
customizations.  Specifying a kernel adds it to the package list.

If no known kernel is specified (neither in the customizations nor the
package list), the default "kernel" is included automatically.

If kernels are specified in both the package list and the
customizations, both are added (even if they're duplicates).
2021-02-16 13:51:26 +00:00
Ondřej Budai
a85511c6de schutzbot: fix the pipeline
https://github.com/osbuild/osbuild-composer/pull/1228 was merged with
a failing Schutzbot's pipeline. The failure is caused because `var` apparently
isn't a right Groovy syntax. Let's use the right word `def` instead.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-16 14:19:25 +01:00
Ondřej Budai
2b823b97ad ci: run the nightly test only on the main branch
Currently, all PRs trigger the daily nightly testing. Let's limit that to just
the main branch.

I found this here:
https://stackoverflow.com/questions/39168861/build-periodically-with-a-multi-branch-pipeline-in-jenkins

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-16 11:00:46 +02:00
Ondřej Budai
8760de26de test/api.sh: set the username for centos-8
This is a merge race condition. Sanne added a new variable, I added a new distro.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-15 09:30:37 +00:00
sanne
9861215ebe test: Let the composer-api testcase also boot the image in aws 2021-02-14 16:09:04 +00:00
Ondřej Budai
bf4280d1e2 distro/rhel84: disable s390x and rhel-commit-edge for Centos
s390x isn't supported on Centos.

rhel-commit-edge sounds just wrong for Centos. We can revert this change any
time. The thing is that I wasn't able to find something like CentOS IoT and
we don't want to be in a position of defining a new distribution spin.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
4492448f49 test/distro/rhel84: invert condition
I'm sorry, I need to extend this condition and my brain isn't powerful enough
to reason about complex negative conditions.

Not a functional change.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
5eb402415d distro/rhel84: add centos 8 stream support
The image definition is shared with the latest RHEL 8.y one (8.4 currently).
I expect that we the introduction of 8.5 support, we point the centos 8
distro at it.

The test repositories and manifests use the official CentOS composes. From
what I can tell, they are persistent. This is not guaranteed though, so we
might need to switch to RPMRepo at some point.

The "classic" CentOS 8 should also be buildable but due to the chicken and egg
issue (this commit will get into Centos "8.4" but Centos "8.4" isn't a thing
yet), we cannot test it and therefore it might be broken.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
6feba70758 test/distro/rhel84: add missing image types to TestImageType_Name
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
4d7b36f08b test/distro/rhel84: add test for Arch.ListImageTypes
We didn't have one so I added it.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
ab773975ea ci: do not run the unit tests with verbose on
When unit tests succeed, no one cares about them.

When unit tests fail, it's hard to find which one failed. This commit removes
the verbose flag, so it's easy to spot what failed.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-13 18:33:09 +00:00
Brian C. Lane
728f2d0f81 weldr: Remove underscores from FreezeHandler error
Error message strings are not guaranteed to be stable, but should
be consistent when used in more than one place.
2021-02-13 08:52:52 +01:00
Tomas Hozza
220c77e37f image-tests: Add RHEL-8.4 test cases for all architectures
Add RHEL-8.4 image test case manifests for all architectures. The set of
image types per architecture is based on what is currently available for
RHEL-8.3.

The image test case for RHEL-8.4 tar image on s390x is not added yet
because of Issue #1220.

Fixes #1167

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-12 18:53:18 +01:00
sanne
31eb215699 test: Fix the compose request in the api test case
Customizations should be alongside image_requests, not inside of it.
2021-02-12 18:46:02 +01:00
Achilleas Koutsou
2e98f13c90 docs/news: add ostree + kernel params news entry 2021-02-12 14:25:48 +01:00
Achilleas Koutsou
d8c292ca79 distro/*: test expected failure for ostree
Test for each distro that runs through all architecture - image type
combinations and calls the Manifest() method with a kernel boot option
customization and checks if the ostree image types produce the expected
error.
2021-02-12 14:25:48 +01:00
Achilleas Koutsou
fd1cc343a5 distro/*: fail on kernel boot params with ostree
Kernel boot parameters have no effect on ostree type images (Fedora IoT
and RHEL for Edge).  Catch this and fail early in the pipeline creation
and communicate the issue to the user.
2021-02-12 14:25:48 +01:00
Martin Sehnoutka
f98e231f64 cloudapi: extend the API spec with /version and /openapi.json
These endpoints are useful for clients while exploring the API. They are
also required for deploying the service into clouddot.
2021-02-12 14:25:19 +01:00
Aleksandar Todorov
f56a07472a schutzbot: Use 8.4 nightly b/c latest is now 8.5 2021-02-12 14:22:04 +01:00
Aleksandar Todorov
4787fce3b7 Unstash rhel8nightly repo & json files
- even if they don't exist sometimes but ignore the errors
- make the nightly repository with a higher priority
- override rhel-8*.json files so that newly built images will
  also consume the nightly content
2021-02-12 14:22:04 +01:00
Alexander Todorov
fb8493b4df tests: Document how to manually replay nightly Pipeline 2021-02-12 14:22:04 +01:00
Alexander Todorov
4cd6b37ea4 Schutzbot nightly: download osbuild-composer-tests from Brew
because -tests.rpm isn't shipped with the distro the prepare
script downloads it from Brew, trying to match the same version
that exists in the actual nightly compose. Then prepares a repo on
S3 for the subsequent test jobs to use!

Use AWS_CREDS for ~/.s3cmd
2021-02-12 14:22:04 +01:00
Alexander Todorov
272cc434ad Schutzbot: send Telegram messages for RHEL nightly test status 2021-02-12 14:22:04 +01:00
Alexander Todorov
d08a8fce69 Use conditional steps in EL8 nightly pipeline. Refs #912
- use detect_build_cause() funtion and set a global env.BUILD_CAUSE
  variable for use in conditionals
- add a cron job trigger - this will work together with the
  GitHub pull request trigger
- use conditional blocks for all steps we want to be executed
  outside of cron jobs
- only EL8 jobs will be executed unconditionally, both in cron
  and for PRs. The preparation stage for cron jobs makes sure to
  use the same name for osbuild-mock.repo so that the jobs can
  unstash it later!
2021-02-12 14:22:04 +01:00
Jacob Kozol
2cae456094 distro/rhel84: remove dbxtool
dbxtool is not available on s390x or ppc64le. So, it cannot be included
for now without breaking image builds for these architectures.
2021-02-11 14:12:35 +01:00
Tom Gundersen
8667d446b1 news: add news entry
Adds news entry for cloudapi package selection.
2021-02-11 14:12:11 +01:00
Tom Gundersen
e513905424 cloudapi: add support for additional packages
Optionally allow a pacakge set to be included in the compose request.

The specified packages are added to the base packages before
depsolving. As the base packages differ between the image types
the package customizations may have different results on the different
images part of the compose request.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-11 14:12:11 +01:00
Aleksandar Todorov
e878752f6e tests: Add info log message for UEFI boot
makes it easier to find in the results log
2021-02-11 11:12:17 +01:00
Tomas Hozza
af3c572f9a distro: Disable RHSM DNF plugins on RHEL qcow2 images
Modify RHEL 8.3 and 8.4 KVM guest images definition to produce osbuild
manifest with `org.osbuild.rhsm` stage to disable both RHSM DNF plugins
(`product-id` and `subscription-manager`).

Update `/docs/news/unreleased/osbuild-rhsm-stage.md` to note that RHEL
8.3 and 8.4 qcow2 image definitions are updated to disable RHSM DNF
plugins by default.

Enhance `tools/image-info` tool to add RHSM-specific section to its
output in case RHSM DNF plugins configurations exist in the tree.

Regenerate all RHEL image test cases affected by the patch set.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-11 11:10:47 +01:00
Tomas Hozza
1d48c92953 Add support for org.osbuild.rhsm stage
Add support to configure `org.osbuild.rhsm` osbuild stage. This stage
allows the configuration of Red Hat Subscription Manager (RHSM) related
components. Currently it is possible to configure only the enablement
status of RHSM DNF plugins.

Add `/docs/news/unreleased/osbuild-rhsm-stage.md` with information about
the added support for `org.osbuild.rhsm` osbuild stage.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-11 11:10:47 +01:00
Achilleas Koutsou
f19e7a9c64 HACKING: Typo fixes 2021-02-11 09:37:36 +01:00
Achilleas Koutsou
444702c10b HACKING: Minor typo fix
Small typo spotted in env variable
2021-02-11 09:37:36 +01:00
Major Hayden
538f64eb67 Send webhooks without credentials
We can now send webhook data to an SQS queue at AWS without signing the
request with credentials. This allows us to trigger Schutzbot from
forks and from branches on the main repository.

Signed-off-by: Major Hayden <major@redhat.com>
2021-02-10 18:31:56 +00:00
Major Hayden
51aa1c771c Trigger schutzbot using github actions
Signed-off-by: Major Hayden <major@redhat.com>
2021-02-10 08:38:42 +01:00
Jacob Kozol
56c4922c8f docs/news: rhel84 add support for org.osbuild.sysconfig stage 2021-02-09 14:13:25 +01:00
Jacob Kozol
eea18fc897 distro/rhel84: add sysconfig stage
The org.osbuild.sysconfig stage is now supported. Config updates can be
made to the kernel and network files. Currently, the same values are
used for all image types in rhel84. The image-info script is updated to
allow testing the sysconfig info.
2021-02-09 14:13:25 +01:00
Jacob Kozol
12f8a1540c docs/news: update osbuild to version 24 2021-02-09 14:13:25 +01:00
Jacob Kozol
52823e3c43 spec: update to osbuild version 24
In order to add the newly supported sysconfig stage, the osbuild
dependency needs to be updated to version 24. The version is updated to
24 in the specfile and README. Also, for testing purposes, the Schutzfile
now points to the commit hash for osbuild version 24 for rhel-8.3 tests.
2021-02-09 14:13:25 +01:00