Commit graph

1792 commits

Author SHA1 Message Date
Achilleas Koutsou
d0923af70d blueprint: test kernel customization
Edit GetPackages test to expect automatic kernel inclusion.

New test for all combinations of adding a kernel to customizations and
package list.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
b861042032 blueprint: add kernel name customization
Blueprints can now be used to specify a kernel as part of the kernel
customizations.  Specifying a kernel adds it to the package list.

If no known kernel is specified (neither in the customizations nor the
package list), the default "kernel" is included automatically.

If kernels are specified in both the package list and the
customizations, both are added (even if they're duplicates).
2021-02-16 13:51:26 +00:00
Ondřej Budai
a85511c6de schutzbot: fix the pipeline
https://github.com/osbuild/osbuild-composer/pull/1228 was merged with
a failing Schutzbot's pipeline. The failure is caused because `var` apparently
isn't a right Groovy syntax. Let's use the right word `def` instead.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-16 14:19:25 +01:00
Ondřej Budai
2b823b97ad ci: run the nightly test only on the main branch
Currently, all PRs trigger the daily nightly testing. Let's limit that to just
the main branch.

I found this here:
https://stackoverflow.com/questions/39168861/build-periodically-with-a-multi-branch-pipeline-in-jenkins

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-16 11:00:46 +02:00
Ondřej Budai
8760de26de test/api.sh: set the username for centos-8
This is a merge race condition. Sanne added a new variable, I added a new distro.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-15 09:30:37 +00:00
sanne
9861215ebe test: Let the composer-api testcase also boot the image in aws 2021-02-14 16:09:04 +00:00
Ondřej Budai
bf4280d1e2 distro/rhel84: disable s390x and rhel-commit-edge for Centos
s390x isn't supported on Centos.

rhel-commit-edge sounds just wrong for Centos. We can revert this change any
time. The thing is that I wasn't able to find something like CentOS IoT and
we don't want to be in a position of defining a new distribution spin.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
4492448f49 test/distro/rhel84: invert condition
I'm sorry, I need to extend this condition and my brain isn't powerful enough
to reason about complex negative conditions.

Not a functional change.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
5eb402415d distro/rhel84: add centos 8 stream support
The image definition is shared with the latest RHEL 8.y one (8.4 currently).
I expect that we the introduction of 8.5 support, we point the centos 8
distro at it.

The test repositories and manifests use the official CentOS composes. From
what I can tell, they are persistent. This is not guaranteed though, so we
might need to switch to RPMRepo at some point.

The "classic" CentOS 8 should also be buildable but due to the chicken and egg
issue (this commit will get into Centos "8.4" but Centos "8.4" isn't a thing
yet), we cannot test it and therefore it might be broken.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
6feba70758 test/distro/rhel84: add missing image types to TestImageType_Name
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
4d7b36f08b test/distro/rhel84: add test for Arch.ListImageTypes
We didn't have one so I added it.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
ab773975ea ci: do not run the unit tests with verbose on
When unit tests succeed, no one cares about them.

When unit tests fail, it's hard to find which one failed. This commit removes
the verbose flag, so it's easy to spot what failed.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-13 18:33:09 +00:00
Brian C. Lane
728f2d0f81 weldr: Remove underscores from FreezeHandler error
Error message strings are not guaranteed to be stable, but should
be consistent when used in more than one place.
2021-02-13 08:52:52 +01:00
Tomas Hozza
220c77e37f image-tests: Add RHEL-8.4 test cases for all architectures
Add RHEL-8.4 image test case manifests for all architectures. The set of
image types per architecture is based on what is currently available for
RHEL-8.3.

The image test case for RHEL-8.4 tar image on s390x is not added yet
because of Issue #1220.

Fixes #1167

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-12 18:53:18 +01:00
sanne
31eb215699 test: Fix the compose request in the api test case
Customizations should be alongside image_requests, not inside of it.
2021-02-12 18:46:02 +01:00
Achilleas Koutsou
2e98f13c90 docs/news: add ostree + kernel params news entry 2021-02-12 14:25:48 +01:00
Achilleas Koutsou
d8c292ca79 distro/*: test expected failure for ostree
Test for each distro that runs through all architecture - image type
combinations and calls the Manifest() method with a kernel boot option
customization and checks if the ostree image types produce the expected
error.
2021-02-12 14:25:48 +01:00
Achilleas Koutsou
fd1cc343a5 distro/*: fail on kernel boot params with ostree
Kernel boot parameters have no effect on ostree type images (Fedora IoT
and RHEL for Edge).  Catch this and fail early in the pipeline creation
and communicate the issue to the user.
2021-02-12 14:25:48 +01:00
Martin Sehnoutka
f98e231f64 cloudapi: extend the API spec with /version and /openapi.json
These endpoints are useful for clients while exploring the API. They are
also required for deploying the service into clouddot.
2021-02-12 14:25:19 +01:00
Aleksandar Todorov
f56a07472a schutzbot: Use 8.4 nightly b/c latest is now 8.5 2021-02-12 14:22:04 +01:00
Aleksandar Todorov
4787fce3b7 Unstash rhel8nightly repo & json files
- even if they don't exist sometimes but ignore the errors
- make the nightly repository with a higher priority
- override rhel-8*.json files so that newly built images will
  also consume the nightly content
2021-02-12 14:22:04 +01:00
Alexander Todorov
fb8493b4df tests: Document how to manually replay nightly Pipeline 2021-02-12 14:22:04 +01:00
Alexander Todorov
4cd6b37ea4 Schutzbot nightly: download osbuild-composer-tests from Brew
because -tests.rpm isn't shipped with the distro the prepare
script downloads it from Brew, trying to match the same version
that exists in the actual nightly compose. Then prepares a repo on
S3 for the subsequent test jobs to use!

Use AWS_CREDS for ~/.s3cmd
2021-02-12 14:22:04 +01:00
Alexander Todorov
272cc434ad Schutzbot: send Telegram messages for RHEL nightly test status 2021-02-12 14:22:04 +01:00
Alexander Todorov
d08a8fce69 Use conditional steps in EL8 nightly pipeline. Refs #912
- use detect_build_cause() funtion and set a global env.BUILD_CAUSE
  variable for use in conditionals
- add a cron job trigger - this will work together with the
  GitHub pull request trigger
- use conditional blocks for all steps we want to be executed
  outside of cron jobs
- only EL8 jobs will be executed unconditionally, both in cron
  and for PRs. The preparation stage for cron jobs makes sure to
  use the same name for osbuild-mock.repo so that the jobs can
  unstash it later!
2021-02-12 14:22:04 +01:00
Jacob Kozol
2cae456094 distro/rhel84: remove dbxtool
dbxtool is not available on s390x or ppc64le. So, it cannot be included
for now without breaking image builds for these architectures.
2021-02-11 14:12:35 +01:00
Tom Gundersen
8667d446b1 news: add news entry
Adds news entry for cloudapi package selection.
2021-02-11 14:12:11 +01:00
Tom Gundersen
e513905424 cloudapi: add support for additional packages
Optionally allow a pacakge set to be included in the compose request.

The specified packages are added to the base packages before
depsolving. As the base packages differ between the image types
the package customizations may have different results on the different
images part of the compose request.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-11 14:12:11 +01:00
Aleksandar Todorov
e878752f6e tests: Add info log message for UEFI boot
makes it easier to find in the results log
2021-02-11 11:12:17 +01:00
Tomas Hozza
af3c572f9a distro: Disable RHSM DNF plugins on RHEL qcow2 images
Modify RHEL 8.3 and 8.4 KVM guest images definition to produce osbuild
manifest with `org.osbuild.rhsm` stage to disable both RHSM DNF plugins
(`product-id` and `subscription-manager`).

Update `/docs/news/unreleased/osbuild-rhsm-stage.md` to note that RHEL
8.3 and 8.4 qcow2 image definitions are updated to disable RHSM DNF
plugins by default.

Enhance `tools/image-info` tool to add RHSM-specific section to its
output in case RHSM DNF plugins configurations exist in the tree.

Regenerate all RHEL image test cases affected by the patch set.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-11 11:10:47 +01:00
Tomas Hozza
1d48c92953 Add support for org.osbuild.rhsm stage
Add support to configure `org.osbuild.rhsm` osbuild stage. This stage
allows the configuration of Red Hat Subscription Manager (RHSM) related
components. Currently it is possible to configure only the enablement
status of RHSM DNF plugins.

Add `/docs/news/unreleased/osbuild-rhsm-stage.md` with information about
the added support for `org.osbuild.rhsm` osbuild stage.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-11 11:10:47 +01:00
Achilleas Koutsou
f19e7a9c64 HACKING: Typo fixes 2021-02-11 09:37:36 +01:00
Achilleas Koutsou
444702c10b HACKING: Minor typo fix
Small typo spotted in env variable
2021-02-11 09:37:36 +01:00
Major Hayden
538f64eb67 Send webhooks without credentials
We can now send webhook data to an SQS queue at AWS without signing the
request with credentials. This allows us to trigger Schutzbot from
forks and from branches on the main repository.

Signed-off-by: Major Hayden <major@redhat.com>
2021-02-10 18:31:56 +00:00
Major Hayden
51aa1c771c Trigger schutzbot using github actions
Signed-off-by: Major Hayden <major@redhat.com>
2021-02-10 08:38:42 +01:00
Jacob Kozol
56c4922c8f docs/news: rhel84 add support for org.osbuild.sysconfig stage 2021-02-09 14:13:25 +01:00
Jacob Kozol
eea18fc897 distro/rhel84: add sysconfig stage
The org.osbuild.sysconfig stage is now supported. Config updates can be
made to the kernel and network files. Currently, the same values are
used for all image types in rhel84. The image-info script is updated to
allow testing the sysconfig info.
2021-02-09 14:13:25 +01:00
Jacob Kozol
12f8a1540c docs/news: update osbuild to version 24 2021-02-09 14:13:25 +01:00
Jacob Kozol
52823e3c43 spec: update to osbuild version 24
In order to add the newly supported sysconfig stage, the osbuild
dependency needs to be updated to version 24. The version is updated to
24 in the specfile and README. Also, for testing purposes, the Schutzfile
now points to the commit hash for osbuild version 24 for rhel-8.3 tests.
2021-02-09 14:13:25 +01:00
Achilleas Koutsou
a7d982d9be image-tests: fix hostname typo in customizations
The test case generation script contained a typo in the blueprint
customizations dictionary: "hosname" instead of "hostname".

Fixed typo and regenerated relevant test cases.
2021-02-08 09:11:07 +01:00
Chloe Kaubisch
899d78f7e1
cloudapi: expose upload status
Expose a more detailed job status result - specifically, include upload status
alongside image status. Expand openapi.yml accordingly and add an UploadStatus
field to the OSBuildJobResult struct. At the moment, only represent the
"success" and "failure" states of UploadStatus - to differentiate between
"pending" and "running" would involve significant design decisions and should be
addressed in a separate commit.
2021-02-05 12:34:28 +01:00
Brian C. Lane
80f833a69b weldr: Fix confusing TOML support for source API
There is some confusion surrounding the format of the source TOML that
can be sent to the server. The format it accepts doesn't match the
output from composer-cli which includes the source id in [] eg.

[k8s]
name = "kubernetes packages"
...

This patch changes the parsing to allow the id to be set as 'id = "k8s"'
or passed as a map in [k8s]. If the id is passed in the body it takes
precedence over the map name.
2021-02-04 15:50:04 -08:00
Brian C. Lane
6bfa7a3e0e weldr: Add tests for V1 source API and better id support 2021-02-04 15:50:04 -08:00
Ondřej Budai
40fc8a8734 27
Release osbuild-composer version 27

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-04 15:44:39 +01:00
Ondřej Budai
6456fbe6a5 image-info: do not include inputhash in the report
The algorithm from calculating changed in osbuild 24, thus breaking the
testsuite. As the the inputhash is merely a implementation detail, there's
no need to test it in the image tests. Nevertheless, the inputhash is also
tested in the osbuild's testsuite.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-04 13:39:23 +01:00
Ondřej Budai
2b41190cf3 go.mod: bump azure-storage-blob-go/azblob
Due to https://github.com/Azure/azure-storage-blob-go/issues/236 , we had to
use a weird version of this library (see 1b051922).

A new release came out yesterday that's tagged correctly so let's use it
so we can remove the hack from go.mod.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-03 12:36:04 +01:00
Brian C. Lane
440753a274 weldr: undo unknown commit should return UnknownCommit 2021-02-03 12:23:58 +01:00
Brian C. Lane
98b2dab128 weldr: Add tests for blueprints/undo
Test undo, and undo of an unknown commit.
2021-02-03 12:23:58 +01:00
Tomas Hozza
0d1c2bc0bf image-tests: fixup for rhel-8 aarch64 edge test case after PR#1178
Tom was too quick to merge PR#1178 and as a result, one image test case
was not properly regenerated in that PR. The CI passed for PR#1178,
because RHEL-8.3 images in CI are not tested on aarch64.

Add properly regenerated image test case for rhel-8 aarch64 edge commit.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-02 11:40:13 +01:00
Tomas Hozza
89df6ab328 image-tests: remove 'subscription' customization, update repos, regenerate
Remove 'subscription' customization from 'format-request-map.json',
because it currently has no effect on image test cases. The reason is
that this customization of image is possible only via Cloud API in
composer and setting imageOptions. However image test cases currently
support image customization only via Blueprint customizations.

Update repo links in 'repos.json' to the latest rpmrepo snapshots.

Regenerate all image test cases affected by updates to 'repos.json'
and 'format-request-map.json'. This includes all `rhel-8` and `rhel-84`
image test cases.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-01 23:20:01 +00:00