Commit graph

1816 commits

Author SHA1 Message Date
Tom Gundersen
e4b9453d40 spec: update osbuild dependency
We now depend on osbuild 26.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 20:01:15 +01:00
Tom Gundersen
5f4a752158 schutzfile: pin osbuild dependency
In order to land a PR before osbuild reaches the stable repositories
we pin the verson to test against (which has been pushed out).

This is an exception from our usual procedures as we would otherwise
not be able to land this bug fix for RHEL8.4.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 20:01:15 +01:00
Tom Gundersen
a76f15648c schutzfile: test against osbuild 26 on RHEL 8.3
So we can update our production setups.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 20:01:15 +01:00
Martin Sehnoutka
8ca6b1ea15 28
Release osbuild-composer version 28
2021-02-20 17:56:23 +00:00
Tom Gundersen
f0b7cc0973 worker: keep output directory in /var/cache
Let's keep this on the same filesystem as the osbuild store, and
in particular stay away from /var/tmp and its scary semantics.

We are not aware of any issues caused by /var/tmp, but getting
rid of it means we don't have to think about that when debugging,
if nothing else.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Tom Gundersen
5ae5cdbf9e docs/news: describe addition to the weldr API
The compose endpoint gained a new `url` parameter for building
ostree-based images.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Tom Gundersen
e77483f007 test/ostree: add a second ostree test to test new features
We are gaining new ostree features that overlap to a great deal
with the current ones. We still need to keep the current features
for backwards compatibility, so add another test run that does the
same but using new API.

For now this simply uses the `url` parameter rather than `parent`
to build update commits. Further changes will be made in follow-up
commits.

Use `curl` rather than `composer-cli` as we have a chicken-and-egg
problem where we can't land this feature without tests, but
`composer-cli` can't add support for it without having it first in
`composer`.
2021-02-20 14:53:49 +01:00
Tom Gundersen
687ac7f615 weldr/compose/ostree: introduce URL parameter
For now this is simply used to resolve the parent commit, in case
one is not provided. In the future it will be used by new image
types to actually pull content from.

This extends the weldr API, so that future work does not have to
modify that.

The logic we now implement for the ostree commit image types is:
If the URL is provided, but the parent commit is not. The parent
commit is taken to be the current HEAD of the ostree repo at the
given url, with the given (or default) ref.

This only provides a small optional convenience, but we will
soon introduce image types where the URL of the repository is
required.

This commit still needs testing.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Tom Gundersen
6cb47862df distro: expose default OSTreeRef()
Rather than setting this automagically, expose it to the caller. For
now the only caller we have simply passes it back in, so this is a
noop.

In follow-up commits this will be used to resolve the parent commit.

This is tested by verifying that the generated manifests do not
change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Tom Gundersen
3c74ea584e gitignore: add config and OSX metadata
Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-02-20 14:53:49 +01:00
Antonio Murdaca
21319520e5 internal/distro/rhel8: un-exclude subman from edge
We aim at shrinking our deps eventually but we need subman for the time
being. This patch basically un-exclude subman which was introduced by
https://github.com/osbuild/osbuild-composer/pull/893

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2021-02-18 13:08:32 +01:00
Ondřej Budai
7f4aaa8db6 schutzbot: add mockbuild for Fedora 34
It's coming!

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-18 13:07:28 +01:00
Ondřej Budai
c2c4bf39c3 schutzfile: test against osbuild 25 on RHEL 8.3
So we can update our production setups.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-17 20:35:13 +01:00
Aleksandar Todorov
748519a104 Schutzbot: ping on nightly test results 2021-02-17 10:15:35 +01:00
Jozef Mikovic
5a9d2fe8b1 test: Add integration test for VMWare upload 2021-02-16 19:06:01 +00:00
Jozef Mikovic
177b86b3a9 docs/news: describe new upload target 2021-02-16 19:06:01 +00:00
Jozef Mikovic
0597ac48a7 upload/vmware: document uploadImage function 2021-02-16 19:06:01 +00:00
Jozef Mikovic
d497bc3b83 osbuild-worker: ensure that uploaded vmdk image has the user specified name
VMDK image has default name 'disk.vmdk' and there is no option to change the name when uploading to vSphere,
so I'm using symlink so that uploaded image has the name user specified instead of the default one.
2021-02-16 19:06:01 +00:00
Jozef Mikovic
08cfc08805 weldr: add vmware upload target to api
Extend Weldr API to allow user to specify option to upload built image to VMWare,
makes use of previously define upload target.
2021-02-16 19:06:01 +00:00
Jozef Mikovic
1a81489ef1 osbuild-worker: add target for upload to vmware
New upload target for VMWare, similar to the ones for AWS and Azure,
allowing users to set credentials for their vSphere instance.
Commit also includes function that performs the actual upload.
2021-02-16 19:06:01 +00:00
Achilleas Koutsou
d686abfffb docs/news: describe new blueprint kernel option 2021-02-16 13:51:26 +00:00
Achilleas Koutsou
8438078c50 tools: test cases for alt kernel selection
Two new test cases added to format-request-map and test cases are
generated.

1. kernel-rt for RHEL images:
Requires new package repositories for RHEL 8.3 and 8.4.
Creates an OSTree commit with the `kernel-rt` as a customization.

2. kernel-debug for Fedora images:
kernel-rt isn't included in the official fedora repositories.  Using
kernel-debug at least tests the feature with the fedora-iot-commit type.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
1ef1bab5a8 distro/*: test kernels in image types
Test that all defined image types return at least one kernel when given
an empty blueprint and exactly one kernel for ostree-commit types.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
8d9753a4c2 distro/*: remove kernel from image types
The kernel now comes from the blueprint packages even when it's not
specified.  Removing from the base packages of the image types avoids
duplication and allows for alternative kernels to be specified without
also including the default.

The latter is necessary for RHEL for Edge and Fedora IoT images (ostree
commits) that fail to build when multiple kernels are installed.

ImageType tests modified to fix expected package order.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
d0923af70d blueprint: test kernel customization
Edit GetPackages test to expect automatic kernel inclusion.

New test for all combinations of adding a kernel to customizations and
package list.
2021-02-16 13:51:26 +00:00
Achilleas Koutsou
b861042032 blueprint: add kernel name customization
Blueprints can now be used to specify a kernel as part of the kernel
customizations.  Specifying a kernel adds it to the package list.

If no known kernel is specified (neither in the customizations nor the
package list), the default "kernel" is included automatically.

If kernels are specified in both the package list and the
customizations, both are added (even if they're duplicates).
2021-02-16 13:51:26 +00:00
Ondřej Budai
a85511c6de schutzbot: fix the pipeline
https://github.com/osbuild/osbuild-composer/pull/1228 was merged with
a failing Schutzbot's pipeline. The failure is caused because `var` apparently
isn't a right Groovy syntax. Let's use the right word `def` instead.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-16 14:19:25 +01:00
Ondřej Budai
2b823b97ad ci: run the nightly test only on the main branch
Currently, all PRs trigger the daily nightly testing. Let's limit that to just
the main branch.

I found this here:
https://stackoverflow.com/questions/39168861/build-periodically-with-a-multi-branch-pipeline-in-jenkins

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-16 11:00:46 +02:00
Ondřej Budai
8760de26de test/api.sh: set the username for centos-8
This is a merge race condition. Sanne added a new variable, I added a new distro.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-15 09:30:37 +00:00
sanne
9861215ebe test: Let the composer-api testcase also boot the image in aws 2021-02-14 16:09:04 +00:00
Ondřej Budai
bf4280d1e2 distro/rhel84: disable s390x and rhel-commit-edge for Centos
s390x isn't supported on Centos.

rhel-commit-edge sounds just wrong for Centos. We can revert this change any
time. The thing is that I wasn't able to find something like CentOS IoT and
we don't want to be in a position of defining a new distribution spin.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
4492448f49 test/distro/rhel84: invert condition
I'm sorry, I need to extend this condition and my brain isn't powerful enough
to reason about complex negative conditions.

Not a functional change.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
5eb402415d distro/rhel84: add centos 8 stream support
The image definition is shared with the latest RHEL 8.y one (8.4 currently).
I expect that we the introduction of 8.5 support, we point the centos 8
distro at it.

The test repositories and manifests use the official CentOS composes. From
what I can tell, they are persistent. This is not guaranteed though, so we
might need to switch to RPMRepo at some point.

The "classic" CentOS 8 should also be buildable but due to the chicken and egg
issue (this commit will get into Centos "8.4" but Centos "8.4" isn't a thing
yet), we cannot test it and therefore it might be broken.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
6feba70758 test/distro/rhel84: add missing image types to TestImageType_Name
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
4d7b36f08b test/distro/rhel84: add test for Arch.ListImageTypes
We didn't have one so I added it.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-14 16:08:08 +00:00
Ondřej Budai
ab773975ea ci: do not run the unit tests with verbose on
When unit tests succeed, no one cares about them.

When unit tests fail, it's hard to find which one failed. This commit removes
the verbose flag, so it's easy to spot what failed.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-02-13 18:33:09 +00:00
Brian C. Lane
728f2d0f81 weldr: Remove underscores from FreezeHandler error
Error message strings are not guaranteed to be stable, but should
be consistent when used in more than one place.
2021-02-13 08:52:52 +01:00
Tomas Hozza
220c77e37f image-tests: Add RHEL-8.4 test cases for all architectures
Add RHEL-8.4 image test case manifests for all architectures. The set of
image types per architecture is based on what is currently available for
RHEL-8.3.

The image test case for RHEL-8.4 tar image on s390x is not added yet
because of Issue #1220.

Fixes #1167

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-02-12 18:53:18 +01:00
sanne
31eb215699 test: Fix the compose request in the api test case
Customizations should be alongside image_requests, not inside of it.
2021-02-12 18:46:02 +01:00
Achilleas Koutsou
2e98f13c90 docs/news: add ostree + kernel params news entry 2021-02-12 14:25:48 +01:00
Achilleas Koutsou
d8c292ca79 distro/*: test expected failure for ostree
Test for each distro that runs through all architecture - image type
combinations and calls the Manifest() method with a kernel boot option
customization and checks if the ostree image types produce the expected
error.
2021-02-12 14:25:48 +01:00
Achilleas Koutsou
fd1cc343a5 distro/*: fail on kernel boot params with ostree
Kernel boot parameters have no effect on ostree type images (Fedora IoT
and RHEL for Edge).  Catch this and fail early in the pipeline creation
and communicate the issue to the user.
2021-02-12 14:25:48 +01:00
Martin Sehnoutka
f98e231f64 cloudapi: extend the API spec with /version and /openapi.json
These endpoints are useful for clients while exploring the API. They are
also required for deploying the service into clouddot.
2021-02-12 14:25:19 +01:00
Aleksandar Todorov
f56a07472a schutzbot: Use 8.4 nightly b/c latest is now 8.5 2021-02-12 14:22:04 +01:00
Aleksandar Todorov
4787fce3b7 Unstash rhel8nightly repo & json files
- even if they don't exist sometimes but ignore the errors
- make the nightly repository with a higher priority
- override rhel-8*.json files so that newly built images will
  also consume the nightly content
2021-02-12 14:22:04 +01:00
Alexander Todorov
fb8493b4df tests: Document how to manually replay nightly Pipeline 2021-02-12 14:22:04 +01:00
Alexander Todorov
4cd6b37ea4 Schutzbot nightly: download osbuild-composer-tests from Brew
because -tests.rpm isn't shipped with the distro the prepare
script downloads it from Brew, trying to match the same version
that exists in the actual nightly compose. Then prepares a repo on
S3 for the subsequent test jobs to use!

Use AWS_CREDS for ~/.s3cmd
2021-02-12 14:22:04 +01:00
Alexander Todorov
272cc434ad Schutzbot: send Telegram messages for RHEL nightly test status 2021-02-12 14:22:04 +01:00
Alexander Todorov
d08a8fce69 Use conditional steps in EL8 nightly pipeline. Refs #912
- use detect_build_cause() funtion and set a global env.BUILD_CAUSE
  variable for use in conditionals
- add a cron job trigger - this will work together with the
  GitHub pull request trigger
- use conditional blocks for all steps we want to be executed
  outside of cron jobs
- only EL8 jobs will be executed unconditionally, both in cron
  and for PRs. The preparation stage for cron jobs makes sure to
  use the same name for osbuild-mock.repo so that the jobs can
  unstash it later!
2021-02-12 14:22:04 +01:00
Jacob Kozol
2cae456094 distro/rhel84: remove dbxtool
dbxtool is not available on s390x or ppc64le. So, it cannot be included
for now without breaking image builds for these architectures.
2021-02-11 14:12:35 +01:00