Commit graph

3027 commits

Author SHA1 Message Date
Achilleas Koutsou
2b34e4003c distro/rhel90: remove rdma-core for simplified-installer
No longer necessary.
See fed8edc51b
2021-12-02 12:12:05 +01:00
Achilleas Koutsou
4035b0d859 test/data: update RHEL 9.0 test repositories 2021-12-02 12:12:05 +01:00
Achilleas Koutsou
13bdde91f7 distro/rhel90: use coreos-installer-dracut
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-12-02 12:12:05 +01:00
Achilleas Koutsou
5d4103af74 distro/rhel90: enable edge-simplified-installer image type
This reverts commit 226ada67f93cdfda73b3cae69d45eaf4fb281d6f.
2021-12-02 12:12:05 +01:00
Alexander Todorov
1547451ee1 ci: Install gssapi/gssapi.h for Coverity Scan 2021-12-02 11:31:41 +01:00
sanne
ee8e551c5f test: Fixes for 8.5-ga 2021-12-02 10:41:04 +01:00
sanne
704c5daf71 8.5
openstack probably won't work
2021-12-02 10:41:04 +01:00
Tomas Hozza
4f4341d774 RHEL-9.0: clean up package sets of commented lines
Remove commented out lines and some comments, kept only as a reference
when we moved away from using the `@core` group.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-01 19:45:36 +01:00
Tomas Hozza
bab13f3c90 RHEL-9.0: replace the @core package group with a static list of packages
Don't use the `@core` package group in image definitions, because it is
not intended as the minimal package set for virtual / cloud images. In
addition, its content is changing without us knowing, which has
consequences such as the recent discovery of the fact that TuneD is no
longer installed by default on RHEL images, while it definitely should be.

Replace the `@core` package group with the `coreOsCommonPackageSet`
package set. The content of it is based on the latest `@core` group
definition with a few modifications, so that image package sets
never end up having the same package listed in the `Include` and `Exclude`
package set at the same time. All additions have been accompanied with a
comment and all removals have been kept commented out with a comment.

The fact that the change does not have any effect on image package sets
was verified by regenerating all RHEL-9.0 image test cases. There is
however one change in the VMDK image. Specifically the
`python3-libselinux` package have been added. The reason is that the
latest `@core` group definition was used when defining the content of
`coreOsCommonPackageSet`, however the `@core` group definition in the
RPMRepo snapshot used for the image test case didn't include the package
yet.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-01 19:45:36 +01:00
Tomas Hozza
aee86c1199 RHEL-9.0: list one package per line in package set definitions
Listing a single package per line in the package set definitions makes
it much more easier to review diffs in code changes and spot potential
issues.

Align EC2 package set functions to use the structure's `.Append()`
method as it is used by all the other package set functions.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-01 19:45:36 +01:00
Tomas Hozza
9e757fbf11 RHEL-9.0: install tuned by default on all images using @core group
The `@core` package group used to include TuneD package by default on
RHEL-8. It has been removed from the group in Fedora as part of [1] and
inherited into RHEL-9. As a result, TuneD is no longer installed by
default on RHEL images.

After a discussion on rhel-devel there seems to be an agreement, that
TuneD should be installed by default on all RHEL virtual images. At
least we should keep the consistency in this regard with RHEL-8.

Regenerate all RHEL-9.0 image test cases.

Related to https://bugzilla.redhat.com/show_bug.cgi?id=2026709

[1] https://pagure.io/fork/adelton/fedora-comps/c/a5d4f1b6c9fcbe20cb0c38eac5048d7d45d1dd17

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-01 19:45:36 +01:00
Ondřej Budai
05c22554d8 schutzbot: introduce packer builds
Every schutzbot run will now build an AMI image that contains the worker.
The composer RPM is taken from the mockbuild phase. The osbuild version
is taken from Schutzfile for RHEL 8 GA.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-12-01 16:08:11 +00:00
Ondřej Budai
8bf2dd55a2 packer: remove osbuild-composer.service override
We no longer use this AMI for composer, so we don't need this override.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-12-01 16:08:11 +00:00
Ondřej Budai
2bd2e3d1bc packer: install just osbuild-composer-worker
We don't actually need a composer in these images, so let's just install
the worker.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-12-01 16:08:11 +00:00
Ondřej Budai
b799605f51 packer: install monit and vector
Previously, monit and vector RPMs were embedded directly in the
image-builder-packer repository. This was not ideal because hosting big
binary files in git is always ugly.

This commit brings back monit and vector:

- monit is installed from EPEL
- vector is installed from the upstream RPM repository

Ansible was dropped because we don't need it in the image.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-12-01 16:08:11 +00:00
Ondřej Budai
fbebe4c2cf packer: adjust ansible playbook filepath
We want an absolute path, otherwise packer doesn't know where to find the
playbook if called from a wrong directory.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-12-01 16:08:11 +00:00
Ondřej Budai
b619e4875e packer: rework variables
osbuild and composer commit SHAs now must be passed into packer using
variables, no defaults are defined. Also, packer is no longer responsible
for naming the AMIs, the name is also passed as a variable.

imagebuilder_packer_sha was dropped entirely as the packer configuration
now lives directly in osbuild-composer repository.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-12-01 16:08:11 +00:00
Ondřej Budai
0fb3634c2c packer: remove forwarding to console
Console support in AWS EC2 is very basic. We now use vector that works much
better than console so we can just drop the forwarding and rely on vector
dumping the logs into cloudwatch.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-12-01 16:08:11 +00:00
Ondřej Budai
15c46544b6 packer/monit: remove verify_worker_connection
This is currently not working because workers in aoc no longer use mTLS.
Definitely something we want to fix in the future I think.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-12-01 16:08:11 +00:00
Ondřej Budai
cc81e919ca packer: drop RH IT certificate
I think it was needed for internal workers - not needed anymore.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-12-01 16:08:11 +00:00
Ondřej Budai
1b289cc27e packer: import image-builder-packer repository
/templates/packer now contains a copy of image-builder-packer repository
as of b8a4b45f93890090de24e3d043e2d958948fc3c5

Changes:
- LICENSE file was dropped (it was redundant)
- README file was dropped (no longer needed)
- GitHub workflows were removed (will be replaced by schutzbot)
- RPMs were removed (they were huge, will be installed in a different way)

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-12-01 16:08:11 +00:00
Martin Sehnoutka
d1029fae69 osbuild2: update cloud-init stage with new options
Related to: https://github.com/osbuild/osbuild/pull/866/

Introduce new fields and move structure validation into the constructor.
This will fail faster and hopefully provide less space for programming
errors. Another advantage is simplified code with less type aliases and
lines.
2021-12-01 12:03:13 +01:00
Alexander Todorov
fbf220707a tests: Small updates to docs 2021-12-01 10:12:04 +01:00
Ondřej Budai
c55ec88505 api/koji: fix /compose/log route
We have been actually unmarshalling into a wrong datatype for a year, by
fixing this, we should get much more logging in Brew.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-11-30 18:14:10 +01:00
Jakub Rusz
8136e0f069 ci: make some jobs interruptible
Jobs that don't interact with clouds can be canceled at any time without
the risk of leaving unused resources in the cloud. This enables the use
of "automatic cancellation of redundant pipelines" which means that if
and update is pushed to an open PR the current running pipeline is
canceled. This is done by adding an "interruptible" flag to the jobs.
Default value is false so only jobs that have it explicitly set to true
will be canceled.
2021-11-30 10:00:22 +01:00
Jakub Rusz
c38ebb1643 tests/regression: skip satellite regression test on subscribed RHEl
This test does not bring any value when running on subscribed RHEL
machines.
2021-11-26 12:20:54 +01:00
Jakub Rusz
35f8707d59 tests: re-enable satellite regression test on nightlies
Test now also works with nightly repositories. Re-enabling so that we
can run it during nightly compose testing.
2021-11-26 12:20:54 +01:00
Jakub Rusz
dbd9285275 schutzbot/prepare-rhel-internal: add trailing slash to repo url
Regression tests were failing without this trailing slash at the end.
2021-11-26 12:20:54 +01:00
Ondřej Budai
fa73586932 nightly: use boston mirror instead of the redirecter
It often takes a lot of time to synchronize the compose across all mirrors,
so it's safer to use just one, otherwise we can get hit by 404s if the
compose isn't available yet by a mirror.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-11-26 12:20:54 +01:00
Juan Abia
42daf0c84d schuzbot: update scheduled cloud cleaner runner
fedora 33 will be EOL, change the runner to centos stream 8
2021-11-26 10:00:42 +01:00
Juan Abia
ca36cf155e schutzbot: remove aws s3 resources
remove old enough and tagged s3 objects
2021-11-26 10:00:42 +01:00
Juan Abia
eed20916b0 test/cases: tag aws s3 resources
all resources on aws originated from a test should be tagged with
"gitlab-ci-test". this commit ensures s3 resources get tagged.
2021-11-26 10:00:42 +01:00
sanne
c6c311cc3d osbuild-worker: Recreate rpmmd for each depsolve
The remote workers are long-running, and the subscription certificates
might expire. Before each depsolve refresh the subscriptions.
2021-11-25 17:55:46 +01:00
Ondřej Budai
14b29ae98a dbjobqueue: don't log when context's deadline was exceeded
This happens rather often as we limit the request job timeout to 20s on the
service.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-11-25 08:20:22 +01:00
Ondřej Budai
76d8a75184 worker: normalize job logging
The format is now always 'JOB_ID' (JOB_TYPE). This means that we also know
the job type when a job is finished or when it failed.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-11-25 08:20:22 +01:00
Ondřej Budai
dc7968d5c7 terraform: update to use the new instance type
The latest commit switched all our AWS x86_64 workloads to the newly announced
c6i.large instance type. I did some measurements and the test suite runs about
10% faster. Nice!

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-11-24 17:13:41 +01:00
Ondřej Budai
d61e1c009b spec: add epoch to nevra only if it's set
This change was tested by running:
```
$ rpm --eval '
    %global noepoch 42
    %{?epoch:%epoch:}
  '
$ rpm --eval '
    %global epoch 42
    %{?epoch:%epoch:}
  '
42:
```

Fixes #2054

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-11-24 16:12:54 +00:00
chloenayon
b0ef1ac256 Post release version bump
[skip ci]
2021-11-24 16:03:02 +00:00
Tomas Hozza
47b1b5b93b Doc: remove unused doc/news directory
The doc/news directory has been re-added by some recent PRs, but it is
not used any more since [1]. Clean up the repository of the unused
files.

[1] https://github.com/osbuild/osbuild-composer/pull/1933

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-11-24 14:55:47 +01:00
Tomas Hozza
8420f34b7a RHEL-9.0: install and enable TuneD by default on all EC2 images
Add TuneD package to the base package set for all EC2 image types,
including the `ami` image type. In addition to installing the package,
also enable the service by default. TuneD will by default auto-detect
the environment in which the image is running and set the most
appropriate TuneD profile, with exception of the `ec2-sap` image, which
explicitly sets a specific TuneD profile.

This change affects the `ami`, `ec2`, and `ec2-ha` image types on all
supported architectures.

Regenerate affected image test cases.

Related to RHELPLAN-102615
Fix #1972

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-11-24 08:51:52 +01:00
Antonio Murdaca
c37b688dd2 test/rhel86-manifest: re-generate
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2021-11-23 18:23:45 +01:00
Diaa Sami
e4d535ea5f Improve contributing.md
use auto-answer -y for all commands
mention the script tools/prepare-source
mention the go generate step
2021-11-23 08:25:07 +01:00
Martin Sehnoutka
04f69db2c0 osbuild2: selinux stage - introduce force_autorelabel option
This was added in osbuild: https://github.com/osbuild/osbuild/pull/875

Introduce the same option in composer and make it optional by specifying
it as a pointer to bool value. It would work the same even if it was
there every time, but as it should be an edge case, don't use it
everywhere. Also osbuild doesn't require it to be present, so it seems
like the right thing to do.
2021-11-22 15:28:19 +01:00
Achilleas Koutsou
841dd233c7 test/data: update image-installer test manifest 2021-11-20 14:31:44 +01:00
Antonio Murdaca
ea5f153f3c internal: drop not needed kargs
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2021-11-20 14:31:44 +01:00
Antonio Murdaca
fed8edc51b internal/rhel86: prune default dracut modules
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2021-11-20 14:31:44 +01:00
sanne
58424b39bc test: Adapt api test metrics to namespacing 2021-11-19 22:48:25 +01:00
Gianluca Zuccarelli
3443fb8771 templates: update dashboard metrics
Update the composer dashboard to make use of the
namespaced metrics.
2021-11-19 22:48:25 +01:00
Gianluca Zuccarelli
91f2457363 metrics: add prometheus namespaces
Make use of the prometheus namespace and subsystem
to give the metrics a consistent namespaces in openshift.
2021-11-19 22:48:25 +01:00
sanne
c8f198166d cloudapi/v2: Add support for edge-container 2021-11-19 21:45:19 +01:00