Commit graph

4681 commits

Author SHA1 Message Date
Achilleas Koutsou
5d61ba5d39 image: add OSTreeRawImage ImageKind
InstantiateManifest is a stub.  No implementation yet.

Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
280d27a1c8 distro/fedora: add iot partition tables
Based on the official images.
Converted to GPT (from DOS).
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
660444fed1 distro/fedora: introduce fedora-iot-raw-image
Image definition is a stub.  No implementation yet and not added to
distro image type list.

Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Ondřej Budai
e2a298723e ci: enable fedora 37 tests
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Ondřej Budai
a8fe0d2620 distro/fedora: add support for Fedora 37 and 38
The most interesting change is the removal of smc-meera-fonts in 37. As
suggested, rit-meera-new-fonts is used instead.

Existing F35 and F36 manifests updated with package changes.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Ondřej Budai
ff8af88a24 distro/fedora: deduplicate Fedora distribution structs
These are just super-simple to construct using a small helper.

It would be great if we can make `distro.go` totally version-agnostic but
that's something for the future.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
c88d0255da test/diff-manifest: fetch base branch from GitHub
Distro version branches aren't synced to GitLab, so we will need to
fetch them from GitHub directly.
This is required for any PR made against any branch other than main.
2022-09-13 10:57:16 +02:00
Xiaofeng Wang
05503c58c2 test: Remove OCP4 test
Since api.ocp.ci.centos.org is going away:
https://lists.centos.org/pipermail/ci-users/2022-June/004547.html.

After discussion, OCP4 test will be removed here, but RHEL for Edge
downstream test will keep OCP4 case
2022-09-12 19:14:13 +02:00
Jakub Rusz
63a57ed383 CI: Introduce x86_64 rules
We have recently added testing on aarch64 but not all of the test are
supported on that architecture. Adding specific rules for all and x86_64
arches to simplify which test jobs run on what arches.
2022-09-12 13:02:41 +02:00
Jakub Rusz
47d0703cb6 test/upgrade: install vdo package
This package is required for the upgrade process.
2022-09-12 13:02:41 +02:00
Lukas Zapletal
a8afca4634 Introduce logging adapter for jobqueue 2022-09-09 16:27:38 +02:00
Lukas Zapletal
c43b4992fe Add IntelliJ IDEA/GoLand to .gitignore 2022-09-09 16:27:38 +02:00
schutzbot
001b8339c5 schutzfile: Update snapshots to 20220906
Updates all snapshots bar centos-stream-9, as tests using libvirt are
broken with the newer snapshots.
2022-09-09 12:59:01 +02:00
Diaa Sami
6df494f810 tools/appsre: disable packer parallel builds 2022-09-09 12:08:29 +02:00
Diaa Sami
819a63e50e templates/packer: reasonable aws_polling limits for rhel AWS builds 2022-09-09 12:08:29 +02:00
Diaa Sami
46d36a0e73 Revert "appsre: disable aarch64 AMI creation until issue is resolved"
This reverts commit 84f46eebdb.
2022-09-09 12:08:29 +02:00
Alexander Todorov
647acfa2f4 Adjust AWS EC2 type in tests depending on CPU arch 2022-09-09 10:49:41 +03:00
Alexander Todorov
769ffc617d tests: Execute vmware testing only on x86_64 2022-09-09 10:49:41 +03:00
Alexander Todorov
3b16145ec9 tests: Terraform RPM available only on x86_64
according to
https://www.hashicorp.com/blog/announcing-the-hashicorp-linux-repository
https://discuss.hashicorp.com/t/rhel-8-aarch64-rpms/24244/3

and
https://github.com/hashicorp/packer/issues/11516

Don't execute azure.sh if the runner is not x86_64
2022-09-09 10:49:41 +03:00
Alexander Todorov
c4f8947e73 Enable integration testing on aarch64
NOTE: yaml can merge maps but not sequences
2022-09-09 10:49:41 +03:00
dependabot[bot]
5088e40795 build(deps): bump github.com/aws/aws-sdk-go from 1.44.44 to 1.44.93
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.44 to 1.44.93.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.44...v1.44.93)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-08 14:16:34 +02:00
Juan Abia
810f3fe0e9 test: change CIV tag
We were using `latest` as tag, this can be dangerous as it's the default
tag, an anyone can accidentally update it. Using `prod` is safer.

Also use dev container image if the test script is running in CIV CI.
2022-09-07 17:59:50 +02:00
Diaa Sami
84f46eebdb appsre: disable aarch64 AMI creation until issue is resolved
after merging of PR #2718, generation of AMIs has been failing with 'ResourceNotReady: exceeded wait attempts'.
issue tracked in #2961
2022-09-07 12:28:40 +02:00
schutzbot
a8d3e07577 Post release version bump
[skip ci]
2022-09-07 08:30:20 +00:00
Tomas Hozza
351bb69d2b distro: use reflection in ImageConfig.InheritFrom()
As it turned out, people make mistakes and forget to write some parts of
code, unless a unit test screams at them. This is true for the
`InheritFrom()` method, which is not handling all members of the
`ImageConfig` structure.

Use reflection, instead of inheriting from each specific hard-coded
structure member. This will make the implementation future-proof in case
the `ImageConfig` structure is extended with additional members.
2022-09-06 14:43:24 +02:00
Tomas Hozza
c8382f1654 distro: use pointers for basic types in ImageConfig
Using basic types as values in the `ImageConfig` structure makes it
impossible to distinguish if the empty value for the type was set
intentionally or if it is just the value the variable was initialized
to. This is very bad especially for `bool` type.

While working on unifying `vhd` and `azure-rhui` image types I found
out, that some newly added variables in the `ImageConfig` structure
were forgotten in the `InheritFrom()` method. This makes it impossible
to inherit their values from a parent configuration. This is however
required for the unification of `vhd` and `azure-rhui` image types. As
described above, it would be impossible to decide whether a `bool` value
should be inherited from the parent configuration or not. The only
solution is to use a pointer to the type. For consistency, use pointer
for all basic types.

Adjust distro implementations accordingly.
2022-09-06 14:43:24 +02:00
Sanne Raymaekers
ab3bd7d94f templates/packer: Increase aws timeouts for rhel-8-aarch64
This job is failing with "ResourceNotReady: exceeded wait attempts".

https://www.packer.io/plugins/builders/amazon#resourcenotready-error
2022-09-05 14:39:12 +02:00
Diaa Sami
a22b0c229a appsre-ansible: subscribe before upgrading packages 2022-09-05 12:08:57 +02:00
Diaa Sami
ec0a1944b4 appsre-ansible: support aarch64
make ansible playbooks arch-agnostic
extract embedded bash script into separate file with parameters
update packer template to support aarch64
Convert parts of bash script to python code that can start multi-arch instances to build RPMS
2022-09-05 12:08:57 +02:00
Lukas Zapletal
b03a131f13 dbjobqueue: use background context when closing listener 2022-09-02 12:52:50 +02:00
Sanne Raymaekers
6a2e719c76 schutzbot: Fast-forward release branch after green main run
The release branch can be tracked by stage deployments to make sure CI
is green before deploying to stage.
2022-09-02 11:05:28 +02:00
Diaa Sami
a6e11b6b3d tests: add aarch64 rhel-9.0 runner to API tests 2022-09-02 10:01:30 +02:00
Achilleas Koutsou
aed7808b03 rhel9: explicitly add containernetworking-plugins to edge
In podman v4.0.0 the default network backend was switched from cni to
netavark.  However, podman will choose cni if there are already
containers, images, or cni networks preset on a system [1].

Starting with podman v4.2.0, containernetworking-plugins is no longer a
hard requirement for podman.  So when an edge commit is built with an
embedded container, podman v4.2.0+ will choose the cni network and fail
with an error because the plugin isn't installed.

Adding the package explicitly alongside podman to avoid this issue with
future RHEL 9.1 edge builds when they include containers.

This change does not affect test manifests.  The package is already
included in manifests as a dependency of podman < v4.2.0.

See rhbz#2123210

[1] a083f790ab/pkg/config/containers.conf (L275-L278)
2022-09-02 10:00:28 +02:00
Diaa Sami
6b2af037a3 test/cases/api/aws.sh: add support for starting aarch64 instances 2022-08-31 15:41:13 +02:00
Diaa Sami
9635155c88 api/common/aws.sh: support arm64 when installing govc 2022-08-31 15:41:13 +02:00
Diaa Sami
615b9ca5f5 .gitlab-ci: add rhel-8.6-ga-aarch64 runner for AWS API tests 2022-08-31 15:41:13 +02:00
Brian C. Lane
c32f94d6f2 weldr: Preload metadata at startup
For each of the supported distros start a goroutine to depsolve
'filesystem' which will preload the metadata making subsequent responses
faster.

This is safe to do without limits because we only supposed a limited
number of distros, and without additional locking because this is the
the same as hitting the API with multiple depsolve requests at the same
time.
2022-08-31 12:18:51 +01:00
dependabot[bot]
c864343770 build(deps): bump github.com/gophercloud/gophercloud
Bumps [github.com/gophercloud/gophercloud](https://github.com/gophercloud/gophercloud) from 0.24.0 to 1.0.0.
- [Release notes](https://github.com/gophercloud/gophercloud/releases)
- [Changelog](https://github.com/gophercloud/gophercloud/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gophercloud/gophercloud/compare/v0.24.0...v1.0.0)

---
updated-dependencies:
- dependency-name: github.com/gophercloud/gophercloud
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-31 12:59:11 +02:00
dependabot[bot]
8f26317115 build(deps): bump github.com/golang-jwt/jwt/v4 from 4.4.1 to 4.4.2
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.4.1 to 4.4.2.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v4.4.1...v4.4.2)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-31 12:58:24 +02:00
dependabot[bot]
48ebe60363 build(deps): bump github.com/Azure/go-autorest/autorest
Bumps [github.com/Azure/go-autorest/autorest](https://github.com/Azure/go-autorest) from 0.11.27 to 0.11.28.
- [Release notes](https://github.com/Azure/go-autorest/releases)
- [Changelog](https://github.com/Azure/go-autorest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Azure/go-autorest/compare/autorest/v0.11.27...autorest/v0.11.28)

---
updated-dependencies:
- dependency-name: github.com/Azure/go-autorest/autorest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-31 12:57:16 +02:00
dependabot[bot]
d75e01aef0 build(deps): bump github.com/labstack/echo/v4 from 4.7.2 to 4.8.0
Bumps [github.com/labstack/echo/v4](https://github.com/labstack/echo) from 4.7.2 to 4.8.0.
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.7.2...v4.8.0)

---
updated-dependencies:
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-31 12:47:24 +02:00
fkolwa
98a7457e08 osbuild-composer.spec: osbuild version bump 64
Schutzfile: Bump osbuild commit hash to include new GCP guest-agent stage
2022-08-30 22:21:29 +02:00
fkolwa
1d9ee1adda test/data/manifests: regenerate test cases RHEL+CentOS 8,9 2022-08-30 22:21:29 +02:00
fkolwa
1b3f509318 internal/distro: Add GCPGuestAgentConf to ImageConfig 2022-08-30 22:21:29 +02:00
fkolwa
93b57652ba internal/distro/rhel90: Add GCPGuestAgentConf
Add NewGcpGuestAgentConfStage to osPipeline
2022-08-30 22:21:29 +02:00
fkolwa
2d3997967f internal/distro/rhel8: Add GCPGuestAgentConf
Add NewGcpGuestAgentConfStage to osPipeline
2022-08-30 22:21:29 +02:00
fkolwa
1fbdb21f6b internal/osbuild: Add GCP guest-agent conf stage
Add GCP guest-agent conf stage test
2022-08-30 22:21:29 +02:00
Sanne Raymaekers
d13347e1ca cloudapi: Add endpoints to clone aws images across regions
Support for creating multiple amis from a single compose. It uses the
AWSEC2* jobs to push images to new regions, and share them with new
accounts.

The compose it depends upon has to have succeeded.
2022-08-30 16:14:52 +02:00
Sanne Raymaekers
599829a3b8 worker: Return dependent jobs in OsbuildJobStatus 2022-08-30 16:14:52 +02:00
Sanne Raymaekers
0fe3f1b2ae jobqueue: Query job dependents 2022-08-30 16:14:52 +02:00