Commit graph

2843 commits

Author SHA1 Message Date
Simon Steinbeiss
dcb5220329 Switch to simple upstream releases
This commit changes our release process from the model of having a
release commit (and pull request) which also updated the NEWS.md file
and bumped the versions in the osbuild.spec and setup.py files to simply
pushing a tag.

After the tag (containing the release notes) is pushed, a GitHub
composite action is triggered that creates a GitHub release with the
contents of the git release tag. Furthermore the bumping of the version
number now always has to happen directly after a release to avoid having
to push a(n untested) commit to main for the release and this is also
handled by the GitHub composite action.

Finally packit pushes directly to dist-git now on pushing the release
tag, so no pull-request needs to be reviewed and merged anymore.

Consequently, we also drop the docs/news folder and its content and
adjust the PR template.
2021-10-27 13:03:53 +02:00
Christian Kellner
93e54cd872 distro/rhel86: special case root user for ssh keys
Add a special case for the root user to the work-around for ssh
keys in OSTree commits. As a little refresher: OSTree does not
support having any content in home directories; we therefore
include a first-boot stage in the commit that will create the
ssh keys on first boot. However, until now we did not special
case the root user, which has a separate root directory (/root,
as a symlink to /var/roothome). This patch fixes this.
2021-10-27 00:41:27 +02:00
Tom Gundersen
f44acd0974 Revert "Revert "cloudapi/v1: Move depsolving to workers""
Workers now depsolve in parallel to image builds, so we can
again move depsolivng to the workers. This will help us deal
with increases in traffic as we currently only have one
depsolve handler per pod. It would also avoid any issues with
composer running out of disk space due to dnf metadata caches.

This reverts commit c65b1e9b26.
2021-10-26 13:58:22 +01:00
Tom Gundersen
cfe9f7a87f templates: image-builder-ci access to composer
This should all move to app-interface, as it is configuration, and
we should distinguish between staging and production.

But for now, enable this where it is.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-10-26 10:39:50 +02:00
Ondřej Budai
83a35fb9b8 ci: remove 8.5 nightly testing
It no longer makes sense because:

- we don't make any changes to 8.5
- we don't regenerate test manifests for 8.5
- osbuild-composer for 8.5 is in the rhel-8.5.0 branch

Also, the latest-8.5.0 symlink was removed, which broke the CI.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-26 09:30:21 +01:00
Diaa Sami
3b6156a240 composer: add json log formatting
Configurable via config file/env
2021-10-25 13:41:36 +02:00
Achilleas Koutsou
aea77ed3de test: timestamp messages in test scripts
The commonly used 'greenprint' function now adds a date + timestamp to
each message for debugging and tracking the duration of segments of each
scripts.
2021-10-24 16:13:53 +01:00
Achilleas Koutsou
03fe1337a5 Add news item for updated CentOS Stream 8 definitions 2021-10-21 15:16:04 +01:00
Achilleas Koutsou
62a9aadfe4 test/api: SSH_USER=ec2-user for centos on AWS 2021-10-21 15:16:04 +01:00
Achilleas Koutsou
63fd590fc2 test/cases: add centos-8 as support to test scripts
Add centos-8 as a supported distro in ostree test scripts.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
a180c06fc4 test/ansible: install greenboot-failing-unit from public source
Use --cache-only to skip trying to update the cache.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
70b70b0da6 CI: enable OSTree tests on CentOS 8
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
0cb95ee9b6 test/data: regenerate manifests for CentOS 8
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
30d1183909 distro/rhel86: skip RHSM config stage for non-RHEL
EC2 RHEL images keep the RHSM DNF plugins enabled, but this shouldn't be
added on CentOS since it doesn't have RHSM.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
93750dd537 distro/rhel86: distro private method isRHEL()
Returns true if the distribution name starts with "rhel".

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
c7df2f1bc7 test/data/repositories: update cs8 rpmrepo snapshot
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
628515a694 tools: add all supported image types for centos-8
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
67e46b888f tools/test-case-generators: fix typo in imagetype-map
image_installer -> image-installer
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
ed0cb5ea24 distro/rhel86: add distro specific package set
A package set that only gets added for some distros.

Now that the rhel86 subpackage defines both RHEL 8.6 and CentOS 8, we
need to be able to separate out packages that are only available on one
of the two distros. Currently, this only includes "insights-client"
which is not available on CentOS.

The packages are added to the OS package sets conditionally based on the
distro name.
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
e4f42066bb test: add RHEL 8.6 image installer test manifest
Installer test manifests don't contain image info but are useful for
detecting changes to the manifest.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
8f84e46ab8 distro/rhel86: add CentOS Stream 8 as alias to RHEL 8.6
Removed old alias from RHEL 8.4.

To make aliasing distributions simpler, all distro-specific strings were
added to the local distribution type.

These were previously global constants of the distribution package and
alias alternatives were used conditionally. Now the two distributions
are predefined in global map.

CentOS exclusions:
- s390x arch
- ec2 image types: rhui client is not available

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
20036e7944 distro/rhel86: remove redundant rhel-86 alias
Accidentally left over from copy from RHEL 8.5
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
c1944095d0 spec: dnf-json conflicts with old composer
dnf-json provides files that osbuild-composer used to provide, so they
should be marked as conflicting.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:07:26 +01:00
Ondřej Budai
ece191f73e ostree: change the URL for OC client temporarily
mirror.openshift.com is temporarily down.

Revert this commit when the server is working as usual.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-21 07:28:46 +02:00
Ondřej Budai
b606e8afca test/aws: remove a no longer needed key from an instance
It's actually not needed, remove it.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-21 07:28:46 +02:00
Ondřej Budai
361ff64bfc ci: pin a specific RHEL 9.0b compose
Otherwise, we get RPM failures. We also have to use a newer terraform
commit. :/

See RHELCMP-7135

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-21 07:28:46 +02:00
Ondřej Budai
58423c262b ci: rotate secret names
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-21 07:28:46 +02:00
Ondřej Budai
65e429fc4a mockbuild: explicitly configure s3cmd
We are no longer using the variable names that s3cmd uses by default
so we need to configure it explicitly.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-21 07:28:46 +02:00
Ondřej Budai
99a8d67812 mockbuild: remove subscriptions
Everything is pre-subscribed.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-21 07:28:46 +02:00
sanne
fa1a4be4d3 .gitlab-ci: Don't save the journal as an artifact 2021-10-19 18:15:07 +02:00
Diaa Sami
751ef84fc1 jobqueue: Better logging
Use appropriate logging levels for different statements
Log important jobqueue events with relevant information
2021-10-19 10:03:39 +01:00
sanne
97fe226c8a templates: Claims based on user_ids 2021-10-19 08:30:15 +01:00
sanne
d25ae71fef worker: Configurable timeout for RequestJob
This is backwards compatible, as long as the timeout is 0 (never
timeout), which is the default.

In case of the dbjobqueue the underlying timeout is due to
context.Canceled, context.DeadlineExceeded, or net.Error with Timeout()
true. For the fsjobqueue only the first two are considered.
2021-10-19 00:12:18 +01:00
dependabot[bot]
9075dbc61d build(deps): bump cloud.google.com/go/storage from 1.16.1 to 1.18.1
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.16.1 to 1.18.1.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/master/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/storage/v1.16.1...storage/v1.18.1)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-18 07:54:09 +01:00
dependabot[bot]
51024c482d build(deps): bump github.com/labstack/echo/v4 from 4.5.0 to 4.6.1
Bumps [github.com/labstack/echo/v4](https://github.com/labstack/echo) from 4.5.0 to 4.6.1.
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.5.0...v4.6.1)

---
updated-dependencies:
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-18 07:50:28 +01:00
dependabot[bot]
a9f36188ee build(deps): bump github.com/gophercloud/gophercloud
Bumps [github.com/gophercloud/gophercloud](https://github.com/gophercloud/gophercloud) from 0.20.0 to 0.22.0.
- [Release notes](https://github.com/gophercloud/gophercloud/releases)
- [Changelog](https://github.com/gophercloud/gophercloud/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gophercloud/gophercloud/compare/v0.20.0...v0.22.0)

---
updated-dependencies:
- dependency-name: github.com/gophercloud/gophercloud
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-17 15:57:40 +01:00
sanne
963688eb07 worker: Separate goroutine for depsolve jobs
The worker client is thread-safe, so we can share it between routines.
2021-10-14 16:27:39 +02:00
sanne
87cc722021 cloudapi/v2: Plural path section when querying a collection 2021-10-14 16:22:24 +02:00
sanne
7e6365742f cloudapi/v2: Do not require auth for /openapi or /errors 2021-10-14 16:22:24 +02:00
Thomas Lavocat
010a1f5022 worker: Configure AWS credentials in the worker 2021-10-14 02:10:54 +01:00
Ondřej Budai
7760ca1c92 cloudapi/v2: ensure only one image per a compose in the API spec
We've never had the ability to build multiple images per a compose, this
commit thus rips out support for this on the API level:

image_requests is now image_request and it accepts only one ImageRequest
object instead of an array of them.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-14 00:14:34 +02:00
Ondřej Budai
b2dc90e404 cloudapi/v2: clean up targets
tbh, I don't fully understand why we wrote the code in this way.

Not a functional change.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-14 00:14:34 +02:00
Ondřej Budai
6b8f758c54 cloudapi/v2: move multi-image compose check to the beginning
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-14 00:14:34 +02:00
Achilleas Koutsou
4e5ce288ab 36
Release osbuild-composer 36

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-13 15:52:08 +01:00
Sanne Raymaekers
c65b1e9b26 Revert "cloudapi/v1: Move depsolving to workers"
This reverts commit 0f44250237.
2021-10-13 12:01:51 +02:00
sanne
cb5acbac74 worker: Prefix https always
Also, with oauth, add a root CA if present.
2021-10-12 17:59:59 +01:00
sanne
0f44250237 cloudapi/v1: Move depsolving to workers
We can't enqueue a job without a worker doing the depsolve step now, so
adapt the tests as well.
2021-10-12 14:21:03 +01:00
Achilleas Koutsou
395a22aa21 cloudapi/v2: fix newV2Server() call in test
Bug caused by two consecutive PR merge rebases, one that added a
function call and another that changed the signature of the same
function.
2021-10-11 23:48:26 +02:00
sanne
93dec413f3 templates: Name service ports 2021-10-11 22:41:36 +01:00
Tomas Hozza
46b8a2e977 SPEC: Exclude armv7hl architecture
Exclude the `armv7hl` architecture, since osbuild-composer does not
support building images for it.

Fix #1839

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-11 15:33:53 +02:00