Commit graph

80 commits

Author SHA1 Message Date
Tomas Hozza
47b1b5b93b Doc: remove unused doc/news directory
The doc/news directory has been re-added by some recent PRs, but it is
not used any more since [1]. Clean up the repository of the unused
files.

[1] https://github.com/osbuild/osbuild-composer/pull/1933

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-11-24 14:55:47 +01:00
Simon Steinbeiss
dcb5220329 Switch to simple upstream releases
This commit changes our release process from the model of having a
release commit (and pull request) which also updated the NEWS.md file
and bumped the versions in the osbuild.spec and setup.py files to simply
pushing a tag.

After the tag (containing the release notes) is pushed, a GitHub
composite action is triggered that creates a GitHub release with the
contents of the git release tag. Furthermore the bumping of the version
number now always has to happen directly after a release to avoid having
to push a(n untested) commit to main for the release and this is also
handled by the GitHub composite action.

Finally packit pushes directly to dist-git now on pushing the release
tag, so no pull-request needs to be reviewed and merged anymore.

Consequently, we also drop the docs/news folder and its content and
adjust the PR template.
2021-10-27 13:03:53 +02:00
Diaa Sami
3b6156a240 composer: add json log formatting
Configurable via config file/env
2021-10-25 13:41:36 +02:00
Achilleas Koutsou
03fe1337a5 Add news item for updated CentOS Stream 8 definitions 2021-10-21 15:16:04 +01:00
sanne
d25ae71fef worker: Configurable timeout for RequestJob
This is backwards compatible, as long as the timeout is 0 (never
timeout), which is the default.

In case of the dbjobqueue the underlying timeout is due to
context.Canceled, context.DeadlineExceeded, or net.Error with Timeout()
true. For the fsjobqueue only the first two are considered.
2021-10-19 00:12:18 +01:00
Achilleas Koutsou
4e5ce288ab 36
Release osbuild-composer 36

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-13 15:52:08 +01:00
Tomas Hozza
eac6b2c083 RHEL-8.6: add support for official EC2 SAP image
Add support for building the official RHEL EC2 SAP image for RHEL-8.6.
The new image type is available under the `ec2-sap` name. Since the
image type's package set includes RHUI client RPMs, which are not
available publicly, it is by default not exposed through the Weldr API.

Extend the relevant files and definitions used for generating image test
cases. This includes addition of repository definitions needed by the
image type, specifically: `sap`, `saphana` and `ansible`.

Generate image test case for the new image type.

Add News entry for the new image type.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-11 09:49:15 +02:00
Achilleas Koutsou
3609c1b54e news: add entry about RHEL 8.6
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-11 09:49:15 +02:00
Tomas Hozza
0e90217fcc RHEL-9.0: add EC2 SAP image definition.
The image is not available via Weldr API, because it requires RHUI
client RPMs.

The content and configuration is based on RHEL-8.6 EC2 SAP image, since
there is no definition for the RHEL-9 SAP image yet.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-08 16:34:55 +01:00
Diaa Sami
aa08e29243 34
Release osbuild-composer 34

Signed-off-by: Diaa Sami <disami@redhat.com>
2021-09-15 21:12:35 +02:00
Tomas Hozza
8970cd23ed Add a new entry to docs/news/unreleased
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-15 14:46:25 +02:00
Ondřej Budai
ca2dc71eea docs/news: add an entry about removing --excludedocs
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-15 10:47:57 +02:00
Tomas Hozza
19eb65e9fd RHEL-8.5/9.0: keep RHSM DNF plugins enabled on ec2, ec2-ha and ami
The RHSM DNF plugins `product-id` and `subscription-manager` are now
by default enabled on the RHEL-8.5 and RHEL-9.0 `ec2`, `ec2-ha` and
`ami` images.

The desired default state of the RHSM DNF plugins has been decided by
the RHSM team.

Related to https://bugzilla.redhat.com/show_bug.cgi?id=1996670

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-14 11:47:09 +02:00
sanne
4a057bf3d5 auth: OpenID/OAUth2 middleware
2 configurations for the listeners are now possible:
- enableJWT=false with client ssl auth
- enableJWT=true with https

Actual verification of the tokens is handled by
https://github.com/openshift-online/ocm-sdk-go.

An authentication handler is run as the top level handler, before any
routing is done. Routes which do not require authentication should be
listed as exceptions.

Authentication can be restricted using an ACL file which allows
filtering based on JWT claims. For more information see the inline
comments in ocm-sdk/authentication.

As an added quirk the `-v` flag for the osbuild-composer executable was
changed to `-verbose` to avoid flag collision with glog which declares
the `-v` flag in the package `init()` function. The ocm-sdk depends on
glog and pulls it in.
2021-09-04 02:48:52 +02:00
Diaa Sami
c35097fd52 Support for temporary credentials
Handle session token so that temporary credentials are supported
2021-09-02 15:13:42 +02:00
Ondřej Budai
8f21f0b873 33
Release osbuild-composer 33

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-30 19:03:46 +02:00
Ondřej Budai
8dc316407c docs/news: add an entry about fixing RHEL 9.0 edge images
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-30 18:22:58 +02:00
Christian Kellner
3dfc997992 distro/rhel85: move payloads to iso root
Move the ostree repository and the tar image to the root of the
boot iso. This has several advantages: we do no longer have to
correctly guess the size of the anaconda image. Also we do not
need to compress the payload within the squashfs.
Update the image installer's test data. NB: the changes to the
package list were introduced earlier and should mostly affect
the build pipeline. Should have caught is in the corresponding
change, but was apparently not picked up by CI.
2021-08-30 12:50:38 +02:00
Simon Steinbeiss
63fe1c4c83 32
Release osbuild-composer 32

Signed-off-by: Simon Steinbeiss <simon.steinbeiss@redhat.com>
Co-Developed-by: Christian Kellner <christian@kellner.me>
Co-Developed-by: Tom Gundersen <teg@jklm.no>
2021-08-28 16:41:26 +02:00
Christian Kellner
1a063680b5 distro/rhel85: add support for edge raw images
OSBuild Composer can now build the RHEL 8.5 Raw Images. This images are
compressed raw images, i.e. a file that has a partition layout with an
deployed OSTree commit in it. It can be used to flash onto a hard drive
or booted in a virtual machine. An existing OSTree commit needs to
be provided.
The following image new types are supported: edge-raw-image.
2021-08-28 09:20:19 +02:00
Christian Kellner
4e80c1bc82 distro/rhel85: add simplified edge installer
This adds a new installer called the "Simplified Installer" for Edge.
In contrast to the existing insaller, which is based on Anaconda, this
new installer based on the CoreOS installer project[1], a small rust
based binary that is executed in the initramfs and will flash a raw
image to a specified installation device. For this a new blueprint
option is introduced. The raw image is created from an existing OSTree
commit and embedded into the resulting bootable iso. When booting the
iso the installation will automatically start witout any interaction
from the user.
NB: As with the existing edge installer, support is currently limited
to x86. The new installer also does not support non-uefi boot.

[1] https://github.com/coreos/coreos-installer

Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
Co-Developed-by: Antonio Murdaca <runcom@linux.com>
2021-08-28 09:20:19 +02:00
Achilleas Koutsou
17a1d38189 news: add entry for edge-container using nginx
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-27 09:29:28 +02:00
Gianluca Zuccarelli
4f4cb012d5 docs: add rhel90 filesystem support release note 2021-08-25 13:18:12 +02:00
Gianluca Zuccarelli
b885b37333 docs: add rhel85 filesystem support release note 2021-08-21 02:54:38 +02:00
Thomas Lavocat
b896ee4d8c 31
Release osbuild-composer 31
2021-08-11 14:04:41 +02:00
Ondřej Budai
c443df5d20 news: add an entry about RHEL 9
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Martin Sehnoutka
04a62d677f docs/news: composer works with multiple subscriptions 2021-08-09 12:40:23 +02:00
Tomas Hozza
a590dd7591 docs/news: reflect EC2 image changes in the RHEL-8.5 images note
The `ami` image type was redefined based on the official RHEL EC2
images. In addition, two new image types `ec2` and `ec2-ha` were
defined. These new image types are by default not exposed via Weldr
API.

Update the docs/news/unreleased entry related to RHEL-8.5 main images to
reflect these changes.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-05 09:34:07 +02:00
Tomas Hozza
0a71054d86 Weldr API: allow globing in distro image type deny list
Allow globing patterns in distro-specific image type deny list of Weldr
API configuration. Extend unit tests to verify simple globing patterns.

Update NEWS entry.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-02 18:51:03 +02:00
Tomas Hozza
b150d57c18 Weldr API: make Image Type denylist distribution-specific
Change the Image Type denylist in Weldr API from being applied to all
distributions to being distribution-specific. A special name `*`
can be used in the configuration to match any distribution
or any image type.

Modify NEWS entry and unit tests to reflect this change.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-02 18:51:03 +02:00
Tomas Hozza
076bbc5456 Weldr API: introduce Image Type denylist for filtering exposed images
Extend Weldr API to accept a list of denied image types, which should
not be exposed via API for any supported distribution. This
functionality will be needed to not expose image types which can't be
successfully built outside of Red Hat VPN. Example of such images are
the official RHEL EC2 images, which include RHUI client packages not
available publicly.

Image Types are filters when listing available compose types and
creating a new compose using Weldr API.

Extend osbuild-composer configuration to allow specifying the list of
denied Image Types for Weldr API.

Add unit tests for implemented changes.

Add NEWS entry describing the newly introduced functionality.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-02 18:51:03 +02:00
Achilleas Koutsou
15ca644a3f news: add entry about RHEL 8.5 images 2021-07-24 15:54:00 +02:00
Tomas Hozza
044fa0ffaf news: document changes to osbuild integration done in PR#1525
Add news entry for changes merged as part of PR#1525 [1].

[1] https://github.com/osbuild/osbuild-composer/pull/1525

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-19 21:37:15 +02:00
Brian C. Lane
1aa5d49696 docs: Add release note about building for other distros 2021-07-12 08:58:42 +02:00
sanne
4385c39d66 worker: Introduce heartbeats
An occupied worker checks about every 15 seconds if it's current job was
cancelled. Use this to introduce a heartbeat mechanism, where if
composer hasn't heard from the worker in 2 minutes, the job times out
and is set to fail.
2021-07-08 21:14:38 +01:00
Ondřej Budai
5ae6203d65 distro: drop Fedora 32
It's EOL.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-05 11:16:08 +02:00
Achilleas Koutsou
c8bf77bad8 news: add entry about RHEL 8.5 Edge images 2021-06-29 17:26:43 +01:00
Achilleas Koutsou
2212fedae3 docs/news: add news item for new feature 2021-06-29 09:33:05 +01:00
Achilleas Koutsou
b825738939 docs/news: add news item for change 2021-06-18 14:02:09 +01:00
sanne
0ea31c39d5 worker: Add identity filter and client oauth support 2021-06-17 10:08:35 +02:00
Ondřej Budai
b5c65b687a 30
Release osbuild-composer 30.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-10 14:59:08 +02:00
Tomas Hozza
dda9cce03e Add support for image type tags in rpmmd and add RepoRegistry API
Extend the RepoConfig structure to contain new field ImageTypeTags.
Extend also other structures and functions as needed, to support loading
repository definitions, which use this new field. The idea is that a
repository should be used for building all image types, unless it has
some ImageTypeTags defined. In such case, it should be used only for
building the specific image types, which names are specified in the new
field.

Add RepoRegistry as a higher-level API to load and manage repository
definitions for each distribution. Currently it provides one method,
which returns a set of repositories needed to build a given image
type. The RepoRegistry uses the new ImageTypeTags field in the RepoConfig
structure and returns all the needed repositories for the image type.

Modify rpmmd unit tests and add unit tests for RepoRegistry.

Add News entry describing the change done to RepoConfig and its JSON
representation.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-05-14 15:43:00 +02:00
Ondřej Budai
973b4b2714 distro/rhel84: build qcow2 images with compat=0.10
By default, `qemu-img convert` creates qcow2 images usable in qemu 1.1 and
newer. RHEL 8 guest images are meant to be bootable on RHEL 6 though.
Unfortunately, RHEL 6 has qemu 0.12, therefore these images cannot be used
there.

To fix this, we need to use the new qcow2_compat option in qemu assembler
to override the default compat version and make qcow2 images that can be used
in qemu 0.10 and newer.

For this, we need osbuild 28 that isn't yet available in of any of
downstreams, therefore we need to pin it everywhere.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-04-10 19:18:13 +01:00
Achilleas Koutsou
2b5187814a docs/news: add news item for fix 2021-03-30 00:22:04 +01:00
Jozef Mikovic
3e566398ec news: add entry about rhel90 support 2021-03-26 14:05:20 +00:00
Tomas Hozza
4a47ad36af Cloud API: fix image_status.status value for running compose
Previously, the Cloud API endpoint `/v1/compose/{id}` return value's
`image_status.status` for a running worker job was "running", which didn't
comply with the Cloud API specification. Equivalents allowed by the API
specification are "building", "uploading" or "registering".

As a result, the Image Builder API also does not comply in this regard
to its specification, because it currently just copies the status value
string returned by osbuild-composer.

Define the `image_status.status` as a reusable type in the Cloud API
specification. This forces openapi to generate an explicit type for it,
which can be then explicitly used in the code, instead of plain strings.

Return "building", instead of "running"  for running compose.

Modify api integration test to check for all valid `image_status.status`
values for a compose.

Add News entry explaining this change.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-03-26 11:59:07 +01:00
Achilleas Koutsou
8c92553623 docs/news: update news item to mention both image types 2021-03-17 18:12:17 +00:00
Achilleas Koutsou
ce963004e0 docs/news: describe osbuild v2 and new image-type 2021-03-17 18:12:17 +00:00
Tomas Hozza
e698080bc7 GCP: Set image name as custom metadata on uploaded image object
Extend StorageObjectUpload() to allow setting custom metadata on the
uploaded object.

Modify worker's osbuild job implementation and GCP CLI upload tool to
set the chosen image name as a custom metadata on the uploaded object.
This will make it possible to connect Storage objects to specific
images.

Add News entry about image name being added as metadata to uploaded GCP
Storage object as part of worker job.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-03-15 16:48:40 +00:00
Tom Gundersen
fd7431276b news: add entry about net-tools in rhel84 qcow2 2021-03-14 15:03:58 +00:00