particle-os/debian-forge-composer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Switch to simple upstream releases

Browse source 
This commit changes our release process from the model of having a
release commit (and pull request) which also updated the NEWS.md file
and bumped the versions in the osbuild.spec and setup.py files to simply
pushing a tag.

After the tag (containing the release notes) is pushed, a GitHub
composite action is triggered that creates a GitHub release with the
contents of the git release tag. Furthermore the bumping of the version
number now always has to happen directly after a release to avoid having
to push a(n untested) commit to main for the release and this is also
handled by the GitHub composite action.

Finally packit pushes directly to dist-git now on pushing the release
tag, so no pull-request needs to be reviewed and merged anymore.

Consequently, we also drop the docs/news folder and its content and
adjust the PR template.
This commit is contained in:
Simon Steinbeiss 2021-10-20 17:17:41 +02:00 • committed by Simon Steinbeiß
parent 93e54cd872
commit dcb5220329
60 changed files with 16 additions and 1766 deletions
Download patch file Download diff file Expand all files Collapse all files

4
.github/PULL_REQUEST_TEMPLATE.md vendored
View file

@ -3,7 +3,6 @@ This pull request includes:
- [ ] adequate testing for the new functionality or fixed issue
- [ ] adequate documentation informing people about the change such as
- [ ] create a file in [news/unreleased](https://github.com/osbuild/osbuild-composer/tree/main/docs/news/unreleased) directory if this change should be mentioned in the release news
- [ ] submit a PR for the [guides](https://github.com/osbuild/guides) repository if this PR changed any behavior described there: https://www.osbuild.org/guides/
<!--
@ -16,9 +15,6 @@ anything.
If the reason for ticking any of the boxes is ambiguous, please add a short
note explaining why.
For user-visible changes, "adequate documentation" is an entry describing the
change for users in docs/news. Please refer to docs/news/README.md for details.
In addition, if this pull request fixes a downstream issue, please refer to
test/README.md and add these additional items:

15
.github/workflows/release.yml vendored Normal file
View file

@ -0,0 +1,15 @@
name: "Create GitHub release"
on:
push:
tags:
- "v*"
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Upstream release
uses: osbuild/release-action@main
with:
token: "${{ secrets.GITHUB_TOKEN }}"

62
Makefile
View file

@ -240,65 +240,3 @@ scratch: $(RPM_SPECFILE) $(RPM_TARBALL)
--nocheck \
$(RPM_SPECFILE)
#
# Releasing
#
NEXT_VERSION := $(shell expr "$(VERSION)" + 1)
.PHONY: release
release:
@echo
@echo "Checklist for release of osbuild-composer-$(NEXT_VERSION):"
@echo
@echo " * Create news entry in NEWS.md with a short description of"
@echo " any changes since the last release, which are relevant to"
@echo " users, packagers, distributors, or dependent projects."
@echo
@echo " Use the following template, break lines at 80ch:"
@echo
@echo "--------------------------------------------------------------------------------"
@echo "## CHANGES WITH $(NEXT_VERSION):"
@echo
@echo " * ..."
@echo
@echo " * ..."
@echo
@echo -n "Contributions from: "
# We omit the contributor list if `git log` fails. If you hit this,
# consider fetching missing tags via `git fetch --tags`, or just copy
# this command and remove the stderr-redirect.
@echo `( git log --format='%an, ' v$(VERSION)..HEAD 2>/dev/null | sort -u | tr -d '\n' | sed 's/, $$//' ) || echo`
@echo
@echo "— Location, YYYY-MM-DD"
@echo "--------------------------------------------------------------------------------"
@echo
@echo " To get a list of changes since the last release, you may use:"
@echo
@echo " git log v$(VERSION)..HEAD"
@echo
@echo " * Bump the project version. The canonical location so far is"
@echo " 'osbuild-composer.spec'."
@echo
@echo " * Make sure the spec-file is updated for the new release and"
@echo " correctly supports all new features. This should already be"
@echo " done by previous commits that introduced the changes, but"
@echo " a sanity check does not hurt."
@echo
@echo " * Commit the version bump, specfile changes and NEWS.md in any"
@echo " order you want."
@echo
@echo " * Tag the release via:"
@echo
@echo " git tag -s -m 'osbuild-composer $(NEXT_VERSION)' v$(NEXT_VERSION) HEAD"
@echo
@echo " * Push main as well as the tag:"
@echo
@echo " git push origin main"
@echo " git push origin v$(NEXT_VERSION)"
@echo
@echo " * Create a release on github. Use 'NEWS.md' verbatim from the"
@echo " top until the end of the section for this release as release"
@echo " notes. Use 'v$(NEXT_VERSION)' as release name and as tag for"
@echo " the release."
@echo

863
NEWS.md
View file

@ -1,863 +0,0 @@
## CHANGES WITH 36:
* Add support for official RHEL EC2 SAP image on RHEL-9.0
* Add support for official RHEL EC2 SAP image on RHEL-8.6
* Add support for RHEL 8.6
Contributions from: Achilleas Koutsou, Diaa Sami, Jakub Rusz, Martin Sehnoutka,
Ondřej Budai, Sanne Raymaekers, Simon Steinbeiss,
Tom Gundersen, Tomas Hozza,
— Berlin, 2021-10-13
## CHANGES WITH 35:
* Change filesystem minsize from int, which is platform dependent, to uint64
* Fix a bug in the startup of composer where e.g. Plymouth would leave it hanging
* RHEL-9.0: Use qcow2 compat 1.1
* RHEL-9.0: Re-add `nss-altfiles` which is needed for OSTree based systems
* RHEL-9.0: Add `gnome-kiosk` package and switch to `python3` from `python36`
* RHEL-8.5: Set bootloader to `none` for Edge raw images
* Improve logging for the Cloud API
* Return a 500 error and write to log on middleware panic
* Return a 400 error when deleting an unknown source in the weldr API
Contributions from: Achilleas Koutsou, Alexander Todorov, Brian C. Lane,
Christian Kellner, Diaa Sami, Gianluca Zuccarelli, Jakub Rusz,
Martin Sehnoutka, Ondřej Budai, Sanne Raymaekers,
Simon Steinbeiss, Tomas Hozza
— Vöcklabruck, 2021-09-30
## CHANGES WITH 34:
* Support temporary AWS credentials
* Composer-api and worker-api: OAuth2 support
* Install docs in RHEL 8.5 and 9.0 images
* RHEL-8.5 / RHEL-9.0: RHSM DNF plugins are now enabled by default on `ec2`
and `ami` images
* Added support for the following osbuild stages
* `org.osbuild.selinux.config` - configures SELinux policy state and type
on the system
* `org.osbuild.tmpfilesd` - creates tmpfiles.d configuration files
* `org.osbuild.pam.limits.conf` - creates configuration files for
pam_limits module
* `org.osbuild.sysctld` - creates sysctl.d configuration files
* `org.osbuild.dnf.config` - configures DNF (currently only variables)
* `org.osbuild.tuned` - sets active tuned profile (or more profiles)
* More accurate HTTP return status for success requests in cloudapi
Contributions from: Alexander Todorov, Antonio Murdaca, Brian C. Lane,
Chloe Kaubisch, Diaa Sami, Gianluca Zuccarelli, Jakub Rusz,
Juan Abia, Martin Sehnoutka, Ondřej Budai, Simon Steinbeiss,
Thomas Lavocat, Tom Gundersen, Tomas Hozza, Sanne Raymaekers,
Yi He
— Berlin, 2021-09-15
## CHANGES WITH 33:
* Bootiso: move payload to iso root
* Fix building of RHEL 9.0 Edge images
Contributions from: Christian Kellner, Ondřej Budai, Xiaofeng Wang
— Liberec, 2021-08-30
## CHANGES WITH 32:
* RHEL-Edge container image now uses nginx and serves on port 8080
* Add support for RHEL 8.5 for Edge Raw images (x86_64 and Aarch64)
* Add support for RHEL 8.5 Simplified installer (x86_64 and Aarch64)
* Add custom file system support for RHEL 8.5
* Add custom file system support for RHEL 9.0
Contributions from: Ondřej Budai, Alexander Todorov, Brian C. Lane, Tomas Hozza,
Chloe Kaubisch, Xiaofeng Wang, Achilleas Koutsou, Juan Abia,
Gianluca Zuccarelli, Diaa Sami, Christian Kellner,
Sanne Raymaekers, Jakub Rusz
— Vöcklabruck & Bruxelles, 2021-08-28
## CHANGES WITH 31:
* Retrieve metadata about a compose through the Cloud API
* Support multiple repository subscriptions
* Support for building artifacts for Fedora 32 is now dropped
* Add support for new / extended osbuild stages
* Building images for other distributions
* Add support for RHEL 8.5 Edge images
* Add support for RHEL 8.5 main image types
* Add support for RHEL 9.0 Beta
* Weldr API: introduce the ability to limit exposed Image Types by configuration
* Add a heartbeat between composer and the worker every 15 seconds.
* Add oauth2 support to get composer into `cloud.redhat.com`.
Contributions from: Achilleas Koutsou, Alexander Todorov, Brian C. Lane,
Chloe Kaubisch, Colin Walters, Gianluca Zuccarelli,
Jakub Rusz, Juan Abia, Lars Karlitski, Martin Sehnoutka,
Ondřej Budai, Sanne Raymaekers, Thomas Lavocat, Tomas Hozza,
Xiaofeng Wang, Yuxin Sun
— Grenoble, 2021-08-11
## CHANGES WITH 30:
* Cloud API now has support for uploading images directly to Azure. Before,
composer only supported uploading to Azure using the Weldr API (used by
cockpit-composer and composer-cli).
* OSBuild Composer can now generate Manifests that conform to the new
OSBuild schema. Two new image types are added that take advantage
of the new schema: rhel-edge-container and rhel-edge-installer.
* OSBuild Composer can now build RHEL 9.0 guest images.
* OSBuild Composer can now build Fedora 34 and Fedora 35 images. Note that
the work on Fedora 34 hasn't finished yet, stay tuned for more updates
in the near future.
* Cloud API has now support for authorization using an Identity header.
This is useful for running composer in a Kubernetes cluster behind
a reverse proxy.
* Several fixes for RHEL 8.4 images are included in this release.
* Worker osbuild jobs with GCP upload target now set the chosen image name as
custom metadata on the uploaded object. This makes finding the uploaded
object using the image name possible. The behavior is useful mainly
for cleaning up cloud resources in case of unexpected failures.
* Upload to GCP is now more reliable because composer now chooses a random
GCE region instead of fixed one which seems to be often exhausted.
Contributions from: Achilleas Koutsou, Alexander Todorov, Brian C. Lane, Chloe
Kaubisch, Christian Kellner, Jakub Rusz, Jozef Mikovic,
Martin Sehnoutka, Ondřej Budai, Sanne Raymaekers, Tomas
Hozza, Tom Gundersen, Xiaofeng Wang, He Yi, Yuxin Sun
— Liberec, 2021-06-10
## CHANGES WITH 29:
* Cloud API supports Google Cloud Platform as an upload target!
* Cloud API now includes additional information about the upload target in
response for "upload status". It works for both AWS and GCP.
* The kernel osbuild-composer installs is now marked as the `saved_entry`
making sure subsequent kernel installations do not change the default.
Contributions from: Alexander Todorov, Chloe Kaubisch, David Rheinsberg, Martin
Sehnoutka, Ondřej Budai, Sanne Raymaekers, Tomas Hozza, Tom
Gundersen
— Brno, 2021-03-04
## CHANGES WITH 28:
* OSBuild Composer can now build CentOS Stream 8 images!
* OSBuild Composer can now upload images to VMWare!
* Cloudapi contains new /openapi.json and /version endpoints. The already
existing `POST /compose` endpoint now supports specification of additional
packages.
* Blueprints now support definition of alternative kernels, such as kernel-rt,
using the `customizations.kernel.name` key.
* Attempt to use the `customization.kernel.append` key in a blueprint for the
OSTree image type now results in error, because the customization was ignored.
* OSBuild dependency now requires version 24 because of its new sysconfig and
RHSM stages.
* RHEL qcow2 images now come with disabled product-id and subscription-manager
plugins in dnf to make them consistent with images produced by imagefactory.
* OSTree `parent` can now be replaced with `url` pointing to a repository
that contains the parent commit.
Contributions from: Achilleas Koutsou, Alexander Todorov, Antonio Murdaca,
Brian C. Lane, Chloe Kaubisch, Jacob Kozol, Jozef Mikovic,
Major Hayden, Martin Sehnoutka, Ondřej Budai, Sanne Raymaekers,
Tomas Hozza, Tom Gundersen
— Brno, 2021-02-20
## CHANGES WITH 27:
* Starting from this release, we are putting longer news posts in
`docs/news/*/`. We strongly encourage everyone to take a look at them!
* OSBuild Composer now ships with a new osbuild-composer-core subpackage.
It neither requires systemd nor a local worker, so it's perfect to use in
a container. Just connect a remote worker to it and you are good to go!
* The RHEL 8.4 guest image is now much closer to its original kickstart
definition.
* The API for Koji now supports retrieving manifests.
* Uploaded AMIs now have a name tag, making them easier to search for in
the AWS console.
* The repository now contains a `Dockerfile` and `docker-compose.yml` for
a quick setup of a development environment. Feel free to use it and let
us know if you find it useful. We strongly recommend reading `HACKING.md`
to learn about some caveats of this setup.
* The bug allowing users to override a system repository in the Weldr API
is now fixed.
* The project now requires Go 1.14.
Contributions from: Achilleas Koutsou, Brian C. Lane, David Rheinsberg,
Jacob Kozol, Lars Karlitski, Major Hayden, Ondřej Budai,
Sanne Raymaekers, Tomas Hozza, Tom Gundersen
— Kořenov, 2020-02-04
## CHANGES WITH 26:
* RHEL 8.4 images got plenty of updates:
* Image building for aarch64, ppc64le and s390x is fixed.
* The root XFS partition now has a random UUID. This change fixes image
builds on an image built by osbuild-composer.
* QCOW2 images are now closer to the old official ones:
* The default size is now set to 10 GiB.
* rng-tools are no longer installed.
* kernel options are now aligned to the old official images.
* org.osbuild.rhel84 runner is now used to build these images.
* Worker crashed in a koji-finalize job when a previous koji-init job
failed. This is now fixed.
* Composer API has now support for mirrorlist and metalink.
* Composer API now supports sharing an Amazon Machine Image with an another
account.
* Upload of aarch64 images to AWS is now fixed.
* Composer API for Koji returns pending status until all images are
finished. Previously, it returned failed as soon as the first image
build failed.
* Composer API for Koji and Worker API now log errors. This should
very much simplify debugging.
* osbuild-composer(7) man page is now included in the RPM.
* The testing got some very nice updates too:
* The CI now runs a subset of tests on Fedora 33 aarch64.
* The CI now runs reverse dependency tests against koji-osbuild.
Contributions from: Chloe Kaubisch, Christian Kellner, Jacob Kozol,
Lars Karlitski, Ondřej Budai, Sanne Raymaekers,
Tomas Hozza
— Liberec, 2020-12-16
## CHANGES WITH 25:
* Composer now supports RHEL 8.4! Big thanks to Jacob Kozol!
If you want to build RHEL 8.4 using Composer API or Composer API for
Koji, remember to pass "rhel-84" as a distribution name.
* Composer can now be started without Weldr API. If you need it, start
`osbuild-composer.socket` before `osbuild-composer.service` is started.
Note that cockpit-composer starts `osbuild-composer.socket` so this change
is backward compatible.
* When Koji call failed, both osbuild-composer and osbuild-worker errored.
This is now fixed.
* The dependency on osbuild in the spec file is now moved to the worker
subpackage. This was a mistake that could cause the worker to use
an incompatible version of osbuild.
* As always, testing got some upgrades. This time, mostly in the way
we build our testing RPMs.
Contributions from: Jacob Kozol, Lars Karlitski, Ondřej Budai, Tom Gundersen
— Liberec, 2020-11-19
## CHANGES WITH 24:
* Composer now internally supports multi-build composes. A big part of the
jobqueue and worker was rewritten to support this feature.
* Composer API for Koji was adjusted to use the new multi-build feature.
All communication with Koji was moved to the worker so there's no need
to have Koji credentials in composer (it's sufficient to have them in
the worker). Additionally, the API can now correctly handle requests with
multiple images.
* Composer API for Koji has now /compose/{id}/logs route exposing logs to
a caller. Keep in mind that the API specification doesn't guarantee the
field structure, so it may change at any point in the future.
* Composer API returned statuses that were not defined in the API
specification. This is now fixed.
* As always, there we are improvements in the testing pipeline. The biggest
change is the introduction of Fedora 33 in composer's CI.
Contributions from: Chloe Kaubisch, Lars Karlitski, Martin Sehnoutka,
Ondřej Budai, Tom Gundersen
— Liberec, 2020-11-11
## CHANGES WITH 23:
* Support for building Fedora 31 images was removed.
* Metadata sent from Composer to Koji were adjusted based on a feedback
from Koji maintainers. More fixes will definitely come in a future
release.
* Composer is now easier to deploy to OpenStack with a new deploy-openstack
script available in the source tree. Note that the previous version
introduced a similar tool for deploying a local qemu VM.
* The testing setup is still being reworked massively. With the help
of deploy-qemu, it should be now very simple to replicate the Schutzbot
tests on a local machine.
Contributions from: Alexander Todorov, Lars Karlitski, Ondřej Budai, Tom
Gundersen, Xiaofeng Wang
— Liberec, 2020-11-04
## CHANGES WITH 22:
* Support for building Fedora 33 images is now available as a tech preview.
* The osbuild-composer-cloud binary is gone. The osbuild-composer binary
now serves the Composer API along with Weldr and Koji APIs.
* The testing setup was reworked. All files related to tests are now shipped
in the tests subpackage. A script to run the test suite locally is now
also available. See HACKING.md for more details.
* GPG keys in Koji API are no longer marked as required.
* Osbuild-composer RPM is now buildable on Fedora 33+ and Fedora ELN.
* Osbuild-composer for Fedora 34 and higher now obsoletes lorax-composer.
Contributions from: Alexander Todorov, Jacob Kozol, Lars Karlitski,
Martin Sehnoutka, Ondřej Budai, Tom Gundersen
— Liberec, 2020-10-16
## CHANGES WITH 21:
* Composer API is now available as a tech preview in the
osbuild-composer-cloud subpackage. It's meant to be a simple API that
allows users build an image and push it to a cloud provider. It doesn't
support advanced features like storing blueprints as Weldr API does. This
is not stable API, and is subject to incompatible change.
* Koji API is now available in the -koji subpackage. It can be used
to perform an image build and push the result directly to a Koji
instance.
* Worker API is now completely overhauled. Support for distinguishing
architectures is added and the whole API is generated from an OpenAPI
spec.
* Weldr API's /projects/source/new route now explicitly requires the url
field.
* The project now requires Go 1.13.
* Testing of vmware and ostree images is now greatly improved.
* All bash scripts are now checked with shellcheck on the CI.
Contributions from: Alexander Todorov, Lars Karlitski, Major Hayden,
Martin Sehnoutka, Ondřej Budai, Peter Robinson,
Sanne Raymaekers, Tom Gundersen, Xiaofeng Wang
— Liberec, 2020-09-24
## CHANGES WITH 20:
* VMDK images are now stream optimized to be compatible with vCenter by
defult.
* RPMs are pulled from the correct repositories on RHEL, depending on whether
the host is running on Beta or GA.
* Cloud credentials can now no longer be returned by the API.
Contributions from: Alexander Todorov, Brian C. Lane, Lars Karlitski,
Major Hayden, Tom Gundersen
— London, 2020-08-23
## CHANGES WITH 19:
* Bug fixes to the weldr API.
* Default image size was increased to be able to build empty blueprints by
default.
* OpenStack images are now tested on the target footprint in CI.
* Other test improvements.
Contributions from: Alexander Todorov, Brian C. Lane, Jenn Giardino,
Major Hayden, Martin Sehnoutka
— London, 2020-08-10
## CHANGES WITH 18:
* Qcow and openstack images for Fedora have now cloudinit service enabled
by default. This change leads to a higher consistency with the official
images.
* Fedora 32 image builds were failing if an installed package shipped
a custom SELinux policy. This is now fixed.
* The DNF integration now uses the fastestmirror plugin. This should lead
to faster and more reliable depsolves.
* Tar archives returned from Weldr routes could have contained files with
a timestamp newer than the current time. This led to warnings when
untarring these archives. The timestamps are now fixed.
* The RCM subpackage was removed. It was never properly finished and will
be superseded by a Koji integration at some point.
Contributions from: Chloe Kaubisch, Christian Kellner, David Rheinsberg,
Lars Karlitski, Major Hayden, Martin Sehnoutka,
Ondřej Budai, Tom Gundersen
— Liberec, 2020-07-22
## CHANGES WITH 17:
* AWS images are now built in the raw format. Previously used vhdx was
space-efficient but actually caused about 30% of uploads to fail.
* The spec file had a wrong version of lorax-composer to obsolete, causing
upgrades to fail. This is now fixed.
Contributions from: Major Hayden, Tom Gundersen
— Liberec, 2020-07-08
## CHANGES WITH 16:
* osbuild-composer now obsoletes lorax-composer on RHEL.
* An upload failure (e.g. due to invalid credentials) now causes the compose
to appear as failed.
* RHEL 8 repositories are switched to the beta ones to allow composer to be
tested on 8.3 Beta. This will be reverted when GA comes.
* OSTree images no longer contains /etc/fstab. The filesystem layout is
determined by the installer and thus it doesn't make any sense to include
it.
* If both group and user customizations were used, the user would be created
before the group, causing a build to fail. This is now fixed.
* Composer now correctly passes UID and GID to org.osbuild.{users,groups}
stages as ints instead of strings.
* The subpackages (worker, tests and rcm) now require a matching version of
osbuild-composer to be installed. Previously, they would be happy with
just an arbitrary one.
* Support for testing OpenStack images in actual OpenStack is now available.
Note that upload to OpenStack is still not available for the end users
(it's on the roadmap though).
* Worker now logs not only job failures but also job successes.
* All DNF errors were mistakenly tagged as RepoError, this is now fixed.
* As always, a lot of test and CI improvements are included in this release.
Contributions from: Alexander Todorov, Christian Kellner, Major Hayden, Martin
Sehnoutka, Ondřej Budai, Tom Gundersen
— Liberec, 2020-06-29
## CHANGES WITH 15:
* Support for building RHEL for Edge is now available.
* Composer has now support for building QCOW2 and tar images for ppc64le and
s390x architectures.
* Tar images for RHEL have returned. The Image Builder team found out that
they are used as a way to install RHEL for Satellite.
* Blueprints containing packages with a wildcard version no longer causes
the built image to have both x86_64 and i686 versions of one package
installed.
* GPG check is now disabled by default. If you have a custom
repository in /etc/osbuild-composer/repositories, just set gpg_check
to true to enable the check. Note that all the pre-defined repositories
have GPG check enabled.
* Composer now supports a cancellation of jobs. This can be done by calling
/compose/cancel route of Weldr API.
* osbuild-composer previously crashed when osbuild didn't return the right
machine-readable output (e.g. because of a disk being out of space). This
is now fixed.
* Because of the GPG check change and RHEL for Edge support, composer
now requires osbuild 17 or higher.
* osbuild-composer previously required the python package to be installed
on RHEL. Now, it uses the always-installed platform-python.
* The buildroot for RHEL 8 didn't have selinux labels before. This is now
fixed.
* When Composer crashed, it left temporary directories in /var/cache. The
temporary directories are now moved to /var/tmp, which is managed by
systemd with PrivateTmp set to true, so they're now correctly removed
after a crash.
* Several weldr API routes were aligned to work in the same way as with
Lorax. /blueprints/freeze now correctly supports option to output TOML.
Projects and modules routes return all fields as Lorax returns.
* AWS upload now logs the current state to the system journal. Emojis are
of course included. 🎉
* As always, amazing improvements in the CI infrastructure happened. Also,
the test coverage went up. Thanks all for doing this!
Contributions from: Alexander Todorov, Brian C. Lane, Christian Kellner,
Jakub Rusz, Lars Karlitski, Major Hayden, Martin
Sehnoutka, Ondřej Budai, Peter Robinson, Tom
Gundersen
— Liberec, 2020-06-12
## CHANGES WITH 14:
* AWS uploads doesn't anymore report to AWS that composer uploads
the image in vhdx format. This surprisingly makes the upload process
more stable.
* Uploads were always in WAITING state. This is now fixed.
* The /projects/source/* routes now correctly supports all the features
of Weldr API v1.
* AWS upload now logs the progress to journal. Even better logging is
hopefully coming soon.
* AWS upload's status is now correctly set to FAILED when ImportSnapshot
fails. Before, this hanged the upload indefinitely.
* Store unmarshalling is now safer in some cases. For example, stored
manifests are now longer checked when loaded from disk. Therefore,
changing of manifest schema doesn't lead to crashes when old manifests
are present in the store.
* When store loading failed in non-verbose mode of osbuild-composer, it
crashed the process because of nil logger. This is now fixed.
* The upstream spec file for building osbuild-composer package now
excludes the i686 architecture. Note that composer never supported
this arch.
* The upstream spec file now correctly specifies the composer's dependency
to osbuild-ostree. This was forgotten in the previous release which
introduced Fedora IoT support.
* The previous version didn't have repositories defined for s390x and
ppc64le architectures. This is now fixed. Note that this only fixes
some codepaths, osbuild-composer still cannot build any images on
these architectures.
Contributions from: Brian C. Lane, Lars Karlitski, Major Hayden, Martin
Sehnoutka, Ondřej Budai, Stef Walter, Tom Gundersen
— Liberec, 2020-06-03
## CHANGES WITH 13:
* Fedora IoT is now supported for Fedora 32 in the form of producing the
commit tarball. Feel free to test it and report any issues you find.
* Support for RHEL was completely revamped. Now, osbuild-composer supports
building images only for the latest RHEL 8. The separate minor versions
are no longer available. Additionally, it now uses the Red Hat CDN which
requires the host system to be properly subscribed. If you need to use
different package repositories to build RHEL from, use a repository
override in /etc/osbuild-composer/repositories.
* Several image types were removed: ext4-filesystem, partitioned-disk,
and tar. The use-cases for these image types were not clearly defined and
without a clear definition, it was very hard to define test cases for
them.
* Support for Fedora 30 was dropped as it is now EOL. So long and thanks
for all the fish!
* The timeout for AWS upload is removed. It's very hard to predict how long
will the AWS upload take. With the timeout in place, it caused the test
suite to produce a lot of false positives.
* Build logs were broken in the previous release, this release fixes it.
This time, they were properly saved but weldr API read them from a wrong
location. This is now fixed and covered with basic tests.
* Weldr API has now support for /compose/metadata and /compose/results
routes. This allows users to easily access a manifest used to build
an image.
* Preliminary support for ppc64le and s390x is added to RHEL distribution.
No images cannot be built yet but at least it won't crash on startup.
* The weldr API socket has now correct permissions. As the result, it can
be read and written only by root and the weldr group. This is the same
behaviour as Lorax has.
* By mistake, workers incorrectly used the default store for every build.
However, this can currently cause the store to grow indefinitely, so
this release switched the osbuild store to use a temporary directory again.
* /status route in weldr API now correctly returns msgs field.
* Handling of json (un)marshalling in store is revamped. It should
make it more stable and simplify the maintenance of the store backwards
compatibility.
* Initial support for koji is now added. It's currently not hooked up
to composer and only supports password authentication. More coming soon.
* Again, the automated testing was greatly improved during this cycle,
big thanks to everyone involved!
Contributions from: Alexander Todorov, Brian C. Lane, David Rheinsberg, Jacob
Kozol, Lars Karlitski, Major Hayden, Ondřej Budai, Tom
Gundersen
— Liberec, 2020-05-28
## CHANGES WITH 12:
* In previous versions support for running remote workers was
broken. This is now fixed and running remote workers is once
again possible. See #568 for more information.
* The job queue and the store are now two separate Go packages.
One of the benefits is that it is now possible to build images
without using the store which is too complicated for some usecases.
* A blueprint name is now checked against the regex
`^[a-zA-Z0-9._-]+$`. This is the same limitation as in
lorax-composer.
* All osbuild calls now use the new --output-directory argument.
This change is a must because the old way of retrieving images from
the osbuild store will soon be deprecated.
* Some routes from the weldr API are now implemented in a more
efficient way.
* As always, the team worked hard on improving the tests and the CI.
Contributions from: Brian C. Lane, David Rheinsberg, Jiri Kortus, Lars
Karlitski, Major Hayden, Ondřej Budai
— Liberec, 2020-05-13
## CHANGES WITH 11:
* The support for uploading VHD images to Azure is now available.
* AMI images are now produced in the vhdx format. This fixes
the issue that those images couldn't be previously booted in EC2.
* In version 10 the logs weren't saved when osbuild failed. This
is now fixed.
* The warnings when upgrading/removing the RPM package are now fixed.
Note that updating to version 11 still produces them because
the upgrade process runs also the scriptlets from version 10.
* The size calculation for Fedora 31 vhd images is fixed.
* The size field was removed from the tar assembler struct.
The field has actually never been supported in osbuild
and it doesn't make any sense.
* The minimal required version of osbuild is bumped to 12.
* This release also got big upgrades to the testing infrastructure,
more tests are run on a CI and they now run faster. Also, the unit
test coverage is improved.
Contributions from: Alexander Todorov, Jacob Kozol, Jakub Rusz,
Jiri Kortus, Lars Karlitski, Major Hayden,
Ondřej Budai, Tom Gundersen
— Liberec, 2020-04-29
## CHANGES WITH 10:
* The correct `metadata_expire` value is now passed to dnf. In the
past, this led to a lot of failed builds, because dnf has the
default expire time set to 48 hours, whereas the Fedora updates
repos have the expire time of 6 hours.
* A decision was made that the minimal Go version required for
building the project is 1.12. This is now enforced by the CI.
* The intermediate s3 object is now deleted after the upload to AWS
is finished. It has no value for users.
* The upload to AWS has now a bigger timeout. The current coronavirus
situation is affecting the AWS responsiveness in a negative way.
* The weldr API has better test coverage. In the process, several
bugs in sources and composes were fixed.
* Worker and jobqueue packages are receiving a big refactoring.
This is the prerequisite for having multiple job queues for building
images for different distributions and architectures.
* The image tests now boot the AWS images in the actual EC2.
Contributions from: Alexander Todorov, Brian C. Lane,
Jacob Kozol, Jakub Rusz, Lars Karlitski,
Major Hayden, Martin Sehnoutka,
Ondřej Budai, Tom Gundersen
— Liberec, 2020-04-15
## CHANGES WITH 9:
* Fedora is now build with updates and modules repositories
enabled, therefore up-to-date images are now produced.
* A new man-page `osbuild-composer(7)` with high-level
description of the project is now available. It can be built
by the new man target in the Makfile.
* All Fedora images have now a generic initramfs. This should
make the images more reproducible and less likely failing to boot
if the image build was done in a less usual environment.
* Metalink is now used to access the Fedora repositories. This change
should hopefully lead to more stable builds.
* Composer is now released to Fedora 32 and 33 in a new
osbuild-composer package. The old golang-github-osbuild-composer
package will be automatically upgraded to the new one.
* The internal osbuild-pipeline command now has a more user-friendly
interface.
* The RCM API (in development, experimental) is reworked to allow
any distribution-architecture-image type combination.
* The work on a high-level description of image types began.
See image-types directory.
* The osbuild-worker arguments are reworked, they are now much more
flexible.
* The image-info tool used in the integration tests can be now run
on Fedora 32.
* The unit test coverage is now much bigger, thanks to all
contributors!
* Internal distribution representation is significantly reworked,
this simplifies the process of adding the support for all currently
missing architectures.
* Integration tests were also improved, the image tests are fully
switched to the new Go implementation and an automatic way
of generating test cases is added. The weldr API coverage is also
much better. Several bugs in it were fixed in the process.
* Codecov.io is now used to monitor the test coverage of the code.
* As always, minor fixes and improvements all over the place.
Contributions from: Alexander Todorov, Brian C. Lane, David
Rheinsberg, Jacob Kozol, Jakub Rusz, Jiri
Kortus, Lars Karlitski, Martin Sehnoutka,
Ondřej Budai, Tom Gundersen
— Liberec, 2020-04-01
## CHANGES WITH 8:
* All generated pipelines now use the `org.osbuild.rpm` stage of
osbuild, rather than `org.osbuild.dnf`. This improves on splitting
resource acquisition from image building and should make image
composition more reliable and faster.
* The `STATE_DIRECTORY` environment variable now allows changing the
state directory path of `osbuild-composer`. This is to support older
systemd versions that do not pass in `StateDirectory=` to the service
executable.
* Minor fixes and improvements all over the place.
Contributions from: Alexander Todorov, Brian C. Lane, Jacob Kozol, Jakub
Rusz, Lars Karlitski, Major Hayden, Martin
Sehnoutka, Ondřej Budai, Tom Gundersen
— Berlin, 2020-03-18
## CHANGES WITH 7:
* Support for `RHEL 8.1` as image type is now available.
* Semantic versioning of blueprints in the lorax API is now enforced.
This was always the case for the original lorax API, and *Composer*
now follows this as well.
* Lots of internal improvements, including many automatic tests,
improved error handling, better cache directory management, as well
as preparations to move over from `org.osbuild.dnf` to
`org.osbuild.rpm` in all build pipelines.
Contributions from: Alexander Todorov, Brian C. Lane, Jacob Kozol, Lars
Karlitski, Major Hayden, Ondřej Budai, Tom Gundersen
— Berlin, 2020-03-05
## CHANGES BEFORE 7:
* Initial implementation of 'osbuild-composer'.
Contributions from: Alexander Todorov, Brian C. Lane, Christian Kellner,
Jacob Kozol, Jakub Rusz, Lars Karlitski, Martin
Sehnoutka, Ondřej Budai, Tom Gundersen

1
README.md
View file

@ -22,6 +22,7 @@ instance, includes a
* **Website**: <https://www.osbuild.org>
* **Bug Tracker**: <https://github.com/osbuild/osbuild-composer/issues>
* **IRC**: #osbuild on [Libera.Chat](https://libera.chat/)
* **Changelog**: <https://github.com/osbuild/osbuild-composer/releases>
#### Contributing

9
docs/news/27/koji-manifests.md
View file

@ -1,9 +0,0 @@
# Koji API: New endpoint for getting the manifests of a compose job
A new endpoint is available in the Koji API: `GET /compose/{ID}/manifests`.
Returns the manifests for a running or finished compose. Returns one manifest
for each image in the request, in the order they were defined.
Relevant PRs:
https://github.com/osbuild/osbuild-composer/pull/1155
https://github.com/osbuild/osbuild-composer/pull/1165

18
docs/news/27/rhel84-differences-fixed.md
View file

@ -1,18 +0,0 @@
# RHEL 8.4: Update rhel-84 distro to better match imagefactory's qcow2
There are minor discrepancies between our nightly image and the imagefactory's
qcow2. These differences are mainly in the installed packages, enabled services,
and disabled services. To remedy these differences the following changes have
been made:
The following packages have been added to our qcow2 image: oddjob,
oddjob-mkhomedir, psmisc, authselect-compat, dbxtool.
The following packages have been removed from our qcow2 image:
dnf-plugin-spacewalk, fwupd, nss, and udisks2.
The following services have been enabled: nfs-convert.service.
The following services have been removed/disabled: mdmonitor.service,
udisks2.service, fwupd-refresh.timer, mdcheck_continue.timer,
mdcheck_start.timer, and mdmonitor-oneshot.timer.

7
docs/news/27/timedatex.md
View file

@ -1,7 +0,0 @@
# RHEL 8.4: Include timedatex in qcow2 images
Timedatex was an excluded package due to an selinux-policy issue that has been
fixed. Therefore, timedatex should be in the qcow2 image we build. Our list of
excluded packages for RHEL 8.4 was not being included in our nightly builds so
we did not realize that timedatex was still being excluded. The issue with the
excluded packages is now fixed and timedatex is now removed from this list.

5
docs/news/28/centos-stream.md
View file

@ -1,5 +0,0 @@
# Add support for CentOS Stream 8
OSBuild Composer can now build CentOS Stream 8 images. The image definitions
are exactly the same as for the latest supported RHEL 8.y release (8.4
currently ).

9
docs/news/28/cloudapi-packages.md
View file

@ -1,9 +0,0 @@
# Cloud API: The compose endopint now allow additional package selection
The `POST /compose` endpoint has now been extended to allow packages to
be requested in addition to the base ones for the image type. Packages
can only be requested by name, and the most recent ones that satisfy
dependency solving will be chosen.
Relevant PR:
https://github.com/osbuild/osbuild-composer/pull/1208

13
docs/news/28/kernel-customizations-name.md
View file

@ -1,13 +0,0 @@
# Blueprint: Kernel name customization
When creating ostree commits, only one kernel package can be installed at a
time, otherwise creating the commit will fail in rpm-ostree. This prevents
ostree type builds (RHEL for Edge and Fedora IoT) to add alternative kernels,
in particular, the real-time kernel (`kernel-rt`).
Blueprints now support defining the name of the kernel to be used in an image,
through the `customizations.kernel.name` key. If not specified, the default
`kernel` package is included as before.
Relevant PRs:
https://github.com/osbuild/osbuild-composer/pull/1175

6
docs/news/28/openapi-endpoint-in-cloudapi.md
View file

@ -1,6 +0,0 @@
# CloudAPI now supports /openapi.json and /version endpoints
These endpoints are useful for deployment in the cloud.redhat.com platform.
The platform will use data from the /openapi.json endpoint to automatically
generate documentation for the REST API. A user can see the documentation
in the cloud.redhat.com web console.

5
docs/news/28/osbuild-24.md
View file

@ -1,5 +0,0 @@
# OSBuild: spec: update to osbuild version 24
In order to add the newly supported sysconfig stage, the osbuild dependency
needs to be updated to version 24. This update is reflected in both the spec
file dependency and in the testing dependency.

13
docs/news/28/osbuild-rhsm-stage.md
View file

@ -1,13 +0,0 @@
# Add support for `org.osbuild.rhsm` osbuild stage
Add support for `org.osbuild.rhsm` osbuild stage. This stage is available in
osbuild since version 24. The stage currently allows only configuring the
enablement status of two RHSM DNF plugins, specifically of `product-id` and
`subscription-manager` DNF plugins.
# RHEL 8.3 & 8.4: Disable all RHSM DNF plugins on qcow2 image
Disable both available RHSM DNF plugins (`product-id` and
`subscription-manager`) on rhel-8 and rhel-84 qcow2 images. The reason for
disabling these DNF plugins is to make the produced images consistent in this
regard, with what had been previously produced by the imagefactory.

9
docs/news/28/ostree-kernel-append.md
View file

@ -1,9 +0,0 @@
# OSTree compose types with kernel boot parameters return error
Previously, specifying Kernel boot parameters in a Blueprint via the
`[customizations.kernel]` section and requesting an OSTree image type
(`rhel-edge-commit` or `fedora-iot-commit`) would produce an image but the boot
parameters would be ignored.
This combination now returns an error message that the configuration is not
supported.

13
docs/news/28/sysconfig.md
View file

@ -1,13 +0,0 @@
# RHEL 8.4: add support for org.osbuild.sysconfig stage
The kernel and network sysconfigs need to have certain values set in RHEL 8.4.
Currently, the following values are set for all image types in 8.4:
kernel:
UPDATEDEFAULT=yes
DEFAULTKERNEL=kernel
network:
NETWORKING=yes
NOZEROCONF=yes

13
docs/news/28/vmware-upload-target.md
View file

@ -1,13 +0,0 @@
# Weldr API: New VMWare upload target
New upload target is available that allows users to push built VMWare images directly to vSphere without the need to download and push them manually.
Upload target requires following options:
`Username`,
`Password`,
`Host`,
`Datacenter`,
`Datastore`,
`Cluster`.
Relevant PR: https://github.com/osbuild/osbuild-composer/pull/1169

40
docs/news/28/weldr-ostree-url.md
View file

@ -1,40 +0,0 @@
# Weldr API: Allow parent OSTree commit to be read from repository
The weldr API for building OSTree based images is extended to optionally take an `url` parameter instead of the current `parent`.
The `parent` parameter contains the OSTree commit SHA of the parent commit when building an update commit. Obtaining this is cumbersome, so instead the `url` of the repository containing the desired parent commit can be specified. In this case, composer will take the current `HEAD` of the given `ref` as the parent.
At most one of `parent` and `url` can be specified in a given compose request.
Before:
curl --silent \
--header "Content-Type: application/json" \
--unix-socket /run/weldr/api.socket \
http://localhost/api/v1/compose \
--data "{ \
\"blueprint_name\": \"foo\", \
\"compose_type\": \"rhel-edge-commit\", \
\"ostree\": {\ \
\"parent\": \"b8a69e5c79be5830bb272356809a52b1660d2013c26f6973d549d0a312a8d21a\", \
\"ref\": \"fedora/stable/x86_64/iot\" \
} \
}"
After:
curl --silent \
--header "Content-Type: application/json" \
--unix-socket /run/weldr/api.socket \
http://localhost/api/v1/compose \
--data "{ \
\"blueprint_name\": \"foo\", \
\"compose_type\": \"rhel-edge-commit\", \
\"ostree\": {\ \
\"url\": \"https://d2ju0wfl996cmc.cloudfront.net/\", \
\"ref\": \"fedora/stable/x86_64/iot\" \
} \
}"
Relevant PRs:
https://github.com/osbuild/osbuild-composer/pull/1235

6
docs/news/29/cloudapi-gcp-support.md
View file

@ -1,6 +0,0 @@
# Cloud API: Add support for Google Cloud Platform target
Added support in Cloud API to upload images to Google Cloud Platform (GCP) and
share them with specified accounts. There is currently no GCP-specific image
type, since GCP supports importing VHD and VMDK image types already supported by
the `osbuild-composer`.

16
docs/news/29/cloudapi-upload-status.md
View file

@ -1,16 +0,0 @@
# Cloud API: include upload target-specific options in `UploadStatus`
The `UploadStatus` now includes additional information in its `options` property.
The information is specific to the chosen target Cloud provider and it is necessary
to successfully identify the built and shared OS image by the end user. Currently
this information is returned for both supported targets, **AWS** and **GCP**.
Information included for **AWS** target:
- AMI
- Region
Information included for **GCP** target:
- Image name
- Image's source Project ID

5
docs/news/29/rhel84-grub2-saved-entry.md
View file

@ -1,5 +0,0 @@
# RHEL8.4: Fix grub2 kernel selection
By marking the kernel we install as the `saved_entry`, we make sure that installing additional/subsequent kernels do not unintentionally change the default kernel to be booted into.
Relevant PR: https://github.com/osbuild/osbuild-composer/pull/1241

13
docs/news/30/cloudapi-azure-support.md
View file

@ -1,13 +0,0 @@
# Cloud API: Add support for uploading to Azure
Cloud API now has support for uploading images directly to Azure. Before,
composer only supported uploading to Azure using the Weldr API (used by
cockpit-composer and composer-cli). Also, it only created a storage
blob requiring the user to do one extra step to run a VM.
The new Azure Image upload target creates a finished Azure Image that can
be immediately used to launch a VM. It also uses the Azure OAuth-based
authentication that doesn't require the user to give composer any credentials.
Note that this is currently only available for the Cloud API. If you are
a user of the Weldr API, you can still use the older method.

10
docs/news/30/cloudapi-image_status-status-value.md
View file

@ -1,10 +0,0 @@
# Cloud API: fix `image_status.status` value for running compose
Previously, the Cloud API endpoint `/v1/compose/{id}` return value's
`image_status.status` for a running worker job was "running", which didn't
comply with the Cloud API specification. Equivalents allowed by the API
specification are "building", "uploading" or "registering".
Return "building" as the `image_status.status` value for a running compose,
instead of "running". Returning the remaining "uploading" and "registering"
values is not yet implemented.

18
docs/news/30/osbuild-schema-2.md
View file

@ -1,18 +0,0 @@
# Support new OSBuild pipelines and new RHEL for Edge image types
OSBuild Composer can now generate Manifests that conform to the new OSBuild
schema. Two new image types are added that take advantage of the new schema:
- `rhel-edge-container`: Creates an OCI container with an embedded
`rhel-edge-commit`. Running the container starts a web server that serves
the commit.
- `rhel-edge-installer`: Creates a boot ISO image that embeds a
`rhel-edge-commit`. The commit is pulled from a URL during the compose of
the boot ISO.
Requesting a `rhel-edge-installer` requires specifying a URL, otherwise the
request will fail. Blueprint customizations have no effect on the boot ISO and
also cause the request to fail if any are specified.
Relevant PR: https://github.com/osbuild/osbuild-composer/pull/1244

3
docs/news/30/ostree-ref-handling.md
View file

@ -1,3 +0,0 @@
# Improve OSTree Repository URL and Ref parsing
If the OSTree Repository URL did not end in a `/` the parsing would fail with a less-than-useful error message. This has been fixed. Error messages for different failure cases have also been improved.

82
docs/news/30/repo-conf-image-type-tags.md
View file

@ -1,82 +0,0 @@
# Allow image type-specific repositories using Image Type Tags
The schema of the repository definitions used by *Weldr API*, located in `/usr/share/osbuild-composer/repositories/` or `/etc/osbuild-composer/repositories` is extended with a new field called **`image_type_tags`** and is expected to be an array of strings representing specific image types.
The behavior of how are defined repositories processed and used by osbuild-composer* is extended in the following way:
1. If the repository definition does not have the `image_type_tags` field specified, then it will be used for building all types of images for a given distribution and architecture. This is how all repository definitions had been used before this change.
1. If the repository definition has the `image_type_tags` field specified and set to a non-empty array of strings, then it will be used **only** for building image types, which names are specified in the array.
An example of a user-defined repository override for Fedora 33 in `/etc/osbuild-composer/repositories/fedora-33.json` follows. In addition to Fedora distribution repositories, it defines an additional repository called `my-custom-repo`, which should be used only for `ami` images built on both architectures.
```json
{
"x86_64": [
{
"name": "fedora",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=fedora-33&arch=x86_64",
"gpgkey": "...",
"check_gpg": true
},
{
"name": "updates",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=updates-released-f33&arch=x86_64",
"gpgkey": "...",
"check_gpg": true
},
{
"name": "fedora-modular",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=fedora-modular-33&arch=x86_64",
"gpgkey": "...",
"check_gpg": true
},
{
"name": "updates-modular",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=updates-released-modular-f33&arch=x86_64",
"gpgkey": "...",
"check_gpg": true
},
{
"name": "my-repo",
"metalink": "https://repos.example.org/f33/x86_64",
"gpgkey": "...",
"check_gpg": true,
"image_type_tags": ["ami"]
}
],
"aarch64": [
{
"name": "fedora",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=fedora-33&arch=aarch64",
"gpgkey": "...",
"check_gpg": true
},
{
"name": "updates",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=updates-released-f33&arch=aarch64",
"gpgkey": "...",
"check_gpg": true
},
{
"name": "fedora-modular",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=fedora-modular-33&arch=aarch64",
"gpgkey": "...",
"check_gpg": true
},
{
"name": "updates-modular",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=updates-released-modular-f33&arch=aarch64",
"gpgkey": "...",
"check_gpg": true
}
{
"name": "my-repo",
"metalink": "https://repos.example.org/f33/aarch64",
"gpgkey": "...",
"check_gpg": true,
"image_type_tags": ["ami"]
}
]
}
```

7
docs/news/30/rhel83-84-difference-fixed.md
View file

@ -1,7 +0,0 @@
# RHEL 8.4: Update rhel-84 distro to better match RHEL 8.3
This restores net-tools to the default package set.
In RHEL8.3 cloud-init depended on net-tools, but in RHEL8.4,
the dependency was dropped. We still want net-tools in the
default package set, so add the dependency explicitly.

6
docs/news/30/rhel84-qcow2-compat.md
View file

@ -1,6 +0,0 @@
# RHEL 8.4: qcow2 images can now be used by older QEMUs
Previously, the guest image for RHEL 8.4 was only usable by QEMU 1.1 and
newer. However, this image should be usable on RHEL 6 that ships an older
version of QEMU. This is now fixed and the guest image can be now used by
QEMU 0.10 and newer.

4
docs/news/30/rhel9.md
View file

@ -1,4 +0,0 @@
# Add support for RHEL 9
OSBuild Composer can now build RHEL 9.0 images.
Support is currently limited to qcow2 image type.

6
docs/news/30/worker-gcp-uploaded-object-meta.md
View file

@ -1,6 +0,0 @@
# Worker: Set image name as custom metadata on the file uploaded to GCP Storage
Worker osbuild jobs with GCP upload target now set the chosen image name as
custom metadata on the uploaded object. This makes finding the uploaded
object using the image name possible. The behavior is useful mainly
for cleaning up cloud resources in case of unexpected failures.

7
docs/news/31/cloud-api-metadata.md
View file

@ -1,7 +0,0 @@
# Retrieve metadata about a compose through the Cloud API
A new endpoint is available in the Cloud API at `compose/id/metadata`. This
endpoint returns a full package list (NEVRA) for the image that was built and
the OSTree commit ID for Edge (OSTree) image types.
PR: https://github.com/osbuild/osbuild-composer/pull/1490

15
docs/news/31/composer-supports-multiple-subscription.md
View file

@ -1,15 +0,0 @@
# Support multiple repository subscriptions
RHEL systems can have multiple subscriptions to different repositories.
Each repository can use its certificate authority and require the users
to authenticate with a client-side TLS certificate.
This is common while using Red Hat Satellite, for example.
osbuild-composer can now work with multiple subscriptions that are available
on the host system. If used with a remote worker, the same subscriptions
must be available on both systems.
Relevant PRs:
https://github.com/osbuild/osbuild-composer/pull/1405
https://github.com/osbuild/osbuild/pull/645

5
docs/news/31/fedora-32-support-dropped.md
View file

@ -1,5 +0,0 @@
# Support for building artifacts for Fedora 32 is now dropped
Fedora 32 is now EOL so we decided to drop support for building images for it.
If you need an image for this distribution, you can use an older version of
osbuild-composer.

5
docs/news/31/hosted-edge-commit.md
View file

@ -1,5 +0,0 @@
# Build Edge commits in Image Builder and upload to S3
Edge commit image types can now be built through the Cloud API (Image Builder). Edge commits are uploaded to an S3 bucket and are downloadable through a presigned URL that is available for up to 7 days.
PR: https://github.com/osbuild/osbuild-composer/pull/1439

16
docs/news/31/new-osbuild-stages.md
View file

@ -1,16 +0,0 @@
# Add support for new / extended osbuild stages
Add support for the following new osbuild stages:
- `org.osbuild.modprobe` - allows to configure modprobe using configuration files
- `org.osbuild.dracut.conf` - allows to create dracut configuration files
- `org.osbuild.systemd-logind` - allows to create system-logind configuration drop-ins
- `org.osbuild.cloud-init` - allows to configure cloud-init
- `org.osbuild.authselect` - allows to set system identity and auth sources using authselect
Add support for new functionality of existing osbuild stages:
- `org.osbuild.sysconfig` - allows to create network-scripts ifcfg files
- `org.osbuild.systemd` - allows to create `.service` file drop-ins
- `org.osbuild.chrony` - allows to configure NTP `servers` with lower level configuration options
- `org.osbuild.keymap` - allows to configure X11 keyboard layout

145
docs/news/31/other-distro-build.md
View file

@ -1,145 +0,0 @@
# Building images for other distributions
Previously osbuild-composer could only build images for the same distribution
as the host. With the addition of the distro field in blueprint it is now
possible to build for any supported distribution shipped with osbuild-composer.
## New API route: /distros/list
The API now supports listing the available distributions. It will return a JSON
object listing the installed distro names that can be used by blueprints,
sources, and the optional `?distro=` selection on API routes.
eg. `curl --unix-socket /run/weldr/api.socket http://localhost/api/v1/distros/list`
{
"distros": [
"centos-8",
"fedora-32",
"fedora-33",
"rhel-8",
"rhel-84",
"rhel-85",
"rhel-90"
]
}
## Distribution selection with blueprints
The blueprint now supports a new `distro` field that will be used to select the
distribution to use when composing images, or depsolving the blueprint. If
`distro` is left blank it will use the host distribution. If you upgrade the
host operating system the blueprints with no `distro` set will build using the
new os.
eg. A blueprint that will always build a fedora-32 image, no matter what
version is running on the host:
name = "tmux"
description = "tmux image with openssh"
version = "1.2.16"
distro = "fedora-32"
[[packages]]
name = "tmux"
version = "*"
[[packages]]
name = "openssh-server"
version = "*"
## Using sources with specific distributions
A new optional field has been added to the repository source format. It is a
list of distribution strings that the source will be used with when depsolving
and building images.
Sources with no `distros` will be used with all composes. If you want to use a
source for a specific distro you set the `distros` list to the distro name(s)
to use it with.
eg. A source that is only used when depsolving or building fedora 32:
check_gpg = true
check_ssl = true
distros = ["fedora-32"]
id = "f32-local"
name = "local packages for fedora32"
system = false
type = "yum-baseurl"
url = "http://local/repos/fedora32/projectrepo/"
This source will be used for any requests that specify fedora-32, eg. listing
packages and specifying fedora-32 will include this source, but listing
packages for the host distro will not.
## Optional distribution selection for routes
Many of the API routes now support selecting the distribution to use when
returning results. Add `?distro=<DISTRO-NAME>` to the API request and it will
return results using `fedora-32` instead of the host distro.
The following routes support distro selection:
* /compose/types
* /modules/list
* /modules/info
* /projects/list
* /projects/info
* /projects/depsolve
The compose start uses the distribution specified by the blueprint to select
which one to use.
eg. Show the image types supported by `centos-8`:
curl --unix-socket /run/weldr/api.socket http://localhost/api/v1/compose/types?distro=centos-8
{
"types": [
{
"name": "ami",
"enabled": true
},
{
"name": "openstack",
"enabled": true
},
{
"name": "qcow2",
"enabled": true
},
{
"name": "tar",
"enabled": true
},
{
"name": "vhd",
"enabled": true
},
{
"name": "vmdk",
"enabled": true
}
]
}
## Unknown Distributions
If an unknown distribution is selected the response from the API server will be
a `DistroError`, like this:
{
"status": false,
"errors": [
{
"id": "DistroError",
"msg": "Invalid distro: fedora-1"
}
]
}

4
docs/news/31/rhel8.5-edge.md
View file

@ -1,4 +0,0 @@
# Add support for RHEL 8.5 Edge images
OSBuild Composer can now build RHEL 8.5 Edge images. The following image types
are supported: edge-commit, edge-container, and edge-installer.

61
docs/news/31/rhel8.5-support.md
View file

@ -1,61 +0,0 @@
# Add support for RHEL 8.5 main image types
OSBuild Composer can now build RHEL 8.5 images. The following new image types
are supported:
- `qcow2`
- `vhd`
- `vmdk`
- `openstack`
- `ami`
- `ec2`
- `ec2-ha`
## RHEL-8.5 AWS images
The `ami` image type have been redefined based on the official RHEL EC2 images.
Notable changes compared to RHEL-8.4 are:
- the default user created by cloud-init is `ec2-user`
- NTP client configuration uses `169.254.169.123` NTP server by default
- the boot mode was changed from hybrid to legacy only
The `ec2` and `ec2-ha` images represent the official RHEL EC2 images, which are
produced as part of RHEL release. These contain RHUI client packages, which are
available only from within Red Hat internal network. For this reason, these
image types are by default not exposed via Weldr API (in the on-premise use
case) for all RHEL releases.
This default configuration can be overridden by placing the following line in
the osbuild-composer configuration `/etc/osbuild-composer/osbuild-composer.toml`:
```toml
[weldr_api.distros."rhel-*"]
# no lines below this section
```
## Extended osbuild support
To support these image types, the following new types were added to support the
functionality in osbuild.
Stages:
- org.osbuild.copy
- org.osbuild.truncate
- org.osbuild.sfdisk
- org.osbuild.qemu
- org.osbuild.mkfs.btrfs
- org.osbuild.mkfs.ext4
- org.osbuild.mkfs.fat
- org.osbuild.mkfs.xfs
- org.osbuild.grub2.inst
Devices:
- org.osbuild.loopback
Mounts:
- org.osbuild.btrfs
- org.osbuild.ext4
- org.osbuild.fat
- org.osbuild.xfs

8
docs/news/31/rhel9.0-support.md
View file

@ -1,8 +0,0 @@
# Add support for RHEL 9.0 Beta
OSBuild Composer can now build RHEL 9.0 Beta images. All image types are based
off RHEL 8.5 ones, thus the same set of image types is supported.
Note that the test coverage isn't complete at this point. Fully supported is
just cross-building RHEL 9 qcow2 images on RHEL 8. Everything else is just
a technical preview.

24
docs/news/31/weldr-image-type-denylist.md
View file

@ -1,24 +0,0 @@
# Weldr API: introduce the ablility to limit exposed Image Types by configuration
Extend Weldr API to accept a map of distribution-specific lists of denied
image types, which should not be exposed via API. It is allowed to use
globing patterns as Distribution and Image Type names. This functionality
is needed to not expose image types which can't be successfully built outside
of Red Hat VPN.
The list of denied Image Types is defined in `osbuild-composer` configuration,
`/etc/osbuild-composer/osbuild-composer.toml`.
Example configuration denying the building of `qcow2` and `vmdk` Image Types
via Weldr API for any distribution:
```toml
[weldr_api.distros."*"]
image_type_denylist = [ "qcow2", "vmdk" ]
```
Example configuration denying the building of `qcow2` and `vmdk` Image Types
via Weldr API for `rhel-84` distribution:
```toml
[weldr_api.distros.rhel-84]
image_type_denylist = [ "qcow2", "vmdk" ]
```

9
docs/news/31/worker-heartbeat.md
View file

@ -1,9 +0,0 @@
# Workers: heartbeat
Workers check in with composer every 15 seconds to see if their job hasn't been
cancelled. We can use this to introduce a heartbeat. If the worker fails to
check in for over 2 minutes, composer assumes the worker crashed or was stopped,
marking the job as failed.
This will mitigate the issue where jobs who had their worker crash or stopped,
would remain in a 'building' state forever.

5
docs/news/31/worker-oauth2-support.md
View file

@ -1,5 +0,0 @@
# Workers: oauth2 support
This change is mainly targeted for getting composer into `cloud.redhat.com`. It
allows remote workers to connect to composer starting from a refresh token, and
is offered as an alternative to the client certificate authentication.

9
docs/news/32/edge-container-nginx.md
View file

@ -1,9 +0,0 @@
# RHEL-Edge container image now uses nginx and serves on port 8080
Previously, the edge-container image type was unable to run in unprivileged
mode which prevented it from being used on OpenShift 4. The container now uses
nginx to serve the commit and a configuration that allows it to run as a
non-root user inside the container. The internal web server now uses port
`8080` instead of `80`.
See rhbz#1945238

10
docs/news/32/rhel8.5-edge-raw-images.md
View file

@ -1,10 +0,0 @@
# Add RHEL for Edge Raw Images for 8.5
OSBuild Composer can now build the RHEL 8.5 Raw Images. This images are
compressed raw images, i.e. a file that has a partition layout with an
deployed OSTree commit in it. It can be used to flash onto a hard drive
or booted in a virtual machine. An existing OSTree commit needs to
be provided.
The following image new types are supported: edge-raw-image.
Relevant PR: https://github.com/osbuild/osbuild-composer/pull/1667

12
docs/news/32/rhel8.5-edge-simplified-installer.md
View file

@ -1,12 +0,0 @@
# Add a new Simplified Installer for RHEL for Edge 8.5
OSBuild Composer can now build the RHEL 8.5 for Edge Simplified Installer.
This installer is optimized for unattended installation to a device, which
can be specified via a new blueprint option, `installation_device`. As for
the existing RHEL for Edge installer, an existing OSTree commit needs to
be provided. A raw image will be created with that commit deployed in it
and the installer will flash this raw image to the specified installation
device.
The following image new types are supported: edge-simplified-installer.
Relevant PR: https://github.com/osbuild/osbuild-composer/pull/1654

20
docs/news/32/rhel8.5-filesystem-support.md
View file

@ -1,20 +0,0 @@
# Add custom file system support for RHEL 8.5
The `weldr` api has been extended to support custom file systems for RHEL 8.5.
Filesystem `mountpoints` and minimum partition `size` can be set under blueprint customizations, as below:
```toml
[[customizations.filesystem]]
mountpoint = "/"
size = 2147483648
```
In addition to the root mountpoint, `/`, the following `mountpoints` and their sub-directories are supported:
- `/var`
- `/home`
- `/opt`
- `/srv`
- `/usr`
- `/app`
- `/data`

20
docs/news/32/rhel9.0-filesystem-support.md
View file

@ -1,20 +0,0 @@
# Add custom file system support for RHEL 9.0
The `weldr` api has been extended to support custom file systems for RHEL 9.0.
Filesystem `mountpoints` and minimum partition `size` can be set under blueprint customizations, as below:
```toml
[[customizations.filesystem]]
mountpoint = "/"
size = 2147483648
```
In addition to the root mountpoint, `/`, the following `mountpoints` and their sub-directories are supported:
- `/var`
- `/home`
- `/opt`
- `/srv`
- `/usr`
- `/app`
- `/data`

6
docs/news/33/bootiso-payload-in-iso-root.md
View file

@ -1,6 +0,0 @@
# Bootiso: move payload to iso root
Instead of including the payload, i.e. ostree commits or live images,
in the anaconda squashfs, they are now located at the root of the iso.
This has several advantages, including shorter build times, more
flexibility in payload size and easier access to the actual payload.

4
docs/news/33/rhel9.0-fix-building.md
View file

@ -1,4 +0,0 @@
# Fix building of RHEL 9.0 Edge images
RHEL 9.0 Beta doesn't ship iwl6000-firmware anymore therefore we had to remove
it from the edge-commit and edge-installer image definitions.

49
docs/news/34/api-oauth2-support.md
View file

@ -1,49 +0,0 @@
# Composer-api and worker-api: OAuth2 support
Adding OAuth2 support to composer means both the composer-api and worker-api are now able to authenticate clients using
the [JWT](https://jwt.io/) set in the "Authorization" HTTP header. This was added with Red Hat Single Sign-On in mind,
but would work for other OAuth2 providers as well (potentially with minor changes).
## Workflow
1. A client makes a request to https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token using
their offline token.
2. sso.redhat.com would respond with an `access_token` which is valid for a certain period.
3. This `access_token` can be set in the "Authorization" HTTP header: "Authorization: Bearer `access_token`".
4. Composer verifies the token against the certificates returned by
https://^Co.redhat.com/auth/realms/redhat-external/protocol/openid-connect/certs.
## Configuration
Using [openshift-online/ocm-sdk](https://github.com/openshift-online/ocm-sdk-go) composer-api now supports oauth2
authentication. To this end there's 4 new config options for the Worker and Composer API:
- EnableJWT: Enable or disable OAuth2 authentication.
- JWTKeysURL: Location where the certs used to verify the JWT tokens are served.
- JWTKeysCA: Path to the CA which should be used when retrieving the certs (optional).
- JWTACLFile: Path to a yaml file containing a series of pattern match rules against the claims
contained within the JWT (optional).
### ACL claims pattern matching format
The ACLFile should contain a list of claims and their required pattern in yaml format. Note that a claim with a specific
name can only be specified once. So if for instance a required pattern for the `email` claim is listed twice, only one
will pattern will be applied.
The pattern is verified using the golang regexp package, and follows the [RE2
syntax](https://github.com/google/re2/wiki/Syntax).
Example:
```
- claim: email
pattern: ^.*@redhat\.com$
- claim: sub
pattern: ^f:b3f7b485-7184-43c8-8169-37bd6d1fe4aa:myuser$
- claim: account_number
pattern: ^(1000|1001|1002)$
- claim: account_id
pattern: ^(5000|5005)$
```

7
docs/news/34/install-docs-in-rhel-85-90.md
View file

@ -1,7 +0,0 @@
# Install docs in RHEL 8.5 and 9.0 images
Previously, all packages in all image types were installed using the
--excludedocs options. This is great for the image size but it actually
causes some issues too: The biggest one is that there are no man pages inside
the images. As that is a pretty big regression, we decided to revert
the --excludedocs setting now.

4
docs/news/34/rhel-ec2-images-rhsm-dnf-plugins.md
View file

@ -1,4 +0,0 @@
# RHEL-8.5 / RHEL-9.0: RHSM DNF plugins are now enabled by default on `ec2` and `ami` images
The RHSM DNF plugins `product-id` and `subscription-manager` are now by default enabled
on the RHEL-8.5 and RHEL-9.0 `ec2`, `ec2-ha` and `ami` images.

10
docs/news/34/support-new-osbuild-stages.md
View file

@ -1,10 +0,0 @@
# Added support for new osbuild stages required for RHEL EC2 SAP images
Added support for the following osbuild stages:
- `org.osbuild.selinux.config` - configures SELinux policy state and type on the system
- `org.osbuild.tmpfilesd` - creates tmpfiles.d configuration files
- `org.osbuild.pam.limits.conf` - creates configuration files for pam_limits module
- `org.osbuild.sysctld` - creates sysctl.d configuration files
- `org.osbuild.dnf.config` - configures DNF (currently only variables)
- `org.osbuild.tuned` - sets active tuned profile (or more profiles)

3
docs/news/34/temp-aws-creds.md
View file

@ -1,3 +0,0 @@
# Support AWS temporary credentials
When using AWS and AWS S3 targets, it is possible to use temporary AWS credentials by specifying a `SessionToken` in addition to the usual `AccessKeyId` and `SecretAccessKey`

6
docs/news/36/rhel-9.0-ec2-sap-image.md
View file

@ -1,6 +0,0 @@
# Add support for official RHEL EC2 SAP image on RHEL-9.0
OSBuild Composer can now build the RHEL 9.0 EC2 SAP image called `ec2-sap`,
which is based on the official RHEL EC2 SAP image. The image type is not
exposed through the Weldr API, because its default package set includes the
RHUI client packages, which are not publicly available.

6
docs/news/36/rhel8.6-ec2-sap-image.md
View file

@ -1,6 +0,0 @@
# Add support for official RHEL EC2 SAP image on RHEL-8.6
OSBuild Composer can now build the RHEL 8.6 EC2 SAP image called `ec2-sap`,
which is based on the official RHEL EC2 SAP image. The image type is not
exposed through the Weldr API, because its default package set includes the
RHUI client packages, which are not publicly available.

4
docs/news/36/rhel8.6-support.md
View file

@ -1,4 +0,0 @@
# Add support for RHEL 8.6
OSBuild Composer can now build RHEL 8.6 images. All image types are based
off RHEL 8.5 ones, thus the same set of image types is supported.

12
docs/news/README.md
View file

@ -1,12 +0,0 @@
# Release Notes
This directory contains release notes for _osbuild-composer_ in the form of
short markdown documents, sorted into subdirectories by release.
When proposing a user-visible change, please add a release note into the
`unreleased` directory. The note should explain the change from the perspective
of somebody using _osbuild-composer_ and ideally how the change affects them or
how they can make use of new functionality.
When preparing a release, a maintainer will rename the `unreleased` directory
and summarize its contents in `NEWS.md`.

0
docs/news/unreleased/.gitkeep
View file

15
docs/news/unreleased/request-job-timeout.md
View file

@ -1,15 +0,0 @@
# Timeout when requesting jobs
When workers request a new job they make a blocking call to the `/api/worker/v1/jobs`
endpoint. There are cases however where a polling approach is more useful, for instance when idle
connections get terminated after a certain period of time.
The new `request_job_timeout` option under the worker config section allows for a timeout on the
`/api/worker/v1/jobs` endpoint. It's a string with `"0"` as default, any string which is parseable
by `time.Duration.ParseDuration()` is allowed however, for instance `"10s"`.
Because this is an expected timeout, "204 No Content" will be returned by the worker server in case
of such a timeout. The worker client will simply poll again straight away.
To maintain backwards compatilibity the default behaviour is still a blocking connection without
timeout.