Commit graph

1323 commits

Author SHA1 Message Date
Diaa Sami
27ca55372a cloudapi: Add extra logging & improve existing
Use different logging levels depending on situation
Log HTTP 5xx errors to stdout with internal error details when possible
2021-09-30 18:34:05 +02:00
Diaa Sami
179009fec4 cloudapi: use Logrus as default logger in Echo
And remove log.Logger references
2021-09-30 18:34:05 +02:00
Ondřej Budai
5ed954bcf5 distro/rhel90: use qcow2 compat 1.1 for qcows
There's no reason to use 0.10 since we don't need to support running
this image on RHEL 6 anymore.
See https://bugzilla.redhat.com/show_bug.cgi?id=2008910

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-30 15:36:15 +02:00
Gianluca Zuccarelli
b8c14a5689 blueprints: change minsize from int to uint64
thozza pointed out that `int` is platform dependent which results in
a fs size that is too small for 32-bit machines. This commit changes
the filesystem custimizations to use `uint64` instead of `int`
2021-09-29 15:08:12 +02:00
Christian Kellner
7a4488e5c4 distro/rhel85: set bootloader to none for edge
Use the new bootloader config for OSTree deployments to set the
bootloader backend to `none` since we use grub2 and BLS in all
currentlysupported architectures. In fact, not setting it to
`none`, will use the default `auto`, which will result in the
run of `grub2-mkconfig`, which is not what we want and might
even fail for us.
2021-09-27 20:10:04 +02:00
Christian Kellner
f6d4228b00 osbuild2: add bootloader option to ostree.config
Expose the new (osbuild > 36) `sysroot.bootloader` option to
configure the bootloader backend for OSTree deployments.
Normally, this should be set to `none` for modern bootloader
that use the BLS, since the BLS snippets are always generated.
2021-09-27 20:10:04 +02:00
Christian Kellner
920863df70 osbuild2: small doc fix for ostree.config 2021-09-27 20:10:04 +02:00
sanne
2f328b0e97 workers: Backwards compatible api.openshift.com spec compliance
The main changes are:
- Kind, Href, Id fields for every object returned
- Attach operationIds to each request, return it for errors
- Errors are predefined and queryable
2021-09-27 13:10:05 +01:00
Martin Sehnoutka
d7d4e02c8c internal/common: introduce git revision and rpm version
The variables are set to the git revision from which the build is
triggered and rpm version from the spec file, if it is build using RPM.

This can be later used to query exact source version while
running osbuild-composer.

It is necessary to use both, because none of them is available in all
possible scenarios.

Use either git-rev (preferably) or RPM version (NEVRA) instead of the
"devel" build type. It was just a placeholder.
2021-09-24 14:13:23 +02:00
Achilleas Koutsou
5368ecf7fe distro/rhel90: explicitly enable greenboot services for edge
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Achilleas Koutsou
23109df500 distro/rhel90: add gnome-kiosk to installer package set
Required for graphical Anaconda installation

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Achilleas Koutsou
0c5a0d6763 distro/rhel90: remove unavailable packages from edge-installer
Packages that are no longer not available in RHEL 9.0.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Diaa Sami
60e403e53e cloudapi: use Recover middleware to handle panics
recover from panics such as out-of-bounds array access & nil
pointer access, print a stack trace and return 5xx error
instead of the service crashing and relying on Execution
framework to handle crashes
2021-09-24 12:11:04 +01:00
Christian Kellner
d9fe813fa5 distro/rhel90: re-include nss-altfiles for edge
The `nss-altfiles` package is actually needed for OSTree based
systems, since in those the user/groups database is located in
the /usr/lib directory (in addition to /etc). It was removed
because it was not available in RHEL 9 (rhbz#1916260). This has
been fixed, so we need to re-include the package.
2021-09-23 10:47:11 +02:00
Brian C. Lane
e92424b5e3 weldr: deleting an unknown source should return an error
This adds a check to see if the source is valid and returns a 400 error
if it doesn't exist.
2021-09-22 11:24:31 +02:00
Tomas Hozza
eef9971027 osbuild2: add support for org.osbuild.tuned stage
Add support for a new osbuild stage `org.osbuild.tuned`, for
setting TuneD profile.

Add unit tests for the new stage.

Related to https://github.com/osbuild/osbuild/pull/797.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-15 14:46:25 +02:00
Tomas Hozza
8b623d9463 osbuild2: add support for org.osbuild.dnf.config stage
Add support for a new osbuild stage `org.osbuild.dnf.config`, for
configuring DNF.

Add unit tests for the new stage.

Related to https://github.com/osbuild/osbuild/pull/798.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-15 14:46:25 +02:00
Tomas Hozza
d9610b97fc osbuild2: add support for org.osbuild.sysctld stage
Add support for a new osbuild stage `org.osbuild.sysctld`, for
creating tmpfiles.d configuration files.

Add unit tests for the new stage.

Related to https://github.com/osbuild/osbuild/pull/804.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-15 14:46:25 +02:00
Tomas Hozza
a5097b2141 osbuild2: add support for org.osbuild.pam.limits.conf stage
Add support for a new osbuild stage `org.osbuild.pam.limits.conf`, for
creating pam_limits module configuration files.

Add unit tests for the new stage.

Related to:
- https://github.com/osbuild/osbuild/pull/802
- https://github.com/osbuild/osbuild/pull/807

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-15 14:46:25 +02:00
Tomas Hozza
3f52af2adb osbuild2: add support for org.osbuild.tmpfilesd stage
Add support for a new osbuild stage `org.osbuild.tmpfilesd`, for
creating tmpfiles.d configuration files.

Add unit tests for the new stage.

Related to https://github.com/osbuild/osbuild/pull/801.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-15 14:46:25 +02:00
Tomas Hozza
fbb70c2d10 osbuild2: add support for org.osbuild.selinux.config stage
Add support for a new osbuid stage `org.osbuild.selinux.config`,
for setting the desired SELinux policy state and type on the system.

Add unit tests for the new stage.

Related to https://github.com/osbuild/osbuild/pull/799.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-15 14:46:25 +02:00
Ondřej Budai
39d77f26b6 distro/rhel90: always install docs in the rpm stage
When we firstly introduced the v2 manifests, we excluded docs from all rpm
stages because it doesn't make sense to include docs in edge images.

However, when we ported the other image types to v2, we left the flag on.
The side effect of --excludedocs is that we no longer install man pages into
the image. This the default behaviour of rpm and can be seen here:
b88f43b9a3/macros.in (L555)

This is actually quite a major regression in non-edge images, thus this commit
reverts this setting. It would be great to have the option not to install
docs in some rpm stages but as this is a difficult topic and we're already
past the deadline, we will consider this in the future.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-15 10:47:57 +02:00
Ondřej Budai
d895c336d6 distro/rhel85: always install docs in the rpm stage
When we firstly introduced the v2 manifests, we excluded docs from all rpm
stages because it doesn't make sense to include docs in edge images.

However, when we ported the other image types to v2, we left the flag on.
The side effect of --excludedocs is that we no longer install man pages into
the image. This the default behaviour of rpm and can be seen here:
b88f43b9a3/macros.in (L555)

This is actually quite a major regression in non-edge images, thus this commit
reverts this setting. It would be great to have the option not to install
docs in some rpm stages but as this is a difficult topic and we're already
past the deadline, we will consider this in the future.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-15 10:47:57 +02:00
Gianluca Zuccarelli
e41e82935d rhel90: enable multi-level mountpoints
Previously it was only possible to configure separate partitions
for mountpoints in the allow list and their immediate subdirectories
only i.e. /var & /var/log
This fix allows for an arbitrary level of mountpoints, i.e. /var/log/audit,
/var/a/b/c/d/e and so on
2021-09-15 10:46:55 +02:00
Gianluca Zuccarelli
16e80ffa33 rhel85: enable multi-level mountpoints
Previously it was only possible to configure separate partitions
for mountpoints in the allow list and their immediate subdirectories
only i.e. /var & /var/log
This fix allows for an arbitrary level of mountpoints, i.e. /var/log/audit,
/var/a/b/c/d/e and so on
2021-09-15 10:46:55 +02:00
sanne
5a9d8c792b cloudapi: V2
V2 is compliant with api.openshift.com design guidelines.

Errors are predefined, have codes, and are queryable.

All requests have an operationId set: a unique identifier which is
sortable by time. This is added to the response in case of an error.

All returned objects have the href, id, and kind field set.
2021-09-14 15:32:21 +02:00
Tomas Hozza
19eb65e9fd RHEL-8.5/9.0: keep RHSM DNF plugins enabled on ec2, ec2-ha and ami
The RHSM DNF plugins `product-id` and `subscription-manager` are now
by default enabled on the RHEL-8.5 and RHEL-9.0 `ec2`, `ec2-ha` and
`ami` images.

The desired default state of the RHSM DNF plugins has been decided by
the RHSM team.

Related to https://bugzilla.redhat.com/show_bug.cgi?id=1996670

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-14 11:47:09 +02:00
Diaa Sami
20501f4e58 Fix package sets for tar installer image type 2021-09-10 15:17:54 +02:00
Chloe Kaubisch
11023cb8a8 cloudapi: change Organization customization to a string
Internally change Organization in the SubscriptionImageOptions
to a string instead of an int.
2021-09-08 17:38:05 +02:00
sanne
4a057bf3d5 auth: OpenID/OAUth2 middleware
2 configurations for the listeners are now possible:
- enableJWT=false with client ssl auth
- enableJWT=true with https

Actual verification of the tokens is handled by
https://github.com/openshift-online/ocm-sdk-go.

An authentication handler is run as the top level handler, before any
routing is done. Routes which do not require authentication should be
listed as exceptions.

Authentication can be restricted using an ACL file which allows
filtering based on JWT claims. For more information see the inline
comments in ocm-sdk/authentication.

As an added quirk the `-v` flag for the osbuild-composer executable was
changed to `-verbose` to avoid flag collision with glog which declares
the `-v` flag in the package `init()` function. The ocm-sdk depends on
glog and pulls it in.
2021-09-04 02:48:52 +02:00
sanne
5e72636331 Revert "cloudapi: Add x-rh-identity header filter"
This reverts commit 19db3ff1d4.
2021-09-04 02:48:52 +02:00
sanne
7a0ea5b244 worker: Remove identity filter
Partially reverts "0ea31c39d5"
2021-09-04 02:48:52 +02:00
Antonio Murdaca
806bb375ee internal/distro/rhel85: move systemd logs kargs to the installer image
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2021-09-03 23:29:08 +02:00
dependabot[bot]
3ccdf85295 build(deps): bump github.com/golang/protobuf from 1.4.3 to 1.5.2
Bumps [github.com/golang/protobuf](https://github.com/golang/protobuf) from 1.4.3 to 1.5.2.
- [Release notes](https://github.com/golang/protobuf/releases)
- [Commits](https://github.com/golang/protobuf/compare/v1.4.3...v1.5.2)

---
updated-dependencies:
- dependency-name: github.com/golang/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Ondřej: I also fixed a deprecated call.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-03 18:23:54 +02:00
Martin Sehnoutka
cd0d450c3a distro: introduce Fedora 36 alias
Fedora 35 has been branched. Introduce an alias for Fedora 36 as it is
the current rawhide. Source of the GPG key:
https://raw.githubusercontent.com/xsuchy/distribution-gpg-keys/main/keys/fedora/RPM-GPG-KEY-fedora-36-primary
2021-09-03 15:05:00 +02:00
Diaa Sami
c35097fd52 Support for temporary credentials
Handle session token so that temporary credentials are supported
2021-09-02 15:13:42 +02:00
Chloe Kaubisch
86082c884d cloudapi: fix compose return value
Correct compose successful return status from http.StatusOK to
http.StatusCreated. Mistakenly set to http.StatusOK in previous
commit.
2021-09-01 22:15:35 +02:00
Thomas Lavocat
8a6e0e2b75 cloudapi: correct format of error strings
Fixes bad formats for the error in the cloudapi.
2021-09-01 20:02:07 +02:00
Ondřej Budai
c5fb118855 distro/rhel90: remove iwl6000-firmware from package sets
This package is no longer available in RHEL 9, let's drop it from our
definition.

Fixes #1696

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-30 18:22:58 +02:00
Christian Kellner
3dfc997992 distro/rhel85: move payloads to iso root
Move the ostree repository and the tar image to the root of the
boot iso. This has several advantages: we do no longer have to
correctly guess the size of the anaconda image. Also we do not
need to compress the payload within the squashfs.
Update the image installer's test data. NB: the changes to the
package list were introduced earlier and should mostly affect
the build pipeline. Should have caught is in the corresponding
change, but was apparently not picked up by CI.
2021-08-30 12:50:38 +02:00
Christian Kellner
1a063680b5 distro/rhel85: add support for edge raw images
OSBuild Composer can now build the RHEL 8.5 Raw Images. This images are
compressed raw images, i.e. a file that has a partition layout with an
deployed OSTree commit in it. It can be used to flash onto a hard drive
or booted in a virtual machine. An existing OSTree commit needs to
be provided.
The following image new types are supported: edge-raw-image.
2021-08-28 09:20:19 +02:00
Christian Kellner
c6b2d3009b distro/rhel85: partition table for edge + aarch64
Define the partition layout for RHEL for Edge images on arm64.
2021-08-28 09:20:19 +02:00
Christian Kellner
1054eab392 distro/rhel85: extract edge image pipelines
Extract the pipelines that create the edge raw image so that we can
expose that as its own image type.
2021-08-28 09:20:19 +02:00
Christian Kellner
0612d14502 distro/rhel85: aarch64 support for edge installers
All RHEL for Edge installer should be supported on aarch64 now.
2021-08-28 09:20:19 +02:00
Christian Kellner
97b20ca0da distro/rhel85: aarch64 support for bootiso mono
Add aarch64 support for the bootiso.mono stage: select the proper efi
architecture and include isolinux only on x86.
2021-08-28 09:20:19 +02:00
Christian Kellner
3e90d66f5c distro/rhel85: no biosdevname on aarch64
Do not include the biosdevname dracut module on aarch64, where it
does not exist.
2021-08-28 09:20:19 +02:00
Christian Kellner
44cf29dad0 distro/rhel85: aarch64 pkgs for installers
Make the package sets for all the installer work on aarch64.
Needed to explicitly pull in fcoe-utils for anaconda on aarch64.
2021-08-28 09:20:19 +02:00
Christian Kellner
fc5c2dbe7d distro/rhel85: refactor installer pkgs set
Split out the common package set of the anaconda based installers and
the coreos installer based one.
2021-08-28 09:20:19 +02:00
Christian Kellner
714250aa9c distro/rhel85: refactor pkg set generation
Instead of using package sets at the distro, arch and image type
level and then merging them in `PackageSets`, store the function
that generates the package set in the image type and have them
return all the package set. In order to do so, they now take an
imageType parameter so that they can also return architecture
dependent packages.
2021-08-28 09:20:19 +02:00
Christian Kellner
0e9e1b32d4 distro/rhel85: move distro build pkgs to pkg set
Instead of having a common build package set defined at distro
struct level and merging them together with build packages in
the image type (and arches), we do the "inheritance" at the
package set level and append more specific packages to base
sets there. We also now ensure that each image type does have
a build package set defined.
The actual package set should not change for anything due to
this commit.
2021-08-28 09:20:19 +02:00