Commit graph

5095 commits

Author SHA1 Message Date
Achilleas Koutsou
c2fe05950c tools: update test case repos for CS9 2022-12-11 12:37:37 +00:00
Achilleas Koutsou
b9098c64c4 test: add valid parent commit ID to test manifests
The previous value for parent was not a valid commit ID.  In a regular
compose request, the parent ref is resolved before being added as a
source in the manifest.  However, when building test manifests, since
the values aren't real, we don't resolve the ostree refs, meaning that
the test manifests weren't valid — they failed validity checks if run in
osbuild.

Replaced the fake parent ref with a fake parent commit ID
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff".
The manifests still aren't buildable, since they don't point to a valid
ostree repository or commit, but now they're at least valid (in the
technical sense).
2022-12-11 12:37:37 +00:00
Sanne Raymaekers
8636c7e1e0 Schutzfile: bump osbuild dependency
This contains a fix to consumer secrets which are required for pulling
protected ostree content.
2022-12-10 13:23:42 +00:00
Ondřej Budai
ff21f2935f distro/rhel9: fix /usr/sbin/zipl unavailable in the buildroot
The big rewrite of rhel9 distro omitted installing s390utils-base into the
buildroot. This caused the org.osbuild.zipl.inst stage because of missing
/usr/sbin/zipl.

This commit introduces s390utils-base back into the buildroot which fixes
building of the s390x images.

I verified it by building the RHEL 9.1 qcow2 image and booting it using
libvirt.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-12-08 16:11:52 +01:00
Achilleas Koutsou
caac5fc7af test: regenerate manifests
ISOLinux removed for Fedora and RHEL/CS 9 installers.
2022-12-08 10:30:09 +01:00
Achilleas Koutsou
9e66ee13e7 image: enable ISOLinux only for x86_64
ISOLinux was unconditionally enabled for all ISOs, which is incorrect
and breaks builds on aarch64.
Enable it only for x86_64.
2022-12-08 10:30:09 +01:00
Brian C. Lane
87b5e38c45 workflows: Update to use actions/checkout@v3 2022-12-07 16:53:25 -08:00
Sanne Raymaekers
4b90cb6fa4 cloudapi/v2: set ostree rhsm option on image options
The ostree options are used during the ostree resolve job, but when
generating the manifest the rhsm value comes from the image options, so
it's necessary to set it on both.
2022-12-07 16:31:46 +01:00
Mario Cattamo
51851bfd49 Firewall feature flag default value 2022-12-06 16:39:00 +01:00
Mario Cattamo
9426b15913 Set up firewall feature flag value 2022-12-06 16:39:00 +01:00
Mario Cattamo
7871ab49e9 Test firewall sources and zones customizations in edge-commit blueprint 2022-12-06 16:39:00 +01:00
Irene Diez
432dab8e78 schutzfile: pin required osbuild commit
Signed-off-by: Irene Diez <idiez@redhat.com>
2022-12-06 16:39:00 +01:00
Irene Diez
6cb8216f09 internal: update firewall stage to allow zones
Updates firewall stage customizations to allow zones
as per the changes made on osbuild/osbuild#1157

Relevant tests and pipelines for rhel8+9 updated.

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-12-06 16:39:00 +01:00
Antonio Murdaca
80a6b1bd23 Extend firewall customizations to add sources
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Co-authored-by: Irene Diez <idiez@redhat.com>
2022-12-06 16:39:00 +01:00
Sayan Paul
4380e70973 fix: regenrate manifest
Signed-off-by: Sayan Paul <saypaul@redhat.com>
2022-12-06 12:15:09 +01:00
Sayan Paul
6f56bee611 build:add journald stage to thel 8/9 pipeline
Adds the new journald stage to rhel 8/9
rpmostree commit pipeline.This will add
the dropin file `10-persitent.conf` with
the storage option set as persistent.

Co-authored-by: Sayan Paul <paul.sayan@gmail.com>
Co-authored-by: Irene Diez <idiez@redhat.com>
Signed-off-by: Sayan Paul <saypaul@redhat.com>
2022-12-06 12:15:09 +01:00
Antonio Murdaca
217cc64425 check_ostree: check persistent logging
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-12-06 12:15:09 +01:00
Sayan Paul
5ce3de214d build:create systemd_journald stage
Creates new stage to configure
journald to persist the journal.
Instead of creating the `/var/log/journal`
directory we explicitly configure journald
via the new stage.This is done in
according to the FCOS norms.
Unit tests also added for functionality
check.

Co-authored-by: Irene Diez <idiez@redhat.com>
Signed-off-by: Sayan Paul <saypaul@redhat.com>
2022-12-06 12:15:09 +01:00
Achilleas Koutsou
8c75975917 osbuild: add masked services to systemd stage 2022-12-02 19:04:26 +00:00
Gianluca Zuccarelli
5457b9fba2 metrics: update status metrics label
Openshift overrides the `service` label for
all metrics in the cluster. Update the label
from `service` to `subsystem` for the status
metrics query. This helps us differentiate
between requests from composer and the worker
server.
2022-12-02 09:25:40 +01:00
Ondřej Budai
fd72a6d16e run-koji-container.sh: drop the duplicate vhd.xz format
When the test upload a vhd.xz image to koji, it returns `koji.GenericError:
multiple matches for file extension: vhd.xz`. It seems like the default
schema is not valid for vhd.xz images because it contains two archive types
for them which koji cannot handle. I reported this issue as

https://pagure.io/koji/issue/3605

This commit works around that by removing one of the archive types, so koji
isn't confused by two same records.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-12-01 20:59:30 +01:00
Ondřej Budai
972da81a91 worker: fix reporting the import error to composer
The result variable wasn't used at all, kojiFinalizeJobResult is what actually
reports the error.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-12-01 20:59:30 +01:00
Xiaofeng Wang
afd0f7ebc7 test: Use new CS9 repo built by PR#3136
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-12-01 11:51:58 +01:00
Antonio Murdaca
64f1ff498b re-enable cs9 runner for simplified installer
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-12-01 11:51:58 +01:00
Gianluca Zuccarelli
113cda7d39 internal/worker: register status middleware
Register the custom middleware function to the worker
server. This function is responsible for recording all
the status codes of all the server's endpoints.

Due to a bug with echo/v4, a request to an endpoint using
the incorrect method should return a `405` error but returns
a `404` error instead when a middleware function is registered.
The worker `server_test` has been updated to reflect this.
2022-11-30 11:14:29 +01:00
Gianluca Zuccarelli
8b5458ae83 internal/cloudapi: register status code middleware
Register the custom middleware function to the cloudapi
server. This function is responsible for recording all
the status codes of all the server's endpoints.
2022-11-30 11:14:29 +01:00
Gianluca Zuccarelli
8756ea717d prometheus: middleware to record 5xx errors
Create a custom middleware function
to measure 5xx requests for all composer
& worker routes and not just the `/composer`
endpoint. The result is a prometheus metric
that contains info on the request status code,
path & method.

A helper function has been added to clean the
dynamic parameters in the path routes to reduce
metric cardinality
2022-11-30 11:14:29 +01:00
Gianluca Zuccarelli
33e53398a6 prometheus: add status metrics
Add a helper function to register the same metrics
for both the worker and composer - the only difference
being the subsystem name. The function checks if the
metric has already been registered and, if so, returns
the already registered metric.
2022-11-30 11:14:29 +01:00
Gianluca Zuccarelli
8e82b223af prometheus: move constants to a single file
Move the constants to a single file and export them.
These can then later be used externally for future use
with the ocm metrics.
2022-11-30 11:14:29 +01:00
Colin Walters
a3a733a638 gcp: Cross-reference to coreos-assembler code
At the moment we have duplicate logic here; ideally of course
we consolidate (since both codebases are Go, perhaps we could
create a tiny little Go library for "RHEL GCP stuff"?) but
for now let's just cross-link for awareness.
2022-11-30 11:13:31 +01:00
Tomáš Hozza
d0a3ad470d internal/rpmmd: replace RepositoryError with plain error
The `RepositoryError` structure was weirdly used instead of a plan
`error` type and added no value. Delete it and replace its only use with
plain `error`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-30 11:04:24 +01:00
Tomáš Hozza
2c794c9e94 internal/rpmmd: delete unused interfaces and structures
Clean up the `rpmmd` package from unused interfaces and data structures.
These are leftovers after the last dnfjson rework.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-30 11:04:24 +01:00
Achilleas Koutsou
75758aee52 distro/test: set RHSM Fact API Type when testing manifests 2022-11-30 10:04:16 +01:00
Achilleas Koutsou
a9efe3ebb9 test: regenerate RHEL 9.x manifests
All RHEL 9.x manifests have been affected except edge-installer,
edge-raw-image, and edge-simplified-installer.  These are image types
that carry a pre-built payload so don't have an OS pipeline and don't
include RHSM facts.
2022-11-30 10:04:16 +01:00
Achilleas Koutsou
5cf0c336d3 distro/rhel9: add RHSM fact to RHEL 9 images
Set the API type RHSM fact in OS customizations for RHEL 9 images.
2022-11-30 10:04:16 +01:00
Achilleas Koutsou
3cff4d0c1d manifest: support RHSM facts in OSCustomizations
Add the RHSM fact to the OSCustomizations and add the stage to the OS
pipeline if the value is set.
2022-11-30 10:04:16 +01:00
Achilleas Koutsou
4bd2c0cb1d test: regenerate manifests
Regenerate manifests with the added RHSM fact.
RHEL 9 manifests haven't changed because the stage is missing in the OS
pipeline (bug).
2022-11-30 10:04:16 +01:00
Achilleas Koutsou
e3e71f01bf cmd/gen-manifests: add package-sets to repo config
Was missing the package sets key from the repo config struct, which
means that the option was being ignored and wasn't being serialised into
the test manifest either.
2022-11-30 10:04:16 +01:00
Achilleas Koutsou
de7bae02f3 cmd/gen-manifests: add RHSM fact to options
Add RHSM fact to image options when generating test manifests.
We add the value "test-manifest" to the API type to indicate it's a test
manifest.  This should never be registered and therefore shouldn't show
up in our data, but it's useful to detect changes and regressions in the
fact creation in the pipelines.
2022-11-30 10:04:16 +01:00
schutzbot
f7334287af Post release version bump
[skip ci]
2022-11-30 08:28:40 +00:00
Tomáš Hozza
c6570f6c94 CloudAPI: add description for Repository definition
Specifically describe that at least one of `baseurl`, `mirrorlist` or
`metalink` properties must be specified. Although this is not enforced
by the schema, it is the reality. Modifying the schema to enforce this
results in the generated data structure for `Repository` to be simply
an `interface{}`, which would just make the code more ugly, Therefore
this is covered by the API implementation logic.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-28 17:30:56 +01:00
Achilleas Koutsou
b3f4d75699 image: panic if unknown compression option is specified
Currently we only support "xz", but keeping it as a `switch` to easily
support more types in the future.
The empty string is also supported as a no-op.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
6374ec022d distro/rhel9: define qcow2 image type in function
The global qcow2ImgType was unconfigured and meant to be used by calling
the mkQcow2ImgType() function.  In the distro initialisation code, the
new variable created from the mkQcow2ImgType() function with the
configuration was shadowing the global variable.  This can lead to
errors in the future where it's not clear which variable is used where.

Putting the definition in a function makes it impossible to use without
configuring.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
7da0cde855 test: regenerate image-installer and edge-installer manifests 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
9ae3d03f88 tools: add package to image-installer manifest request
Add podman to the image installer's packages when generating test
manifests.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
fa948b9e0b distro: set the workload for the image installer pipelines 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c6bfb22f54 manifest: support specifying kickstart file location
Add support for specifying where the kickstart file will be created:
either in the default location in the anaconda tree or at the root of
the ISO like we did so far in RHEL.

From the distro level, this is controlled by a boolean that when set to
true puts the kickstart file in the root of the ISO tree at /osbuild.ks
(the old behaviour) and if unset falls back to the default location
/usr/share/anaconda/interactive-defaults.ks.

Changed the name of the kickstart file for ostree back to osbuild.ks
Was ostree.ks for a while since a1bfcfe91c.

For Fedora we use the default and for RHEL we maintain the old behaviour
for now.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
d4a8775751 test: update installer manifests
- Added users module.
- Set squashfs compression to xz for RHEL 9.x and CentOS Stream 9.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
5d55ccf109 manifest: support specifying squashfs compression for ISOs
Add support for specifying the squashfs compression method for ISOTree
pipelines from the caller.
Build Fedora ISOs with lz4 compression and RHEL with xz.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
cb0280c8c9 image: always enable users module in anaconda
Unconditionally enable the users module in Anaconda.
The module serves two purposes:
- It will create a user that's defined in the blueprint by adding the
  user creation stage in the kickstart file.
- It allows users to create user accounts interactively during
  installation.
2022-11-28 17:20:49 +01:00