Commit graph

5666 commits

Author SHA1 Message Date
Achilleas Koutsou
5b4e77797c distro/rhel8: remove outdated comment 2023-03-30 18:40:12 +02:00
Achilleas Koutsou
18ef5ae32d distro/rhel8: distro workload implementation and EAP
Add a local, private implementation of the Workload interface for RHEL
8.  The type should be used for concrete workloads that provide package
package lists for specific image types.

The eapWorkload() function creates a workload that provides the packages
required for EAP image type variants.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
77b8328999 distro: add workload to image type
Support specifying workload on an image type.
This isn't used yet, but when set, the workload will define packages and
services and not allow any blueprint customizations.

In the future, there will be multiple workloads with each image type
specifying one (but multiple image types can have the same default
workload).  The custom workload will allow all customizations that are
currently supported.
2023-03-30 18:40:12 +02:00
Jakub Rusz
c59a92b2d9 rpmbuild: add fedora-38 2023-03-29 14:32:06 +03:00
Ondřej Budai
7e13d20ca6 packit: build RPMs also for EPEL 9 and RHEL
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-03-29 12:24:16 +02:00
dependabot[bot]
4bd6f82cc9 build(deps): bump github.com/aws/aws-sdk-go from 1.44.114 to 1.44.230
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.114 to 1.44.230.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.114...v1.44.230)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-28 11:11:05 +02:00
Tomáš Hozza
58e3716b89 distro: add support for RHEL-8.9 and RHEL-9.3
Add a basic support for building RHEL-8.9 and RHEL-9.3 images with
composer.

Add 8.9 and 9.3 repositories to the multitude of places where we have
them.

Generate image test manifests for 8.9 and 9.3. No functional testing is
added for 8.9 or 9.3 at this moment.

This change unblocks the RHEL Gating (since unit tests are currently
failing on 8.9 and 9.3 as on unknown distribution).

Related to https://issues.redhat.com/browse/COMPOSER-1924

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-27 11:39:24 +02:00
Juan Abia
e3594f7234 Start using CIV config file
This way, all CIV parameters can be provided from a file stored in
gitlab secrets without the need to change the script.
2023-03-24 23:10:55 +01:00
Mario Cattamo
00de277d3a Add Butane test and Ignition logs 2023-03-24 17:07:36 +01:00
Tomáš Hozza
dc637d9a57 Manifest: always set kernel options in grub2 stage
It turned out that when we stopped setting the kernel options in grubenv
file, we also stopped setting them in /etc/default/grub under
`GRUB_CMDLINE_LINUX`. This file is used by grub2-mkconfig when
generating grub configuration.

10_linux script executed by grub2-mkconfig recently started to overwrite
the /etc/kernel/cmdline, if its timestamp is older than the timestamp of
/etc/default/grub [1]. As a result, all kernel options were wiped out from
/etc/kernel/cmdline.

Make sure that we always set the `KernelOptions` in the grub2 stage
options, even if the `WriteCmdLine` is set to `false`.

In addition, unify the way we concatenate kernel options set in the
grub2 stage options. Some pipeline implementations were previously using
space, other were using comma. Space is now used everywhere.

Regenerate all affected image manifests.

[1] https://src.fedoraproject.org/rpms/grub2/c/fc76aed5333f56dd05400521a35b944a5df52ebc

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-23 16:32:34 +01:00
Sanne Raymaekers
03973acc1f distro/rhel: add payload repos to os package set
Before instantiating the manifest, any repositories that contain a
package set key contained in `PayloadPackageSets()` should be added to
the os package set in order to correctly generate the `org.osbuild.rpm`
stage for the os pipeline.

Otherwise options like GPG keys are not set correctly.

Fixes #3326
2023-03-23 13:39:04 +01:00
Brian C. Lane
f731ab53d0 dnfjson: Cleanup old distro cache dirs
This adds a function, CleanupOldCacheDirs, that checks the dirs under
/var/cache/osbuild-composer/rpmmd/ and removes files and directories
that don't match the current list of supported distros.

This will clean up the cache from old releases as the are retired, and
will also cleanup the old top level cache directory structure after an
upgrade.

NOTE: This function does not return errors, any real problems it
encounters will also be caught by the cache initialization code and
handled there.
2023-03-23 11:26:40 +01:00
Brian C. Lane
8f20b550ea dnfjson: Adjust cache size info for new layout
The repo id hash directories are now down one level, below a directory
named for the distribution.
2023-03-23 11:26:40 +01:00
Brian C. Lane
3481e1d3ba Change the rpmmd cache directory structure to include the distro name
This causes dnf-json to use separate caches, allowing them to run in
parallel, with one lock per distribution. Multiple depsolves with the
same distribution in the blueprint will continue to be serial.
2023-03-23 11:26:40 +01:00
Brian C. Lane
2ab0430d5d weldr: Log when starting and finishing metadata preload
It is difficult to tell if these are really running in parallel or not,
even with loggin, but it helps. They will all always start at the same
time (because they are run concurrently with goroutines) and if things
work right should be finished at about the same time.
2023-03-23 11:26:40 +01:00
Brian C. Lane
c6a8d89da6 remotefile: Use io.ReadAll not ioutil.ReadAll 2023-03-22 08:14:01 -07:00
schutzbot
2040e93e56 Post release version bump
[skip ci]
2023-03-22 08:21:56 +00:00
Sanne Raymaekers
a2a3a2602c templates/dashboards/worker: add arch label to job wait duration
Display the wait duration of jobs per architecture.
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
b13865d361 templates/dashboards/worker: edit thresholds
95th percentile duration is now a fixed colour, as it's tricky to get
dynamic thresholds based on the job type.

Budget remaining thresholds are now only green at infinity, turn yellow
below 4 weeks, and turn red when budget consumption would only last 3
weeks (out of 4).
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
63d5132aa6 templates/dashboards/worker: change panel alignment
This aligns vertical dividers between panels across rows.
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
865bb98034 templates/dashboards/worker: bump version 2023-03-21 12:34:09 +01:00
Sanne Raymaekers
5a9f8d3457 templates/dashboards/worker: show request throughput per path 2023-03-21 12:34:09 +01:00
Sanne Raymaekers
26a521f54d templates/dashboards/worker: use jobtype variable for job stats
This removes the rows of panels per job type, and uses the jobtype
variable.
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
5d2f84cb9e templates/dashboards/worker: add target duration 2023-03-21 12:34:09 +01:00
Sanne Raymaekers
0b7e94b097 templates/dashboard/worker: add job type variable 2023-03-21 12:34:09 +01:00
Juan Abia
06d83d3c07 Change civ repo name 2023-03-21 11:56:57 +01:00
Jakub Rusz
ecd69ca25b tests/ostree: Change centos-8 BOOT_LOCATION to a working boot.iso
The latest boot.iso stopped working. Changing to an older version to
unblock CI. RHBZ#2180024 opened with this issue.
2023-03-20 17:59:20 -07:00
Simon de Vlieger
6e4893ad78 test/data: regenerate f39 manifests
Since F39 got enabled on `main` after the rebase they have different
sizes in this branch.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
ca836a87f4 internal/disk: Remove RequiredSizes property.
The property was only used in a single method where it is passed as an
argument. There is no need to expose it.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
6704dfd609 distro/fedora: clarify requiredPartitionSizes 2023-03-20 11:04:14 +01:00
Simon de Vlieger
7e93df725c distro: move RequiredPartitionSizes into ImageType
It used to be defined on ImageOptions, but ImageType is a better fit for
this.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
34f7f6c7e2 disk: make testcase less confusing
This testcase tested for a / size of 1 GiB but better is to test for a
minimal size of 4 GiB which is the actual lower limit when the required
sizes are added together.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
92cdb0f082 iot-raw-image: change partition table to DOS
For parity with ImageFactory built images there should be only a DOS
partition table in the raw images, with a non-standard ID set for the
boot partition for Raspberry Pi compatibility.

This only applies to the aarch64 images.

As noted in #3220.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
39879a9f60 disk: add RequiredSizes to ImageOptions
These RequiredSizes are a map that is passed on to the partition table
logic which had hardcoded defaults. This makes it possible to define
either no RequiredSizes (`nil`) or empty RequiredSizes which means no
further constraint checks or partition resizes will be done.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
d7f5fac183 disk: clarify comment 2023-03-20 11:04:14 +01:00
Simon de Vlieger
053bbfcb98 iot-raw-image: set size to 4 GiB
The current Fedora IoT raw images are 4 GiB but we were using 10 GiB raw
images. Make this consistent to current.
2023-03-20 11:04:14 +01:00
dependabot[bot]
88f863f14b build(deps): bump actions/setup-go from 3 to 4
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 10:56:34 +01:00
Gianluca Zuccarelli
4fee181fec worker: add file resolve job
Implement a file resolution job which fetches
the contents of a remote file.
2023-03-16 09:55:39 +00:00
Gianluca Zuccarelli
b7e7bafb2e internal/remotefile: create a remote file resolver
Create a resolver to fetch the contents of a
remote file which can be used at build time.
The initial usecase for this resolver is to
resolve remote gpgkeys but the resolver has
been made more generic for general files.
2023-03-16 09:55:39 +00:00
Gianluca Zuccarelli
c056db4811 worker/server: add file resolver job
Add a file resolver job to the worker server in
order for us to resolve the contents of a remote
file.
2023-03-16 09:55:39 +00:00
Gianluca Zuccarelli
98d611d34f worker/server: fix container resolver job error
The container job resolve job error message was printing
the wrong error type to the error string.
2023-03-16 09:55:39 +00:00
Gianluca Zuccarelli
b493da1878 internal/clienterrors: add file resolution error
Add a client error type for an error in the remote
file resolution job.
2023-03-16 09:55:39 +00:00
Antonio Murdaca
414424cd30 simplified-installer: enable isolinux
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2023-03-15 12:35:50 +01:00
Sanne Raymaekers
861e1473d9 test/vmware: boot VMs with EFI and use SCSI as a disk controller
Since we use streamOptimized images everywhere, we no longer need to use
IDE and boot with bios. Let's test a more realistic scenario.
2023-03-15 10:37:21 +01:00
Simon de Vlieger
23627e9ce0 distro/fedora: add support for Fedora 39 2023-03-15 09:31:23 +01:00
yih
4023b61e1a test fdo create user 2023-03-10 14:40:49 +01:00
Sanne Raymaekers
a25e0f4adb prometheus:: add arch label to dequeue metrics
Only add the arch label for osbuild job types, as the finish metrics
behave similarly. Having arch labels on dequeue metrics for any other
job type (but not on the finish metrics) would produce weird results.
2023-03-09 18:47:57 +01:00
yih
d7e2e5d740 add sysroot permission test 2023-03-09 18:54:06 +08:00
Simon de Vlieger
a28d535577 image-installer: switch payload to minimal-rpm
With the addition of the minimal rpm package set and image type in #3181
the previously added bare metal package set is a duplicate. It's not
used elsewhere within fedora and the minimal rpm package set is
officially defined.
2023-03-08 12:15:23 +01:00
Simon de Vlieger
5f0d273248 distro: remove duplicate version checks for fonts
The new names of the packages are being added twice in two different
checks, remove the redundant code.
2023-03-08 12:12:18 +01:00