Commit graph

5477 commits

Author SHA1 Message Date
Achilleas Koutsou
4e3d94f9a7 test-case-generators: add request for Azure EAP 7 image type 2023-03-30 18:40:12 +02:00
Achilleas Koutsou
520e455088 test: add eap7 repo mirror to RHEL 8.6+ tests 2023-03-30 18:40:12 +02:00
Achilleas Koutsou
81d6f526d0 config: don't show azure-eap7-rhui in weldr
The azure-eap7-rhui image type should only be available for internal
builds (for now).
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
21d13a2ae3 distro/rhel8: specify ports for azure EAP
See COMPOSER-1859
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
fb5accc227 distro/rhel8: add env vars to azure EAP
Add the required environment variables to Azure EAP.
See COMPOSER-1860
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
3f299c7d43 distro: add ShellVars field to ImageConfig 2023-03-30 18:40:12 +02:00
Achilleas Koutsou
816d23988c manifest: add support for shell vars in OSCustomizations
ShellVars are added as a nested map of the form
filename->key->value.  A stage is generated and added if any are
defined.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
92bbf5fb8b osbuild: add generator function for shell.init stage
Generates a stage from a basic nested map.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
a024b923a3 shell: new package for representing shell init configs internally
New types for the internal representation of the shell init configs,
which currently consists of files and key-value pairs for environment
global variables.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
2fcf3582b5 osbuild: add shell.init stage
Add support for the org.osbuild.shell.init stage and test validator.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
636f3ff237 distro/rhel8: add azure EAP 7 image type
Initialise the image type as an Azure RHUI type with the name
"azure-eap7-rhui" and a workload attached that provides the extra
packages required by the EAP workload.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
5b4e77797c distro/rhel8: remove outdated comment 2023-03-30 18:40:12 +02:00
Achilleas Koutsou
18ef5ae32d distro/rhel8: distro workload implementation and EAP
Add a local, private implementation of the Workload interface for RHEL
8.  The type should be used for concrete workloads that provide package
package lists for specific image types.

The eapWorkload() function creates a workload that provides the packages
required for EAP image type variants.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
77b8328999 distro: add workload to image type
Support specifying workload on an image type.
This isn't used yet, but when set, the workload will define packages and
services and not allow any blueprint customizations.

In the future, there will be multiple workloads with each image type
specifying one (but multiple image types can have the same default
workload).  The custom workload will allow all customizations that are
currently supported.
2023-03-30 18:40:12 +02:00
Jakub Rusz
c59a92b2d9 rpmbuild: add fedora-38 2023-03-29 14:32:06 +03:00
Ondřej Budai
7e13d20ca6 packit: build RPMs also for EPEL 9 and RHEL
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-03-29 12:24:16 +02:00
dependabot[bot]
4bd6f82cc9 build(deps): bump github.com/aws/aws-sdk-go from 1.44.114 to 1.44.230
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.114 to 1.44.230.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.114...v1.44.230)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-28 11:11:05 +02:00
Tomáš Hozza
58e3716b89 distro: add support for RHEL-8.9 and RHEL-9.3
Add a basic support for building RHEL-8.9 and RHEL-9.3 images with
composer.

Add 8.9 and 9.3 repositories to the multitude of places where we have
them.

Generate image test manifests for 8.9 and 9.3. No functional testing is
added for 8.9 or 9.3 at this moment.

This change unblocks the RHEL Gating (since unit tests are currently
failing on 8.9 and 9.3 as on unknown distribution).

Related to https://issues.redhat.com/browse/COMPOSER-1924

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-27 11:39:24 +02:00
Juan Abia
e3594f7234 Start using CIV config file
This way, all CIV parameters can be provided from a file stored in
gitlab secrets without the need to change the script.
2023-03-24 23:10:55 +01:00
Mario Cattamo
00de277d3a Add Butane test and Ignition logs 2023-03-24 17:07:36 +01:00
Tomáš Hozza
dc637d9a57 Manifest: always set kernel options in grub2 stage
It turned out that when we stopped setting the kernel options in grubenv
file, we also stopped setting them in /etc/default/grub under
`GRUB_CMDLINE_LINUX`. This file is used by grub2-mkconfig when
generating grub configuration.

10_linux script executed by grub2-mkconfig recently started to overwrite
the /etc/kernel/cmdline, if its timestamp is older than the timestamp of
/etc/default/grub [1]. As a result, all kernel options were wiped out from
/etc/kernel/cmdline.

Make sure that we always set the `KernelOptions` in the grub2 stage
options, even if the `WriteCmdLine` is set to `false`.

In addition, unify the way we concatenate kernel options set in the
grub2 stage options. Some pipeline implementations were previously using
space, other were using comma. Space is now used everywhere.

Regenerate all affected image manifests.

[1] https://src.fedoraproject.org/rpms/grub2/c/fc76aed5333f56dd05400521a35b944a5df52ebc

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-23 16:32:34 +01:00
Sanne Raymaekers
03973acc1f distro/rhel: add payload repos to os package set
Before instantiating the manifest, any repositories that contain a
package set key contained in `PayloadPackageSets()` should be added to
the os package set in order to correctly generate the `org.osbuild.rpm`
stage for the os pipeline.

Otherwise options like GPG keys are not set correctly.

Fixes #3326
2023-03-23 13:39:04 +01:00
Brian C. Lane
f731ab53d0 dnfjson: Cleanup old distro cache dirs
This adds a function, CleanupOldCacheDirs, that checks the dirs under
/var/cache/osbuild-composer/rpmmd/ and removes files and directories
that don't match the current list of supported distros.

This will clean up the cache from old releases as the are retired, and
will also cleanup the old top level cache directory structure after an
upgrade.

NOTE: This function does not return errors, any real problems it
encounters will also be caught by the cache initialization code and
handled there.
2023-03-23 11:26:40 +01:00
Brian C. Lane
8f20b550ea dnfjson: Adjust cache size info for new layout
The repo id hash directories are now down one level, below a directory
named for the distribution.
2023-03-23 11:26:40 +01:00
Brian C. Lane
3481e1d3ba Change the rpmmd cache directory structure to include the distro name
This causes dnf-json to use separate caches, allowing them to run in
parallel, with one lock per distribution. Multiple depsolves with the
same distribution in the blueprint will continue to be serial.
2023-03-23 11:26:40 +01:00
Brian C. Lane
2ab0430d5d weldr: Log when starting and finishing metadata preload
It is difficult to tell if these are really running in parallel or not,
even with loggin, but it helps. They will all always start at the same
time (because they are run concurrently with goroutines) and if things
work right should be finished at about the same time.
2023-03-23 11:26:40 +01:00
Brian C. Lane
c6a8d89da6 remotefile: Use io.ReadAll not ioutil.ReadAll 2023-03-22 08:14:01 -07:00
schutzbot
2040e93e56 Post release version bump
[skip ci]
2023-03-22 08:21:56 +00:00
Sanne Raymaekers
a2a3a2602c templates/dashboards/worker: add arch label to job wait duration
Display the wait duration of jobs per architecture.
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
b13865d361 templates/dashboards/worker: edit thresholds
95th percentile duration is now a fixed colour, as it's tricky to get
dynamic thresholds based on the job type.

Budget remaining thresholds are now only green at infinity, turn yellow
below 4 weeks, and turn red when budget consumption would only last 3
weeks (out of 4).
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
63d5132aa6 templates/dashboards/worker: change panel alignment
This aligns vertical dividers between panels across rows.
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
865bb98034 templates/dashboards/worker: bump version 2023-03-21 12:34:09 +01:00
Sanne Raymaekers
5a9f8d3457 templates/dashboards/worker: show request throughput per path 2023-03-21 12:34:09 +01:00
Sanne Raymaekers
26a521f54d templates/dashboards/worker: use jobtype variable for job stats
This removes the rows of panels per job type, and uses the jobtype
variable.
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
5d2f84cb9e templates/dashboards/worker: add target duration 2023-03-21 12:34:09 +01:00
Sanne Raymaekers
0b7e94b097 templates/dashboard/worker: add job type variable 2023-03-21 12:34:09 +01:00
Juan Abia
06d83d3c07 Change civ repo name 2023-03-21 11:56:57 +01:00
Jakub Rusz
ecd69ca25b tests/ostree: Change centos-8 BOOT_LOCATION to a working boot.iso
The latest boot.iso stopped working. Changing to an older version to
unblock CI. RHBZ#2180024 opened with this issue.
2023-03-20 17:59:20 -07:00
Simon de Vlieger
6e4893ad78 test/data: regenerate f39 manifests
Since F39 got enabled on `main` after the rebase they have different
sizes in this branch.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
ca836a87f4 internal/disk: Remove RequiredSizes property.
The property was only used in a single method where it is passed as an
argument. There is no need to expose it.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
6704dfd609 distro/fedora: clarify requiredPartitionSizes 2023-03-20 11:04:14 +01:00
Simon de Vlieger
7e93df725c distro: move RequiredPartitionSizes into ImageType
It used to be defined on ImageOptions, but ImageType is a better fit for
this.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
34f7f6c7e2 disk: make testcase less confusing
This testcase tested for a / size of 1 GiB but better is to test for a
minimal size of 4 GiB which is the actual lower limit when the required
sizes are added together.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
92cdb0f082 iot-raw-image: change partition table to DOS
For parity with ImageFactory built images there should be only a DOS
partition table in the raw images, with a non-standard ID set for the
boot partition for Raspberry Pi compatibility.

This only applies to the aarch64 images.

As noted in #3220.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
39879a9f60 disk: add RequiredSizes to ImageOptions
These RequiredSizes are a map that is passed on to the partition table
logic which had hardcoded defaults. This makes it possible to define
either no RequiredSizes (`nil`) or empty RequiredSizes which means no
further constraint checks or partition resizes will be done.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
d7f5fac183 disk: clarify comment 2023-03-20 11:04:14 +01:00
Simon de Vlieger
053bbfcb98 iot-raw-image: set size to 4 GiB
The current Fedora IoT raw images are 4 GiB but we were using 10 GiB raw
images. Make this consistent to current.
2023-03-20 11:04:14 +01:00
dependabot[bot]
88f863f14b build(deps): bump actions/setup-go from 3 to 4
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 10:56:34 +01:00
Gianluca Zuccarelli
4fee181fec worker: add file resolve job
Implement a file resolution job which fetches
the contents of a remote file.
2023-03-16 09:55:39 +00:00
Gianluca Zuccarelli
b7e7bafb2e internal/remotefile: create a remote file resolver
Create a resolver to fetch the contents of a
remote file which can be used at build time.
The initial usecase for this resolver is to
resolve remote gpgkeys but the resolver has
been made more generic for general files.
2023-03-16 09:55:39 +00:00