Commit graph

5452 commits

Author SHA1 Message Date
Brian C. Lane
2ab0430d5d weldr: Log when starting and finishing metadata preload
It is difficult to tell if these are really running in parallel or not,
even with loggin, but it helps. They will all always start at the same
time (because they are run concurrently with goroutines) and if things
work right should be finished at about the same time.
2023-03-23 11:26:40 +01:00
Brian C. Lane
c6a8d89da6 remotefile: Use io.ReadAll not ioutil.ReadAll 2023-03-22 08:14:01 -07:00
schutzbot
2040e93e56 Post release version bump
[skip ci]
2023-03-22 08:21:56 +00:00
Sanne Raymaekers
a2a3a2602c templates/dashboards/worker: add arch label to job wait duration
Display the wait duration of jobs per architecture.
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
b13865d361 templates/dashboards/worker: edit thresholds
95th percentile duration is now a fixed colour, as it's tricky to get
dynamic thresholds based on the job type.

Budget remaining thresholds are now only green at infinity, turn yellow
below 4 weeks, and turn red when budget consumption would only last 3
weeks (out of 4).
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
63d5132aa6 templates/dashboards/worker: change panel alignment
This aligns vertical dividers between panels across rows.
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
865bb98034 templates/dashboards/worker: bump version 2023-03-21 12:34:09 +01:00
Sanne Raymaekers
5a9f8d3457 templates/dashboards/worker: show request throughput per path 2023-03-21 12:34:09 +01:00
Sanne Raymaekers
26a521f54d templates/dashboards/worker: use jobtype variable for job stats
This removes the rows of panels per job type, and uses the jobtype
variable.
2023-03-21 12:34:09 +01:00
Sanne Raymaekers
5d2f84cb9e templates/dashboards/worker: add target duration 2023-03-21 12:34:09 +01:00
Sanne Raymaekers
0b7e94b097 templates/dashboard/worker: add job type variable 2023-03-21 12:34:09 +01:00
Juan Abia
06d83d3c07 Change civ repo name 2023-03-21 11:56:57 +01:00
Jakub Rusz
ecd69ca25b tests/ostree: Change centos-8 BOOT_LOCATION to a working boot.iso
The latest boot.iso stopped working. Changing to an older version to
unblock CI. RHBZ#2180024 opened with this issue.
2023-03-20 17:59:20 -07:00
Simon de Vlieger
6e4893ad78 test/data: regenerate f39 manifests
Since F39 got enabled on `main` after the rebase they have different
sizes in this branch.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
ca836a87f4 internal/disk: Remove RequiredSizes property.
The property was only used in a single method where it is passed as an
argument. There is no need to expose it.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
6704dfd609 distro/fedora: clarify requiredPartitionSizes 2023-03-20 11:04:14 +01:00
Simon de Vlieger
7e93df725c distro: move RequiredPartitionSizes into ImageType
It used to be defined on ImageOptions, but ImageType is a better fit for
this.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
34f7f6c7e2 disk: make testcase less confusing
This testcase tested for a / size of 1 GiB but better is to test for a
minimal size of 4 GiB which is the actual lower limit when the required
sizes are added together.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
92cdb0f082 iot-raw-image: change partition table to DOS
For parity with ImageFactory built images there should be only a DOS
partition table in the raw images, with a non-standard ID set for the
boot partition for Raspberry Pi compatibility.

This only applies to the aarch64 images.

As noted in #3220.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
39879a9f60 disk: add RequiredSizes to ImageOptions
These RequiredSizes are a map that is passed on to the partition table
logic which had hardcoded defaults. This makes it possible to define
either no RequiredSizes (`nil`) or empty RequiredSizes which means no
further constraint checks or partition resizes will be done.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
d7f5fac183 disk: clarify comment 2023-03-20 11:04:14 +01:00
Simon de Vlieger
053bbfcb98 iot-raw-image: set size to 4 GiB
The current Fedora IoT raw images are 4 GiB but we were using 10 GiB raw
images. Make this consistent to current.
2023-03-20 11:04:14 +01:00
dependabot[bot]
88f863f14b build(deps): bump actions/setup-go from 3 to 4
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 10:56:34 +01:00
Gianluca Zuccarelli
4fee181fec worker: add file resolve job
Implement a file resolution job which fetches
the contents of a remote file.
2023-03-16 09:55:39 +00:00
Gianluca Zuccarelli
b7e7bafb2e internal/remotefile: create a remote file resolver
Create a resolver to fetch the contents of a
remote file which can be used at build time.
The initial usecase for this resolver is to
resolve remote gpgkeys but the resolver has
been made more generic for general files.
2023-03-16 09:55:39 +00:00
Gianluca Zuccarelli
c056db4811 worker/server: add file resolver job
Add a file resolver job to the worker server in
order for us to resolve the contents of a remote
file.
2023-03-16 09:55:39 +00:00
Gianluca Zuccarelli
98d611d34f worker/server: fix container resolver job error
The container job resolve job error message was printing
the wrong error type to the error string.
2023-03-16 09:55:39 +00:00
Gianluca Zuccarelli
b493da1878 internal/clienterrors: add file resolution error
Add a client error type for an error in the remote
file resolution job.
2023-03-16 09:55:39 +00:00
Antonio Murdaca
414424cd30 simplified-installer: enable isolinux
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2023-03-15 12:35:50 +01:00
Sanne Raymaekers
861e1473d9 test/vmware: boot VMs with EFI and use SCSI as a disk controller
Since we use streamOptimized images everywhere, we no longer need to use
IDE and boot with bios. Let's test a more realistic scenario.
2023-03-15 10:37:21 +01:00
Simon de Vlieger
23627e9ce0 distro/fedora: add support for Fedora 39 2023-03-15 09:31:23 +01:00
yih
4023b61e1a test fdo create user 2023-03-10 14:40:49 +01:00
Sanne Raymaekers
a25e0f4adb prometheus:: add arch label to dequeue metrics
Only add the arch label for osbuild job types, as the finish metrics
behave similarly. Having arch labels on dequeue metrics for any other
job type (but not on the finish metrics) would produce weird results.
2023-03-09 18:47:57 +01:00
yih
d7e2e5d740 add sysroot permission test 2023-03-09 18:54:06 +08:00
Simon de Vlieger
a28d535577 image-installer: switch payload to minimal-rpm
With the addition of the minimal rpm package set and image type in #3181
the previously added bare metal package set is a duplicate. It's not
used elsewhere within fedora and the minimal rpm package set is
officially defined.
2023-03-08 12:15:23 +01:00
Simon de Vlieger
5f0d273248 distro: remove duplicate version checks for fonts
The new names of the packages are being added twice in two different
checks, remove the redundant code.
2023-03-08 12:12:18 +01:00
schutzbot
365e8242f9 Post release version bump
[skip ci]
2023-03-08 08:23:28 +00:00
Brian C. Lane
b1c2dbdfc6 Remove old build tag comments
Go is transitioning from the old '// +build' form to '//go:build', this
removes all uses of the old form, adding the new one where needed.

See https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
2023-03-07 09:22:23 -08:00
Brian C. Lane
7a4bb863dd Update deprecated io/ioutil functions
ioutil has been deprecated since go 1.16, this fixes all of the
deprecated functions we are using:

ioutil.ReadFile -> os.ReadFile
ioutil.ReadAll -> io.ReadAll
ioutil.WriteFile -> os.WriteFile
ioutil.TempFile -> os.CreateTemp
ioutil.TempDir -> os.MkdirTemp

All of the above are a simple name change, the function arguments and
results are exactly the same as before.

ioutil.ReadDir -> os.ReadDir

now returns a os.DirEntry but the IsDir and Name functions work the
same. The difference is that the FileInfo must be retrieved with the
Info() function which can also return an error.

These were identified by running:
golangci-lint run --build-tags=integration ./...
2023-03-07 09:22:23 -08:00
Tomáš Hozza
0e4a5b34b2 worker: allow configuring number of upload threads for Azure
The default number of threads (16) is OK for general use case. However,
we are being asked by RH IT to lower the number of threads when
uploading the image to Azure using proxy server.

Make the number of threads configurable in the worker configuration and
default to the currently used value if it is not provided.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-03 18:15:53 +01:00
Sanne Raymaekers
a08eb69b2e templates/packer/ansible: fix enabling cdn repos on aarch64 2023-03-03 17:58:49 +01:00
Sanne Raymaekers
c1032f31e4 templates/packer/ansible: fix unregister
The community redhat_subscription module calls `subscription-manager
unsusbscribe`, which doesn't exist. Use shell for now.
2023-03-03 17:58:49 +01:00
schutzbot
3f9da9df33 schutzfile: Update snapshots to 20230301 2023-03-03 13:11:42 +01:00
Sanne Raymaekers
ca8a05bd3a templates/packer: subscribe packer machines
To avoid a mismatch between the RPMs (which are build using CDN content)
and the packer instances (RHUI, which might be older).
2023-03-03 13:00:05 +01:00
Sanne Raymaekers
41d98c85a7 tools/appsre-ansible: revert sub removal on rpmbuild machines
The rhel-9 templates used in mock don't work with RHUI, they require the
CDN. This reintroduces the RHUI-CDN repo sync issue again.

This reverts commit d9f2ac14bc.
2023-03-03 13:00:05 +01:00
yih
266cfdcc44 fix ansible error on centos9 2023-03-02 16:03:11 +01:00
Diaa Sami
19f9ab7f58 koji: log unsuccessful requests only once 2023-03-02 15:48:12 +01:00
Tomáš Hozza
f8b39cd34e manifest/os: fix SUPPRESSED_ERROR issue reported by Coverity
Fix the following error reported by Coverity:
```
Defect type: SUPPRESSED_ERROR
1. osbuild-composer-75/_build/src/github.com/osbuild/osbuild-composer/internal/manifest/os.go:546: suppressed_error: The error returned by the function "GetVerStrFromPackageSpecList(p.packageSpecs, "dracut-config-rescue")" is not explicitly checked.
2. osbuild-composer-75/_build/src/github.com/osbuild/osbuild-composer/internal/manifest/os.go:546: remediation: Explicitly check the return error.
   544|   				}
   545|
   546|-> 				rescueVer, _ := rpmmd.GetVerStrFromPackageSpecList(p.packageSpecs, "dracut-config-rescue")
   547|   				hasRescue := rescueVer != ""
   548|   				bootloader = osbuild.NewGrub2LegacyStage(
```

Change the code to check if the called function returned an error or
not. If not error is returned, then the function returns a package
version string.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-01 20:20:52 +01:00
Sanne Raymaekers
75e6333b9e Revert "tools/appsre-ansible: workaround missing authselect-compat"
This reverts commit 4833f99070.
2023-03-01 20:05:38 +01:00
Sanne Raymaekers
0096ff3689 Revert "Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos"
This reverts commit 0a4a75e19e.
2023-03-01 20:05:38 +01:00