Commit graph

7208 commits

Author SHA1 Message Date
Achilleas Koutsou
c77ca66191 go.mod: update osbuild/images to v0.148.0
tag v0.145.0
Tagger: imagebuilder-bot <imagebuilder-bots+imagebuilder-bot@redhat.com>

Changes with 0.145.0

----------------
  * github: run dependabot gomod action weekly (osbuild/images#1476)
    * Author: Achilleas Koutsou, Reviewers: Lukáš Zapletal

— Somewhere on the Internet, 2025-05-12

---

tag v0.146.0
Tagger: imagebuilder-bot <imagebuilder-bots+imagebuilder-bot@redhat.com>

Changes with 0.146.0

----------------
  * Fixes for ESP partition: Make optional, set label (osbuild/images#1525)
    * Author: Alexander Larsson, Reviewers: Achilleas Koutsou, Brian C. Lane
  * Initial automotive work: custom selinux policy, separate build container for bootc, and ext4 verity (osbuild/images#1519)
    * Author: Alexander Larsson, Reviewers: Achilleas Koutsou, Simon de Vlieger
  * Update snapshots to 20250512 (osbuild/images#1515)
    * Author: SchutzBot, Reviewers: Achilleas Koutsou, Simon de Vlieger
  * disk: make auto-generated /boot 1 GiB big (osbuild/images#1499)
    * Author: Ondřej Budai, Reviewers: Achilleas Koutsou, Michael Vogt
  * distro.yaml: Clean up yamllint errors and warnings (osbuild/images#1523)
    * Author: Brian C. Lane, Reviewers: Michael Vogt, Simon de Vlieger
  * distro/rhel9: make /boot 1 GiB everywhere (osbuild/images#1498)
    * Author: Ondřej Budai, Reviewers: Michael Vogt, Simon de Vlieger
  * distro: move disk/container image types into pure YAML (COMPOSER-2533) (osbuild/images#1508)
    * Author: Michael Vogt, Reviewers: Achilleas Koutsou, Simon de Vlieger
  * fedora: move all image types into pure YAML (osbuild/images#1514)
    * Author: Michael Vogt, Reviewers: Brian C. Lane, Simon de Vlieger
  * fsnode: fix go-1.24 errors  (osbuild/images#1521)
    * Author: Michael Vogt, Reviewers: Ondřej Budai, Tomáš Hozza
  * osbuild: add JSON/YAML unmarshal to UdevRulesStageOptions (osbuild/images#1489)
    * Author: Michael Vogt, Reviewers: Achilleas Koutsou, Simon de Vlieger
  * test: Run more distro tests in parallel (osbuild/images#1483)
    * Author: Brian C. Lane, Reviewers: Michael Vogt, Simon de Vlieger

— Somewhere on the Internet, 2025-05-19

---

tag v0.147.0
Tagger: imagebuilder-bot <imagebuilder-bots+imagebuilder-bot@redhat.com>

Changes with 0.147.0

----------------
  * Add support for setting partition uuid and label (osbuild/images#1543)
    * Author: Alexander Larsson, Reviewers: Achilleas Koutsou, Simon de Vlieger
  * Cleanup of new APIs (mkfs options and build container) (osbuild/images#1526)
    * Author: Alexander Larsson, Reviewers: Achilleas Koutsou, Simon de Vlieger
  * distro/rhel: remove the user/group warnings for edge-commits (osbuild/images#1538)
    * Author: Achilleas Koutsou, Reviewers: Brian C. Lane, Simon de Vlieger

— Somewhere on the Internet, 2025-05-20

---

tag v0.148.0
Tagger: imagebuilder-bot <imagebuilder-bots+imagebuilder-bot@redhat.com>

Changes with 0.148.0

----------------
  * Makefile: add vet command to check for consistent struct tags (osbuild/images#1554)
    * Author: Michael Vogt, Reviewers: Lukáš Zapletal, Simon de Vlieger
  * disk: tiny tweaks for the new MkfsOptions support (osbuild/images#1545)
    * Author: Michael Vogt, Reviewers: Achilleas Koutsou, Alexander Larsson, Lukáš Zapletal
  * fedora/many: increase `/boot` to 1 GiB (HMS-8604) (osbuild/images#1557)
    * Author: Simon de Vlieger, Reviewers: Achilleas Koutsou, Ondřej Budai
  * fedora/wsl: include `wsl-setup` (HMS-8573) (osbuild/images#1550)
    * Author: Simon de Vlieger, Reviewers: Brian C. Lane, Michael Vogt
  * fedora: add `anaconda.ModuleUsers` to ImageInstallerImage (osbuild/images#1558)
    * Author: Michael Vogt, Reviewers: Brian C. Lane, Simon de Vlieger
  * fedora: implement setting of the RootfsType via YAML (osbuild/images#1544)
    * Author: Michael Vogt, Reviewers: Brian C. Lane, Simon de Vlieger
  * rhel10: move ImageConfig into pure YAML (osbuild/images#1542)
    * Author: Michael Vogt, Reviewers: Brian C. Lane, Simon de Vlieger

— Somewhere on the Internet, 2025-05-26

---
2025-06-05 10:35:58 +02:00
Brian C. Lane
12dd0b0be4 cloudapi: Fix listing composes with empty result
It was returning a null body instead of an empty list.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
b33f86783f cloudapi: Add artifact cleanup to delete handler
Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
56fc58cca3 cloudapi: Add DeleteCompose to delete a job by UUID
This adds the handler for DELETE /composes/{id} which will delete a job and
all of its dependencies, and any artifacts.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
a569ac0f7b cloudapi: Add DELETE /composes/{id} route to delete a job
This will be used to delete jobs and their artifacts.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
f1a2c24563 worker: Add CleanupArtifacts function
This removes all artifact directories, and their contents, if there
isn't an associated Job. This is used to clean up local artifacts after
the compose job has been deleted.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
c3394ae589 jobqueuetest: Add job delete tests
This adds tests for retrieving all root jobs, and deleting jobs
and unused dependencies. These tests are run against the fsjobqueue for
unit testing, and against dbjobqueue for integration testing.

Resolves: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
c06064c1e2 dbjobqueue: Add DeleteJob to database job queue
This adds SQL to delete jobs and dependencies, and implements the
database version of the DeleteJob function.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
5cddc4223d dbjobqueue: Add AllRootJobIDs implementation
Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
d8285a0b74 jobqueue: Add DeleteJob function
This allows jobs to be deleted from the database.
Currently only implemented by fsjobqueue. The function for
dbjobqueue currently returns nil.

This will remove all the job files used by the root job UUID as long as
no other job depends on them. ie. It starts at the top, and moves down
the dependency tree until it finds a job that is also used by another
job, removes the job to be deleted from its dependants list, and moves
back up the tree only deleting jobs with empty dependants lists.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
5961b69caa jsondb: Add Delete function
This allows database entries to be deleted.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Achilleas Koutsou
fd7dc96d06 gitlab: enable OCI tests
The problem with the test infrastructure should be resolved.

This reverts commit d0e805e8be.
2025-06-05 09:29:55 +02:00
Michael Vogt
acec29dea2 osbuild-store-dump: use distrofactory instead of importing fedora
This commit removes the direct use of `pkg/distro/fedora` and
uses `distrofactory` instead. This is the more generic interface,
it also helps with PR#1563 as `pkg/distro/fedora` goes away.

Note that I did not test this but @bcl was kind enough to
mention how this is used:
```
The output from this is used in Test_upgrade in
internal/store/json_test.go, the files there are manually
created when we change the format of the store and add a new
parser to internal/store/json.go -- which hasn't happened in
a long time :)
```
I updated the cmd/README with a snippet of the above.
2025-06-04 12:41:39 +02:00
Tom Koscielniak
488a53e106 tests: Fix failing cross distro test due to osbuild/images pseudo-version
The cross distro test started failing on RHEL 10.1, because the osbuild/images version was a Go pseudo-version that the sed command parsing the version did not recognize.
2025-06-03 17:06:52 +02:00
Tom Koscielniak
b3fab64dcc tests/CI: Fix missing slashes in RHEL 9.7/10.1 baseurl
In repository files for RHEL 9.7 and 10.1, there were missing slashes at the end of the baseurl which caused the satellite regression tests to fail. The refresh job only subs the snapshot dates and leaves the rest of the url unchanged.
2025-06-03 17:06:52 +02:00
Tom Koscielniak
7ed60dbe2e CI: Add tkosciel ssh key 2025-06-03 17:06:52 +02:00
Tom Koscielniak
93684fcef9 tests: Improve logging in api.sh test
Add compose request output print in case of failure.
2025-06-03 17:06:52 +02:00
Tom Koscielniak
557d5f440c tests/CI: Add runners for RHEL 9.7 and 10.1
Add runners for RHEL 9.7 and 10.1 and remove 9.6 and 10.0 runners.
2025-06-03 17:06:52 +02:00
Achilleas Koutsou
d0e805e8be gitlab: disable OCI tests
OCI tests are constantly failing due to a known issue.  A fix is coming
but it's taking a while so let's disable them so we can keep merging
PRs.
2025-05-30 18:56:40 +02:00
Tomáš Hozza
1fc5e2ad18 Packer: use latest RHEL-9 GA Cloud Access images for workers
Update the RHEL-9 Cloud Access images used for our workers from 9.0 to
9.6, which is the latest GA. We do upgrade all packages in our Ansible
playbook, but that is just waste of resources if we can use the latest
GA images.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-05-30 15:28:37 +02:00
Tomáš Hozza
73ceb94b51 Packer: update Fedora images to F42 and remove workarounds
Update Fedora workers from EOL F40 to F42.

Remove workarounds that should not be needed any more (i.e. the Packer
upstream issue has been closed).

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-05-30 15:28:37 +02:00
Tom Koscielniak
dee4a5a21b CI: Run RHEL 10 nightly pipeline on PRs
Add RHEL 10 nightly pipeline to PRs as well to avoid test breaking changes being introduced into main.
2025-05-28 11:06:28 +02:00
schutzbot
b825fd8f77 Post release version bump
[skip ci]
2025-05-28 08:14:53 +00:00
schutzbot
8441138d2d schutzfile: Update snapshots to 20250515 2025-05-27 09:22:16 +02:00
Sanne Raymaekers
c3cb3ba785 templates/packer: set wanted-by to cloud-init.target
The `cloud-init.target` in 9.6 has `After=multi-user.target` in its unit
config. The worker initialization service was set to run before
`multi-user.target`, but after `cloud-final.service`. This created an
impossible situation and systemd just disabling the initialization
service.

So this changes:
`multi-user.target -> worker-*.service -> cloud-final.service -> multi-user.target`
to
`cloud-init.target -> worker-*.service -> cloud-final.service -> multi-user.target`.

Thus resolving the loop.
2025-05-14 21:01:39 +02:00
schutzbot
92363cee22 Post release version bump
[skip ci]
2025-05-14 08:15:07 +00:00
Brian C. Lane
4cb47a424d cloudapi: Fix missing specs for koji ContainerResolve
It looks like this got lost in a refactor back in commit 6e4efabf24
when it used to populate the Specs list in a loop.
2025-05-13 20:38:38 +02:00
Lukas Zapletal
0e5d8a94f2 cmd: use log in service-maintenance 2025-05-12 15:20:16 +02:00
Lukas Zapletal
46f0a71053 cmd: use log in jobsite 2025-05-12 15:20:16 +02:00
Lukas Zapletal
71117f3ccc cmd: use log in koji 2025-05-12 15:20:16 +02:00
Lukas Zapletal
650e7b64c2 cmd: use log in upload-gcp 2025-05-12 15:20:16 +02:00
Lukas Zapletal
b87d3d78ec docs: add README to cmd/ 2025-05-12 15:20:16 +02:00
Sanne Raymaekers
af97ca6fe7 go.mod: bump images to v0.144.0
tag v0.144.0
Tagger: imagebuilder-bot <imagebuilder-bots+imagebuilder-bot@redhat.com>

Changes with 0.144.0

----------------
  * customizations: drop `FsNode` interface as it is not used (#1490)
    * Author: Michael Vogt, Reviewers: Brian C. Lane, Simon de Vlieger
  * distro/rhel10/ami: unset authselect in ImageConfig (COMPOSER-2517) (#1469)
    * Author: Achilleas Koutsou, Reviewers: Brian C. Lane, Michael Vogt
  * manifest/os: immediately create directories (HMS-6043) (#1502)
    * Author: Simon de Vlieger, Reviewers: Achilleas Koutsou, Michael Vogt
  * osbuild: Add config to InsightsClientConfigStageOptions (#1503)
    * Author: rverdile, Reviewers: Michael Vogt, Sanne Raymaekers
  * refactor: replace logrus with stdlib log package (#1481)
    * Author: Lukáš Zapletal, Reviewers: Michael Vogt, Simon de Vlieger

— Somewhere on the Internet, 2025-05-08
2025-05-12 14:06:21 +02:00
rverdile
1559b4d760 cloudapi/v2: add templates to subscription image options
Adds optional templateName and templateUUID paramaters to image options,
each used if registering a system to a template a boot.
2025-05-12 10:07:19 +02:00
Achilleas Koutsou
943e147a67 go.mod: update osbuild/images to v0.143.0
tag v0.142.0
Tagger: imagebuilder-bot <imagebuilder-bots+imagebuilder-bot@redhat.com>

Changes with 0.142.0

----------------
  * distro: move `kernelOptions` into `ImageConfig` (osbuild/images#1470)
    * Author: Michael Vogt, Reviewers: Achilleas Koutsou, Tomáš Hozza
  * manifest: register insights to template on boot (HMS-5994) (osbuild/images#1443)
    * Author: rverdile, Reviewers: Achilleas Koutsou, Tomáš Hozza
  * many: add custom unmarshalers for osbuild, platform (osbuild/images#1477)
    * Author: Michael Vogt, Reviewers: Achilleas Koutsou, Tomáš Hozza

— Somewhere on the Internet, 2025-05-05

---

tag v0.143.0
Tagger: imagebuilder-bot <imagebuilder-bots+imagebuilder-bot@redhat.com>

Changes with 0.143.0

----------------
  * distro/rhel9/azure: drop net.ifnames=0 kernel arg (RHEL-89440) (osbuild/images#1487)
    * Author: Achilleas Koutsou, Reviewers: Michael Vogt, Sanne Raymaekers
  * github: don't run manifest checksum validation on the merge queue (osbuild/images#1478)
    * Author: Achilleas Koutsou, Reviewers: Brian C. Lane, Florian Schüller
  * repositories: Set 6h expire on main repo, use default for updates (osbuild/images#1482)
    * Author: Brian C. Lane, Reviewers: Achilleas Koutsou, Neal Gompa (ニール・ゴンパ), Simon de Vlieger

— Somewhere on the Internet, 2025-05-05

---
2025-05-06 14:00:09 +02:00
Michael Vogt
1a6c70b649 internal: drop internal/fsnode package
This package has an identical copy in `images` and is also (AFAICT)
unused. Hence this commit removes it.
2025-05-06 10:45:21 +03:00
Tomáš Hozza
b0d2d47180 Test/cross-distro: add version check for cross-building el8 on el9
The issue has been fixed in 9.6 0day ZStream in v132.1, so running the
test against anything older will make the test fail. This is currently
the case for RHEL-9-nightly pipeline.

Let's solve this annoyance.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-05-06 02:27:52 +03:00
Achilleas Koutsou
e621386caf cloudapi: drop ImageRequest.GetImageOptions() method
This method is not particularly useful anymore.  Its purpose was to
initialise the ImageOptions from an ImageRequest with the appropriate
size and partitioning mode.  However, the partitioning mode was also
being set later using request.GetPartitioningMode().  More importantly,
setting the size on the ImageOptions caused issues with the interaction
between filesystem and partitioning customizations as well as the image
request size (see #4705).  The correct thing to do here is to map the
ImageRequest.Size directly onto ImageOptions.Size, without taking into
account ImageType or the Blueprint Customizations.  The rest are
considered when generating the manifest in images, either when preparing
the Manifest() call or when generating the partition table.  This makes
it easier to trace and reason about the effect of each option.  This
kind of decision making in the API layer makes it difficult to maintain
the logic, since it requires duplicating the decision making or, as we
had now, making certain specific combinations impossible.
2025-05-05 20:37:19 +02:00
Tomáš Hozza
c587e723a9 Test/cross-distro: take the images version used to compile composer
After the move of the repo configs to `osbuild/images`, we changed the
way to determine the all available distro repo configs by taking the
last release of 'images'. This turns out to be fragile, especially when
the test case is being run on a "rhel-x.y.0" branch or basically with
any old osbuild-composer binary. It could happen also in Nightly test
pipelines after the devel freeze.

We always compile osbuild-composer binary with debug information, so we
can determine the vendored 'images' version from the binary. We also
always ship repo configs from the same 'images' version in the RPMs.

Therefore modify the test case to check out the 'images' version that
osbuild-composer was compiled with and use repo configs from that
version.

In addition, don't reduce the depth when cloning the images repository,
because even 200 commits may be too little on some older RHEL ZStream
branches.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-04-30 14:36:48 +03:00
Tomáš Hozza
ad68e81f1f Test/cross-distro: fix processing of repo configs
Previously, the code would remove everything after the first dot in the
repo config filename. So 'rhel-10.0.json' -> 'rhel-10'. This means that
there was just a bunch of 'rhel-X' repo config names to compare. This is
probably a leftover from the days before we introduced dot-notation for
distro names.

Modify the test to effectivelly strip only the filename extension (the
part after the last dot, including the dot).

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-04-30 14:36:48 +03:00
schutzbot
55cd6a9df5 Post release version bump
[skip ci]
2025-04-30 08:14:51 +00:00
Tomáš Hozza
746151803a GH actions: move from soon to be EOL F40 to F42
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-04-30 09:14:48 +02:00
Achilleas Koutsou
045364cbf2 cloudapi: reduce minsize type to just string
The type of the minsize parameter in the new disk customization was
meant to support both integers (size in bytes) and strings (size with
unit).  However, the schema wasn't done properly, which made any input
result in an error:

    GenericError: Failed to create the compose request:
    {"code":"IMAGE-BUILDER-COMPOSER-30","details":"request body has an
    error: doesn't match schema #/components/schemas/ComposeRequest:
    Error at \"/customizations/disk/partitions/0\": doesn't match schema
    due to: Error at \"/minsize\": input matches more than one oneOf
    schemas

Reducing it to just support strings simplifies the schema.  It's also
not an important feature reduction since sizes as integers (for
filesystems that are typically in GiB) aren't very convenient.
2025-04-29 23:40:59 +02:00
Achilleas Koutsou
84dfe7e21b go.mod: update osbuild/images to v0.141.0
tag v0.140.0
Tagger: imagebuilder-bot <imagebuilder-bots+imagebuilder-bot@redhat.com>

Changes with 0.140.0

----------------
  * Loader part override followups (osbuild/images#1466)
    * Author: Michael Vogt, Reviewers: Tomáš Hozza
  * RHEL-10/gce: install cloud-utils-growpart by default (osbuild/images#1463)
    * Author: Tomáš Hozza, Reviewers: Michael Vogt, Simon de Vlieger
  * base-host-check: expand to cover more (COMPOSER-2498) (osbuild/images#1432)
    * Author: Simon de Vlieger, Reviewers: Tomáš Hozza
  * rhel8,9: move partitioning into YAML (osbuild/images#1390)
    * Author: Michael Vogt, Reviewers: Tomáš Hozza

— Somewhere on the Internet, 2025-04-28

---

tag v0.141.0
Tagger: imagebuilder-bot <imagebuilder-bots+imagebuilder-bot@redhat.com>

Changes with 0.141.0

----------------
  * Shrink the current installer images a bit (osbuild/images#1423)
    * Author: Brian C. Lane, Reviewers: Achilleas Koutsou, Simon de Vlieger, Tomáš Hozza
  * Update EL10 Azure images with changes made in RHEL 9.6 (osbuild/images#1472)
    * Author: Achilleas Koutsou, Reviewers: Michael Vogt, Tomáš Hozza
  * distro: add DNFSetReleaseVerVar to ImageConfig (osbuild/images#1465)
    * Author: Michael Vogt, Reviewers: Achilleas Koutsou, Tomáš Hozza
  * test/base-host-check.sh: fix disabled firewall services check (osbuild/images#1471)
    * Author: Sanne Raymaekers, Reviewers: Achilleas Koutsou, Tomáš Hozza

— Somewhere on the Internet, 2025-04-29

---
2025-04-29 18:20:17 +02:00
Tomáš Hozza
7036e2bcb9 Many: expose rhsm.subscription_manager.rhsm.auto_enable_yum_plugins
Expose the ${SUBJ} customization in Cloud API. Weldr API inherited it
automatically with the update of `osbuild/blueprint` to v1.6.0.

Adjust relevant test cases and unit tests to use this new customization.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-04-29 18:22:40 +03:00
Irina Gulina
6f4c2899f8 remove workaround for CLOUDX-994 2025-04-25 13:42:27 +02:00
Ondřej Budai
401f832cc8 client: adjust the tests to new image type names in Fedora
See the comment inline.
2025-04-25 09:26:59 +02:00
Simon de Vlieger
242564cc5c test: update type names to new names
The image type names for Fedora were changed in images 0.138.0, aliases
were left behind so the old names still exist.

Split out the testcases between rhel and fedora so we test for the new
names being returned.

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
2025-04-25 09:26:59 +02:00
Ondřej Budai
31d1665f7e go mod: update osbuild/images to v0.139.0
Includes:

0.139.0

RHEL 9 Azure kernel command line arguments (#1457)
  Author: Achilleas Koutsou, Reviewers: Ondřej Budai, Simon de Vlieger, Tomáš Hozza

0.138.0

Update dependencies 2025-04-24 (#1460)
  Author: SchutzBot, Reviewers: Achilleas Koutsou
ci: add missing gh tool to gobump action (#1452)
  Author: Lukáš Zapletal, Reviewers: Achilleas Koutsou, Simon de Vlieger
ci: keep GITHUB_TOKEN in gobump action (#1458)
  Author: Lukáš Zapletal, Reviewers: Achilleas Koutsou, Simon de Vlieger
ci: remove unnecessary git config command (#1451)
  Author: Lukáš Zapletal, Reviewers: Achilleas Koutsou
fedora: image type names follow variants (#1233)
  Author: Simon de Vlieger, Reviewers: Achilleas Koutsou
many: propagate masked services (#1456)
  Author: Simon de Vlieger, Reviewers: Achilleas Koutsou, Ondřej Budai, Tomáš Hozza
2025-04-25 09:26:59 +02:00
Sanne Raymaekers
7c0aecdd82 .gitlab-ci.yml: test WSL images on RHEL 10 2025-04-24 16:59:44 +02:00