Commit graph

2847 commits

Author SHA1 Message Date
Andre Marianiello
f6ae58151e stages/org.osbuild.users: add expire date
Add the expiredate field to the users option in the org.osbuild.users
stage. This option maps to the --expiredate option of useradd/usermod,
which can be useful when creating users whose password must be changed
upon first login.
2024-03-12 07:38:31 +01:00
Gianluca Zuccarelli
8b601d146b util/containers: remount containers store as rw 2024-03-11 18:15:24 +02:00
Gianluca Zuccarelli
9258bda89d util/mnt: add a remount paramater
This is just a temporary workaround to get the containers-storage
input working in `bootc-image-builder`
2024-03-11 18:15:24 +02:00
Gianluca Zuccarelli
36d1187c35 util/containers: update storage location
Change the contianers store from `/containers/storage` to
`/var/tmp/containers/storage` since technically `/containers/storage`
isn't ostree compatible.
2024-03-11 18:15:24 +02:00
Michael Vogt
945209a80c test: use del os.environ[] instead of os.unsetenv (thanks Simon)
Simon discovered that `os.unsetenv()` will not remove the env
from os.environ so this commit fixes this.

Thanks for figuring this out!
2024-03-11 14:20:27 +01:00
Michael Vogt
2efe2de09c workflow: run all unit tests in parallel
There were special cases in the workflow for only running specific
tests in parallel. However how that the test_assemblers parallel
runs are fixed [0] there is really no need for special cases anymore
and we can just run them all in parallel.

[0] https://github.com/osbuild/osbuild/pull/1641
2024-03-11 13:48:47 +01:00
Michael Vogt
2f0ed8c755 osbuild: ensure a usable /var/tmp is available inside the buildroot
Colin asked for this in
https://github.com/osbuild/bootc-image-builder/issues/223 and
it's easy enough.
2024-03-11 13:09:28 +01:00
Simon de Vlieger
a895aa177c stages/kickstart: add noswap to autopart
Allow passing the `noswap` option to `autopart`.
2024-03-11 11:50:34 +01:00
Sanne Raymaekers
29159189f1 sources/curl: add org.osbuild.mtls secrets support
If `org.osbuild.mtls` is passed as a secret name, look for the mtls data
in the environment.
2024-03-11 11:09:37 +01:00
Michael Vogt
c990c07f79 osbuild: reuse loop_for_fd() in LoopServer
The LoopServer._create_device() duplicates the code from
`Loop.loop_for_fd()` right now. Given that this is relatively
subtle code it's better to not duplicate it and instead just
reuse the loop_for_fd() implementation in LoopServer.
2024-03-11 10:56:49 +01:00
Michael Vogt
d43194c0b3 test: add unit-{type,path} test coverage 2024-03-08 16:20:48 -08:00
Gianluca Zuccarelli
6c0973238d utils/mnt: fix mount permissions
This is a follow up to #1550 where we enabled a `rw` permissions mode,
which is not ideal since it would theoretically be possible to set both
`ro` and `rw` modes at the same time. This commit fixes the issue by only
allowing one option at a time.

Fixes #1588
2024-03-07 13:01:47 +00:00
Sayan Paul
2d2cdd8097 stage/systemd-unit-create:paremeterize file location
User can now customize the systemd unit load path.
User can select between etc or usr , defaults to 'usr'.
Also user can customize the scope of the service between global
or system, defaults to system.

Signed-off-by: Sayan Paul <paul.sayan@gmail.com>
2024-03-07 14:04:00 +02:00
Simon Steinbeiss
144b0563d6 tests/manifest: Don't retain image-info artifacts
Nothing else seems to rely on the image-info generated by the manifest
tests, so let's not retain those artifacts. This currently makes the
`finish` stage take 6 minutes for no good reason.
2024-03-06 13:06:10 +01:00
Michael Vogt
1278e5d217 test: fix test_assembler to support parallel runs and run in parallel
The `test_assembler.py` hardcods some filesystem and partition
UUIDs. This leads to hard to diagnose test failures when the
test is run in parallel. The btrfs and xfs filesystem drivers
will see the same uuid for multi created images and error sometimes with
someting like:
```
Mar 06 10:22:54 top kernel: BTRFS error: device /dev/loop104 belongs to fsid aff010e9-df95-4f81-be6b-e22317251033, and the fs is already mounted, scanned by mount (123856)
```
Its a race that only happens when two images are checked at the
same time.

This commit fixes the issue by just using a randomized UUID in
the test_assemblers.py. It also re-enables running the test in
parallel (which make it run a lot faster, from 34min to 14min).
2024-03-06 12:28:10 +01:00
Michael Vogt
87636878da Revert "workflow: run assembler tests in parallel"
This reverts commit ea36e25b09.
2024-03-06 10:43:21 +01:00
Simon Steinbeiss
37d5a23a64 tests: Cancel in-progress tests on PR updates
Before this commit GitHub Action runs that were triggered by a PR were
not canceled when updates were made to the same PR. This lead to even
more clogging of our pipelines and not enough runners being available.

This changes the behavior in a way that whenever a PR gets updated all
still-in-progress runs get canceled and new runs get spawned.
2024-03-05 16:40:34 +01:00
schutzbot
b8b81280ff schutzfile: Update snapshots to 20240301 2024-03-05 17:10:53 +02:00
Simon de Vlieger
ef612d592c lint: add lint-quick target
This runs `ruff` locally, outside a special created environment which
makes it super quick but requires `ruff` to be installed.
2024-03-05 16:10:27 +01:00
Simon de Vlieger
82218c91a1 tox: remove isort
The role of the `isort` check is now performed by `ruff` instead of
`isort`.
2024-03-05 16:10:27 +01:00
Simon de Vlieger
6536a45d0a lint: explicit check
Add an explicit check to the return value of subprocess.run.
2024-03-05 16:10:27 +01:00
Simon de Vlieger
c9739dbd2a lint: sort import(s) 2024-03-05 16:10:27 +01:00
Simon de Vlieger
a779d3268d ruff: configure
Newer versions of ruff support more linters and autoformatting, let's
configure those to do the same as our current linter selection.
2024-03-05 16:10:27 +01:00
Simon de Vlieger
7fc06acffc tox: update ruff dependency
Renames a property that should now live under `lint` to silence a
warning in newer `ruff`.
2024-03-05 16:10:27 +01:00
Achilleas Koutsou
a0f5262701 gitlab: don't update GitHub status from individual jobs
Never update the GitHub status at the end of a job.  Instead, when
everything is done, the 'fail' job will run in the 'finish' stage and
update the status accordingly.
2024-03-05 16:04:20 +01:00
Achilleas Koutsou
f7dd0878b9 gitlab: new pipeline 'fail'
New pipeline called fail that runs in the 'finish' stage but only when a
job fails.  The job updates the github status to indicate failure.
2024-03-05 16:04:20 +01:00
Achilleas Koutsou
b0abe9725d schutzbot/update_github_status.sh: new verb: fail
New script subcommand that explicitly reports a failure.
2024-03-05 16:04:20 +01:00
Ondřej Budai
57bba13752 gitlab: parallelize the manifest db tests even more
Why not, we can run a lot of VMs in parallel, and this will speed up the test
runs.
2024-03-05 12:49:21 +02:00
Ondřej Budai
08d4bbf4dd test/ostree: parallelize the test
Building 7 images on one machine is quite slow. Instead, let's spawn build
them on separate ones to save some time.
2024-03-05 12:49:21 +02:00
Ondřej Budai
6750c0fd6a gitlab: Stop running the ostree tests in the internal network
It's just Fedora, there's nothing internal.
2024-03-05 12:49:21 +02:00
Ondřej Budai
8632269f79 gitlab: Run the ostree tests just on Fedora
These tests are building just Fedora manifests, thus there's basically
no point in running them on RHEL, since we never support building
Fedora on RHEL.
2024-03-05 12:49:21 +02:00
Simon Steinbeiss
b8e0253493 actions: Add a PR best practices check
This pipeline tests:
1. If the PR description is not empty (blocking)
2. If the PR title follows our format (non-blocking)
   `component: This is the change (JIRA-001)`
3. If 1. and 2. are True, it adds a 'best practice' label to the PR
2024-03-05 12:01:10 +02:00
Michael Vogt
ea36e25b09 workflow: run assembler tests in parallel
The assembler tests currently run around 45min, try to run in
parallel to see how much we can win from that.
2024-03-05 07:44:26 +01:00
Michael Vogt
4d2476a26d test: use OSBUILD_TEST_STORE in test_assemblers.py too
Use the OSBUILD_TEST_STORE in the test_assemblers.py file too
and re-use already downloaded sources.
2024-03-05 07:44:26 +01:00
Jakub Rusz
95c1b90d33 Schutzbot: Remove RH-IT-Root-CA 2024-03-04 23:53:36 +01:00
Simon Steinbeiss
75bb706cb0 actions/test: Only test assemblers on Python3.6
Assemblers are only part of v1 manifests, so the only Python version we
need to test is 3.6 (RHEL8).
2024-03-04 14:39:50 +01:00
Simon Steinbeiss
3de3bddc10 actions/sonarqube: Drop SonarQube in favor of Snyk 2024-03-04 14:14:03 +01:00
Ondřej Budai
ec096f449b osbuild-mpp: disable zchunks
See the comment.
2024-03-04 14:06:33 +01:00
Michael Vogt
ba732b8532 workflow, osbuild-mpp: run with cache and use GH cache
Share cache between runs and also put into GH cache accross runs.
2024-03-04 14:06:33 +01:00
Michael Vogt
1ea7b4943c workflow: drop python3.11 from matrix (HMS-3697)
With fedora moving to python3.12 we can stop testing on py311.

As a drive-by this commit also documents why we need to test on
py36 and py39.
2024-03-04 13:51:41 +01:00
Michael Vogt
9baca1fe90 stages: add small unit test for the gzip stage 2024-03-04 11:18:13 +01:00
Luke Yang
cfaabe618f stages/org.osbuild.gzip: add compression level option
Allow compression level to be specified instead of defaulting to 1. This is needed for CoreOS Assembler.
2024-03-04 11:18:13 +01:00
Michael Vogt
30f740ec9f stages: add test for coreos.platform generate_console_settings_file
And a tiny tweak to avoid an empty line at the start of the
`boot/grub2/console.cfg`
2024-02-28 10:37:01 +01:00
Michael Vogt
e9c31c035b stages: tweak process_platforms_json to be slightly shorter
With the test the helper can now be slightly simplified. Because
we only have two results it seems easier to just use them directly
than to store them in an intermediate result struct.
2024-02-28 10:37:01 +01:00
Michael Vogt
6cc7309890 stages: add unit test for coreos.platform process_platforms_json
Add unit test for `coreos.platform` process_platforms_json helper.
2024-02-28 10:37:01 +01:00
Michael Vogt
d38665a2af util: tweak bls.options_append() support no/multiple options
The BLS specification [0] says the `options` field is optional and
can also appear multiple times. This commit tweaks the code to
deal with these corner cases and also adds tests that ensure that
this works correctly.

It also tweaks the file handling to be atomic.

[0] https://uapi-group.org/specifications/specs/boot_loader_specification/
2024-02-28 10:37:01 +01:00
schutzbot
87dce84b90 Post release version bump
[skip ci]
2024-02-28 08:13:32 +00:00
Luke Yang
7709b7caeb stages/org.osbuild.tar: add sparse option
Add a sparse option to the tar stage which make files in the archive sparse if they are sparse on disk. This is needed for CoreOS Assember.
2024-02-27 16:46:19 -05:00
Michael Vogt
b0091c0470 inputs: simplify/comment clean of test_containers_local_inputs_integration
Thanks to Ondrej fot the suggestions
2024-02-27 15:07:42 +01:00
Michael Vogt
82f2414637 sources: tweak ContainersStorageSources.exists to return False
When an images does not exist just return `False` instead of
raising a RuntimeError. If anything else goes wrong (unknown
output or hash mismatch) keep the RuntimeError as this is an
unexpected exception.
2024-02-27 15:07:42 +01:00