Add testing on pythong 3.14 and keep testing on python 3.13 for the
current stable Fedora version (42 at this time).
Use osbuild-ci:latest-202506112350 for tests.yml
The main reason is that it comes with license_expression python package
needed for newly added SBOM test cases.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The test_host.py:test_signals_on_separate_fd (and more) runs
itself but that run will happen without the tox env so a pip/tox
installed pytest will not be found, install the pytest package
as a workaround.
This commit adds a tiny smoke test that runs the unit tests as
a normal user on a regular github runner. This should ensure that
we catch issues like in osbuild#1936.
This will bring the latest image builds with new dependencies for unit
tests.
In addition, pin down the osbuild-ci-c*s container to a specific tag, to
ensure stability of the CI.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Instead of hard-coding the Python version that the installed
python3-dnf has been built against on the latest Fedora, read the
value from the osbuild-ci container. The container now has the version
written in /osb/libdnf-python-version.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
'dnf' Python package can't be installed using pip in the tox
environment. In order to test the code which uses it, we need to use the
system version. Our testing environment uses Fedora as the system,
therefore we can reasonably use the system version of 'dnf' only with
Python version which is on Fedora.
Enable site packages in tox for Python 3.12 when testing osbuild
internals.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This will include the latest osbuild-ci container changes from
https://github.com/osbuild/containers/pull/73
This includes a bunch of binaries that were missing and lead
to skpped tests.
In PR#1752 Florian noticed that `make_containers` is buggy [0] in
one specific testcase. However the GH actions did not trigger and
uppon inspection the test is skipped:
```
stages/test/test_container_deploy.py::test_container_deploy_integration SKIPPED [ 7%]
```
This commit adds `-rs` to get a summary why the tests are skipped.
This is useful in general, I suspect the test container might miss
podman but this commit will bring clarity.
[0] https://github.com/osbuild/osbuild/pull/1752#discussion_r1578294358
Run only two jobs in the GH runner for the unittest. The `test_stage.py`
because it takes a very long time and needs to run in parallel and
all the other tests.
This split avoid that we forget to add new unittests to the matrix
as we did before (see e.g. https://github.com/osbuild/osbuild/pull/1731)
and it will also enable the tests in:
- sources/test
- inputs/tests
- mounts/test
to run.
This will reduce the "granularity" of the test output a bit, in the
GH runner we only see two unit test matrix jobs now. However that
should not be too bad because the non-stage tests are really quick
to run.
Using an environ for passing the tests is a bit of a headache when
it comes to quoting which is important when trying to write something
like: `-k "not test_stages.py"`.
I (personally) also find it slightly nicer/more intuitive to be able to do:
```
$ tox -e py36 -- ./test/mod
```
compared to
```
$ TEST_CATEGORY="./test/mod" tox -e py36
```
The other tests have concurrency issues at the moment.
It seems in https://github.com/osbuild/osbuild/pull/1655 we lost
parallel running. The issue seems to be that `env` does not
actually define a shell environment but only a github environment.
This sets commit sets the shell env explicitly to unbreak us
again.
There were special cases in the workflow for only running specific
tests in parallel. However how that the test_assemblers parallel
runs are fixed [0] there is really no need for special cases anymore
and we can just run them all in parallel.
[0] https://github.com/osbuild/osbuild/pull/1641
The `test_assembler.py` hardcods some filesystem and partition
UUIDs. This leads to hard to diagnose test failures when the
test is run in parallel. The btrfs and xfs filesystem drivers
will see the same uuid for multi created images and error sometimes with
someting like:
```
Mar 06 10:22:54 top kernel: BTRFS error: device /dev/loop104 belongs to fsid aff010e9-df95-4f81-be6b-e22317251033, and the fs is already mounted, scanned by mount (123856)
```
Its a race that only happens when two images are checked at the
same time.
This commit fixes the issue by just using a randomized UUID in
the test_assemblers.py. It also re-enables running the test in
parallel (which make it run a lot faster, from 34min to 14min).
Before this commit GitHub Action runs that were triggered by a PR were
not canceled when updates were made to the same PR. This lead to even
more clogging of our pipelines and not enough runners being available.
This changes the behavior in a way that whenever a PR gets updated all
still-in-progress runs get canceled and new runs get spawned.
The default overlayfs doesn't work inside the container runner which
causes the test to fail with
Invalid destination name
containers-storage:localhost/osbuild-skopeo-test-...:
'overlay' is not supported over overlayfs, a mount_program is
required: backing file system is unsupported for this graph driver
Changing the driver to vfs solves the issue.
With pytest 8.0.0 the `-k` option seem to have changed it's
behavior. Drop `-k` therefore.
Tests started to fail recently and it looks like this is because
pytest 8.0.0 changes the semantic of the `-k` option. We used
to pass `-k stages/test` but that seems to no longer work. So
pin pytest to the last good version until this is better
understood.
We've temporarily disabled the merge queue because our tests often
require retries to go all-green and this isn't possible to do on the
queue, meaning that it's close to impossible to get a PR merged.
Run tests on push so that they run in main when a PR is merged.
This reverts commit 63feab7d86.
Run the `test_stages` test in parallel in the github runner. This
test currently takes about 1:30h to 2:30h and running it in parallel
will give us big wins in terms of test time. The time is observed
to go down to 0:30h to 1h.
Note that the other tests are not run in parallel. The reason is
that they fail randomly, it looks like insufficient isolation
between them. Some are easy to fix, e.g.:
721521220b
but it's probably not worth it as the other tests run a lot faster.
Include the new `stages/test` test category to the CI runs as well.
Note that because `stages/__init__.py` and `stages/test/__init__.py`
are missing it is not possible to use the existing style of just
doing `stages.test`. Adding `stages/__init__.py` feels wrong and
the desire is to have the stages tests close to the stages so this
seems the least invasive way.
A simple test to make sure that our btrfs stages work.
Updating the testing container is needed, because the old version
didn't have btrfs-progs in it.
`tox` is a standard testing tool for Python projects, this allows you to
test locally with all your installed Python version with the following
command:
`tox -m test -p all`
To run the tests in parallel for all supported Python versions.
To run linters or type analysis:
```
tox -m lint -p all
tox -m type -p all
```
This commit *also* disables the `import-error` warning from `pylint`,
not all Python versions have the system-installed Python libraries
available and they can't be fetched from PyPI.
Some linters have been added and the general order linters run in has
been changed. This allows for quicker test failure when running
`tox -m lint`. As a consequence the `test_pylint` test has been removed
as it's role can now be fulfilled by `tox`.
Other assorted linter fixes due to newer versions:
- use a str.join method (`consider-using-join`)
- fix various (newer) mypy and pylint issues
- comments starting with `#` and no space due to `autopep8`
This also changes our CI to use the new `tox` setup and on top of that
pins the versions of linters used. This might move into separate
requirements.txt files later on to allow for easier updating of those
dependencies.
2023-08-01 15:01:13 +02:00
Renamed from .github/workflows/tests.yml (Browse further)
