Commit graph

20 commits

Author SHA1 Message Date
Ondřej Budai
af0e849081 sources/curl: Use our own User-Agent
Currently, osbuild downloads are identified as coming from `curl`. This
is unfortunate because some RPM mirrors block requests from curl. Let's
"fix" that by introducing our own user-agent. While this can certainly
be seen as "circumventing" a policy, I think that this change is
actually helpful: Now, the mirror maintainers can actually distinguish
osbuild requests from regular curl calls. If they want to block osbuild,
they certainly can, we have no power there, but at least this allows
more fine-grained filtering. Also, our new user-agent contains our
domain name, so if there's a problem, they can contact us.
2024-04-30 03:10:44 +02:00
Michael Vogt
2586a748fd testutil: skip tests for missing ThreadingHTTPServer in py36
Only py3.7+ has ThreadingHTTPServer and SimpleHTTPRequestHandler
that can take a directory argument. We could reimplement this
on py36 (easy for threading, harder for missing directory) but
instead this commit just skips tests that try to use a
ThreadingHTTPServer.

Remove once we no longer support py3.6.
2024-04-16 15:16:49 +02:00
Michael Vogt
dbe7039674 sources(curl): tweak tests to use monkeypatch.setenv()
Using pytests support for changing setenv() in tests makes things
a little bit more concise.
2024-04-10 16:13:12 -07:00
Michael Vogt
34ad069757 sources(curl): tweak tests to use monkeypatch.setenv()
Using pytests support for changing setenv() in tests makes things
a little bit more concise.
2024-04-09 03:03:38 +02:00
Michael Vogt
b9b296a7e5 testutil: add AtomicCounter() as a threadsafe counter
The existing code in the reqs counting is not really thread safe,
this commit fixes that.
2024-04-09 03:02:45 +02:00
Michael Vogt
98f5904181 source: add curl test in preparation for #1573
When moving to parallel downloads in curl we will need more
comprehensive tests. This commits starts with building some
infrastructure for this.
2024-04-05 16:42:07 +02:00
Michael Vogt
79d788ac23 tests: use tmp_path fixture in test_curl_source.py
Tiny tweak to remove some boilerplate related to tmpfile handling.
The pytest `tmp_path` fixture gives us the tmpdir without having
to worry about cleanup etc (and in a slightly more concise way).
2024-04-03 15:06:07 +02:00
Michael Vogt
fb701d6db5 sources: simplify test_curl_source_amend_* tests a little bit
The `amend_secrets()` does not work with real files so there is
no need to mock cachedirs or create fake input files. This commit
just removes those.

It also changes the checksum to `"1"*64` to make it very clear
that the checksum has no significance in this test.
2024-04-03 15:06:07 +02:00
Michael Vogt
fe05b3084b sources: add regression test for issue##1693
There was a regression with the secrets adding of rhsm for the
curl source. This was my mistake (sorry!). Here is a regression
test that would have prevented this (if we have had it earlier).
2024-04-03 13:55:00 +02:00
Michael Vogt
1d4f2dc53b testutil: extract find_one_subclass_in_module() helper
A small refactor to avoid shipping this duplicated code (this
one is easy to extract/reuse).
2024-04-03 11:36:01 +02:00
Michael Vogt
79360b529a sources: add new sources_service fixture
Similar to the previous commit to include a `inputs_service` fixture
this does the same for `source.SourcesService` imports.

Note that we cannot easily share the helpers so we have to life with
a bit of very similar but duplicated code. To fix this we would have
to have a shared confftest.py that pytest can find. Which would mean
that we need to put the tests under a common dir that is reachable
via __init__.py files (which we currently not have because stages,
inputs etc do not have a __init__.py so python does not considers
them modules).
2024-04-03 11:36:01 +02:00
Michael Vogt
5f31ccf9f2 test: add/use new testutil.make_fake_service_fd()
All inputs/sources tests need a fake service fd to instanciate
their services. Consolidate the creation in a single helper.
2024-04-03 11:36:01 +02:00
Michael Vogt
352bf5cd52 curl: rename "transform" to "amend_secrets"
The curl source is the only source left that uses "transform". And
here the name is very generic but in fact we only do a single thing:
we add secrets for subscriptions for for mtls to the download.

So rename to make it clear what this is all about.
2024-03-19 14:21:57 +01:00
Michael Vogt
fd0167f130 test: return container_id in make_container
The current `make_container()` helper is a bit silly (which is
entirely my fault). It requires a container tag as input but all
tests end up creating a random number for this input. So instead
just remove the input and return the container_id from the podman
build in the contextmanager and use that.
2024-03-18 20:36:19 +01:00
Sanne Raymaekers
29159189f1 sources/curl: add org.osbuild.mtls secrets support
If `org.osbuild.mtls` is passed as a secret name, look for the mtls data
in the environment.
2024-03-11 11:09:37 +01:00
Michael Vogt
82f2414637 sources: tweak ContainersStorageSources.exists to return False
When an images does not exist just return `False` instead of
raising a RuntimeError. If anything else goes wrong (unknown
output or hash mismatch) keep the RuntimeError as this is an
unexpected exception.
2024-02-27 15:07:42 +01:00
Michael Vogt
5ab0b41456 sources: add test for non-existing id in containers-storage 2024-02-27 15:07:42 +01:00
Achilleas Koutsou
591593ea00 testutil: make_container context manager
Make make_container a context manager so we can reliably clean up
containers that were created in tests.
2024-02-21 17:55:37 +01:00
Michael Vogt
d6cd4b93ba test: add minimal test for ContainersStorageSource.from_args() 2024-02-21 17:55:37 +01:00
Michael Vogt
119172e8dd test: add sources_module fixture for sources unit tests
Similar to the `stage_module` fixture for stage tests this adds
a fixture to test sources modules of osbuild.

The code from `stage_module` and `sources_module` is similar and
could be combined but pytest makes it hard to do this without
having a shared root dir. Given that it's just four lines it
seems easier to just life with the tiny bit of code duplication.
2024-02-21 17:55:37 +01:00