Commit graph

13 commits

Author SHA1 Message Date
Ondřej Budai
af0e849081 sources/curl: Use our own User-Agent
Currently, osbuild downloads are identified as coming from `curl`. This
is unfortunate because some RPM mirrors block requests from curl. Let's
"fix" that by introducing our own user-agent. While this can certainly
be seen as "circumventing" a policy, I think that this change is
actually helpful: Now, the mirror maintainers can actually distinguish
osbuild requests from regular curl calls. If they want to block osbuild,
they certainly can, we have no power there, but at least this allows
more fine-grained filtering. Also, our new user-agent contains our
domain name, so if there's a problem, they can contact us.
2024-04-30 03:10:44 +02:00
Michael Vogt
2586a748fd testutil: skip tests for missing ThreadingHTTPServer in py36
Only py3.7+ has ThreadingHTTPServer and SimpleHTTPRequestHandler
that can take a directory argument. We could reimplement this
on py36 (easy for threading, harder for missing directory) but
instead this commit just skips tests that try to use a
ThreadingHTTPServer.

Remove once we no longer support py3.6.
2024-04-16 15:16:49 +02:00
Michael Vogt
dbe7039674 sources(curl): tweak tests to use monkeypatch.setenv()
Using pytests support for changing setenv() in tests makes things
a little bit more concise.
2024-04-10 16:13:12 -07:00
Michael Vogt
34ad069757 sources(curl): tweak tests to use monkeypatch.setenv()
Using pytests support for changing setenv() in tests makes things
a little bit more concise.
2024-04-09 03:03:38 +02:00
Michael Vogt
b9b296a7e5 testutil: add AtomicCounter() as a threadsafe counter
The existing code in the reqs counting is not really thread safe,
this commit fixes that.
2024-04-09 03:02:45 +02:00
Michael Vogt
98f5904181 source: add curl test in preparation for #1573
When moving to parallel downloads in curl we will need more
comprehensive tests. This commits starts with building some
infrastructure for this.
2024-04-05 16:42:07 +02:00
Michael Vogt
79d788ac23 tests: use tmp_path fixture in test_curl_source.py
Tiny tweak to remove some boilerplate related to tmpfile handling.
The pytest `tmp_path` fixture gives us the tmpdir without having
to worry about cleanup etc (and in a slightly more concise way).
2024-04-03 15:06:07 +02:00
Michael Vogt
fb701d6db5 sources: simplify test_curl_source_amend_* tests a little bit
The `amend_secrets()` does not work with real files so there is
no need to mock cachedirs or create fake input files. This commit
just removes those.

It also changes the checksum to `"1"*64` to make it very clear
that the checksum has no significance in this test.
2024-04-03 15:06:07 +02:00
Michael Vogt
fe05b3084b sources: add regression test for issue##1693
There was a regression with the secrets adding of rhsm for the
curl source. This was my mistake (sorry!). Here is a regression
test that would have prevented this (if we have had it earlier).
2024-04-03 13:55:00 +02:00
Michael Vogt
79360b529a sources: add new sources_service fixture
Similar to the previous commit to include a `inputs_service` fixture
this does the same for `source.SourcesService` imports.

Note that we cannot easily share the helpers so we have to life with
a bit of very similar but duplicated code. To fix this we would have
to have a shared confftest.py that pytest can find. Which would mean
that we need to put the tests under a common dir that is reachable
via __init__.py files (which we currently not have because stages,
inputs etc do not have a __init__.py so python does not considers
them modules).
2024-04-03 11:36:01 +02:00
Michael Vogt
5f31ccf9f2 test: add/use new testutil.make_fake_service_fd()
All inputs/sources tests need a fake service fd to instanciate
their services. Consolidate the creation in a single helper.
2024-04-03 11:36:01 +02:00
Michael Vogt
352bf5cd52 curl: rename "transform" to "amend_secrets"
The curl source is the only source left that uses "transform". And
here the name is very generic but in fact we only do a single thing:
we add secrets for subscriptions for for mtls to the download.

So rename to make it clear what this is all about.
2024-03-19 14:21:57 +01:00
Sanne Raymaekers
29159189f1 sources/curl: add org.osbuild.mtls secrets support
If `org.osbuild.mtls` is passed as a secret name, look for the mtls data
in the environment.
2024-03-11 11:09:37 +01:00