sources(curl): error if curl exists 0 but there are downloads left
As part of the investigation of the CI failure in https://github.com/osbuild/osbuild-composer/pull/4247 we noticed that curl can return a return_code of `0` even when it did not downloaded all the urls in a `--config` provided file. This seems to be curl version dependent, I had a hard time writing a test-case with the real curl (8.6.0) that reproduces this so I went with mocking it. We definietly saw this failure with the centos 9 version (7.76). Our current code is buggy and assumes that the exit status of curl is always non-zero if any download fails but that is only the case when `--fail-early` is used. The extra paranoia will not hurt even when relying on the exit code of curl is fixed.
This commit is contained in:
parent
2b2fec85b2
commit
16667ef260
2 changed files with 17 additions and 0 deletions
|
|
@ -317,6 +317,9 @@ class CurlSource(sources.SourceService):
|
|||
failed_urls = ",".join([itm[1]["url"] for itm in dl_pairs])
|
||||
raise RuntimeError(f"curl: error downloading {failed_urls}: error code {return_code}")
|
||||
|
||||
if len(dl_pairs) > 0:
|
||||
raise RuntimeError(f"curl: finished with return_code {return_code} but {dl_pairs} left to download")
|
||||
|
||||
def _fetch_all_old_curl(self, amended):
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers) as executor:
|
||||
for _ in executor.map(self.fetch_one, *zip(*amended)):
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue