No description
2bdb287731
This changes the sources module to explicitly cleanup event-loops.
Additionally, the implementation is protected against re-entrency which
we do not support (and do not need).
We did occasionally get the following exception when running
source-servers:
/usr/lib/python3.8/asyncio/base_events.py:654: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
_warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Exception ignored in: <function BaseEventLoop.__del__ at 0x7f92589d14c0>
Traceback (most recent call last):
File "/usr/lib/python3.8/asyncio/base_events.py", line 656, in __del__
self.close()
File "/usr/lib/python3.8/asyncio/unix_events.py", line 58, in close
super().close()
File "/usr/lib/python3.8/asyncio/selector_events.py", line 92, in close
self._close_self_pipe()
File "/usr/lib/python3.8/asyncio/selector_events.py", line 99, in _close_self_pipe
self._remove_reader(self._ssock.fileno())
File "/usr/lib/python3.8/asyncio/selector_events.py", line 274, in _remove_reader
key = self._selector.get_key(fd)
File "/usr/lib/python3.8/selectors.py", line 190, in get_key
return mapping[fileobj]
File "/usr/lib/python3.8/selectors.py", line 71, in __getitem__
fd = self._selector._fileobj_lookup(fileobj)
File "/usr/lib/python3.8/selectors.py", line 225, in _fileobj_lookup
return _fileobj_to_fd(fileobj)
File "/usr/lib/python3.8/selectors.py", line 42, in _fileobj_to_fd
raise ValueError("Invalid file descriptor: {}".format(fd))
ValueError: Invalid file descriptor: -1
This is triggered when an event-loop is not closed explicitly via
`event_loop.close()`. It then tries to cleanup explicitly. The problem
here is that python has no knowledge of in which order it should
collect GC'ed objects. This might end up more or less random. Therefore,
file-descriptors might be closed in arbitrary order, leading to the
event-loop being unable to unregister its internal objects.
I am not entirely sure whether this is the case here. However, the error
definitely triggers on the internal event-loop socketpair, which there
is no other external access to. Furthermore, this socketpair is only set
to -1 in its own __del__ function. So unless we have a memory
corruption, I see nothing else that could trigger this.
With this fix in place, I can run `test_sources.py` in a loop without
triggering the bug.
It is quite likely that our other `*Server` classes need the same fix. I
did not verify, yet.
|
||
|---|---|---|
| .github/workflows | ||
| assemblers | ||
| docs | ||
| jenkins | ||
| osbuild | ||
| runners | ||
| samples | ||
| schemas | ||
| sources | ||
| stages | ||
| test | ||
| .editorconfig | ||
| .gitignore | ||
| .pylintrc | ||
| .travis.yml | ||
| LICENSE | ||
| Makefile | ||
| NEWS.md | ||
| osbuild.spec | ||
| README.md | ||
| setup.py | ||
| tree-diff | ||
OSBuild
Build-Pipelines for Operating System Artifacts
OSBuild is a pipeline-based build system for operating system artifacts. It defines a universal pipeline description and a build system to execute them, producing artifacts like operating system images, working towards an image build pipeline that is more comprehensible, reproducible, and extendable.
See the osbuild(1) man-page for details on how to run osbuild, the definition
of the pipeline description, and more.
Project
- Website: https://www.osbuild.org
- Bug Tracker: https://github.com/osbuild/osbuild/issues
Requirements
The requirements for this project are:
python >= 3.7systemd-nspawn >= 244
Additionally, the built-in stages require:
bash >= 5.0coreutils >= 8.31curl >= 7.68qemu-img >= 4.2.0rpm >= 4.15tar >= 1.32util-linux >= 235
At build-time, the following software is required:
python-docutils >= 0.13pkg-config >= 0.29
Build
The standard python package system is used. Consult upstream documentation for detailed help. In most situations the following commands are sufficient to build and install from source:
python setup.py build
python setup.py install --skip-build --root=/
The man-pages require python-docutils and can be built via:
rst2man docs/<input-file>.rst <output-file>
Repository:
- web: https://github.com/osbuild/osbuild
- https:
https://github.com/osbuild/osbuild.git - ssh:
git@github.com:osbuild/osbuild.git
License:
- Apache-2.0
- See LICENSE file for details.