No description
88c35ea306
We recently hit the issue that `osbuild` crashed with:
```
Unable to decode response body "Traceback (most recent call last):
File \"/usr/bin/osbuild\", line 33, in <module>
sys.exit(load_entry_point('osbuild==124', 'console_scripts', 'osbuild')())
File \"/usr/lib/python3.9/site-packages/osbuild/main_cli.py\", line 181, in osbuild_cli
r = manifest.build(
File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 477, in build
res = pl.run(store, monitor, libdir, debug_break, stage_timeout)
File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 376, in run
results = self.build_stages(store,
File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 348, in build_stages
r = stage.run(tree,
File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 213, in run
data = ipmgr.map(ip, store)
File \"/usr/lib/python3.9/site-packages/osbuild/inputs.py\", line 94, in map
reply, _ = client.call_with_fds(\"map\", {}, fds)
File \"/usr/lib/python3.9/site-packages/osbuild/host.py\", line 373, in call_with_fds
kind, data = self.protocol.decode_message(ret)
File \"/usr/lib/python3.9/site-packages/osbuild/host.py\", line 83, in decode_message
raise ProtocolError(\"message empty\")
osbuild.host.ProtocolError: message empty
cannot run osbuild: exit status 1" into osbuild result: invalid character 'T' looking for beginning of value
...
input/packages (org.osbuild.files): Traceback (most recent call last):
input/packages (org.osbuild.files): File "/usr/lib/osbuild/inputs/org.osbuild.files", line 226, in <module>
input/packages (org.osbuild.files): main()
input/packages (org.osbuild.files): File "/usr/lib/osbuild/inputs/org.osbuild.files", line 222, in main
input/packages (org.osbuild.files): service.main()
input/packages (org.osbuild.files): File "/usr/lib/python3.11/site-packages/osbuild/host.py", line 250, in main
input/packages (org.osbuild.files): self.serve()
input/packages (org.osbuild.files): File "/usr/lib/python3.11/site-packages/osbuild/host.py", line 284, in serve
input/packages (org.osbuild.files): self.sock.send(reply, fds=reply_fds)
input/packages (org.osbuild.files): File "/usr/lib/python3.11/site-packages/osbuild/util/jsoncomm.py", line 407, in send
input/packages (org.osbuild.files): n = self._socket.sendmsg([serialized], cmsg, 0)
input/packages (org.osbuild.files): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
input/packages (org.osbuild.files): OSError: [Errno 90] Message too long
```
The underlying issue is that the reply of the `map()` call is too
big for the buffer that `jsoncomm` uses. This problem existed before
for the args of map and was fixed by introducing a temporary file
in https://github.com/osbuild/osbuild/pull/1331 (and similarly
before in https://github.com/osbuild/osbuild/pull/824).
This commit writes the return values also into a file. This should
fix the crash above and make the function more symetrical as well.
Alternative/complementary version of
https://github.com/osbuild/osbuild/pull/1833
Closes: HMS-4537
|
||
|---|---|---|
| .devcontainer | ||
| .github | ||
| assemblers | ||
| data | ||
| devices | ||
| docs | ||
| inputs | ||
| mounts | ||
| osbuild | ||
| runners | ||
| schemas | ||
| schutzbot | ||
| selinux | ||
| sources | ||
| stages | ||
| test | ||
| tools | ||
| .bandit | ||
| .editorconfig | ||
| .git-blame-ignore-revs | ||
| .gitignore | ||
| .gitlab-ci.yml | ||
| .mypy.ini | ||
| .packit.yaml | ||
| .ruff.toml | ||
| LICENSE | ||
| Makefile | ||
| osbuild.spec | ||
| README.md | ||
| requirements.txt | ||
| samples | ||
| Schutzfile | ||
| setup.cfg | ||
| setup.py | ||
| tox.ini | ||
OSBuild
Build-Pipelines for Operating System Artifacts
OSBuild is a pipeline-based build system for operating system artifacts. It defines a universal pipeline description and a build system to execute them, producing artifacts like operating system images, working towards an image build pipeline that is more comprehensible, reproducible, and extendable.
See the osbuild(1) man-page for details on how to run osbuild, the definition
of the pipeline description, and more.
Project
- Website: https://www.osbuild.org
- Bug Tracker: https://github.com/osbuild/osbuild/issues
- Matrix: #image-builder on fedoraproject.org
- Mailing List: image-builder@redhat.com
- Changelog: https://github.com/osbuild/osbuild/releases
Principles
- OSBuild stages are never broken, only deprecated. The same manifest should always produce the same output.
- OSBuild stages should be explicit whenever possible instead of e.g. relying on the state of the tree.
- Pipelines are independent, so the tree is expected to be empty at the beginning of each.
- Manifests are expected to be machine-generated, so OSBuild has no convenience functions to support manually created manifests.
- The build environment is confined against accidental misuse, but this should not be considered a security boundary.
- OSBuild may only use Python language features supported by the oldest target distribution.
Contributing
Please refer to the developer guide to learn about our workflow, code style and more.
Requirements
The requirements for this project are:
bubblewrap >= 0.4.0python >= 3.6
Additionally, the built-in stages require:
bash >= 5.0coreutils >= 8.31curl >= 7.68qemu-img >= 4.2.0rpm >= 4.15tar >= 1.32util-linux >= 235skopeo
At build-time, the following software is required:
python-docutils >= 0.13pkg-config >= 0.29
Testing requires additional software:
pytest
Build
Osbuild is a python script so it is not compiled. To verify changes made to the code use included makefile rules:
make lintto run linter on top of the codemake test-allto run base set of testssudo make test-runto run extended set of tests (takes long time)
Installation
Installing osbuild requires to not only install the osbuild module, but also
additional artifacts such as tools (i.e: osbuild-mpp) sources, stages, schemas
and SELinux policies.
For this reason, doing an installation from source is not trivial and the easier way to install it is to create the set of RPMs that contain all these components.
This can be done with the rpm make target, i.e:
make rpm
A set of RPMs will be created in the ./rpmbuild/RPMS/noarch/ directory and can
be installed in the system using the distribution package manager, i.e:
sudo dnf install ./rpmbuild/RPMS/noarch/*.rpm
Repository
- web: https://github.com/osbuild/osbuild
- https:
https://github.com/osbuild/osbuild.git - ssh:
git@github.com:osbuild/osbuild.git
License
- Apache-2.0
- See LICENSE file for details.