debian-forge/osbuild/util
Michael Vogt 88c35ea306 osbuild: make inputs map() function use fd for reply as well
We recently hit the issue that `osbuild` crashed with:
```
Unable to decode response body "Traceback (most recent call last):
  File \"/usr/bin/osbuild\", line 33, in <module>
    sys.exit(load_entry_point('osbuild==124', 'console_scripts', 'osbuild')())
  File \"/usr/lib/python3.9/site-packages/osbuild/main_cli.py\", line 181, in osbuild_cli
    r = manifest.build(
  File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 477, in build
    res = pl.run(store, monitor, libdir, debug_break, stage_timeout)
  File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 376, in run
    results = self.build_stages(store,
  File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 348, in build_stages
    r = stage.run(tree,
  File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 213, in run
    data = ipmgr.map(ip, store)
  File \"/usr/lib/python3.9/site-packages/osbuild/inputs.py\", line 94, in map
    reply, _ = client.call_with_fds(\"map\", {}, fds)
  File \"/usr/lib/python3.9/site-packages/osbuild/host.py\", line 373, in call_with_fds
    kind, data = self.protocol.decode_message(ret)
  File \"/usr/lib/python3.9/site-packages/osbuild/host.py\", line 83, in decode_message
    raise ProtocolError(\"message empty\")
osbuild.host.ProtocolError: message empty
cannot run osbuild: exit status 1" into osbuild result: invalid character 'T' looking for beginning of value
...
input/packages (org.osbuild.files): Traceback (most recent call last):
input/packages (org.osbuild.files):   File "/usr/lib/osbuild/inputs/org.osbuild.files", line 226, in <module>
input/packages (org.osbuild.files):     main()
input/packages (org.osbuild.files):   File "/usr/lib/osbuild/inputs/org.osbuild.files", line 222, in main
input/packages (org.osbuild.files):     service.main()
input/packages (org.osbuild.files):   File "/usr/lib/python3.11/site-packages/osbuild/host.py", line 250, in main
input/packages (org.osbuild.files):     self.serve()
input/packages (org.osbuild.files):   File "/usr/lib/python3.11/site-packages/osbuild/host.py", line 284, in serve
input/packages (org.osbuild.files):     self.sock.send(reply, fds=reply_fds)
input/packages (org.osbuild.files):   File "/usr/lib/python3.11/site-packages/osbuild/util/jsoncomm.py", line 407, in send
input/packages (org.osbuild.files):     n = self._socket.sendmsg([serialized], cmsg, 0)
input/packages (org.osbuild.files):         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
input/packages (org.osbuild.files): OSError: [Errno 90] Message too long
```

The underlying issue is that the reply of the `map()` call is too
big for the buffer that `jsoncomm` uses. This problem existed before
for the args of map and was fixed by introducing a temporary file
in https://github.com/osbuild/osbuild/pull/1331 (and similarly
before in https://github.com/osbuild/osbuild/pull/824).

This commit writes the return values also into a file. This should
fix the crash above and make the function more symetrical as well.

Alternative/complementary version of
https://github.com/osbuild/osbuild/pull/1833

Closes: HMS-4537
2024-08-13 13:13:24 +02:00
..
__init__.py util: mark as module 2020-04-21 17:00:04 +02:00
bls.py util: tweak bls.options_append() support no/multiple options 2024-02-28 10:37:01 +01:00
checksum.py osbuild: run isort on all files 2022-09-12 13:32:51 +02:00
containers.py util/containers: remount containers store as rw 2024-03-11 18:15:24 +02:00
ctx.py osbuild: run isort on all files 2022-09-12 13:32:51 +02:00
fscache.py fscache: use remove_lru() to reclaim space when the cache is full 2024-02-06 17:16:48 +01:00
jsoncomm.py osbuild: make inputs map() function use fd for reply as well 2024-08-13 13:13:24 +02:00
linux.py osutil: add Libc.futimens() wrapper for futimens() call 2023-12-12 22:57:21 +01:00
lorax.py autopep8: Update with changes to make autopep8 -a -a -a happy 2023-08-10 13:04:14 +02:00
lvm2.py autopep8: Update with changes to make autopep8 -a -a -a happy 2023-08-10 13:04:14 +02:00
mnt.py util/mnt: add a remount paramater 2024-03-11 18:15:24 +02:00
osrelease.py osbuild: run isort on all files 2022-09-12 13:32:51 +02:00
ostree.py lint: sort import(s) 2024-03-05 16:10:27 +01:00
parsing.py util: Rename function 2024-03-25 18:26:53 +01:00
path.py util/path: new clamp mtime function 2022-12-15 13:10:35 +00:00
pe32p.py util: add module to parse PE32+ files 2022-11-14 20:10:59 +01:00
rhsm.py util: fix typo in get_consumer_secrets 2022-12-09 21:46:43 +01:00
rmrf.py ostuild: fix new pylint error with latest pylint version 2023-11-15 18:31:56 +01:00
runners.py Revert "runners: clean up temp files before exiting the runner" 2024-05-20 11:55:24 -07:00
selinux.py osbuild: add support to exclude_paths to setfiles() 2024-03-20 18:05:51 +01:00
term.py osbuild: run isort on all files 2022-09-12 13:32:51 +02:00
types.py osbuild: fix optional-types 2022-07-13 17:31:37 +02:00
udev.py osbuild: run isort on all files 2022-09-12 13:32:51 +02:00