debian-forge/test/mod
Michael Vogt 88c35ea306 osbuild: make inputs map() function use fd for reply as well
We recently hit the issue that `osbuild` crashed with:
```
Unable to decode response body "Traceback (most recent call last):
  File \"/usr/bin/osbuild\", line 33, in <module>
    sys.exit(load_entry_point('osbuild==124', 'console_scripts', 'osbuild')())
  File \"/usr/lib/python3.9/site-packages/osbuild/main_cli.py\", line 181, in osbuild_cli
    r = manifest.build(
  File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 477, in build
    res = pl.run(store, monitor, libdir, debug_break, stage_timeout)
  File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 376, in run
    results = self.build_stages(store,
  File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 348, in build_stages
    r = stage.run(tree,
  File \"/usr/lib/python3.9/site-packages/osbuild/pipeline.py\", line 213, in run
    data = ipmgr.map(ip, store)
  File \"/usr/lib/python3.9/site-packages/osbuild/inputs.py\", line 94, in map
    reply, _ = client.call_with_fds(\"map\", {}, fds)
  File \"/usr/lib/python3.9/site-packages/osbuild/host.py\", line 373, in call_with_fds
    kind, data = self.protocol.decode_message(ret)
  File \"/usr/lib/python3.9/site-packages/osbuild/host.py\", line 83, in decode_message
    raise ProtocolError(\"message empty\")
osbuild.host.ProtocolError: message empty
cannot run osbuild: exit status 1" into osbuild result: invalid character 'T' looking for beginning of value
...
input/packages (org.osbuild.files): Traceback (most recent call last):
input/packages (org.osbuild.files):   File "/usr/lib/osbuild/inputs/org.osbuild.files", line 226, in <module>
input/packages (org.osbuild.files):     main()
input/packages (org.osbuild.files):   File "/usr/lib/osbuild/inputs/org.osbuild.files", line 222, in main
input/packages (org.osbuild.files):     service.main()
input/packages (org.osbuild.files):   File "/usr/lib/python3.11/site-packages/osbuild/host.py", line 250, in main
input/packages (org.osbuild.files):     self.serve()
input/packages (org.osbuild.files):   File "/usr/lib/python3.11/site-packages/osbuild/host.py", line 284, in serve
input/packages (org.osbuild.files):     self.sock.send(reply, fds=reply_fds)
input/packages (org.osbuild.files):   File "/usr/lib/python3.11/site-packages/osbuild/util/jsoncomm.py", line 407, in send
input/packages (org.osbuild.files):     n = self._socket.sendmsg([serialized], cmsg, 0)
input/packages (org.osbuild.files):         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
input/packages (org.osbuild.files): OSError: [Errno 90] Message too long
```

The underlying issue is that the reply of the `map()` call is too
big for the buffer that `jsoncomm` uses. This problem existed before
for the args of map and was fixed by introducing a temporary file
in https://github.com/osbuild/osbuild/pull/1331 (and similarly
before in https://github.com/osbuild/osbuild/pull/824).

This commit writes the return values also into a file. This should
fix the crash above and make the function more symetrical as well.

Alternative/complementary version of
https://github.com/osbuild/osbuild/pull/1833

Closes: HMS-4537
2024-08-13 13:13:24 +02:00
..
__init__.py test: '{. -> ./mod}/test_util_selinux.py' 2020-04-24 15:50:44 +02:00
test_api.py pipeline,api: write metadata directly 2022-12-09 12:03:40 +01:00
test_buildroot.py Test/buildroot: fix checking /var/tmp mode 2024-05-16 09:58:38 +02:00
test_fmt_v1.py test: fix autopep8 issues 2024-01-02 19:31:31 +01:00
test_fmt_v2.py test: add test that checks that partition are added to mount stage 2023-12-22 10:18:29 -05:00
test_host.py host: add support for emitting signals 2022-02-22 10:38:43 +01:00
test_inputs.py osbuild: make inputs map() function use fd for reply as well 2024-08-13 13:13:24 +02:00
test_loop.py Revert "osbuild: ensure loop.Loop() has the required device node" 2024-03-13 22:16:51 +02:00
test_meta.py test: add "functional" test for devices/mounts acceptance 2024-05-03 11:26:22 +02:00
test_mixins.py osbuild: error when {Device,Mount} is modified after creation 2024-01-19 02:54:26 +01:00
test_monitor.py main,monitor: fix total steps in progress reporting 2024-07-31 23:00:33 +02:00
test_mounts.py test: fix isort issues 2024-01-02 19:31:31 +01:00
test_objectstore.py test: fix all pylint issues 2024-01-02 19:31:31 +01:00
test_osbuild.py test/osbuild: use proper object in stage run test 2022-12-09 12:03:40 +01:00
test_testutil_fake_tree.py testutil: extrace new make_fake_tree() helper 2024-01-03 11:25:48 -08:00
test_testutil_imports.py test: fix isort issues 2024-01-02 19:31:31 +01:00
test_testutil_jsonschema.py test: add new testutil.assert_jsonschema_error_contains() helper 2024-01-23 12:00:59 +01:00
test_testutil_make_container.py testutil: fix make_container() cleanup 2024-04-25 21:59:40 +02:00
test_testutil_mock_command.py testutil: tweak mock_command to write a call_log 2024-03-18 20:37:21 +01:00
test_testutil_net.py test: add new https_serve_directory() and test certs 2024-07-29 16:51:48 +02:00
test_util_bls.py util: tweak bls.options_append() support no/multiple options 2024-02-28 10:37:01 +01:00
test_util_checksum.py osbuild: run isort on all files 2022-09-12 13:32:51 +02:00
test_util_ctx.py util/ctx: extract suppress_oserror() 2020-05-11 18:05:12 +02:00
test_util_fscache.py fscache: use remove_lru() to reclaim space when the cache is full 2024-02-06 17:16:48 +01:00
test_util_fscache_coherency.py test: fix autopep8 issues 2024-01-02 19:31:31 +01:00
test_util_jsoncomm.py jsoncom: gracefully report EMSGSIZE errors 2024-08-13 09:38:59 +02:00
test_util_linux.py test: fix test_libc_futimes_works 2024-01-18 08:44:25 +01:00
test_util_lorax.py osbuild: run isort on all files 2022-09-12 13:32:51 +02:00
test_util_lvm2.py autopep8: Update with changes to make autopep8 -a -a -a happy 2023-08-10 13:04:14 +02:00
test_util_mnt.py test: tweak test_osbuild_mount_failure_msg() for tmt/fc40 2024-05-23 14:56:41 +02:00
test_util_osrelease.py test: make TestBase inherit unittest.TestCase 2020-06-05 09:27:40 +02:00
test_util_ostree.py ostree: add convenience function for using default OSTree deployment 2024-02-07 18:50:38 -05:00
test_util_parsing.py stages,test: fix lint errors and add basic unit tests 2024-03-25 18:26:53 +01:00
test_util_path.py test_clamp_mtime: avoid clashing with chrony 2024-04-27 00:00:10 +02:00
test_util_pe32p.py util: add module to parse PE32+ files 2022-11-14 20:10:59 +01:00
test_util_rhsm.py global: fix PEP-8 formatting 2022-08-05 09:41:05 +02:00
test_util_rmrf.py test: convert to shared helpers 2020-05-13 14:26:05 +02:00
test_util_selinux.py osbuild: add support to exclude_paths to setfiles() 2024-03-20 18:05:51 +01:00
test_util_udev.py devices: add custom udev rule inhibitor mechanism 2021-12-09 00:44:21 +00:00