From adf5989de2d8693fa87d97f5042e0c96f8224e6b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ond=C5=99ej=20Budai?= Date: Tue, 1 Oct 2019 08:46:01 +0200 Subject: [PATCH] osbuild/pipeline: Fix crashes when running multiple builds at once Storytime! I tried to run multiple osbuilds at once. It failed when unmounting the buildtree. Weird. It turned out the buildtree was not there anymore when osbuild tried to unmount it. But who unmounted it? We need to deep dive into mount-types. Nowadays, the / directory is shared-mounted by systemd. See: https://serverfault.com/questions/868682/implications-of-mount-make-private This has interesting implications, see the following example: we start osbuild1 with /var/tmp/os1 as its store osbuild1 creates /var/tmp/os1/tmp osbuild1 bind-mounts / onto /var/tmp/os1/tmp we start osbuild2 with /var/tmp/os2 as its store osbuild2 creates /var/tmp/os2/tmp osbuild2 bind-mounts / onto /var/tmp/os2/tmp Now, the shared-mounting goes into effect: The second mount-event gets propagated into the first mount, where it creates another mount, so we get something like this: /var/tmp/os1/tmp/var/tmp/os2/tmp But this is just a start! Imagine running three osbuilds at once. The event would get propagated to those 3 mounts created by two osbuilds, creating 3 extra mounts, 7 in total. It turns out this mounting strategy creates an *exponential number* of mounts. Crazy, right? This commit mounts the root inside build root using private bind, which doesn't propagate bind-events. This solves the problem with the exponential growth. But the original problem was different, mount points were disappearing. So how does this fix solve the problem? Honestly, I don't know. Something with mount-event propagation is probably responsible, but I cannot imagine how it is actually affecting the unbinding. --- osbuild/pipeline.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/osbuild/pipeline.py b/osbuild/pipeline.py index ee6a4a4a..94b96030 100644 --- a/osbuild/pipeline.py +++ b/osbuild/pipeline.py @@ -189,7 +189,7 @@ class Pipeline: yield tree else: with tempfile.TemporaryDirectory(dir=object_store.store) as tmp: - subprocess.run(["mount", "-o", "bind,ro,mode=0755", "/", tmp], check=True) + subprocess.run(["mount", "--make-private", "-o", "bind,ro,mode=0755", "/", tmp], check=True) try: yield tmp finally: