linux: add accessor for fcntl file locking ops

This adds a new accessor-function for the file-locking operations
through `fcntl(2)`. In particular, it adds the new function
`fcntl_flock()`, which wraps the `F_OFD_SETLK` command on `fcntl(2)`.

There were a few design considerations:

  * The name `fcntl_flock` comes from the `struct flock` structure that
    is the argument type of all file-locking syscalls. Furthermore, it
    mirrors what the `fcntl` module already provides as a wrapper for
    the classic file-locking syscall.

  * The wrapper only exposes very limited access to the file-locking
    commands. There already is `fcntl.fcntl()` and `fcntl.fcntl_flock()`
    in the standard library, which expose the classic file-locks.
    However, those are implemented in C, which gives much more freedom
    and access to architecture dependent types and functions.
    We do not have that freedom (see the in-code comments for the
    things to consider when exposing more fcntl-locking features).
    Hence, this only exposes a very limited set of functionality,
    exactly the parts we need in the objectstore rework.

  * We cannot use `fcntl.fcntl_flock()` from the standard library,
    because we really want the `OFD` version. OFD stands for
    `open-file-description`. These locks were introduced in 2014 to the
    linux kernel and mirror what the non-OFD locks do, but bind the
    locks to the file-description, rather than to a process. Therefore,
    closing a file-description will release all held locks on that
    file-description.
    This is so much more convenient to work with, and much less
    error-prone than the old-style locks. Hence, we really want these,
    even if it means that we have to introduce this new helper.

  * There is an open bug to add this to the python standard library:

        https://bugs.python.org/issue22367

    This is unresolved since 2014.

The implementation of the `fcntl_flock()` helper is straighforward and
should be easy to understand. However, the reasoning behind the design
decisions are not. Hence, the code contains a rather elaborate comment
explaining why it is done this way.

Lastly, this adds a small, but I think sufficient unit-test suite which
makes sure the API works as expected. It does not test for full
functionality of the underlying locking features, but that is not the
job of a wrapping layer, I think. But more tests can always be added.

osbuild/util/linux.py

@@ -18,9 +18,11 @@ import ctypes.util
 import fcntl
 import os
 import platform
+import struct
 import threading
 
 __all__ = [
+    "fcntl_flock",
     "ioctl_get_immutable",
     "ioctl_toggle_immutable",
 ]
@@ -274,3 +276,142 @@ def cap_mask_to_set(mask: int) -> set:
     }
 
     return res
+
+
+def fcntl_flock(fd: int, lock_type: int, wait: bool = False):
+    """Perform File-locking Operation
+
+    This function performs a linux file-locking operation on the specified
+    file-descriptor. The specific type of lock must be specified by the caller.
+    This function does not allow to specify the byte-range of the file to lock.
+    Instead, it always applies the lock operations to the entire file.
+
+    For system-level documentation, see the `fcntl(2)` man-page, especially the
+    section about `struct flock` and the locking commands.
+
+    This function always uses the open-file-description locks provided by
+    modern linux kernels. This means, locks are tied to the
+    open-file-description. That is, they are shared between duplicated
+    file-descriptors. Furthermore, acquiring a lock while already holding a
+    lock will update the lock to the new specified lock type, rather than
+    acquiring a new lock.
+
+    If `wait` is `False` a non-blocking operation is performed. In case the lock
+    is contested a `BlockingIOError` is raised by the python standard library.
+    If `Wait` is `True`, the kernel will suspend execution until the lock is
+    acquired.
+
+    If a synchronous exception is raised, the operation will be canceled and the
+    exception is forwarded.
+
+    Parameters
+    ----------
+    fd
+        The file-descriptor to use for the locking operation.
+    lock_type
+        The type of lock to use. This can be one of: `fcntl.F_RDLCK`,
+        `fcntl.F_WRLCK`, `fcntl.F_UNLCK`.
+    wait
+        Whether to suspend execution until the lock is acquired in case of
+        contested locks.
+
+    Raises
+    ------
+    OSError
+        If the underlying `fcntl(2)` syscall fails, a matching `OSError` is
+        raised. In particular, `BlockingIOError` signals contested locks. The
+        POSIX error code is `EAGAIN`.
+    """
+
+    valid_types = [fcntl.F_RDLCK, fcntl.F_WRLCK, fcntl.F_UNLCK]
+    if lock_type not in valid_types:
+        raise ValueError("Unknown lock type")
+    if not isinstance(fd, int):
+        raise ValueError("File-descriptor is not an integer")
+    if fd < 0:
+        raise ValueError("File-descriptor is negative")
+
+    #
+    # The `OFD` constants are not available through the `fcntl` module, so we
+    # need to use their integer representations directly. They are the same
+    # across all linux ABIs:
+    #
+    #     F_OFD_GETLK = 36
+    #     F_OFD_SETLK = 37
+    #     F_OFD_SETLKW = 38
+    #
+
+    if wait:
+        lock_cmd = 38
+    else:
+        lock_cmd = 37
+
+    #
+    # We use the linux open-file-descriptor (OFD) version of the POSIX file
+    # locking operations. They attach locks to an open file description, rather
+    # than to a process. They have clear, useful semantics.
+    # This means, we need to use the `fcntl(2)` operation with `struct flock`,
+    # which is rather unfortunate, since it varies depending on compiler
+    # arguments used for the python library, as well as depends on the host
+    # architecture, etc.
+    #
+    # The structure layout of the locking argument is:
+    #
+    #     struct flock {
+    #         short int l_type;
+    #         short int l_whence;
+    #         off_t l_start;
+    #         off_t l_len;
+    #         pid_t int l_pid;
+    #     }
+    #
+    # The possible options for `l_whence` are `SEEK_SET`, `SEEK_CUR`, and
+    # `SEEK_END`. All are provided by the `fcntl` module. Same for the possible
+    # options for `l_type`, which are `L_RDLCK`, `L_WRLCK`, and `L_UNLCK`.
+    #
+    # Depending on which architecture you run on, but also depending on whether
+    # large-file mode was enabled to compile the python library, the values of
+    # the constants as well as the sizes of `off_t` can change. What we know is
+    # that `short int` is always 16-bit on linux, and we know that `fcntl(2)`
+    # does not take a `size` parameter. Therefore, the kernel will just fetch
+    # the structure from user-space with the correct size. The python wrapper
+    # `fcntl.fcntl()` always uses a 1024-bytes buffer and thus we can just pad
+    # our argument with trailing zeros to provide a valid argument to the
+    # kernel. Note that your libc might also do automatic translation to
+    # `fcntl64(2)` and `struct flock64` (if you run on 32bit machines with
+    # large-file support enabled). Also, random architectures change trailing
+    # padding of the structure (MIPS-ABI32 adds 128-byte trailing padding,
+    # SPARC adds 16?).
+    #
+    # To avoid all this mess, we use the fact that we only care for `l_type`.
+    # Everything else is always set to 0 in all our needed locking calls.
+    # Therefore, we simply use the largest possible `struct flock` for your
+    # libc and set everything to 0. The `l_type` field is guaranteed to be
+    # 16-bit, so it will have the correct offset, alignment, and endianness
+    # without us doing anything. Downside of all this is that all our locks
+    # always affect the entire file. However, we do not need locks for specific
+    # sub-regions of a file, so we should be fine. Eventually, what we end up
+    # with passing to libc is:
+    #
+    #     struct flock {
+    #         uint16_t l_type;
+    #         uint16_t l_whence;
+    #         uint32_t pad0;
+    #         uint64_t pad1;
+    #         uint64_t pad2;
+    #         uint32_t pad3;
+    #         uint32_t pad4;
+    #     }
+    #
+
+    type_flock64 = struct.Struct('=HHIQQII')
+    arg_flock64 = type_flock64.pack(lock_type, 0, 0, 0, 0, 0, 0)
+
+    #
+    # Since python-3.5 (PEP475) the standard library guards around `EINTR` and
+    # automatically retries the operation. Hence, there is no need to retry
+    # waiting calls. If a python signal handler raises an exception, the
+    # operation is not retried and the exception is forwarded.
+    #
+
+    fcntl.fcntl(fd, lock_cmd, arg_flock64)