particle-os/debian-forge
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

linux: add accessor for fcntl file locking ops

Browse source 
This adds a new accessor-function for the file-locking operations
through `fcntl(2)`. In particular, it adds the new function
`fcntl_flock()`, which wraps the `F_OFD_SETLK` command on `fcntl(2)`.

There were a few design considerations:

  * The name `fcntl_flock` comes from the `struct flock` structure that
    is the argument type of all file-locking syscalls. Furthermore, it
    mirrors what the `fcntl` module already provides as a wrapper for
    the classic file-locking syscall.

  * The wrapper only exposes very limited access to the file-locking
    commands. There already is `fcntl.fcntl()` and `fcntl.fcntl_flock()`
    in the standard library, which expose the classic file-locks.
    However, those are implemented in C, which gives much more freedom
    and access to architecture dependent types and functions.
    We do not have that freedom (see the in-code comments for the
    things to consider when exposing more fcntl-locking features).
    Hence, this only exposes a very limited set of functionality,
    exactly the parts we need in the objectstore rework.

  * We cannot use `fcntl.fcntl_flock()` from the standard library,
    because we really want the `OFD` version. OFD stands for
    `open-file-description`. These locks were introduced in 2014 to the
    linux kernel and mirror what the non-OFD locks do, but bind the
    locks to the file-description, rather than to a process. Therefore,
    closing a file-description will release all held locks on that
    file-description.
    This is so much more convenient to work with, and much less
    error-prone than the old-style locks. Hence, we really want these,
    even if it means that we have to introduce this new helper.

  * There is an open bug to add this to the python standard library:

        https://bugs.python.org/issue22367

    This is unresolved since 2014.

The implementation of the `fcntl_flock()` helper is straighforward and
should be easy to understand. However, the reasoning behind the design
decisions are not. Hence, the code contains a rather elaborate comment
explaining why it is done this way.

Lastly, this adds a small, but I think sufficient unit-test suite which
makes sure the API works as expected. It does not test for full
functionality of the underlying locking features, but that is not the
job of a wrapping layer, I think. But more tests can always be added.
This commit is contained in:
David Rheinsberg 2020-03-26 18:36:14 +01:00 • committed by Christian Kellner
parent 41851f7762
commit aefaf21411
2 changed files with 211 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

70
test/mod/test_util_linux.py
View file

@ -95,3 +95,73 @@ def test_capabilities():
assert not linux.cap_is_supported("CAP_GICMO")
with pytest.raises(OSError):
lib.from_name("CAP_GICMO")
def test_fcntl_flock():
#
# This tests the `linux.fcntl_flock()` file-locking helper. Note
# that file-locks are on the open-file-description, so they are shared
# between dupped file-descriptors. We explicitly create a separate
# file-description via `/proc/self/fd/`.
#
with tempfile.TemporaryFile() as f:
fd1 = f.fileno()
fd2 = os.open(os.path.join("/proc/self/fd/", str(fd1)), os.O_RDWR | os.O_CLOEXEC)
# Test: unlock
linux.fcntl_flock(fd1, linux.fcntl.F_UNLCK)
# Test: write-lock + unlock
linux.fcntl_flock(fd1, linux.fcntl.F_WRLCK)
linux.fcntl_flock(fd1, linux.fcntl.F_UNLCK)
# Test: read-lock1 + read-lock2 + unlock1 + unlock2
linux.fcntl_flock(fd1, linux.fcntl.F_RDLCK)
linux.fcntl_flock(fd2, linux.fcntl.F_RDLCK)
linux.fcntl_flock(fd1, linux.fcntl.F_UNLCK)
linux.fcntl_flock(fd2, linux.fcntl.F_UNLCK)
# Test: write-lock1 + write-lock2 + unlock
linux.fcntl_flock(fd1, linux.fcntl.F_WRLCK)
with pytest.raises(BlockingIOError):
linux.fcntl_flock(fd2, linux.fcntl.F_WRLCK)
linux.fcntl_flock(fd1, linux.fcntl.F_UNLCK)
# Test: write-lock1 + read-lock2 + unlock
linux.fcntl_flock(fd1, linux.fcntl.F_WRLCK)
with pytest.raises(BlockingIOError):
linux.fcntl_flock(fd2, linux.fcntl.F_RDLCK)
linux.fcntl_flock(fd1, linux.fcntl.F_UNLCK)
# Test: read-lock1 + write-lock2 + unlock
linux.fcntl_flock(fd1, linux.fcntl.F_RDLCK)
with pytest.raises(BlockingIOError):
linux.fcntl_flock(fd2, linux.fcntl.F_WRLCK)
linux.fcntl_flock(fd1, linux.fcntl.F_UNLCK)
# Test: write-lock1 + read-lock1 + read-lock2 + unlock
linux.fcntl_flock(fd1, linux.fcntl.F_WRLCK)
linux.fcntl_flock(fd1, linux.fcntl.F_RDLCK)
linux.fcntl_flock(fd2, linux.fcntl.F_RDLCK)
linux.fcntl_flock(fd1, linux.fcntl.F_UNLCK)
# Test: read-lock1 + read-lock2 + write-lock1 + unlock1 + unlock2
linux.fcntl_flock(fd1, linux.fcntl.F_RDLCK)
linux.fcntl_flock(fd2, linux.fcntl.F_RDLCK)
with pytest.raises(BlockingIOError):
linux.fcntl_flock(fd1, linux.fcntl.F_WRLCK)
linux.fcntl_flock(fd1, linux.fcntl.F_UNLCK)
linux.fcntl_flock(fd2, linux.fcntl.F_UNLCK)
# Test: write-lock3 + write-lock1 + close3 + write-lock1 + unlock1
fd3 = os.open(os.path.join("/proc/self/fd/", str(fd1)), os.O_RDWR | os.O_CLOEXEC)
linux.fcntl_flock(fd3, linux.fcntl.F_WRLCK)
with pytest.raises(BlockingIOError):
linux.fcntl_flock(fd1, linux.fcntl.F_WRLCK)
os.close(fd3)
linux.fcntl_flock(fd1, linux.fcntl.F_WRLCK)
linux.fcntl_flock(fd1, linux.fcntl.F_UNLCK)
# Cleanup
os.close(fd2)