Provide a way to read the current contents of the object, in a way
the follows the copy-on-write semantics: If `base` is set but the
object has not yet been written to, the `base` content will be
exposed. If no base is set or the object has been written to, the
current (temporary) tree will be exposed. In either way it is done
via a bind mount so it is assured that the contents indeed can only
be read from, but not written to.
The code also currently make sure that there is no write operation
started as long as there is at least one reader.
Additionally, also introduce checks that the object is intact, i.e.
not cleaned up, for all operations that require such a state.
Analogous to `_path`, it is not possible to identify the intended
mode of the i/o operation from using `open` (whether it is a read
or a write operation) and thus make it an internal method and only
use it for read operations.
Since it is hard to infer the intended modus of the i/o operation,
i.e. whether it is going to be a read or a write from accessing the
`path` property make it an internal method. Do not initialize the
method on property access but return the writable tree, if Object
is initialized, the path to its base tree otherwise.
Adapt all the usage internally: Use `path` for read operations and
initialize the object and then directly use `_tree` for write ops.
As a result of the previous commits that implement copy on write
semantics, `commit` can now be used to create snapshots. Whenever
an Object is committed, its tree is moved to the store and it is
being reset, i.e. a new clean workdir is created and the old one
discarded. The moved tree is then set as the base of the reset
Object. On the next call to `write` the moved tree will be copied
over and forms the basis of the Object again. Should nobody want
to write to Object after the snapshot, i.e. the `commit`, no copy
will be made.
NB: snapshots/commits will act now act as synchronization points:
if a object with the same treesum, i.e. the very same content
already exists, the move (i.e. `store_tree`) will gracefully fail
and the existing content will be set as the base for Object.
Since Object knows its base now, the initialization of the tree
with the content of its base can be delayed until the moment
someone wants to actually modify the tree, thus implementing
copy on write semantics. For this a new `write` method is added
that will initialize the base and return the writable tree. It
should be used instead of `path` whenever the a client wants to
write to the tree of the Object.
Adapt the pipeline and the tests to use the new `write` method
in all the appropriate places.
NB: since the intention can not be inferred when using `path`
directly, the Object is still being initialized there.
When a new Object is created it can have a `base`, i.e. another
object that is already committed to the store, which is then used
to initialize the tree of the new object. That is, the contents
of the new Object will be based on the contents of the existing.
The initialization of an Object with its base (if any) was done
by the ObjectStore. Move all of that logic inside `Object`:
The Object will store its base, which `Object.init` will use to
initialize itself. Additionally, if `Object.path` is accessed
`init` is being called as well to make sure it is properly
initialized, i.e. the tree initialized with the base content.
Refactor the `ObjectStore.snapshot` method to take an `Object` not
a plain filesystem tree, so the latter is more encapsulated from
the ObjectStore user (e.g. the pipeline) and prepares a unified
code-path for `snapshot` and `commit` in the future.
Now that Object manages its work directory itself, re-create the
latter when the its tree is moved, i.e. when the object is being
committed to the store. This means that after the object has been
written to the store it is in the same state is if it was new and
can be used in the very same way.
If the move itself fails (the rename(2) fails), the tree and its
contents is cleaned up with the reset of the work directory.
Rename the `move` method to `store_tree` to better reflect how the
method should be used, i.e. to store the tree corresponding to the
Object instance.
When a new object is being created, a corresponding temporary
directory is created within the file system tree of the object
store, which shall be called the "work dir". Within that dir a
well-known directory ('tree') is created that then is the root
of the filesystem tree[1] that clients use to store the tree
or the resulting image in.
Previously, the work dir was managed, i.e. created and cleaned
up (via a context manager) by the ObjectStore. Now the Object
itself manages the tree and thus the lifetime of the work dir
is more directly integrated and controlled by it. As a result
the Object itself is now a context manager. On exit of the
context the work dir is cleaned up.
[1] For the assembler this is the output directory that will
contain the final image.
Instead of just returning the path of the temporary object that is
created in .new() the actual instance of the new `Object` is being
returned, which can then provide a richer interface for clients
than a plain directory path.
Keep are reference to the parent store, which this object is tied
to. It is currently not yet used directly but is a preparation for
a closer Object and ObjectStore integration that will happen in
commits to follow.
As the name implies, the ObjectStore stores objects, which can be
trees but also everything an Assembler can make of the input tree,
like qcow2 images, tarballs and other non tree-like outputs.
Therefore rename the TreeObject to Object to better reflect that it
is representing any object, not only trees, in the store.
When the tree is committed to the objects directory of the object-
store, it is done via rename(3). The two possible errors that can
be raised in case that a non-empty tree with the same name already
exist is [EEXIST] or [ENOTEMPTY]. The latter was already ignored
but the former was not. At least on btrfs former will be raised
File "/home/gicmo/Code/src/osbuild/osbuild/objectstore.py",
os.rename(tree.root, output_tree)
FileExistsError: [Errno 17] File exists: 'store/tmpyyi3yvie/tree' -> 'store/objects/…'
Add a new method to the ObjectStore that takes a path to a file
system tree, which is currently being built, and commits it to
the store and references it via a given object_id.
The tree is copied to a temporary location (co-located in the
store to enable fast copying via reflinks) and then atomically
moved into the ObjectStore's objects path via rename(3).
Extract the code from ObjectStore.new that will commit the filled
tree to the store into its own method so it can be used from a
future method to snapshot trees at random points in time.
Introduce a small `TreeObject` class that is the representation of
a tree during its construction. It supports calculating its treesum
as well initialize the new tree with an existing one.
Import modules between files using the syntax `from . import foobar`,
renaming what used to be `FooBar` to `foobar.FooBar` when moved to a
separate file.
In __init__.py only import what is meant to be public API.
Signed-off-by: Tom Gundersen <teg@jklm.no>
