particle-os/debian-forge
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Pipeline: reuse existing trees if they exist in the object store

Browse source 
Don't do this only for the base, but for any prefix of the current
pipeline.

Note that if two pipelines share a prefix, but one is not the prefix
of another, no sharing is possible. Only a proper prefix can be
reused by another pipeline, as only the result of the last pipeline
is saved to the object store (this restriction could be changed in
the future).

Signed-off-by: Tom Gundersen <teg@jklm.no>
This commit is contained in:
Tom Gundersen 2019-08-12 23:37:39 +02:00
parent 0a9223b6f2
commit 67da9e0cca
1 changed files with 11 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

13
osbuild/__init__.py
View file

@ -434,13 +434,22 @@ class Pipeline:
with self.get_buildtree(object_store) as build_tree:
if self.stages:
if not object_store.has_tree(self.get_id()):
# Find the last stage that already exists in the object store, and use
# that as the base.
base = self.base
base_idx = -1
for i in range(len(self.stages) - 1, 0, -1):
if object_store.has_tree(self.stages[i].id):
base = self.stages[i].id
base_idx = i
break
# The tree does not exist. Create it and save it to the object store. If
# two run() calls race each-other, two trees may be generated, and it
# is nondeterministic which of them will end up referenced by the tree_id
# in the content store. However, we guarantee that all tree_id's and all
# generated trees remain valid.
with object_store.new_tree(self.get_id(), base_id=self.base) as tree:
for stage in self.stages:
with object_store.new_tree(self.get_id(), base_id=base) as tree:
for stage in self.stages[base_idx + 1:]:
r = stage.run(tree,
build_tree,
interactive=interactive,