pipeline: drop output_id and pull in output-directory
Now that no caller requires the "output_id" anymore, drop it from our results-dictionary. Instead, pass the output-directory through and copy outputs where we produce / fetch them. This still uses `objectstore.resolve_ref()`, since we do not have the outputs pinned at the places where we want to copy. This needs a little bit more rework, but we might just delay that until we have the cache rework landed. This already simplifies the output-directory path and drops the slight hack which checked very late for produced outputs. Note that we must be careful not to copy things too early, because we do not want remnants in the output-directory if we return failure. Hence, keep the copy-operation close to the commit-operation on the store.
This commit is contained in:
parent
18b16acd3f
commit
43ddcf895d
2 changed files with 23 additions and 12 deletions
|
|
@ -195,6 +195,20 @@ class Object:
|
|||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
self.cleanup()
|
||||
|
||||
def export(self, to_directory):
|
||||
"""Copy object into an external directory"""
|
||||
with self.read() as from_directory:
|
||||
subprocess.run(
|
||||
[
|
||||
"cp",
|
||||
"--reflink=auto",
|
||||
"-a",
|
||||
f"{from_directory}/.",
|
||||
to_directory,
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
class HostTree:
|
||||
"""Read-only access to the host file system
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue