api: introduce pipeline monitoring
Introduce the concept of pipeline monitoring: A new monitor class is passed to the pipeline.run() function. The main idea is to separate the monitoring from the code that builds pipeline. Through the build process various methods will be called on that object, representing the different steps and their targets during the build process. This can be used to fully stream the output of the various stages or just indicate the start and finish of the individual stages. This replaces the 'interactive' argument throughout the pipeline code. The old interactive behavior is replicated via the new `LogMonitor` class that logs the beginning of stages/assembler, but also streams all the output of them to stdout. The non-interactive behavior of not reporting anything is done by using the `NullMonitor` class, which in turn outputs nothing.
This commit is contained in:
parent
5d55bc9aca
commit
3e18d8118c
4 changed files with 159 additions and 33 deletions
|
|
@ -2,19 +2,18 @@ import asyncio
|
|||
import io
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
from .util import jsoncomm
|
||||
|
||||
|
||||
class API:
|
||||
def __init__(self, socket_address, args, interactive):
|
||||
def __init__(self, socket_address, args, monitor):
|
||||
self.socket_address = socket_address
|
||||
self.input = args
|
||||
self.interactive = interactive
|
||||
self._output_data = io.StringIO()
|
||||
self._output_pipe = None
|
||||
self.monitor = monitor
|
||||
self.event_loop = asyncio.new_event_loop()
|
||||
self.thread = threading.Thread(target=self._run_event_loop)
|
||||
self.barrier = threading.Barrier(2)
|
||||
|
|
@ -41,9 +40,7 @@ class API:
|
|||
raw = os.read(self._output_pipe, 4096)
|
||||
data = raw.decode("utf-8")
|
||||
self._output_data.write(data)
|
||||
|
||||
if self.interactive:
|
||||
sys.stdout.write(data)
|
||||
self.monitor.log(data)
|
||||
|
||||
def _setup_stdio(self, server, addr):
|
||||
with self._prepare_input() as stdin, \
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue