api: introduce pipeline monitoring

Introduce the concept of pipeline monitoring: A new monitor class is
passed to the pipeline.run() function. The main idea is to separate
the monitoring from the code that builds pipeline. Through the build
process various methods will be called on that object, representing
the different steps and their targets during the build process. This
can be used to fully stream the output of the various stages or just
indicate the start and finish of the individual stages.

This replaces the 'interactive' argument throughout the pipeline
code. The old interactive behavior is replicated via the new
`LogMonitor` class that logs the beginning of stages/assembler,
but also streams all the output of them to stdout.
The non-interactive behavior of not reporting anything is done by
using the `NullMonitor` class, which in turn outputs nothing.
This commit is contained in:
Christian Kellner 2020-07-10 15:19:36 +02:00
parent 5d55bc9aca
commit 3e18d8118c
4 changed files with 159 additions and 33 deletions

View file

@ -2,19 +2,18 @@ import asyncio
import io
import json
import os
import sys
import tempfile
import threading
from .util import jsoncomm
class API:
def __init__(self, socket_address, args, interactive):
def __init__(self, socket_address, args, monitor):
self.socket_address = socket_address
self.input = args
self.interactive = interactive
self._output_data = io.StringIO()
self._output_pipe = None
self.monitor = monitor
self.event_loop = asyncio.new_event_loop()
self.thread = threading.Thread(target=self._run_event_loop)
self.barrier = threading.Barrier(2)
@ -41,9 +40,7 @@ class API:
raw = os.read(self._output_pipe, 4096)
data = raw.decode("utf-8")
self._output_data.write(data)
if self.interactive:
sys.stdout.write(data)
self.monitor.log(data)
def _setup_stdio(self, server, addr):
with self._prepare_input() as stdin, \