worker: remove race from WatchJob()

WatchJob() regularly checks if a job was canceled in a goroutine. It
does so by accessing composer's `/jobs/{token}` route. However, once the
main goroutine marks the job as done (by sending PATCH to that same
route), the `token` is no longer valid and thus the route not accessible
anymore.

main() does cancel the goroutine running WatchJob, but it's not
guaranteed that it gets scheduled in time to actually stop watching the
job.

Thus, don't cancel the job when fetching the `/jobs/{token}` fails. This
means that it won't cancel the job anymore when the connection to
composer goes down.

Also, we will be able to move job.Update() into RunJob().
This commit is contained in:
Lars Karlitski 2020-11-01 17:03:47 +01:00
parent 299a5e52ab
commit 1184d78494

View file

@ -353,11 +353,7 @@ func WatchJob(ctx context.Context, job worker.Job) {
select {
case <-time.After(15 * time.Second):
canceled, err := job.Canceled()
if err != nil {
log.Printf("Error fetching job status: %v", err)
os.Exit(0)
}
if canceled {
if err == nil && canceled {
log.Println("Job was canceled. Exiting.")
os.Exit(0)
}