c4c7f44fcb
Previously, all dequeuers (goroutines waiting for a job to be dequeued) were listening for new messages on postgres channel jobs (LISTEN jobs). This didn't scale well as each dequeuer required to have its own DB connection and the number of DB connections is hard-limited in the pool's config. I changed the logic to work somewhat differently: dbjobqueue.New() now spawns a goroutine that listens on the postgres channel. If there's a new message, the goroutine just wakes up all dequeuers using a standard go channel. Go channels are cheap so this should scale much better. A test was added that confirms that 100 dequeuers are not a big deal now. This test failed when I tried to run on it on the previous commit. I tried even 1000 locally and it was still fine. Signed-off-by: Ondřej Budai <ondrej@budai.cz> |
||
|---|---|---|
| .. | ||
| cloud.google.com/go | ||
| github.com | ||
| go.opencensus.io | ||
| golang.org/x | ||
| google.golang.org | ||
| gopkg.in | ||
| modules.txt | ||