c4c7f44fcb
Previously, all dequeuers (goroutines waiting for a job to be dequeued) were listening for new messages on postgres channel jobs (LISTEN jobs). This didn't scale well as each dequeuer required to have its own DB connection and the number of DB connections is hard-limited in the pool's config. I changed the logic to work somewhat differently: dbjobqueue.New() now spawns a goroutine that listens on the postgres channel. If there's a new message, the goroutine just wakes up all dequeuers using a standard go channel. Go channels are cheap so this should scale much better. A test was added that confirms that 100 dequeuers are not a big deal now. This test failed when I tried to run on it on the previous commit. I tried even 1000 locally and it was still fine. Signed-off-by: Ondřej Budai <ondrej@budai.cz>
59 lines
2.4 KiB
Modula-2
59 lines
2.4 KiB
Modula-2
module github.com/osbuild/osbuild-composer
|
|
|
|
go 1.16
|
|
|
|
require (
|
|
cloud.google.com/go v0.100.0
|
|
cloud.google.com/go/cloudbuild v1.0.0
|
|
cloud.google.com/go/storage v1.18.2
|
|
github.com/Azure/azure-pipeline-go v0.2.3
|
|
github.com/Azure/azure-sdk-for-go v57.4.0+incompatible
|
|
github.com/Azure/azure-storage-blob-go v0.14.0
|
|
github.com/Azure/go-autorest/autorest v0.11.21
|
|
github.com/Azure/go-autorest/autorest/adal v0.9.15 // indirect
|
|
github.com/Azure/go-autorest/autorest/azure/auth v0.5.8
|
|
github.com/Azure/go-autorest/autorest/azure/cli v0.4.3 // indirect
|
|
github.com/Azure/go-autorest/autorest/to v0.4.0 // indirect
|
|
github.com/Azure/go-autorest/autorest/validation v0.3.1 // indirect
|
|
github.com/BurntSushi/toml v0.4.1
|
|
github.com/aws/aws-sdk-go v1.42.25
|
|
github.com/cenkalti/backoff/v4 v4.1.1 // indirect
|
|
github.com/cespare/xxhash/v2 v2.1.2 // indirect
|
|
github.com/coreos/go-semver v0.3.0
|
|
github.com/coreos/go-systemd v0.0.0-20190719114852-fd7a80b32e1f
|
|
github.com/deepmap/oapi-codegen v1.8.2
|
|
github.com/getkin/kin-openapi v0.61.0
|
|
github.com/gobwas/glob v0.2.3
|
|
github.com/golang-jwt/jwt v3.2.2+incompatible
|
|
github.com/golang/glog v1.0.0 // indirect
|
|
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
|
|
github.com/google/go-cmp v0.5.6
|
|
github.com/google/uuid v1.3.0
|
|
github.com/gophercloud/gophercloud v0.22.0
|
|
github.com/hashicorp/go-retryablehttp v0.7.0
|
|
github.com/jackc/pgtype v1.8.1
|
|
github.com/jackc/pgx/v4 v4.13.0
|
|
github.com/julienschmidt/httprouter v1.3.0
|
|
github.com/kolo/xmlrpc v0.0.0-20201022064351-38db28db192b
|
|
github.com/labstack/echo/v4 v4.6.1
|
|
github.com/labstack/gommon v0.3.0
|
|
github.com/openshift-online/ocm-sdk-go v0.1.214
|
|
github.com/oracle/oci-go-sdk/v54 v54.0.0
|
|
github.com/prometheus/client_golang v1.11.0
|
|
github.com/prometheus/common v0.30.0 // indirect
|
|
github.com/prometheus/procfs v0.7.3 // indirect
|
|
github.com/segmentio/ksuid v1.0.4
|
|
github.com/sirupsen/logrus v1.8.1
|
|
github.com/spf13/cobra v0.0.3
|
|
github.com/stretchr/testify v1.7.0
|
|
github.com/ubccr/kerby v0.0.0-20170626144437-201a958fc453
|
|
github.com/vmware/govmomi v0.26.1
|
|
golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8
|
|
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
|
|
golang.org/x/sys v0.0.0-20211210111614-af8b64212486
|
|
google.golang.org/api v0.63.0
|
|
google.golang.org/genproto v0.0.0-20211221195035-429b39de9b1c
|
|
google.golang.org/protobuf v1.27.1
|
|
gopkg.in/ini.v1 v1.63.0
|
|
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
|
|
)
|