Version 5.22 introduced a new option to /etc/containers/policy.json called
keyPaths, see
https://github.com/containers/image/pull/1609
EL9 immediately took advantage of this new feature and started using it, see
04645c4a84
This quickly became an issue in our code: The go library (containers/image)
parses the configuration file very strictly and refuses to create a client
when policy.json with an unknown key is present on the filesystem. As we
used 5.21.1 that doesn't know the new key, our unit tests started to
failing when containers-common was present.
Reproducer:
podman run --pull=always --rm -it centos:stream9
dnf install -y dnf-plugins-core
dnf config-manager --set-enabled crb
dnf install -y gpgme-devel libassuan-devel krb5-devel golang git-core
git clone https://github.com/osbuild/osbuild-composer
cd osbuild-composer
# install the new containers-common and run the test
dnf install -y https://kojihub.stream.centos.org/kojifiles/packages/containers-common/1/44.el9/x86_64/containers-common-1-44.el9.x86_64.rpm
go test -count 1 ./...
# this returns:
--- FAIL: TestClientResolve (0.00s)
client_test.go:31:
Error Trace: client_test.go:31
Error: Received unexpected error:
Unknown key "keyPaths"
invalid policy in "/etc/containers/policy.json"
github.com/containers/image/v5/signature.NewPolicyFromFile
/osbuild-composer/vendor/github.com/containers/image/v5/signature/policy_config.go:88
github.com/osbuild/osbuild-composer/internal/container.NewClient
/osbuild-composer/internal/container/client.go:123
github.com/osbuild/osbuild-composer/internal/container_test.TestClientResolve
/osbuild-composer/internal/container/client_test.go:29
testing.tRunner
/usr/lib/golang/src/testing/testing.go:1439
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1571
Test: TestClientResolve
client_test.go:32:
Error Trace: client_test.go:32
Error: Expected value not to be nil.
Test: TestClientResolve
When run with an older containers-common, it succeeds:
dnf install -y https://kojihub.stream.centos.org/kojifiles/packages/containers-common/1/40.el9/x86_64/containers-common-1-40.el9.x86_64.rpm
go test -count 1 ./...
PASS
To sum it up, I had to upgrade github.com/containers/image/v5 to v5.22.0.
Unfortunately, this wasn't so simple, see
go get github.com/containers/image/v5@latest
go: github.com/containers/image/v5@v5.22.0 requires
github.com/letsencrypt/boulder@v0.0.0-20220331220046-b23ab962616e requires
github.com/honeycombio/beeline-go@v1.1.1 requires
github.com/gobuffalo/pop/v5@v5.3.1 requires
github.com/mattn/go-sqlite3@v2.0.3+incompatible: reading github.com/mattn/go-sqlite3/go.mod at revision v2.0.3: unknown revision v2.0.3
It turns out that github.com/mattn/go-sqlite3@v2.0.3+incompatible has been
recently retracted https://github.com/mattn/go-sqlite3/pull/998 and this
broke a ton of packages depending on it. I was able to fix it by adding
exclude github.com/mattn/go-sqlite3 v2.0.3+incompatible
to our go.mod, see
https://github.com/mattn/go-sqlite3/issues/975#issuecomment-955661657
After adding it,
go get github.com/containers/image/v5@latest
succeeded and tools/prepare-source.sh took care of the rest.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
227 lines
8.7 KiB
Go
227 lines
8.7 KiB
Go
// Copyright 2014 Google LLC
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
/*
|
|
Package cloud is the root of the packages used to access Google Cloud
|
|
Services. See https://godoc.org/cloud.google.com/go for a full list
|
|
of sub-packages.
|
|
|
|
|
|
Client Options
|
|
|
|
All clients in sub-packages are configurable via client options. These options are
|
|
described here: https://godoc.org/google.golang.org/api/option.
|
|
|
|
|
|
Authentication and Authorization
|
|
|
|
All the clients in sub-packages support authentication via Google Application Default
|
|
Credentials (see https://cloud.google.com/docs/authentication/production), or
|
|
by providing a JSON key file for a Service Account. See examples below.
|
|
|
|
Google Application Default Credentials (ADC) is the recommended way to authorize
|
|
and authenticate clients. For information on how to create and obtain
|
|
Application Default Credentials, see
|
|
https://cloud.google.com/docs/authentication/production. Here is an example
|
|
of a client using ADC to authenticate:
|
|
client, err := secretmanager.NewClient(context.Background())
|
|
if err != nil {
|
|
// TODO: handle error.
|
|
}
|
|
_ = client // Use the client.
|
|
|
|
You can use a file with credentials to authenticate and authorize, such as a JSON
|
|
key file associated with a Google service account. Service Account keys can be
|
|
created and downloaded from
|
|
https://console.cloud.google.com/iam-admin/serviceaccounts. This example uses
|
|
the Secret Manger client, but the same steps apply to the other client libraries
|
|
underneath this package. Example:
|
|
client, err := secretmanager.NewClient(context.Background(),
|
|
option.WithCredentialsFile("/path/to/service-account-key.json"))
|
|
if err != nil {
|
|
// TODO: handle error.
|
|
}
|
|
_ = client // Use the client.
|
|
|
|
In some cases (for instance, you don't want to store secrets on disk), you can
|
|
create credentials from in-memory JSON and use the WithCredentials option.
|
|
The google package in this example is at golang.org/x/oauth2/google.
|
|
This example uses the Secret Manager client, but the same steps apply to
|
|
the other client libraries underneath this package. Note that scopes can be
|
|
found at https://developers.google.com/identity/protocols/oauth2/scopes, and
|
|
are also provided in all auto-generated libraries: for example,
|
|
cloud.google.com/go/secretmanager/apiv1 provides DefaultAuthScopes. Example:
|
|
ctx := context.Background()
|
|
creds, err := google.CredentialsFromJSON(ctx, []byte("JSON creds"), secretmanager.DefaultAuthScopes()...)
|
|
if err != nil {
|
|
// TODO: handle error.
|
|
}
|
|
client, err := secretmanager.NewClient(ctx, option.WithCredentials(creds))
|
|
if err != nil {
|
|
// TODO: handle error.
|
|
}
|
|
_ = client // Use the client.
|
|
|
|
|
|
Timeouts and Cancellation
|
|
|
|
By default, non-streaming methods, like Create or Get, will have a default deadline applied to the
|
|
context provided at call time, unless a context deadline is already set. Streaming
|
|
methods have no default deadline and will run indefinitely. To set timeouts or
|
|
arrange for cancellation, use contexts. Transient
|
|
errors will be retried when correctness allows.
|
|
|
|
Here is an example of how to set a timeout for an RPC, use context.WithTimeout:
|
|
ctx := context.Background()
|
|
// Do not set a timeout on the context passed to NewClient: dialing happens
|
|
// asynchronously, and the context is used to refresh credentials in the
|
|
// background.
|
|
client, err := secretmanager.NewClient(ctx)
|
|
if err != nil {
|
|
// TODO: handle error.
|
|
}
|
|
// Time out if it takes more than 10 seconds to create a dataset.
|
|
tctx, cancel := context.WithTimeout(ctx, 10*time.Second)
|
|
defer cancel() // Always call cancel.
|
|
|
|
req := &secretmanagerpb.DeleteSecretRequest{Name: "projects/project-id/secrets/name"}
|
|
if err := client.DeleteSecret(tctx, req); err != nil {
|
|
// TODO: handle error.
|
|
}
|
|
|
|
Here is an example of how to arrange for an RPC to be canceled, use context.WithCancel:
|
|
ctx := context.Background()
|
|
// Do not cancel the context passed to NewClient: dialing happens asynchronously,
|
|
// and the context is used to refresh credentials in the background.
|
|
client, err := secretmanager.NewClient(ctx)
|
|
if err != nil {
|
|
// TODO: handle error.
|
|
}
|
|
cctx, cancel := context.WithCancel(ctx)
|
|
defer cancel() // Always call cancel.
|
|
|
|
// TODO: Make the cancel function available to whatever might want to cancel the
|
|
// call--perhaps a GUI button.
|
|
req := &secretmanagerpb.DeleteSecretRequest{Name: "projects/proj/secrets/name"}
|
|
if err := client.DeleteSecret(cctx, req); err != nil {
|
|
// TODO: handle error.
|
|
}
|
|
|
|
To opt out of default deadlines, set the temporary environment variable
|
|
GOOGLE_API_GO_EXPERIMENTAL_DISABLE_DEFAULT_DEADLINE to "true" prior to client
|
|
creation. This affects all Google Cloud Go client libraries. This opt-out
|
|
mechanism will be removed in a future release. File an issue at
|
|
https://github.com/googleapis/google-cloud-go if the default deadlines
|
|
cannot work for you.
|
|
|
|
Do not attempt to control the initial connection (dialing) of a service by setting a
|
|
timeout on the context passed to NewClient. Dialing is non-blocking, so timeouts
|
|
would be ineffective and would only interfere with credential refreshing, which uses
|
|
the same context.
|
|
|
|
|
|
Connection Pooling
|
|
|
|
Connection pooling differs in clients based on their transport. Cloud
|
|
clients either rely on HTTP or gRPC transports to communicate
|
|
with Google Cloud.
|
|
|
|
Cloud clients that use HTTP (bigquery, compute, storage, and translate) rely on the
|
|
underlying HTTP transport to cache connections for later re-use. These are cached to
|
|
the default http.MaxIdleConns and http.MaxIdleConnsPerHost settings in
|
|
http.DefaultTransport.
|
|
|
|
For gRPC clients (all others in this repo), connection pooling is configurable. Users
|
|
of cloud client libraries may specify option.WithGRPCConnectionPool(n) as a client
|
|
option to NewClient calls. This configures the underlying gRPC connections to be
|
|
pooled and addressed in a round robin fashion.
|
|
|
|
|
|
Using the Libraries with Docker
|
|
|
|
Minimal docker images like Alpine lack CA certificates. This causes RPCs to appear to
|
|
hang, because gRPC retries indefinitely. See https://github.com/googleapis/google-cloud-go/issues/928
|
|
for more information.
|
|
|
|
|
|
Debugging
|
|
|
|
To see gRPC logs, set the environment variable GRPC_GO_LOG_SEVERITY_LEVEL. See
|
|
https://godoc.org/google.golang.org/grpc/grpclog for more information.
|
|
|
|
For HTTP logging, set the GODEBUG environment variable to "http2debug=1" or "http2debug=2".
|
|
|
|
|
|
Inspecting errors
|
|
|
|
Most of the errors returned by the generated clients are wrapped in an
|
|
`apierror.APIError` (https://pkg.go.dev/github.com/googleapis/gax-go/v2/apierror)
|
|
and can be further unwrapped into a `grpc.Status` or `googleapi.Error` depending
|
|
on the transport used to make the call (gRPC or REST). Converting your errors to
|
|
these types can be a useful way to get more information about what went wrong
|
|
while debugging.
|
|
|
|
`apierror.APIError` gives access to specific details in the
|
|
error. The transport-specific errors can still be unwrapped using the
|
|
`apierror.APIError`.
|
|
if err != nil {
|
|
var ae *apierror.APIError
|
|
if errors.As(err, &ae) {
|
|
log.Println(ae.Reason())
|
|
log.Println(ae.Details().Help.GetLinks())
|
|
}
|
|
}
|
|
|
|
If the gRPC transport was used, the `grpc.Status` can still be parsed using the
|
|
`status.FromError` function.
|
|
if err != nil {
|
|
if s, ok := status.FromError(err); ok {
|
|
log.Println(s.Message())
|
|
for _, d := range s.Proto().Details {
|
|
log.Println(d)
|
|
}
|
|
}
|
|
}
|
|
|
|
If the REST transport was used, the `googleapi.Error` can be parsed in a similar
|
|
way.
|
|
if err != nil {
|
|
var gerr *googleapi.Error
|
|
if errors.As(err, &gerr) {
|
|
log.Println(gerr.Message)
|
|
}
|
|
}
|
|
|
|
Client Stability
|
|
|
|
Clients in this repository are considered alpha or beta unless otherwise
|
|
marked as stable in the README.md. Semver is not used to communicate stability
|
|
of clients.
|
|
|
|
Alpha and beta clients may change or go away without notice.
|
|
|
|
Clients marked stable will maintain compatibility with future versions for as
|
|
long as we can reasonably sustain. Incompatible changes might be made in some
|
|
situations, including:
|
|
|
|
- Security bugs may prompt backwards-incompatible changes.
|
|
|
|
- Situations in which components are no longer feasible to maintain without
|
|
making breaking changes, including removal.
|
|
|
|
- Parts of the client surface may be outright unstable and subject to change.
|
|
These parts of the surface will be labeled with the note, "It is EXPERIMENTAL
|
|
and subject to change or removal without notice."
|
|
*/
|
|
package cloud // import "cloud.google.com/go"
|