build(deps): bump the go-deps group with 10 updates

Bumps the go-deps group with 10 updates:

| Package | From | To |
| --- | --- | --- |
| [cloud.google.com/go/compute](https://github.com/googleapis/google-cloud-go) | `1.23.1` | `1.23.3` |
| [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) | `1.33.0` | `1.35.1` |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.45.27` | `1.47.9` |
| [github.com/hashicorp/go-retryablehttp](https://github.com/hashicorp/go-retryablehttp) | `0.7.4` | `0.7.5` |
| [github.com/labstack/echo/v4](https://github.com/labstack/echo) | `4.11.2` | `4.11.3` |
| [github.com/labstack/gommon](https://github.com/labstack/gommon) | `0.4.0` | `0.4.1` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.374` | `0.1.385` |
| [github.com/osbuild/images](https://github.com/osbuild/images) | `0.12.0` | `0.15.0` |
| [github.com/spf13/cobra](https://github.com/spf13/cobra) | `1.7.0` | `1.8.0` |
| [golang.org/x/oauth2](https://github.com/golang/oauth2) | `0.13.0` | `0.14.0` |


Updates `cloud.google.com/go/compute` from 1.23.1 to 1.23.3
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/documentai/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.23.1...compute/v1.23.3)

Updates `cloud.google.com/go/storage` from 1.33.0 to 1.35.1
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.33.0...storage/v1.35.1)

Updates `github.com/aws/aws-sdk-go` from 1.45.27 to 1.47.9
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.45.27...v1.47.9)

Updates `github.com/hashicorp/go-retryablehttp` from 0.7.4 to 0.7.5
- [Changelog](https://github.com/hashicorp/go-retryablehttp/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/go-retryablehttp/compare/v0.7.4...v0.7.5)

Updates `github.com/labstack/echo/v4` from 4.11.2 to 4.11.3
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.11.2...v4.11.3)

Updates `github.com/labstack/gommon` from 0.4.0 to 0.4.1
- [Release notes](https://github.com/labstack/gommon/releases)
- [Commits](https://github.com/labstack/gommon/compare/v0.4.0...v0.4.1)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.374 to 0.1.385
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.374...v0.1.385)

Updates `github.com/osbuild/images` from 0.12.0 to 0.15.0
- [Release notes](https://github.com/osbuild/images/releases)
- [Commits](https://github.com/osbuild/images/compare/v0.12.0...v0.15.0)

Updates `github.com/spf13/cobra` from 1.7.0 to 1.8.0
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.7.0...v1.8.0)

Updates `golang.org/x/oauth2` from 0.13.0 to 0.14.0
- [Commits](https://github.com/golang/oauth2/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/compute
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/hashicorp/go-retryablehttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/labstack/gommon
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
This commit is contained in:
dependabot[bot] 2023-11-13 04:25:22 +00:00 committed by Tomáš Hozza
parent 13d642bb46
commit a1e428fc53
565 changed files with 10888 additions and 5729 deletions

View file

@ -15,4 +15,4 @@
package internal
// Version is the current tagged release of the library.
const Version = "1.23.1"
const Version = "1.23.3"

View file

@ -1,6 +1,13 @@
# Changes
## [1.1.3](https://github.com/googleapis/google-cloud-go/compare/iam/v1.1.2...iam/v1.1.3) (2023-10-12)
### Bug Fixes
* **iam:** Update golang.org/x/net to v0.17.0 ([174da47](https://github.com/googleapis/google-cloud-go/commit/174da47254fefb12921bbfc65b7829a453af6f5d))
## [1.1.2](https://github.com/googleapis/google-cloud-go/compare/iam/v1.1.1...iam/v1.1.2) (2023-08-08)

View file

@ -14,7 +14,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.30.0
// protoc-gen-go v1.31.0
// protoc v4.23.2
// source: google/iam/v1/iam_policy.proto

View file

@ -14,7 +14,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.30.0
// protoc-gen-go v1.31.0
// protoc v4.23.2
// source: google/iam/v1/options.proto

View file

@ -14,7 +14,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.30.0
// protoc-gen-go v1.31.0
// protoc v4.23.2
// source: google/iam/v1/policy.proto

View file

@ -1,6 +1,46 @@
# Changes
## [1.35.1](https://github.com/googleapis/google-cloud-go/compare/storage/v1.35.0...storage/v1.35.1) (2023-11-09)
### Bug Fixes
* **storage:** Rename aux.go to auxiliary.go fixing windows build ([ba23673](https://github.com/googleapis/google-cloud-go/commit/ba23673da7707c31292e4aa29d65b7ac1446d4a6))
## [1.35.0](https://github.com/googleapis/google-cloud-go/compare/storage/v1.34.1...storage/v1.35.0) (2023-11-09)
### Features
* **storage:** Change gRPC writes to use bi-directional streams ([#8930](https://github.com/googleapis/google-cloud-go/issues/8930)) ([3e23a36](https://github.com/googleapis/google-cloud-go/commit/3e23a364b1a20c4fda7aef257e4136586ec769a4))
## [1.34.1](https://github.com/googleapis/google-cloud-go/compare/storage/v1.34.0...storage/v1.34.1) (2023-11-01)
### Bug Fixes
* **storage:** Bump google.golang.org/api to v0.149.0 ([8d2ab9f](https://github.com/googleapis/google-cloud-go/commit/8d2ab9f320a86c1c0fab90513fc05861561d0880))
## [1.34.0](https://github.com/googleapis/google-cloud-go/compare/storage/v1.33.0...storage/v1.34.0) (2023-10-31)
### Features
* **storage/internal:** Add match_glob field to ListObjectsRequest ([#8618](https://github.com/googleapis/google-cloud-go/issues/8618)) ([e9ae601](https://github.com/googleapis/google-cloud-go/commit/e9ae6018983ae09781740e4ff939e6e365863dbb))
* **storage/internal:** Add terminal_storage_class fields to Autoclass message ([57fc1a6](https://github.com/googleapis/google-cloud-go/commit/57fc1a6de326456eb68ef25f7a305df6636ed386))
* **storage/internal:** Adds the RestoreObject operation ([56ce871](https://github.com/googleapis/google-cloud-go/commit/56ce87195320634b07ae0b012efcc5f2b3813fb0))
* **storage:** Support autoclass v2.1 ([#8721](https://github.com/googleapis/google-cloud-go/issues/8721)) ([fe1e195](https://github.com/googleapis/google-cloud-go/commit/fe1e19590a252c6adc6ca6c51a69b6e561e143b8))
* **storage:** Support MatchGlob for gRPC ([#8670](https://github.com/googleapis/google-cloud-go/issues/8670)) ([3df0287](https://github.com/googleapis/google-cloud-go/commit/3df0287f88d5e2c4526e9e6b8dc2a4ca54f88918)), refs [#7727](https://github.com/googleapis/google-cloud-go/issues/7727)
### Bug Fixes
* **storage:** Drop stream reference after closing it for gRPC writes ([#8872](https://github.com/googleapis/google-cloud-go/issues/8872)) ([525abde](https://github.com/googleapis/google-cloud-go/commit/525abdee433864d4d456f1f1fff5599017b557ff))
* **storage:** Update golang.org/x/net to v0.17.0 ([174da47](https://github.com/googleapis/google-cloud-go/commit/174da47254fefb12921bbfc65b7829a453af6f5d))
* **storage:** Update grpc-go to v1.56.3 ([343cea8](https://github.com/googleapis/google-cloud-go/commit/343cea8c43b1e31ae21ad50ad31d3b0b60143f8c))
* **storage:** Update grpc-go to v1.59.0 ([81a97b0](https://github.com/googleapis/google-cloud-go/commit/81a97b06cb28b25432e4ece595c55a9857e960b7))
## [1.33.0](https://github.com/googleapis/google-cloud-go/compare/storage/v1.32.0...storage/v1.33.0) (2023-09-07)

View file

@ -740,6 +740,13 @@ type Autoclass struct {
// If Autoclass is enabled when the bucket is created, the ToggleTime
// is set to the bucket creation time. This field is read-only.
ToggleTime time.Time
// TerminalStorageClass: The storage class that objects in the bucket
// eventually transition to if they are not read for a certain length of
// time. Valid values are NEARLINE and ARCHIVE.
TerminalStorageClass string
// TerminalStorageClassUpdateTime represents the time of the most recent
// update to "TerminalStorageClass".
TerminalStorageClassUpdateTime time.Time
}
func newBucket(b *raw.Bucket) (*BucketAttrs, error) {
@ -1241,9 +1248,11 @@ func (ua *BucketAttrsToUpdate) toRawBucket() *raw.Bucket {
}
if ua.Autoclass != nil {
rb.Autoclass = &raw.BucketAutoclass{
Enabled: ua.Autoclass.Enabled,
ForceSendFields: []string{"Enabled"},
Enabled: ua.Autoclass.Enabled,
TerminalStorageClass: ua.Autoclass.TerminalStorageClass,
ForceSendFields: []string{"Enabled"},
}
rb.ForceSendFields = append(rb.ForceSendFields, "Autoclass")
}
if ua.PredefinedACL != "" {
// Clear ACL or the call will fail.
@ -1954,9 +1963,10 @@ func (a *Autoclass) toRawAutoclass() *raw.BucketAutoclass {
if a == nil {
return nil
}
// Excluding read only field ToggleTime.
// Excluding read only fields ToggleTime and TerminalStorageClassUpdateTime.
return &raw.BucketAutoclass{
Enabled: a.Enabled,
Enabled: a.Enabled,
TerminalStorageClass: a.TerminalStorageClass,
}
}
@ -1964,27 +1974,34 @@ func (a *Autoclass) toProtoAutoclass() *storagepb.Bucket_Autoclass {
if a == nil {
return nil
}
// Excluding read only field ToggleTime.
return &storagepb.Bucket_Autoclass{
// Excluding read only fields ToggleTime and TerminalStorageClassUpdateTime.
ba := &storagepb.Bucket_Autoclass{
Enabled: a.Enabled,
}
if a.TerminalStorageClass != "" {
ba.TerminalStorageClass = &a.TerminalStorageClass
}
return ba
}
func toAutoclassFromRaw(a *raw.BucketAutoclass) *Autoclass {
if a == nil || a.ToggleTime == "" {
return nil
}
// Return Autoclass.ToggleTime only if parsed with a valid value.
ac := &Autoclass{
Enabled: a.Enabled,
TerminalStorageClass: a.TerminalStorageClass,
}
// Return ToggleTime and TSCUpdateTime only if parsed with valid values.
t, err := time.Parse(time.RFC3339, a.ToggleTime)
if err != nil {
return &Autoclass{
Enabled: a.Enabled,
}
if err == nil {
ac.ToggleTime = t
}
return &Autoclass{
Enabled: a.Enabled,
ToggleTime: t,
ut, err := time.Parse(time.RFC3339, a.TerminalStorageClassUpdateTime)
if err == nil {
ac.TerminalStorageClassUpdateTime = ut
}
return ac
}
func toAutoclassFromProto(a *storagepb.Bucket_Autoclass) *Autoclass {
@ -1992,8 +2009,10 @@ func toAutoclassFromProto(a *storagepb.Bucket_Autoclass) *Autoclass {
return nil
}
return &Autoclass{
Enabled: a.GetEnabled(),
ToggleTime: a.GetToggleTime().AsTime(),
Enabled: a.GetEnabled(),
ToggleTime: a.GetToggleTime().AsTime(),
TerminalStorageClass: a.GetTerminalStorageClass(),
TerminalStorageClassUpdateTime: a.GetTerminalStorageClassUpdateTime().AsTime(),
}
}

View file

@ -141,11 +141,11 @@ func (c *grpcStorageClient) GetServiceAccount(ctx context.Context, project strin
Project: toProjectResource(project),
}
var resp *storagepb.ServiceAccount
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
var err error
resp, err = c.raw.GetServiceAccount(ctx, req, s.gax...)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
return "", err
}
@ -173,13 +173,13 @@ func (c *grpcStorageClient) CreateBucket(ctx context.Context, project, bucket st
}
var battrs *BucketAttrs
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
res, err := c.raw.CreateBucket(ctx, req, s.gax...)
battrs = newBucketFromProto(res)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
return battrs, err
}
@ -193,26 +193,26 @@ func (c *grpcStorageClient) ListBuckets(ctx context.Context, project string, opt
var gitr *gapic.BucketIterator
fetch := func(pageSize int, pageToken string) (token string, err error) {
// Initialize GAPIC-based iterator when pageToken is empty, which
// indicates that this fetch call is attempting to get the first page.
//
// Note: Initializing the GAPIC-based iterator lazily is necessary to
// capture the BucketIterator.Prefix set by the user *after* the
// BucketIterator is returned to them from the veneer.
if pageToken == "" {
req := &storagepb.ListBucketsRequest{
Parent: toProjectResource(it.projectID),
Prefix: it.Prefix,
}
gitr = c.raw.ListBuckets(it.ctx, req, s.gax...)
}
var buckets []*storagepb.Bucket
var next string
err = run(it.ctx, func() error {
err = run(it.ctx, func(ctx context.Context) error {
// Initialize GAPIC-based iterator when pageToken is empty, which
// indicates that this fetch call is attempting to get the first page.
//
// Note: Initializing the GAPIC-based iterator lazily is necessary to
// capture the BucketIterator.Prefix set by the user *after* the
// BucketIterator is returned to them from the veneer.
if pageToken == "" {
req := &storagepb.ListBucketsRequest{
Parent: toProjectResource(it.projectID),
Prefix: it.Prefix,
}
gitr = c.raw.ListBuckets(ctx, req, s.gax...)
}
buckets, next, err = gitr.InternalFetch(pageSize, pageToken)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
return "", err
}
@ -246,9 +246,9 @@ func (c *grpcStorageClient) DeleteBucket(ctx context.Context, bucket string, con
ctx = setUserProjectMetadata(ctx, s.userProject)
}
return run(ctx, func() error {
return run(ctx, func(ctx context.Context) error {
return c.raw.DeleteBucket(ctx, req, s.gax...)
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
}
func (c *grpcStorageClient) GetBucket(ctx context.Context, bucket string, conds *BucketConditions, opts ...storageOption) (*BucketAttrs, error) {
@ -265,13 +265,13 @@ func (c *grpcStorageClient) GetBucket(ctx context.Context, bucket string, conds
}
var battrs *BucketAttrs
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
res, err := c.raw.GetBucket(ctx, req, s.gax...)
battrs = newBucketFromProto(res)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if s, ok := status.FromError(err); ok && s.Code() == codes.NotFound {
return nil, ErrBucketNotExist
@ -369,11 +369,11 @@ func (c *grpcStorageClient) UpdateBucket(ctx context.Context, bucket string, uat
req.UpdateMask = fieldMask
var battrs *BucketAttrs
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
res, err := c.raw.UpdateBucket(ctx, req, s.gax...)
battrs = newBucketFromProto(res)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
return battrs, err
}
@ -386,10 +386,10 @@ func (c *grpcStorageClient) LockBucketRetentionPolicy(ctx context.Context, bucke
return err
}
return run(ctx, func() error {
return run(ctx, func(ctx context.Context) error {
_, err := c.raw.LockBucketRetentionPolicy(ctx, req, s.gax...)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
}
func (c *grpcStorageClient) ListObjects(ctx context.Context, bucket string, q *Query, opts ...storageOption) *ObjectIterator {
@ -408,23 +408,21 @@ func (c *grpcStorageClient) ListObjects(ctx context.Context, bucket string, q *Q
LexicographicStart: it.query.StartOffset,
LexicographicEnd: it.query.EndOffset,
IncludeTrailingDelimiter: it.query.IncludeTrailingDelimiter,
MatchGlob: it.query.MatchGlob,
ReadMask: q.toFieldMask(), // a nil Query still results in a "*" FieldMask
}
if s.userProject != "" {
ctx = setUserProjectMetadata(ctx, s.userProject)
}
gitr := c.raw.ListObjects(it.ctx, req, s.gax...)
fetch := func(pageSize int, pageToken string) (token string, err error) {
// MatchGlob not yet supported for gRPC.
// TODO: add support when b/287306063 resolved.
if q != nil && q.MatchGlob != "" {
return "", status.Errorf(codes.Unimplemented, "MatchGlob is not supported for gRPC")
}
var objects []*storagepb.Object
err = run(it.ctx, func() error {
var gitr *gapic.ObjectIterator
err = run(it.ctx, func(ctx context.Context) error {
gitr = c.raw.ListObjects(ctx, req, s.gax...)
it.ctx = ctx
objects, token, err = gitr.InternalFetch(pageSize, pageToken)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
if st, ok := status.FromError(err); ok && st.Code() == codes.NotFound {
err = ErrBucketNotExist
@ -467,9 +465,9 @@ func (c *grpcStorageClient) DeleteObject(ctx context.Context, bucket, object str
if s.userProject != "" {
ctx = setUserProjectMetadata(ctx, s.userProject)
}
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
return c.raw.DeleteObject(ctx, req, s.gax...)
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if s, ok := status.FromError(err); ok && s.Code() == codes.NotFound {
return ErrObjectNotExist
}
@ -495,12 +493,12 @@ func (c *grpcStorageClient) GetObject(ctx context.Context, bucket, object string
}
var attrs *ObjectAttrs
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
res, err := c.raw.GetObject(ctx, req, s.gax...)
attrs = newObjectFromProto(res)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if s, ok := status.FromError(err); ok && s.Code() == codes.NotFound {
return nil, ErrObjectNotExist
@ -577,11 +575,11 @@ func (c *grpcStorageClient) UpdateObject(ctx context.Context, bucket, object str
req.UpdateMask = fieldMask
var attrs *ObjectAttrs
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
res, err := c.raw.UpdateObject(ctx, req, s.gax...)
attrs = newObjectFromProto(res)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if e, ok := status.FromError(err); ok && e.Code() == codes.NotFound {
return nil, ErrObjectNotExist
}
@ -820,10 +818,10 @@ func (c *grpcStorageClient) ComposeObject(ctx context.Context, req *composeObjec
var obj *storagepb.Object
var err error
if err := run(ctx, func() error {
if err := run(ctx, func(ctx context.Context) error {
obj, err = c.raw.ComposeObject(ctx, rawReq, s.gax...)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx)); err != nil {
}, s.retry, s.idempotent); err != nil {
return nil, err
}
@ -870,9 +868,9 @@ func (c *grpcStorageClient) RewriteObject(ctx context.Context, req *rewriteObjec
var res *storagepb.RewriteResponse
var err error
retryCall := func() error { res, err = c.raw.RewriteObject(ctx, call, s.gax...); return err }
retryCall := func(ctx context.Context) error { res, err = c.raw.RewriteObject(ctx, call, s.gax...); return err }
if err := run(ctx, retryCall, s.retry, s.idempotent, setRetryHeaderGRPC(ctx)); err != nil {
if err := run(ctx, retryCall, s.retry, s.idempotent); err != nil {
return nil, err
}
@ -936,7 +934,7 @@ func (c *grpcStorageClient) NewRangeReader(ctx context.Context, params *newRange
var msg *storagepb.ReadObjectResponse
var err error
err = run(cc, func() error {
err = run(cc, func(ctx context.Context) error {
stream, err = c.raw.ReadObject(cc, req, s.gax...)
if err != nil {
return err
@ -950,7 +948,7 @@ func (c *grpcStorageClient) NewRangeReader(ctx context.Context, params *newRange
}
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
// Close the stream context we just created to ensure we don't leak
// resources.
@ -1068,7 +1066,7 @@ func (c *grpcStorageClient) OpenWriter(params *openWriterParams, opts ...storage
}
}
o, off, finalized, err := gw.uploadBuffer(recvd, offset, doneReading)
o, off, err := gw.uploadBuffer(recvd, offset, doneReading)
if err != nil {
err = checkCanceled(err)
errorf(err)
@ -1087,9 +1085,9 @@ func (c *grpcStorageClient) OpenWriter(params *openWriterParams, opts ...storage
progress(offset)
}
// When we are done reading data and the chunk has been finalized,
// we are done.
if doneReading && finalized {
// When we are done reading data without errors, set the object and
// finish.
if doneReading {
// Build Object from server's response.
setObj(newObjectFromProto(o))
return
@ -1112,11 +1110,11 @@ func (c *grpcStorageClient) GetIamPolicy(ctx context.Context, resource string, v
},
}
var rp *iampb.Policy
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
var err error
rp, err = c.raw.GetIamPolicy(ctx, req, s.gax...)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
return rp, err
}
@ -1130,10 +1128,10 @@ func (c *grpcStorageClient) SetIamPolicy(ctx context.Context, resource string, p
Policy: policy,
}
return run(ctx, func() error {
return run(ctx, func(ctx context.Context) error {
_, err := c.raw.SetIamPolicy(ctx, req, s.gax...)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
}
func (c *grpcStorageClient) TestIamPermissions(ctx context.Context, resource string, permissions []string, opts ...storageOption) ([]string, error) {
@ -1144,11 +1142,11 @@ func (c *grpcStorageClient) TestIamPermissions(ctx context.Context, resource str
Permissions: permissions,
}
var res *iampb.TestIamPermissionsResponse
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
var err error
res, err = c.raw.TestIamPermissions(ctx, req, s.gax...)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -1167,11 +1165,11 @@ func (c *grpcStorageClient) GetHMACKey(ctx context.Context, project, accessID st
ctx = setUserProjectMetadata(ctx, s.userProject)
}
var metadata *storagepb.HmacKeyMetadata
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
var err error
metadata, err = c.raw.GetHmacKey(ctx, req, s.gax...)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -1193,13 +1191,13 @@ func (c *grpcStorageClient) ListHMACKeys(ctx context.Context, project, serviceAc
projectID: project,
retry: s.retry,
}
gitr := c.raw.ListHmacKeys(it.ctx, req, s.gax...)
fetch := func(pageSize int, pageToken string) (token string, err error) {
var hmacKeys []*storagepb.HmacKeyMetadata
err = run(it.ctx, func() error {
err = run(it.ctx, func(ctx context.Context) error {
gitr := c.raw.ListHmacKeys(ctx, req, s.gax...)
hmacKeys, token, err = gitr.InternalFetch(pageSize, pageToken)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
return "", err
}
@ -1246,11 +1244,11 @@ func (c *grpcStorageClient) UpdateHMACKey(ctx context.Context, project, serviceA
ctx = setUserProjectMetadata(ctx, s.userProject)
}
var metadata *storagepb.HmacKeyMetadata
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
var err error
metadata, err = c.raw.UpdateHmacKey(ctx, req, s.gax...)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -1267,11 +1265,11 @@ func (c *grpcStorageClient) CreateHMACKey(ctx context.Context, project, serviceA
ctx = setUserProjectMetadata(ctx, s.userProject)
}
var res *storagepb.CreateHmacKeyResponse
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
var err error
res, err = c.raw.CreateHmacKey(ctx, req, s.gax...)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -1290,9 +1288,9 @@ func (c *grpcStorageClient) DeleteHMACKey(ctx context.Context, project string, a
if s.userProject != "" {
ctx = setUserProjectMetadata(ctx, s.userProject)
}
return run(ctx, func() error {
return run(ctx, func(ctx context.Context) error {
return c.raw.DeleteHmacKey(ctx, req, s.gax...)
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
}
// Notification methods.
@ -1309,7 +1307,7 @@ func (c *grpcStorageClient) ListNotifications(ctx context.Context, bucket string
Parent: bucketResourceName(globalProjectAlias, bucket),
}
var notifications []*storagepb.NotificationConfig
err = run(ctx, func() error {
err = run(ctx, func(ctx context.Context) error {
gitr := c.raw.ListNotificationConfigs(ctx, req, s.gax...)
for {
// PageSize is not set and fallbacks to the API default pageSize of 100.
@ -1324,7 +1322,7 @@ func (c *grpcStorageClient) ListNotifications(ctx context.Context, bucket string
}
req.PageToken = nextPageToken
}
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -1342,11 +1340,11 @@ func (c *grpcStorageClient) CreateNotification(ctx context.Context, bucket strin
NotificationConfig: toProtoNotification(n),
}
var pbn *storagepb.NotificationConfig
err = run(ctx, func() error {
err = run(ctx, func(ctx context.Context) error {
var err error
pbn, err = c.raw.CreateNotificationConfig(ctx, req, s.gax...)
return err
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -1359,9 +1357,9 @@ func (c *grpcStorageClient) DeleteNotification(ctx context.Context, bucket strin
s := callSettings(c.settings, opts...)
req := &storagepb.DeleteNotificationConfigRequest{Name: id}
return run(ctx, func() error {
return run(ctx, func(ctx context.Context) error {
return c.raw.DeleteNotificationConfig(ctx, req, s.gax...)
}, s.retry, s.idempotent, setRetryHeaderGRPC(ctx))
}, s.retry, s.idempotent)
}
// setUserProjectMetadata appends a project ID to the outgoing Context metadata
@ -1539,7 +1537,7 @@ type gRPCWriter struct {
chunkSize int
// The gRPC client-stream used for sending buffers.
stream storagepb.Storage_WriteObjectClient
stream storagepb.Storage_BidiWriteObjectClient
// The Resumable Upload ID started by a gRPC-based Writer.
upid string
@ -1560,68 +1558,79 @@ func (w *gRPCWriter) startResumableUpload() error {
// the upload, but in the future, we must also support sending it
// on the *last* message of the stream.
req.ObjectChecksums = toProtoChecksums(w.sendCRC32C, w.attrs)
return run(w.ctx, func() error {
return run(w.ctx, func(ctx context.Context) error {
upres, err := w.c.raw.StartResumableWrite(w.ctx, req)
w.upid = upres.GetUploadId()
return err
}, w.settings.retry, w.settings.idempotent, setRetryHeaderGRPC(w.ctx))
}, w.settings.retry, w.settings.idempotent)
}
// queryProgress is a helper that queries the status of the resumable upload
// associated with the given upload ID.
func (w *gRPCWriter) queryProgress() (int64, error) {
var persistedSize int64
err := run(w.ctx, func() error {
err := run(w.ctx, func(ctx context.Context) error {
q, err := w.c.raw.QueryWriteStatus(w.ctx, &storagepb.QueryWriteStatusRequest{
UploadId: w.upid,
})
persistedSize = q.GetPersistedSize()
return err
}, w.settings.retry, true, setRetryHeaderGRPC(w.ctx))
}, w.settings.retry, true)
// q.GetCommittedSize() will return 0 if q is nil.
return persistedSize, err
}
// uploadBuffer opens a Write stream and uploads the buffer at the given offset (if
// uploading a chunk for a resumable uploadBuffer), and will mark the write as
// finished if we are done receiving data from the user. The resulting write
// offset after uploading the buffer is returned, as well as a boolean
// indicating if the Object has been finalized. If it has been finalized, the
// final Object will be returned as well. Finalizing the upload is primarily
// important for Resumable Uploads. A simple or multi-part upload will always
// be finalized once the entire buffer has been written.
func (w *gRPCWriter) uploadBuffer(recvd int, start int64, doneReading bool) (*storagepb.Object, int64, bool, error) {
var err error
var finishWrite bool
var sent, limit int = 0, maxPerMessageWriteSize
// uploadBuffer uploads the buffer at the given offset using a bi-directional
// Write stream. It will open a new stream if necessary (on the first call or
// after resuming from failure). The resulting write offset after uploading the
// buffer is returned, as well as well as the final Object if the upload is
// completed.
//
// Returns object, persisted size, and any error that is not retriable.
func (w *gRPCWriter) uploadBuffer(recvd int, start int64, doneReading bool) (*storagepb.Object, int64, error) {
var shouldRetry = ShouldRetry
if w.settings.retry != nil && w.settings.retry.shouldRetry != nil {
shouldRetry = w.settings.retry.shouldRetry
}
offset := start
var err error
var lastWriteOfEntireObject bool
sent := 0
writeOffset := start
toWrite := w.buf[:recvd]
// Send a request with as many bytes as possible.
// Loop until all bytes are sent.
for {
// This indicates that this is the last message and the remaining
// data fits in one message.
belowLimit := recvd-sent <= limit
if belowLimit {
limit = recvd - sent
bytesNotYetSent := recvd - sent
remainingDataFitsInSingleReq := bytesNotYetSent <= maxPerMessageWriteSize
if remainingDataFitsInSingleReq && doneReading {
lastWriteOfEntireObject = true
}
if belowLimit && doneReading {
finishWrite = true
// Send the maximum amount of bytes we can, unless we don't have that many.
bytesToSendInCurrReq := maxPerMessageWriteSize
if remainingDataFitsInSingleReq {
bytesToSendInCurrReq = bytesNotYetSent
}
// Prepare chunk section for upload.
data := toWrite[sent : sent+limit]
req := &storagepb.WriteObjectRequest{
Data: &storagepb.WriteObjectRequest_ChecksummedData{
data := toWrite[sent : sent+bytesToSendInCurrReq]
req := &storagepb.BidiWriteObjectRequest{
Data: &storagepb.BidiWriteObjectRequest_ChecksummedData{
ChecksummedData: &storagepb.ChecksummedData{
Content: data,
},
},
WriteOffset: offset,
FinishWrite: finishWrite,
WriteOffset: writeOffset,
FinishWrite: lastWriteOfEntireObject,
Flush: remainingDataFitsInSingleReq,
StateLookup: remainingDataFitsInSingleReq,
}
// Open a new stream if necessary and set the first_message field on
@ -1630,19 +1639,20 @@ func (w *gRPCWriter) uploadBuffer(recvd int, start int64, doneReading bool) (*st
if w.stream == nil {
hds := []string{"x-goog-request-params", fmt.Sprintf("bucket=projects/_/buckets/%s", url.QueryEscape(w.bucket))}
ctx := gax.InsertMetadataIntoOutgoingContext(w.ctx, hds...)
w.stream, err = w.c.raw.WriteObject(ctx)
w.stream, err = w.c.raw.BidiWriteObject(ctx)
if err != nil {
return nil, 0, false, err
return nil, 0, err
}
if w.upid != "" {
req.FirstMessage = &storagepb.WriteObjectRequest_UploadId{UploadId: w.upid}
} else {
if w.upid != "" { // resumable upload
req.FirstMessage = &storagepb.BidiWriteObjectRequest_UploadId{UploadId: w.upid}
} else { // non-resumable
spec, err := w.writeObjectSpec()
if err != nil {
return nil, 0, false, err
return nil, 0, err
}
req.FirstMessage = &storagepb.WriteObjectRequest_WriteObjectSpec{
req.FirstMessage = &storagepb.BidiWriteObjectRequest_WriteObjectSpec{
WriteObjectSpec: spec,
}
req.CommonObjectRequestParams = toProtoCommonObjectRequestParams(w.encryptionKey)
@ -1652,38 +1662,53 @@ func (w *gRPCWriter) uploadBuffer(recvd int, start int64, doneReading bool) (*st
// on the *last* message of the stream (instead of the first).
req.ObjectChecksums = toProtoChecksums(w.sendCRC32C, w.attrs)
}
}
err = w.stream.Send(req)
if err == io.EOF {
// err was io.EOF. The client-side of a stream only gets an EOF on Send
// when the backend closes the stream and wants to return an error
// status. Closing the stream receives the status as an error.
_, err = w.stream.CloseAndRecv()
// status.
// Receive from the stream Recv() until it returns a non-nil error
// to receive the server's status as an error. We may get multiple
// messages before the error due to buffering.
err = nil
for err == nil {
_, err = w.stream.Recv()
}
// Drop the stream reference as a new one will need to be created if
// we retry.
w.stream = nil
// Drop the stream reference as a new one will need to be created if
// we can retry the upload
w.stream = nil
// Retriable errors mean we should start over and attempt to
// resend the entire buffer via a new stream.
// If not retriable, falling through will return the error received
// from closing the stream.
// If not retriable, falling through will return the error received.
if shouldRetry(err) {
sent = 0
finishWrite = false
// TODO: Add test case for failure modes of querying progress.
offset, err = w.determineOffset(start)
if err == nil {
continue
writeOffset, err = w.determineOffset(start)
if err != nil {
return nil, 0, err
}
sent = int(writeOffset) - int(start)
// Continue sending requests, opening a new stream and resending
// any bytes not yet persisted as per QueryWriteStatus
continue
}
}
if err != nil {
return nil, 0, false, err
return nil, 0, err
}
// Update the immediate stream's sent total and the upload offset with
// the data sent.
sent += len(data)
offset += int64(len(data))
writeOffset += int64(len(data))
// Not done sending data, do not attempt to commit it yet, loop around
// and send more data.
@ -1692,31 +1717,81 @@ func (w *gRPCWriter) uploadBuffer(recvd int, start int64, doneReading bool) (*st
}
// The buffer has been uploaded and there is still more data to be
// uploaded, but this is not a resumable upload session. Therefore
// keep the stream open and don't commit yet.
if !finishWrite && w.chunkSize == 0 {
return nil, offset, false, nil
// uploaded, but this is not a resumable upload session. Therefore,
// don't check persisted data.
if !lastWriteOfEntireObject && w.chunkSize == 0 {
return nil, writeOffset, nil
}
// Done sending data. Close the stream to "commit" the data sent.
resp, finalized, err := w.commit()
// Done sending data (remainingDataFitsInSingleReq should == true if we
// reach this code). Receive from the stream to confirm the persisted data.
resp, err := w.stream.Recv()
// Retriable errors mean we should start over and attempt to
// resend the entire buffer via a new stream.
// If not retriable, falling through will return the error received
// from closing the stream.
if shouldRetry(err) {
sent = 0
finishWrite = false
offset, err = w.determineOffset(start)
if err == nil {
continue
writeOffset, err = w.determineOffset(start)
if err != nil {
return nil, 0, err
}
sent = int(writeOffset) - int(start)
// Drop the stream reference as a new one will need to be created.
w.stream = nil
continue
}
if err != nil {
return nil, 0, false, err
return nil, 0, err
}
return resp.GetResource(), offset, finalized, nil
// Confirm the persisted data if we have not finished uploading the object.
if !lastWriteOfEntireObject {
if resp.GetPersistedSize() != writeOffset {
// Retry if not all bytes were persisted.
writeOffset = resp.GetPersistedSize()
sent = int(writeOffset) - int(start)
continue
}
} else {
// If the object is done uploading, close the send stream to signal
// to the server that we are done sending so that we can receive
// from the stream without blocking.
err = w.stream.CloseSend()
if err != nil {
// CloseSend() retries the send internally. It never returns an
// error in the current implementation, but we check it anyway in
// case that it does in the future.
return nil, 0, err
}
// Stream receives do not block once send is closed, but we may not
// receive the response with the object right away; loop until we
// receive the object or error out.
var obj *storagepb.Object
for obj == nil {
resp, err := w.stream.Recv()
if err != nil {
return nil, 0, err
}
obj = resp.GetResource()
}
// Even though we received the object response, continue reading
// until we receive a non-nil error, to ensure the stream does not
// leak even if the context isn't cancelled. See:
// https://pkg.go.dev/google.golang.org/grpc#ClientConn.NewStream
for err == nil {
_, err = w.stream.Recv()
}
return obj, writeOffset, nil
}
return nil, writeOffset, nil
}
}
@ -1736,26 +1811,6 @@ func (w *gRPCWriter) determineOffset(offset int64) (int64, error) {
return offset, nil
}
// commit closes the stream to commit the data sent and potentially receive
// the finalized object if finished uploading. If the last request sent
// indicated that writing was finished, the Object will be finalized and
// returned. If not, then the Object will be nil, and the boolean returned will
// be false.
func (w *gRPCWriter) commit() (*storagepb.WriteObjectResponse, bool, error) {
finalized := true
resp, err := w.stream.CloseAndRecv()
if err == io.EOF {
// Closing a stream for a resumable upload finish_write = false results
// in an EOF which can be ignored, as we aren't done uploading yet.
finalized = false
err = nil
}
// Drop the stream reference as it has been closed.
w.stream = nil
return resp, finalized, err
}
// writeObjectSpec constructs a WriteObjectSpec proto using the Writer's
// ObjectAttrs and applies its Conditions. This is only used for gRPC.
func (w *gRPCWriter) writeObjectSpec() (*storagepb.WriteObjectSpec, error) {

View file

@ -289,12 +289,11 @@ func (it *HMACKeysIterator) fetch(pageSize int, pageToken string) (token string,
call = call.MaxResults(int64(pageSize))
}
ctx := it.ctx
var resp *raw.HmacKeysMetadata
err = run(it.ctx, func() error {
err = run(it.ctx, func(ctx context.Context) error {
resp, err = call.Context(ctx).Do()
return err
}, it.retry, true, setRetryHeaderHTTP(call))
}, it.retry, true)
if err != nil {
return "", err
}

View file

@ -148,11 +148,11 @@ func (c *httpStorageClient) GetServiceAccount(ctx context.Context, project strin
s := callSettings(c.settings, opts...)
call := c.raw.Projects.ServiceAccount.Get(project)
var res *raw.ServiceAccount
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
var err error
res, err = call.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(call))
}, s.retry, s.idempotent)
if err != nil {
return "", err
}
@ -182,14 +182,14 @@ func (c *httpStorageClient) CreateBucket(ctx context.Context, project, bucket st
req.PredefinedDefaultObjectAcl(attrs.PredefinedDefaultObjectACL)
}
var battrs *BucketAttrs
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
b, err := req.Context(ctx).Do()
if err != nil {
return err
}
battrs, err = newBucket(b)
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
return battrs, err
}
@ -210,10 +210,10 @@ func (c *httpStorageClient) ListBuckets(ctx context.Context, project string, opt
req.MaxResults(int64(pageSize))
}
var resp *raw.Buckets
err = run(it.ctx, func() error {
resp, err = req.Context(it.ctx).Do()
err = run(it.ctx, func(ctx context.Context) error {
resp, err = req.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
if err != nil {
return "", err
}
@ -248,7 +248,7 @@ func (c *httpStorageClient) DeleteBucket(ctx context.Context, bucket string, con
req.UserProject(s.userProject)
}
return run(ctx, func() error { return req.Context(ctx).Do() }, s.retry, s.idempotent, setRetryHeaderHTTP(req))
return run(ctx, func(ctx context.Context) error { return req.Context(ctx).Do() }, s.retry, s.idempotent)
}
func (c *httpStorageClient) GetBucket(ctx context.Context, bucket string, conds *BucketConditions, opts ...storageOption) (*BucketAttrs, error) {
@ -264,10 +264,10 @@ func (c *httpStorageClient) GetBucket(ctx context.Context, bucket string, conds
}
var resp *raw.Bucket
err = run(ctx, func() error {
err = run(ctx, func(ctx context.Context) error {
resp, err = req.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
var e *googleapi.Error
if ok := errors.As(err, &e); ok && e.Code == http.StatusNotFound {
@ -298,10 +298,10 @@ func (c *httpStorageClient) UpdateBucket(ctx context.Context, bucket string, uat
}
var rawBucket *raw.Bucket
err = run(ctx, func() error {
err = run(ctx, func(ctx context.Context) error {
rawBucket, err = req.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -317,10 +317,10 @@ func (c *httpStorageClient) LockBucketRetentionPolicy(ctx context.Context, bucke
}
req := c.raw.Buckets.LockRetentionPolicy(bucket, metageneration)
return run(ctx, func() error {
return run(ctx, func(ctx context.Context) error {
_, err := req.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
}
func (c *httpStorageClient) ListObjects(ctx context.Context, bucket string, q *Query, opts ...storageOption) *ObjectIterator {
s := callSettings(c.settings, opts...)
@ -357,10 +357,10 @@ func (c *httpStorageClient) ListObjects(ctx context.Context, bucket string, q *Q
}
var resp *raw.Objects
var err error
err = run(it.ctx, func() error {
resp, err = req.Context(it.ctx).Do()
err = run(it.ctx, func(ctx context.Context) error {
resp, err = req.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
if err != nil {
var e *googleapi.Error
if ok := errors.As(err, &e); ok && e.Code == http.StatusNotFound {
@ -395,7 +395,7 @@ func (c *httpStorageClient) DeleteObject(ctx context.Context, bucket, object str
if s.userProject != "" {
req.UserProject(s.userProject)
}
err := run(ctx, func() error { return req.Context(ctx).Do() }, s.retry, s.idempotent, setRetryHeaderHTTP(req))
err := run(ctx, func(ctx context.Context) error { return req.Context(ctx).Do() }, s.retry, s.idempotent)
var e *googleapi.Error
if ok := errors.As(err, &e); ok && e.Code == http.StatusNotFound {
return ErrObjectNotExist
@ -417,10 +417,10 @@ func (c *httpStorageClient) GetObject(ctx context.Context, bucket, object string
}
var obj *raw.Object
var err error
err = run(ctx, func() error {
err = run(ctx, func(ctx context.Context) error {
obj, err = req.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
var e *googleapi.Error
if ok := errors.As(err, &e); ok && e.Code == http.StatusNotFound {
return nil, ErrObjectNotExist
@ -499,7 +499,7 @@ func (c *httpStorageClient) UpdateObject(ctx context.Context, bucket, object str
rawObj := attrs.toRawObject(bucket)
rawObj.ForceSendFields = forceSendFields
rawObj.NullFields = nullFields
call := c.raw.Objects.Patch(bucket, object, rawObj).Projection("full").Context(ctx)
call := c.raw.Objects.Patch(bucket, object, rawObj).Projection("full")
if err := applyConds("Update", gen, conds, call); err != nil {
return nil, err
}
@ -514,7 +514,7 @@ func (c *httpStorageClient) UpdateObject(ctx context.Context, bucket, object str
}
var obj *raw.Object
var err error
err = run(ctx, func() error { obj, err = call.Do(); return err }, s.retry, s.idempotent, setRetryHeaderHTTP(call))
err = run(ctx, func(ctx context.Context) error { obj, err = call.Context(ctx).Do(); return err }, s.retry, s.idempotent)
var e *googleapi.Error
if errors.As(err, &e) && e.Code == http.StatusNotFound {
return nil, ErrObjectNotExist
@ -531,7 +531,7 @@ func (c *httpStorageClient) DeleteDefaultObjectACL(ctx context.Context, bucket s
s := callSettings(c.settings, opts...)
req := c.raw.DefaultObjectAccessControls.Delete(bucket, string(entity))
configureACLCall(ctx, s.userProject, req)
return run(ctx, func() error { return req.Context(ctx).Do() }, s.retry, s.idempotent, setRetryHeaderHTTP(req))
return run(ctx, func(ctx context.Context) error { return req.Context(ctx).Do() }, s.retry, s.idempotent)
}
func (c *httpStorageClient) ListDefaultObjectACLs(ctx context.Context, bucket string, opts ...storageOption) ([]ACLRule, error) {
@ -540,10 +540,10 @@ func (c *httpStorageClient) ListDefaultObjectACLs(ctx context.Context, bucket st
var err error
req := c.raw.DefaultObjectAccessControls.List(bucket)
configureACLCall(ctx, s.userProject, req)
err = run(ctx, func() error {
acls, err = req.Do()
err = run(ctx, func(ctx context.Context) error {
acls, err = req.Context(ctx).Do()
return err
}, s.retry, true, setRetryHeaderHTTP(req))
}, s.retry, true)
if err != nil {
return nil, err
}
@ -560,14 +560,13 @@ func (c *httpStorageClient) UpdateDefaultObjectACL(ctx context.Context, bucket s
Entity: string(entity),
Role: string(role),
}
var req setRequest
var err error
req = c.raw.DefaultObjectAccessControls.Update(bucket, string(entity), acl)
req := c.raw.DefaultObjectAccessControls.Update(bucket, string(entity), acl)
configureACLCall(ctx, s.userProject, req)
return run(ctx, func() error {
_, err = req.Do()
return run(ctx, func(ctx context.Context) error {
_, err = req.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
}
// Bucket ACL methods.
@ -576,7 +575,7 @@ func (c *httpStorageClient) DeleteBucketACL(ctx context.Context, bucket string,
s := callSettings(c.settings, opts...)
req := c.raw.BucketAccessControls.Delete(bucket, string(entity))
configureACLCall(ctx, s.userProject, req)
return run(ctx, func() error { return req.Context(ctx).Do() }, s.retry, s.idempotent, setRetryHeaderHTTP(req))
return run(ctx, func(ctx context.Context) error { return req.Context(ctx).Do() }, s.retry, s.idempotent)
}
func (c *httpStorageClient) ListBucketACLs(ctx context.Context, bucket string, opts ...storageOption) ([]ACLRule, error) {
@ -585,10 +584,10 @@ func (c *httpStorageClient) ListBucketACLs(ctx context.Context, bucket string, o
var err error
req := c.raw.BucketAccessControls.List(bucket)
configureACLCall(ctx, s.userProject, req)
err = run(ctx, func() error {
acls, err = req.Do()
err = run(ctx, func(ctx context.Context) error {
acls, err = req.Context(ctx).Do()
return err
}, s.retry, true, setRetryHeaderHTTP(req))
}, s.retry, true)
if err != nil {
return nil, err
}
@ -605,10 +604,10 @@ func (c *httpStorageClient) UpdateBucketACL(ctx context.Context, bucket string,
req := c.raw.BucketAccessControls.Update(bucket, string(entity), acl)
configureACLCall(ctx, s.userProject, req)
var err error
return run(ctx, func() error {
_, err = req.Do()
return run(ctx, func(ctx context.Context) error {
_, err = req.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
}
// configureACLCall sets the context, user project and headers on the apiary library call.
@ -628,7 +627,7 @@ func (c *httpStorageClient) DeleteObjectACL(ctx context.Context, bucket, object
s := callSettings(c.settings, opts...)
req := c.raw.ObjectAccessControls.Delete(bucket, object, string(entity))
configureACLCall(ctx, s.userProject, req)
return run(ctx, func() error { return req.Context(ctx).Do() }, s.retry, s.idempotent, setRetryHeaderHTTP(req))
return run(ctx, func(ctx context.Context) error { return req.Context(ctx).Do() }, s.retry, s.idempotent)
}
// ListObjectACLs retrieves object ACL entries. By default, it operates on the latest generation of this object.
@ -639,10 +638,10 @@ func (c *httpStorageClient) ListObjectACLs(ctx context.Context, bucket, object s
var err error
req := c.raw.ObjectAccessControls.List(bucket, object)
configureACLCall(ctx, s.userProject, req)
err = run(ctx, func() error {
acls, err = req.Do()
err = run(ctx, func(ctx context.Context) error {
acls, err = req.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -661,14 +660,13 @@ func (c *httpStorageClient) UpdateObjectACL(ctx context.Context, bucket, object
Entity: string(entity),
Role: string(role),
}
var req setRequest
var err error
req = c.raw.ObjectAccessControls.Update(bucket, object, string(entity), acl)
req := c.raw.ObjectAccessControls.Update(bucket, object, string(entity), acl)
configureACLCall(ctx, s.userProject, req)
return run(ctx, func() error {
_, err = req.Do()
return run(ctx, func(ctx context.Context) error {
_, err = req.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(req))
}, s.retry, s.idempotent)
}
// Media operations.
@ -692,7 +690,7 @@ func (c *httpStorageClient) ComposeObject(ctx context.Context, req *composeObjec
rawReq.SourceObjects = append(rawReq.SourceObjects, srcObj)
}
call := c.raw.Objects.Compose(req.dstBucket, req.dstObject.name, rawReq).Context(ctx)
call := c.raw.Objects.Compose(req.dstBucket, req.dstObject.name, rawReq)
if err := applyConds("ComposeFrom destination", defaultGen, req.dstObject.conds, call); err != nil {
return nil, err
}
@ -709,9 +707,9 @@ func (c *httpStorageClient) ComposeObject(ctx context.Context, req *composeObjec
setClientHeader(call.Header())
var err error
retryCall := func() error { obj, err = call.Do(); return err }
retryCall := func(ctx context.Context) error { obj, err = call.Context(ctx).Do(); return err }
if err := run(ctx, retryCall, s.retry, s.idempotent, setRetryHeaderHTTP(call)); err != nil {
if err := run(ctx, retryCall, s.retry, s.idempotent); err != nil {
return nil, err
}
return newObject(obj), nil
@ -721,7 +719,7 @@ func (c *httpStorageClient) RewriteObject(ctx context.Context, req *rewriteObjec
rawObject := req.dstObject.attrs.toRawObject("")
call := c.raw.Objects.Rewrite(req.srcObject.bucket, req.srcObject.name, req.dstObject.bucket, req.dstObject.name, rawObject)
call.Context(ctx).Projection("full")
call.Projection("full")
if req.token != "" {
call.RewriteToken(req.token)
}
@ -757,9 +755,9 @@ func (c *httpStorageClient) RewriteObject(ctx context.Context, req *rewriteObjec
var err error
setClientHeader(call.Header())
retryCall := func() error { res, err = call.Do(); return err }
retryCall := func(ctx context.Context) error { res, err = call.Context(ctx).Do(); return err }
if err := run(ctx, retryCall, s.retry, s.idempotent, setRetryHeaderHTTP(call)); err != nil {
if err := run(ctx, retryCall, s.retry, s.idempotent); err != nil {
return nil, err
}
@ -801,7 +799,6 @@ func (c *httpStorageClient) newRangeReaderXML(ctx context.Context, params *newRa
if err != nil {
return nil, err
}
req = req.WithContext(ctx)
if s.userProject != "" {
req.Header.Set("X-Goog-User-Project", s.userProject)
@ -821,7 +818,7 @@ func (c *httpStorageClient) newRangeReaderXML(ctx context.Context, params *newRa
}
reopen := readerReopen(ctx, req.Header, params, s,
func() (*http.Response, error) { return c.hc.Do(req) },
func(ctx context.Context) (*http.Response, error) { return c.hc.Do(req.WithContext(ctx)) },
func() error { return setConditionsHeaders(req.Header, params.conds) },
func() { req.URL.RawQuery = fmt.Sprintf("generation=%d", params.gen) })
@ -836,7 +833,6 @@ func (c *httpStorageClient) newRangeReaderJSON(ctx context.Context, params *newR
call := c.raw.Objects.Get(params.bucket, params.object)
setClientHeader(call.Header())
call.Context(ctx)
call.Projection("full")
if s.userProject != "" {
@ -847,7 +843,7 @@ func (c *httpStorageClient) newRangeReaderJSON(ctx context.Context, params *newR
return nil, err
}
reopen := readerReopen(ctx, call.Header(), params, s, func() (*http.Response, error) { return call.Download() },
reopen := readerReopen(ctx, call.Header(), params, s, func(ctx context.Context) (*http.Response, error) { return call.Context(ctx).Download() },
func() error { return applyConds("NewReader", params.gen, params.conds, call) },
func() { call.Generation(params.gen) })
@ -957,11 +953,11 @@ func (c *httpStorageClient) GetIamPolicy(ctx context.Context, resource string, v
call.UserProject(s.userProject)
}
var rp *raw.Policy
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
var err error
rp, err = call.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(call))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -978,10 +974,10 @@ func (c *httpStorageClient) SetIamPolicy(ctx context.Context, resource string, p
call.UserProject(s.userProject)
}
return run(ctx, func() error {
return run(ctx, func(ctx context.Context) error {
_, err := call.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(call))
}, s.retry, s.idempotent)
}
func (c *httpStorageClient) TestIamPermissions(ctx context.Context, resource string, permissions []string, opts ...storageOption) ([]string, error) {
@ -992,11 +988,11 @@ func (c *httpStorageClient) TestIamPermissions(ctx context.Context, resource str
call.UserProject(s.userProject)
}
var res *raw.TestIamPermissionsResponse
err := run(ctx, func() error {
err := run(ctx, func(ctx context.Context) error {
var err error
res, err = call.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(call))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -1014,10 +1010,10 @@ func (c *httpStorageClient) GetHMACKey(ctx context.Context, project, accessID st
var metadata *raw.HmacKeyMetadata
var err error
if err := run(ctx, func() error {
if err := run(ctx, func(ctx context.Context) error {
metadata, err = call.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(call)); err != nil {
}, s.retry, s.idempotent); err != nil {
return nil, err
}
hk := &raw.HmacKey{
@ -1054,10 +1050,10 @@ func (c *httpStorageClient) ListHMACKeys(ctx context.Context, project, serviceAc
}
var resp *raw.HmacKeysMetadata
err = run(it.ctx, func() error {
resp, err = call.Context(it.ctx).Do()
err = run(it.ctx, func(ctx context.Context) error {
resp, err = call.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(call))
}, s.retry, s.idempotent)
if err != nil {
return "", err
}
@ -1099,10 +1095,10 @@ func (c *httpStorageClient) UpdateHMACKey(ctx context.Context, project, serviceA
var metadata *raw.HmacKeyMetadata
var err error
if err := run(ctx, func() error {
if err := run(ctx, func(ctx context.Context) error {
metadata, err = call.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(call)); err != nil {
}, s.retry, s.idempotent); err != nil {
return nil, err
}
hk := &raw.HmacKey{
@ -1119,11 +1115,11 @@ func (c *httpStorageClient) CreateHMACKey(ctx context.Context, project, serviceA
}
var hk *raw.HmacKey
if err := run(ctx, func() error {
if err := run(ctx, func(ctx context.Context) error {
h, err := call.Context(ctx).Do()
hk = h
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(call)); err != nil {
}, s.retry, s.idempotent); err != nil {
return nil, err
}
return toHMACKeyFromRaw(hk, true)
@ -1135,9 +1131,9 @@ func (c *httpStorageClient) DeleteHMACKey(ctx context.Context, project string, a
if s.userProject != "" {
call = call.UserProject(s.userProject)
}
return run(ctx, func() error {
return run(ctx, func(ctx context.Context) error {
return call.Context(ctx).Do()
}, s.retry, s.idempotent, setRetryHeaderHTTP(call))
}, s.retry, s.idempotent)
}
// Notification methods.
@ -1156,10 +1152,10 @@ func (c *httpStorageClient) ListNotifications(ctx context.Context, bucket string
call.UserProject(s.userProject)
}
var res *raw.Notifications
err = run(ctx, func() error {
err = run(ctx, func(ctx context.Context) error {
res, err = call.Context(ctx).Do()
return err
}, s.retry, true, setRetryHeaderHTTP(call))
}, s.retry, true)
if err != nil {
return nil, err
}
@ -1176,10 +1172,10 @@ func (c *httpStorageClient) CreateNotification(ctx context.Context, bucket strin
call.UserProject(s.userProject)
}
var rn *raw.Notification
err = run(ctx, func() error {
err = run(ctx, func(ctx context.Context) error {
rn, err = call.Context(ctx).Do()
return err
}, s.retry, s.idempotent, setRetryHeaderHTTP(call))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}
@ -1195,9 +1191,9 @@ func (c *httpStorageClient) DeleteNotification(ctx context.Context, bucket strin
if s.userProject != "" {
call.UserProject(s.userProject)
}
return run(ctx, func() error {
return run(ctx, func(ctx context.Context) error {
return call.Context(ctx).Do()
}, s.retry, s.idempotent, setRetryHeaderHTTP(call))
}, s.retry, s.idempotent)
}
type httpReader struct {
@ -1246,7 +1242,7 @@ func setRangeReaderHeaders(h http.Header, params *newRangeReaderParams) error {
// readerReopen initiates a Read with offset and length, assuming we
// have already read seen bytes.
func readerReopen(ctx context.Context, header http.Header, params *newRangeReaderParams, s *settings,
doDownload func() (*http.Response, error), applyConditions func() error, setGeneration func()) func(int64) (*http.Response, error) {
doDownload func(context.Context) (*http.Response, error), applyConditions func() error, setGeneration func()) func(int64) (*http.Response, error) {
return func(seen int64) (*http.Response, error) {
// If the context has already expired, return immediately without making a
// call.
@ -1273,8 +1269,8 @@ func readerReopen(ctx context.Context, header http.Header, params *newRangeReade
var err error
var res *http.Response
err = run(ctx, func() error {
res, err = doDownload()
err = run(ctx, func(ctx context.Context) error {
res, err = doDownload(ctx)
if err != nil {
var e *googleapi.Error
if errors.As(err, &e) {
@ -1328,7 +1324,7 @@ func readerReopen(ctx context.Context, header http.Header, params *newRangeReade
params.gen = gen64
}
return nil
}, s.retry, s.idempotent, setRetryHeaderHTTP(nil))
}, s.retry, s.idempotent)
if err != nil {
return nil, err
}

View file

@ -0,0 +1,210 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Code generated by protoc-gen-go_gapic. DO NOT EDIT.
package storage
import (
storagepb "cloud.google.com/go/storage/internal/apiv2/storagepb"
"google.golang.org/api/iterator"
)
// BucketIterator manages a stream of *storagepb.Bucket.
type BucketIterator struct {
items []*storagepb.Bucket
pageInfo *iterator.PageInfo
nextFunc func() error
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response interface{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch func(pageSize int, pageToken string) (results []*storagepb.Bucket, nextPageToken string, err error)
}
// PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
func (it *BucketIterator) PageInfo() *iterator.PageInfo {
return it.pageInfo
}
// Next returns the next result. Its second return value is iterator.Done if there are no more
// results. Once Next returns Done, all subsequent calls will return Done.
func (it *BucketIterator) Next() (*storagepb.Bucket, error) {
var item *storagepb.Bucket
if err := it.nextFunc(); err != nil {
return item, err
}
item = it.items[0]
it.items = it.items[1:]
return item, nil
}
func (it *BucketIterator) bufLen() int {
return len(it.items)
}
func (it *BucketIterator) takeBuf() interface{} {
b := it.items
it.items = nil
return b
}
// HmacKeyMetadataIterator manages a stream of *storagepb.HmacKeyMetadata.
type HmacKeyMetadataIterator struct {
items []*storagepb.HmacKeyMetadata
pageInfo *iterator.PageInfo
nextFunc func() error
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response interface{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch func(pageSize int, pageToken string) (results []*storagepb.HmacKeyMetadata, nextPageToken string, err error)
}
// PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
func (it *HmacKeyMetadataIterator) PageInfo() *iterator.PageInfo {
return it.pageInfo
}
// Next returns the next result. Its second return value is iterator.Done if there are no more
// results. Once Next returns Done, all subsequent calls will return Done.
func (it *HmacKeyMetadataIterator) Next() (*storagepb.HmacKeyMetadata, error) {
var item *storagepb.HmacKeyMetadata
if err := it.nextFunc(); err != nil {
return item, err
}
item = it.items[0]
it.items = it.items[1:]
return item, nil
}
func (it *HmacKeyMetadataIterator) bufLen() int {
return len(it.items)
}
func (it *HmacKeyMetadataIterator) takeBuf() interface{} {
b := it.items
it.items = nil
return b
}
// NotificationConfigIterator manages a stream of *storagepb.NotificationConfig.
type NotificationConfigIterator struct {
items []*storagepb.NotificationConfig
pageInfo *iterator.PageInfo
nextFunc func() error
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response interface{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch func(pageSize int, pageToken string) (results []*storagepb.NotificationConfig, nextPageToken string, err error)
}
// PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
func (it *NotificationConfigIterator) PageInfo() *iterator.PageInfo {
return it.pageInfo
}
// Next returns the next result. Its second return value is iterator.Done if there are no more
// results. Once Next returns Done, all subsequent calls will return Done.
func (it *NotificationConfigIterator) Next() (*storagepb.NotificationConfig, error) {
var item *storagepb.NotificationConfig
if err := it.nextFunc(); err != nil {
return item, err
}
item = it.items[0]
it.items = it.items[1:]
return item, nil
}
func (it *NotificationConfigIterator) bufLen() int {
return len(it.items)
}
func (it *NotificationConfigIterator) takeBuf() interface{} {
b := it.items
it.items = nil
return b
}
// ObjectIterator manages a stream of *storagepb.Object.
type ObjectIterator struct {
items []*storagepb.Object
pageInfo *iterator.PageInfo
nextFunc func() error
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response interface{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch func(pageSize int, pageToken string) (results []*storagepb.Object, nextPageToken string, err error)
}
// PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
func (it *ObjectIterator) PageInfo() *iterator.PageInfo {
return it.pageInfo
}
// Next returns the next result. Its second return value is iterator.Done if there are no more
// results. Once Next returns Done, all subsequent calls will return Done.
func (it *ObjectIterator) Next() (*storagepb.Object, error) {
var item *storagepb.Object
if err := it.nextFunc(); err != nil {
return item, err
}
item = it.items[0]
it.items = it.items[1:]
return item, nil
}
func (it *ObjectIterator) bufLen() int {
return len(it.items)
}
func (it *ObjectIterator) takeBuf() interface{} {
b := it.items
it.items = nil
return b
}

View file

@ -17,7 +17,17 @@
// Package storage is an auto-generated package for the
// Cloud Storage API.
//
// Lets you store and retrieve potentially-large, immutable data objects.
// Stop. This folder is likely not what you are looking for. This folder
// contains protocol buffer definitions for an unreleased API for accessing
// Cloud Storage. Unless told otherwise by a Google Cloud representative, do
// not use any of the contents of this folder. If you would like to use Cloud
// Storage, please consult our official documentation (at
// https://cloud.google.com/storage/docs/apis) for details on our XML and
// JSON APIs, or else consider one of our client libraries (at
// https://cloud.google.com/storage/docs/reference/libraries). This API
// defined in this folder is unreleased and may shut off, break, or fail at
// any time for any users who are not registered as a part of a private
// preview program.
//
// # General documentation
//
@ -66,15 +76,32 @@
// // TODO: Handle error.
// }
// defer c.Close()
//
// req := &storagepb.DeleteBucketRequest{
// // TODO: Fill request struct fields.
// // See https://pkg.go.dev/cloud.google.com/go/storage/internal/apiv2/storagepb#DeleteBucketRequest.
// }
// err = c.DeleteBucket(ctx, req)
// stream, err := c.BidiWriteObject(ctx)
// if err != nil {
// // TODO: Handle error.
// }
// go func() {
// reqs := []*storagepb.BidiWriteObjectRequest{
// // TODO: Create requests.
// }
// for _, req := range reqs {
// if err := stream.Send(req); err != nil {
// // TODO: Handle error.
// }
// }
// stream.CloseSend()
// }()
// for {
// resp, err := stream.Recv()
// if err == io.EOF {
// break
// }
// if err != nil {
// // TODO: handle error.
// }
// // TODO: Use resp.
// _ = resp
// }
//
// # Use of Context
//

View file

@ -10,6 +10,11 @@
"grpc": {
"libraryClient": "Client",
"rpcs": {
"BidiWriteObject": {
"methods": [
"BidiWriteObject"
]
},
"CancelResumableWrite": {
"methods": [
"CancelResumableWrite"
@ -120,6 +125,11 @@
"ReadObject"
]
},
"RestoreObject": {
"methods": [
"RestoreObject"
]
},
"RewriteObject": {
"methods": [
"RewriteObject"

View file

@ -56,11 +56,13 @@ type CallOptions struct {
ListNotificationConfigs []gax.CallOption
ComposeObject []gax.CallOption
DeleteObject []gax.CallOption
RestoreObject []gax.CallOption
CancelResumableWrite []gax.CallOption
GetObject []gax.CallOption
ReadObject []gax.CallOption
UpdateObject []gax.CallOption
WriteObject []gax.CallOption
BidiWriteObject []gax.CallOption
ListObjects []gax.CallOption
RewriteObject []gax.CallOption
StartResumableWrite []gax.CallOption
@ -282,6 +284,19 @@ func defaultCallOptions() *CallOptions {
})
}),
},
RestoreObject: []gax.CallOption{
gax.WithTimeout(60000 * time.Millisecond),
gax.WithRetry(func() gax.Retryer {
return gax.OnCodes([]codes.Code{
codes.DeadlineExceeded,
codes.Unavailable,
}, gax.Backoff{
Initial: 1000 * time.Millisecond,
Max: 60000 * time.Millisecond,
Multiplier: 2.00,
})
}),
},
CancelResumableWrite: []gax.CallOption{
gax.WithTimeout(60000 * time.Millisecond),
gax.WithRetry(func() gax.Retryer {
@ -345,6 +360,18 @@ func defaultCallOptions() *CallOptions {
})
}),
},
BidiWriteObject: []gax.CallOption{
gax.WithRetry(func() gax.Retryer {
return gax.OnCodes([]codes.Code{
codes.DeadlineExceeded,
codes.Unavailable,
}, gax.Backoff{
Initial: 1000 * time.Millisecond,
Max: 60000 * time.Millisecond,
Multiplier: 2.00,
})
}),
},
ListObjects: []gax.CallOption{
gax.WithTimeout(60000 * time.Millisecond),
gax.WithRetry(func() gax.Retryer {
@ -498,11 +525,13 @@ type internalClient interface {
ListNotificationConfigs(context.Context, *storagepb.ListNotificationConfigsRequest, ...gax.CallOption) *NotificationConfigIterator
ComposeObject(context.Context, *storagepb.ComposeObjectRequest, ...gax.CallOption) (*storagepb.Object, error)
DeleteObject(context.Context, *storagepb.DeleteObjectRequest, ...gax.CallOption) error
RestoreObject(context.Context, *storagepb.RestoreObjectRequest, ...gax.CallOption) (*storagepb.Object, error)
CancelResumableWrite(context.Context, *storagepb.CancelResumableWriteRequest, ...gax.CallOption) (*storagepb.CancelResumableWriteResponse, error)
GetObject(context.Context, *storagepb.GetObjectRequest, ...gax.CallOption) (*storagepb.Object, error)
ReadObject(context.Context, *storagepb.ReadObjectRequest, ...gax.CallOption) (storagepb.Storage_ReadObjectClient, error)
UpdateObject(context.Context, *storagepb.UpdateObjectRequest, ...gax.CallOption) (*storagepb.Object, error)
WriteObject(context.Context, ...gax.CallOption) (storagepb.Storage_WriteObjectClient, error)
BidiWriteObject(context.Context, ...gax.CallOption) (storagepb.Storage_BidiWriteObjectClient, error)
ListObjects(context.Context, *storagepb.ListObjectsRequest, ...gax.CallOption) *ObjectIterator
RewriteObject(context.Context, *storagepb.RewriteObjectRequest, ...gax.CallOption) (*storagepb.RewriteResponse, error)
StartResumableWrite(context.Context, *storagepb.StartResumableWriteRequest, ...gax.CallOption) (*storagepb.StartResumableWriteResponse, error)
@ -598,16 +627,16 @@ func (c *Client) LockBucketRetentionPolicy(ctx context.Context, req *storagepb.L
// GetIamPolicy gets the IAM policy for a specified bucket or object.
// The resource field in the request should be
// projects//buckets/<bucket_name> for a bucket or
// projects//buckets/<bucket_name>/objects/<object_name> for an object.
// projects/_/buckets/{bucket} for a bucket or
// projects/_/buckets/{bucket}/objects/{object} for an object.
func (c *Client) GetIamPolicy(ctx context.Context, req *iampb.GetIamPolicyRequest, opts ...gax.CallOption) (*iampb.Policy, error) {
return c.internalClient.GetIamPolicy(ctx, req, opts...)
}
// SetIamPolicy updates an IAM policy for the specified bucket or object.
// The resource field in the request should be
// projects//buckets/<bucket_name> for a bucket or
// projects//buckets/<bucket_name>/objects/<object_name> for an object.
// projects/_/buckets/{bucket} for a bucket or
// projects/_/buckets/{bucket}/objects/{object} for an object.
func (c *Client) SetIamPolicy(ctx context.Context, req *iampb.SetIamPolicyRequest, opts ...gax.CallOption) (*iampb.Policy, error) {
return c.internalClient.SetIamPolicy(ctx, req, opts...)
}
@ -615,8 +644,8 @@ func (c *Client) SetIamPolicy(ctx context.Context, req *iampb.SetIamPolicyReques
// TestIamPermissions tests a set of permissions on the given bucket or object to see which, if
// any, are held by the caller.
// The resource field in the request should be
// projects//buckets/<bucket_name> for a bucket or
// projects//buckets/<bucket_name>/objects/<object_name> for an object.
// projects/_/buckets/{bucket} for a bucket or
// projects/_/buckets/{bucket}/objects/{object} for an object.
func (c *Client) TestIamPermissions(ctx context.Context, req *iampb.TestIamPermissionsRequest, opts ...gax.CallOption) (*iampb.TestIamPermissionsResponse, error) {
return c.internalClient.TestIamPermissions(ctx, req, opts...)
}
@ -665,6 +694,11 @@ func (c *Client) DeleteObject(ctx context.Context, req *storagepb.DeleteObjectRe
return c.internalClient.DeleteObject(ctx, req, opts...)
}
// RestoreObject restores a soft-deleted object.
func (c *Client) RestoreObject(ctx context.Context, req *storagepb.RestoreObjectRequest, opts ...gax.CallOption) (*storagepb.Object, error) {
return c.internalClient.RestoreObject(ctx, req, opts...)
}
// CancelResumableWrite cancels an in-progress resumable upload.
//
// Any attempts to write to the resumable upload after cancelling the upload
@ -752,10 +786,33 @@ func (c *Client) UpdateObject(ctx context.Context, req *storagepb.UpdateObjectRe
// Attempting to resume an already finalized object will result in an OK
// status, with a WriteObjectResponse containing the finalized objects
// metadata.
//
// Alternatively, the BidiWriteObject operation may be used to write an
// object with controls over flushing and the ability to fetch the ability to
// determine the current persisted size.
func (c *Client) WriteObject(ctx context.Context, opts ...gax.CallOption) (storagepb.Storage_WriteObjectClient, error) {
return c.internalClient.WriteObject(ctx, opts...)
}
// BidiWriteObject stores a new object and metadata.
//
// This is similar to the WriteObject call with the added support for
// manual flushing of persisted state, and the ability to determine current
// persisted size without closing the stream.
//
// The client may specify one or both of the state_lookup and flush fields
// in each BidiWriteObjectRequest. If flush is specified, the data written
// so far will be persisted to storage. If state_lookup is specified, the
// service will respond with a BidiWriteObjectResponse that contains the
// persisted size. If both flush and state_lookup are specified, the flush
// will always occur before a state_lookup, so that both may be set in the
// same request and the returned state will be the state of the object
// post-flush. When the stream is closed, a BidiWriteObjectResponse will
// always be sent to the client, regardless of the value of state_lookup.
func (c *Client) BidiWriteObject(ctx context.Context, opts ...gax.CallOption) (storagepb.Storage_BidiWriteObjectClient, error) {
return c.internalClient.BidiWriteObject(ctx, opts...)
}
// ListObjects retrieves a list of objects matching the criteria.
func (c *Client) ListObjects(ctx context.Context, req *storagepb.ListObjectsRequest, opts ...gax.CallOption) *ObjectIterator {
return c.internalClient.ListObjects(ctx, req, opts...)
@ -1375,6 +1432,33 @@ func (c *gRPCClient) DeleteObject(ctx context.Context, req *storagepb.DeleteObje
return err
}
func (c *gRPCClient) RestoreObject(ctx context.Context, req *storagepb.RestoreObjectRequest, opts ...gax.CallOption) (*storagepb.Object, error) {
routingHeaders := ""
routingHeadersMap := make(map[string]string)
if reg := regexp.MustCompile("(?P<bucket>.*)"); reg.MatchString(req.GetBucket()) && len(url.QueryEscape(reg.FindStringSubmatch(req.GetBucket())[1])) > 0 {
routingHeadersMap["bucket"] = url.QueryEscape(reg.FindStringSubmatch(req.GetBucket())[1])
}
for headerName, headerValue := range routingHeadersMap {
routingHeaders = fmt.Sprintf("%s%s=%s&", routingHeaders, headerName, headerValue)
}
routingHeaders = strings.TrimSuffix(routingHeaders, "&")
hds := []string{"x-goog-request-params", routingHeaders}
hds = append(c.xGoogHeaders, hds...)
ctx = gax.InsertMetadataIntoOutgoingContext(ctx, hds...)
opts = append((*c.CallOptions).RestoreObject[0:len((*c.CallOptions).RestoreObject):len((*c.CallOptions).RestoreObject)], opts...)
var resp *storagepb.Object
err := gax.Invoke(ctx, func(ctx context.Context, settings gax.CallSettings) error {
var err error
resp, err = c.client.RestoreObject(ctx, req, settings.GRPC...)
return err
}, opts...)
if err != nil {
return nil, err
}
return resp, nil
}
func (c *gRPCClient) CancelResumableWrite(ctx context.Context, req *storagepb.CancelResumableWriteRequest, opts ...gax.CallOption) (*storagepb.CancelResumableWriteResponse, error) {
routingHeaders := ""
routingHeadersMap := make(map[string]string)
@ -1498,6 +1582,21 @@ func (c *gRPCClient) WriteObject(ctx context.Context, opts ...gax.CallOption) (s
return resp, nil
}
func (c *gRPCClient) BidiWriteObject(ctx context.Context, opts ...gax.CallOption) (storagepb.Storage_BidiWriteObjectClient, error) {
ctx = gax.InsertMetadataIntoOutgoingContext(ctx, c.xGoogHeaders...)
var resp storagepb.Storage_BidiWriteObjectClient
opts = append((*c.CallOptions).BidiWriteObject[0:len((*c.CallOptions).BidiWriteObject):len((*c.CallOptions).BidiWriteObject)], opts...)
err := gax.Invoke(ctx, func(ctx context.Context, settings gax.CallSettings) error {
var err error
resp, err = c.client.BidiWriteObject(ctx, settings.GRPC...)
return err
}, opts...)
if err != nil {
return nil, err
}
return resp, nil
}
func (c *gRPCClient) ListObjects(ctx context.Context, req *storagepb.ListObjectsRequest, opts ...gax.CallOption) *ObjectIterator {
routingHeaders := ""
routingHeadersMap := make(map[string]string)
@ -1822,191 +1921,3 @@ func (c *gRPCClient) UpdateHmacKey(ctx context.Context, req *storagepb.UpdateHma
}
return resp, nil
}
// BucketIterator manages a stream of *storagepb.Bucket.
type BucketIterator struct {
items []*storagepb.Bucket
pageInfo *iterator.PageInfo
nextFunc func() error
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response interface{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch func(pageSize int, pageToken string) (results []*storagepb.Bucket, nextPageToken string, err error)
}
// PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
func (it *BucketIterator) PageInfo() *iterator.PageInfo {
return it.pageInfo
}
// Next returns the next result. Its second return value is iterator.Done if there are no more
// results. Once Next returns Done, all subsequent calls will return Done.
func (it *BucketIterator) Next() (*storagepb.Bucket, error) {
var item *storagepb.Bucket
if err := it.nextFunc(); err != nil {
return item, err
}
item = it.items[0]
it.items = it.items[1:]
return item, nil
}
func (it *BucketIterator) bufLen() int {
return len(it.items)
}
func (it *BucketIterator) takeBuf() interface{} {
b := it.items
it.items = nil
return b
}
// HmacKeyMetadataIterator manages a stream of *storagepb.HmacKeyMetadata.
type HmacKeyMetadataIterator struct {
items []*storagepb.HmacKeyMetadata
pageInfo *iterator.PageInfo
nextFunc func() error
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response interface{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch func(pageSize int, pageToken string) (results []*storagepb.HmacKeyMetadata, nextPageToken string, err error)
}
// PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
func (it *HmacKeyMetadataIterator) PageInfo() *iterator.PageInfo {
return it.pageInfo
}
// Next returns the next result. Its second return value is iterator.Done if there are no more
// results. Once Next returns Done, all subsequent calls will return Done.
func (it *HmacKeyMetadataIterator) Next() (*storagepb.HmacKeyMetadata, error) {
var item *storagepb.HmacKeyMetadata
if err := it.nextFunc(); err != nil {
return item, err
}
item = it.items[0]
it.items = it.items[1:]
return item, nil
}
func (it *HmacKeyMetadataIterator) bufLen() int {
return len(it.items)
}
func (it *HmacKeyMetadataIterator) takeBuf() interface{} {
b := it.items
it.items = nil
return b
}
// NotificationConfigIterator manages a stream of *storagepb.NotificationConfig.
type NotificationConfigIterator struct {
items []*storagepb.NotificationConfig
pageInfo *iterator.PageInfo
nextFunc func() error
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response interface{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch func(pageSize int, pageToken string) (results []*storagepb.NotificationConfig, nextPageToken string, err error)
}
// PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
func (it *NotificationConfigIterator) PageInfo() *iterator.PageInfo {
return it.pageInfo
}
// Next returns the next result. Its second return value is iterator.Done if there are no more
// results. Once Next returns Done, all subsequent calls will return Done.
func (it *NotificationConfigIterator) Next() (*storagepb.NotificationConfig, error) {
var item *storagepb.NotificationConfig
if err := it.nextFunc(); err != nil {
return item, err
}
item = it.items[0]
it.items = it.items[1:]
return item, nil
}
func (it *NotificationConfigIterator) bufLen() int {
return len(it.items)
}
func (it *NotificationConfigIterator) takeBuf() interface{} {
b := it.items
it.items = nil
return b
}
// ObjectIterator manages a stream of *storagepb.Object.
type ObjectIterator struct {
items []*storagepb.Object
pageInfo *iterator.PageInfo
nextFunc func() error
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response interface{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch func(pageSize int, pageToken string) (results []*storagepb.Object, nextPageToken string, err error)
}
// PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
func (it *ObjectIterator) PageInfo() *iterator.PageInfo {
return it.pageInfo
}
// Next returns the next result. Its second return value is iterator.Done if there are no more
// results. Once Next returns Done, all subsequent calls will return Done.
func (it *ObjectIterator) Next() (*storagepb.Object, error) {
var item *storagepb.Object
if err := it.nextFunc(); err != nil {
return item, err
}
item = it.items[0]
it.items = it.items[1:]
return item, nil
}
func (it *ObjectIterator) bufLen() int {
return len(it.items)
}
func (it *ObjectIterator) takeBuf() interface{} {
b := it.items
it.items = nil
return b
}

File diff suppressed because it is too large Load diff

View file

@ -15,4 +15,4 @@
package internal
// Version is the current tagged release of the library.
const Version = "1.33.0"
const Version = "1.35.1"

View file

@ -20,7 +20,6 @@ import (
"fmt"
"io"
"net"
"net/http"
"net/url"
"strings"
@ -29,6 +28,7 @@ import (
sinternal "cloud.google.com/go/storage/internal"
"github.com/google/uuid"
gax "github.com/googleapis/gax-go/v2"
"github.com/googleapis/gax-go/v2/callctx"
"google.golang.org/api/googleapi"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
@ -37,10 +37,15 @@ import (
var defaultRetry *retryConfig = &retryConfig{}
var xGoogDefaultHeader = fmt.Sprintf("gl-go/%s gccl/%s", version.Go(), sinternal.Version)
const (
xGoogHeaderKey = "x-goog-api-client"
idempotencyHeaderKey = "x-goog-gcs-idempotency-token"
)
// run determines whether a retry is necessary based on the config and
// idempotency information. It then calls the function with or without retries
// as appropriate, using the configured settings.
func run(ctx context.Context, call func() error, retry *retryConfig, isIdempotent bool, setHeader func(string, int)) error {
func run(ctx context.Context, call func(ctx context.Context) error, retry *retryConfig, isIdempotent bool) error {
attempts := 1
invocationID := uuid.New().String()
@ -48,8 +53,8 @@ func run(ctx context.Context, call func() error, retry *retryConfig, isIdempoten
retry = defaultRetry
}
if (retry.policy == RetryIdempotent && !isIdempotent) || retry.policy == RetryNever {
setHeader(invocationID, attempts)
return call()
ctxWithHeaders := setInvocationHeaders(ctx, invocationID, attempts)
return call(ctxWithHeaders)
}
bo := gax.Backoff{}
if retry.backoff != nil {
@ -63,35 +68,22 @@ func run(ctx context.Context, call func() error, retry *retryConfig, isIdempoten
}
return internal.Retry(ctx, bo, func() (stop bool, err error) {
setHeader(invocationID, attempts)
err = call()
ctxWithHeaders := setInvocationHeaders(ctx, invocationID, attempts)
err = call(ctxWithHeaders)
attempts++
return !errorFunc(err), err
})
}
func setRetryHeaderHTTP(req interface{ Header() http.Header }) func(string, int) {
return func(invocationID string, attempts int) {
if req == nil {
return
}
header := req.Header()
// TODO(b/274504690): Consider dropping gccl-invocation-id key since it
// duplicates the X-Goog-Gcs-Idempotency-Token header (added in v1.31.0).
invocationHeader := fmt.Sprintf("gccl-invocation-id/%v gccl-attempt-count/%v", invocationID, attempts)
xGoogHeader := strings.Join([]string{invocationHeader, xGoogDefaultHeader}, " ")
header.Set("x-goog-api-client", xGoogHeader)
// Also use the invocationID for the idempotency token header, which will
// enable idempotent retries for more operations.
header.Set("x-goog-gcs-idempotency-token", invocationID)
}
}
// Sets invocation ID headers on the context which will be propagated as
// headers in the call to the service (for both gRPC and HTTP).
func setInvocationHeaders(ctx context.Context, invocationID string, attempts int) context.Context {
invocationHeader := fmt.Sprintf("gccl-invocation-id/%v gccl-attempt-count/%v", invocationID, attempts)
xGoogHeader := strings.Join([]string{invocationHeader, xGoogDefaultHeader}, " ")
// TODO: Implement method setting header via context for gRPC
func setRetryHeaderGRPC(_ context.Context) func(string, int) {
return func(_ string, _ int) {
return
}
ctx = callctx.SetHeaders(ctx, xGoogHeaderKey, xGoogHeader)
ctx = callctx.SetHeaders(ctx, idempotencyHeaderKey, invocationID)
return ctx
}
// ShouldRetry returns true if an error is retryable, based on best practice

View file

@ -187,16 +187,6 @@ func setConditionsHeaders(headers http.Header, conds *Conditions) error {
return nil
}
// Wrap a request to look similar to an apiary library request, in order to
// be used by run().
type readerRequestWrapper struct {
req *http.Request
}
func (w *readerRequestWrapper) Header() http.Header {
return w.req.Header
}
var emptyBody = ioutil.NopCloser(strings.NewReader(""))
// Reader reads a Cloud Storage object.

View file

@ -1818,6 +1818,12 @@ var awsPartition = partition{
endpointKey{
Region: "ap-northeast-1",
}: endpoint{},
endpointKey{
Region: "ap-northeast-2",
}: endpoint{},
endpointKey{
Region: "ap-northeast-3",
}: endpoint{},
endpointKey{
Region: "ap-south-1",
}: endpoint{},
@ -1827,12 +1833,27 @@ var awsPartition = partition{
endpointKey{
Region: "ap-southeast-2",
}: endpoint{},
endpointKey{
Region: "ap-southeast-4",
}: endpoint{},
endpointKey{
Region: "ca-central-1",
}: endpoint{},
endpointKey{
Region: "eu-central-1",
}: endpoint{},
endpointKey{
Region: "eu-north-1",
}: endpoint{},
endpointKey{
Region: "eu-west-1",
}: endpoint{},
endpointKey{
Region: "eu-west-3",
}: endpoint{},
endpointKey{
Region: "sa-east-1",
}: endpoint{},
endpointKey{
Region: "us-east-1",
}: endpoint{},
@ -3145,6 +3166,9 @@ var awsPartition = partition{
endpointKey{
Region: "ap-northeast-1",
}: endpoint{},
endpointKey{
Region: "ap-south-1",
}: endpoint{},
endpointKey{
Region: "ap-southeast-1",
}: endpoint{},
@ -3157,6 +3181,12 @@ var awsPartition = partition{
endpointKey{
Region: "eu-west-1",
}: endpoint{},
endpointKey{
Region: "eu-west-2",
}: endpoint{},
endpointKey{
Region: "eu-west-3",
}: endpoint{},
endpointKey{
Region: "fips-us-east-1",
}: endpoint{
@ -4492,6 +4522,14 @@ var awsPartition = partition{
Region: "ap-southeast-1",
},
},
endpointKey{
Region: "bedrock-eu-central-1",
}: endpoint{
Hostname: "bedrock.eu-central-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "eu-central-1",
},
},
endpointKey{
Region: "bedrock-fips-us-east-1",
}: endpoint{
@ -4524,6 +4562,14 @@ var awsPartition = partition{
Region: "ap-southeast-1",
},
},
endpointKey{
Region: "bedrock-runtime-eu-central-1",
}: endpoint{
Hostname: "bedrock-runtime.eu-central-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "eu-central-1",
},
},
endpointKey{
Region: "bedrock-runtime-fips-us-east-1",
}: endpoint{
@ -4572,6 +4618,9 @@ var awsPartition = partition{
Region: "us-west-2",
},
},
endpointKey{
Region: "eu-central-1",
}: endpoint{},
endpointKey{
Region: "us-east-1",
}: endpoint{},
@ -7396,6 +7445,9 @@ var awsPartition = partition{
endpointKey{
Region: "ap-southeast-3",
}: endpoint{},
endpointKey{
Region: "ap-southeast-4",
}: endpoint{},
endpointKey{
Region: "ca-central-1",
}: endpoint{},
@ -11312,6 +11364,12 @@ var awsPartition = partition{
endpointKey{
Region: "us-east-2",
}: endpoint{},
endpointKey{
Region: "us-east-2",
Variant: dualStackVariant,
}: endpoint{
Hostname: "elasticmapreduce.us-east-2.api.aws",
},
endpointKey{
Region: "us-east-2",
Variant: fipsVariant,
@ -11525,6 +11583,9 @@ var awsPartition = partition{
endpointKey{
Region: "ap-northeast-2",
}: endpoint{},
endpointKey{
Region: "ap-northeast-3",
}: endpoint{},
endpointKey{
Region: "ap-south-1",
}: endpoint{},
@ -12321,6 +12382,9 @@ var awsPartition = partition{
},
"finspace": service{
Endpoints: serviceEndpoints{
endpointKey{
Region: "ap-northeast-1",
}: endpoint{},
endpointKey{
Region: "ap-southeast-1",
}: endpoint{},
@ -13440,16 +13504,6 @@ var awsPartition = partition{
}: endpoint{},
},
},
"gamesparks": service{
Endpoints: serviceEndpoints{
endpointKey{
Region: "ap-northeast-1",
}: endpoint{},
endpointKey{
Region: "us-east-1",
}: endpoint{},
},
},
"geo": service{
Endpoints: serviceEndpoints{
endpointKey{
@ -15710,12 +15764,45 @@ var awsPartition = partition{
},
"iottwinmaker": service{
Endpoints: serviceEndpoints{
endpointKey{
Region: "ap-northeast-1",
}: endpoint{},
endpointKey{
Region: "ap-northeast-2",
}: endpoint{},
endpointKey{
Region: "ap-south-1",
}: endpoint{},
endpointKey{
Region: "ap-southeast-1",
}: endpoint{},
endpointKey{
Region: "ap-southeast-2",
}: endpoint{},
endpointKey{
Region: "api-ap-northeast-1",
}: endpoint{
Hostname: "api.iottwinmaker.ap-northeast-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "ap-northeast-1",
},
},
endpointKey{
Region: "api-ap-northeast-2",
}: endpoint{
Hostname: "api.iottwinmaker.ap-northeast-2.amazonaws.com",
CredentialScope: credentialScope{
Region: "ap-northeast-2",
},
},
endpointKey{
Region: "api-ap-south-1",
}: endpoint{
Hostname: "api.iottwinmaker.ap-south-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "ap-south-1",
},
},
endpointKey{
Region: "api-ap-southeast-1",
}: endpoint{
@ -15764,6 +15851,30 @@ var awsPartition = partition{
Region: "us-west-2",
},
},
endpointKey{
Region: "data-ap-northeast-1",
}: endpoint{
Hostname: "data.iottwinmaker.ap-northeast-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "ap-northeast-1",
},
},
endpointKey{
Region: "data-ap-northeast-2",
}: endpoint{
Hostname: "data.iottwinmaker.ap-northeast-2.amazonaws.com",
CredentialScope: credentialScope{
Region: "ap-northeast-2",
},
},
endpointKey{
Region: "data-ap-south-1",
}: endpoint{
Hostname: "data.iottwinmaker.ap-south-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "ap-south-1",
},
},
endpointKey{
Region: "data-ap-southeast-1",
}: endpoint{
@ -18130,6 +18241,9 @@ var awsPartition = partition{
endpointKey{
Region: "il-central-1",
}: endpoint{},
endpointKey{
Region: "me-central-1",
}: endpoint{},
endpointKey{
Region: "me-south-1",
}: endpoint{},
@ -19094,6 +19208,9 @@ var awsPartition = partition{
endpointKey{
Region: "ap-northeast-2",
}: endpoint{},
endpointKey{
Region: "ap-northeast-3",
}: endpoint{},
endpointKey{
Region: "ap-south-1",
}: endpoint{},
@ -19146,6 +19263,9 @@ var awsPartition = partition{
endpointKey{
Region: "ap-northeast-2",
}: endpoint{},
endpointKey{
Region: "ap-northeast-3",
}: endpoint{},
endpointKey{
Region: "ap-south-1",
}: endpoint{},
@ -19198,6 +19318,9 @@ var awsPartition = partition{
endpointKey{
Region: "ap-northeast-2",
}: endpoint{},
endpointKey{
Region: "ap-northeast-3",
}: endpoint{},
endpointKey{
Region: "ap-south-1",
}: endpoint{},
@ -23617,6 +23740,16 @@ var awsPartition = partition{
},
},
Endpoints: serviceEndpoints{
endpointKey{
Region: "af-south-1",
}: endpoint{
Hostname: "resource-explorer-2.af-south-1.api.aws",
},
endpointKey{
Region: "ap-east-1",
}: endpoint{
Hostname: "resource-explorer-2.ap-east-1.api.aws",
},
endpointKey{
Region: "ap-northeast-1",
}: endpoint{
@ -23682,6 +23815,11 @@ var awsPartition = partition{
}: endpoint{
Hostname: "resource-explorer-2.eu-north-1.api.aws",
},
endpointKey{
Region: "eu-south-1",
}: endpoint{
Hostname: "resource-explorer-2.eu-south-1.api.aws",
},
endpointKey{
Region: "eu-west-1",
}: endpoint{
@ -23702,6 +23840,11 @@ var awsPartition = partition{
}: endpoint{
Hostname: "resource-explorer-2.il-central-1.api.aws",
},
endpointKey{
Region: "me-central-1",
}: endpoint{
Hostname: "resource-explorer-2.me-central-1.api.aws",
},
endpointKey{
Region: "me-south-1",
}: endpoint{
@ -26024,6 +26167,9 @@ var awsPartition = partition{
endpointKey{
Region: "ap-northeast-2",
}: endpoint{},
endpointKey{
Region: "ap-northeast-3",
}: endpoint{},
endpointKey{
Region: "ap-south-1",
}: endpoint{},
@ -26033,15 +26179,24 @@ var awsPartition = partition{
endpointKey{
Region: "ap-southeast-2",
}: endpoint{},
endpointKey{
Region: "ca-central-1",
}: endpoint{},
endpointKey{
Region: "eu-central-1",
}: endpoint{},
endpointKey{
Region: "eu-north-1",
}: endpoint{},
endpointKey{
Region: "eu-west-1",
}: endpoint{},
endpointKey{
Region: "eu-west-2",
}: endpoint{},
endpointKey{
Region: "eu-west-3",
}: endpoint{},
endpointKey{
Region: "sa-east-1",
}: endpoint{},
@ -30333,6 +30488,9 @@ var awsPartition = partition{
endpointKey{
Region: "ap-southeast-1",
}: endpoint{},
endpointKey{
Region: "ap-southeast-2",
}: endpoint{},
endpointKey{
Region: "ca-central-1",
}: endpoint{},
@ -33264,9 +33422,21 @@ var awscnPartition = partition{
endpointKey{
Region: "cn-north-1",
}: endpoint{},
endpointKey{
Region: "cn-north-1",
Variant: dualStackVariant,
}: endpoint{
Hostname: "elasticmapreduce.cn-north-1.api.amazonwebservices.com.cn",
},
endpointKey{
Region: "cn-northwest-1",
}: endpoint{},
endpointKey{
Region: "cn-northwest-1",
Variant: dualStackVariant,
}: endpoint{
Hostname: "elasticmapreduce.cn-northwest-1.api.amazonwebservices.com.cn",
},
},
},
"emr-containers": service{
@ -34299,9 +34469,21 @@ var awscnPartition = partition{
endpointKey{
Region: "cn-north-1",
}: endpoint{},
endpointKey{
Region: "cn-north-1",
Variant: dualStackVariant,
}: endpoint{
Hostname: "states.cn-north-1.api.amazonwebservices.com.cn",
},
endpointKey{
Region: "cn-northwest-1",
}: endpoint{},
endpointKey{
Region: "cn-northwest-1",
Variant: dualStackVariant,
}: endpoint{
Hostname: "states.cn-northwest-1.api.amazonwebservices.com.cn",
},
},
},
"storagegateway": service{
@ -36790,6 +36972,12 @@ var awsusgovPartition = partition{
endpointKey{
Region: "us-gov-east-1",
}: endpoint{},
endpointKey{
Region: "us-gov-east-1",
Variant: dualStackVariant,
}: endpoint{
Hostname: "elasticmapreduce.us-gov-east-1.api.aws",
},
endpointKey{
Region: "us-gov-east-1",
Variant: fipsVariant,
@ -36801,6 +36989,13 @@ var awsusgovPartition = partition{
}: endpoint{
Protocols: []string{"https"},
},
endpointKey{
Region: "us-gov-west-1",
Variant: dualStackVariant,
}: endpoint{
Hostname: "elasticmapreduce.us-gov-west-1.api.aws",
Protocols: []string{"https"},
},
endpointKey{
Region: "us-gov-west-1",
Variant: fipsVariant,
@ -37210,21 +37405,45 @@ var awsusgovPartition = partition{
endpointKey{
Region: "us-gov-east-1",
}: endpoint{},
endpointKey{
Region: "us-gov-east-1",
Variant: dualStackVariant,
}: endpoint{
Hostname: "glue.us-gov-east-1.api.aws",
},
endpointKey{
Region: "us-gov-east-1",
Variant: fipsVariant,
}: endpoint{
Hostname: "glue-fips.us-gov-east-1.amazonaws.com",
},
endpointKey{
Region: "us-gov-east-1",
Variant: fipsVariant | dualStackVariant,
}: endpoint{
Hostname: "glue-fips.us-gov-east-1.api.aws",
},
endpointKey{
Region: "us-gov-west-1",
}: endpoint{},
endpointKey{
Region: "us-gov-west-1",
Variant: dualStackVariant,
}: endpoint{
Hostname: "glue.us-gov-west-1.api.aws",
},
endpointKey{
Region: "us-gov-west-1",
Variant: fipsVariant,
}: endpoint{
Hostname: "glue-fips.us-gov-west-1.amazonaws.com",
},
endpointKey{
Region: "us-gov-west-1",
Variant: fipsVariant | dualStackVariant,
}: endpoint{
Hostname: "glue-fips.us-gov-west-1.api.aws",
},
},
},
"greengrass": service{
@ -39069,6 +39288,46 @@ var awsusgovPartition = partition{
},
},
},
"resiliencehub": service{
Endpoints: serviceEndpoints{
endpointKey{
Region: "fips-us-gov-east-1",
}: endpoint{
Hostname: "resiliencehub-fips.us-gov-east-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "us-gov-east-1",
},
Deprecated: boxedTrue,
},
endpointKey{
Region: "fips-us-gov-west-1",
}: endpoint{
Hostname: "resiliencehub-fips.us-gov-west-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "us-gov-west-1",
},
Deprecated: boxedTrue,
},
endpointKey{
Region: "us-gov-east-1",
}: endpoint{},
endpointKey{
Region: "us-gov-east-1",
Variant: fipsVariant,
}: endpoint{
Hostname: "resiliencehub-fips.us-gov-east-1.amazonaws.com",
},
endpointKey{
Region: "us-gov-west-1",
}: endpoint{},
endpointKey{
Region: "us-gov-west-1",
Variant: fipsVariant,
}: endpoint{
Hostname: "resiliencehub-fips.us-gov-west-1.amazonaws.com",
},
},
},
"resource-explorer-2": service{
Defaults: endpointDefaults{
defaultKey{}: endpoint{
@ -41828,6 +42087,13 @@ var awsisobPartition = partition{
},
},
},
"api.sagemaker": service{
Endpoints: serviceEndpoints{
endpointKey{
Region: "us-isob-east-1",
}: endpoint{},
},
},
"appconfig": service{
Endpoints: serviceEndpoints{
endpointKey{
@ -41866,6 +42132,13 @@ var awsisobPartition = partition{
}: endpoint{},
},
},
"cloudcontrolapi": service{
Endpoints: serviceEndpoints{
endpointKey{
Region: "us-isob-east-1",
}: endpoint{},
},
},
"cloudformation": service{
Endpoints: serviceEndpoints{
endpointKey{
@ -42289,6 +42562,13 @@ var awsisobPartition = partition{
}: endpoint{},
},
},
"runtime.sagemaker": service{
Endpoints: serviceEndpoints{
endpointKey{
Region: "us-isob-east-1",
}: endpoint{},
},
},
"s3": service{
Defaults: endpointDefaults{
defaultKey{}: endpoint{

View file

@ -171,6 +171,12 @@ type envConfig struct {
// AWS_EC2_METADATA_SERVICE_ENDPOINT_MODE=IPv6
EC2IMDSEndpointMode endpoints.EC2IMDSEndpointModeState
// Specifies that IMDS clients should not fallback to IMDSv1 if token
// requests fail.
//
// AWS_EC2_METADATA_V1_DISABLED=true
EC2IMDSv1Disabled *bool
// Specifies that SDK clients must resolve a dual-stack endpoint for
// services.
//
@ -251,6 +257,9 @@ var (
ec2IMDSEndpointModeEnvKey = []string{
"AWS_EC2_METADATA_SERVICE_ENDPOINT_MODE",
}
ec2MetadataV1DisabledEnvKey = []string{
"AWS_EC2_METADATA_V1_DISABLED",
}
useCABundleKey = []string{
"AWS_CA_BUNDLE",
}
@ -393,6 +402,7 @@ func envConfigLoad(enableSharedConfig bool) (envConfig, error) {
if err := setEC2IMDSEndpointMode(&cfg.EC2IMDSEndpointMode, ec2IMDSEndpointModeEnvKey); err != nil {
return envConfig{}, err
}
setBoolPtrFromEnvVal(&cfg.EC2IMDSv1Disabled, ec2MetadataV1DisabledEnvKey)
if err := setUseDualStackEndpointFromEnvVal(&cfg.UseDualStackEndpoint, awsUseDualStackEndpoint); err != nil {
return cfg, err
@ -414,6 +424,24 @@ func setFromEnvVal(dst *string, keys []string) {
}
}
func setBoolPtrFromEnvVal(dst **bool, keys []string) {
for _, k := range keys {
value := os.Getenv(k)
if len(value) == 0 {
continue
}
switch {
case strings.EqualFold(value, "false"):
*dst = new(bool)
**dst = false
case strings.EqualFold(value, "true"):
*dst = new(bool)
**dst = true
}
}
}
func setEC2IMDSEndpointMode(mode *endpoints.EC2IMDSEndpointModeState, keys []string) error {
for _, k := range keys {
value := os.Getenv(k)

View file

@ -779,6 +779,14 @@ func mergeConfigSrcs(cfg, userCfg *aws.Config,
cfg.EndpointResolver = wrapEC2IMDSEndpoint(cfg.EndpointResolver, ec2IMDSEndpoint, endpointMode)
}
cfg.EC2MetadataEnableFallback = userCfg.EC2MetadataEnableFallback
if cfg.EC2MetadataEnableFallback == nil && envCfg.EC2IMDSv1Disabled != nil {
cfg.EC2MetadataEnableFallback = aws.Bool(!*envCfg.EC2IMDSv1Disabled)
}
if cfg.EC2MetadataEnableFallback == nil && sharedCfg.EC2IMDSv1Disabled != nil {
cfg.EC2MetadataEnableFallback = aws.Bool(!*sharedCfg.EC2IMDSv1Disabled)
}
cfg.S3UseARNRegion = userCfg.S3UseARNRegion
if cfg.S3UseARNRegion == nil {
cfg.S3UseARNRegion = &envCfg.S3UseARNRegion

View file

@ -80,6 +80,9 @@ const (
// EC2 IMDS Endpoint
ec2MetadataServiceEndpointKey = "ec2_metadata_service_endpoint"
// ECS IMDSv1 disable fallback
ec2MetadataV1DisabledKey = "ec2_metadata_v1_disabled"
// Use DualStack Endpoint Resolution
useDualStackEndpoint = "use_dualstack_endpoint"
@ -179,6 +182,12 @@ type sharedConfig struct {
// ec2_metadata_service_endpoint=http://fd00:ec2::254
EC2IMDSEndpoint string
// Specifies that IMDS clients should not fallback to IMDSv1 if token
// requests fail.
//
// ec2_metadata_v1_disabled=true
EC2IMDSv1Disabled *bool
// Specifies that SDK clients must resolve a dual-stack endpoint for
// services.
//
@ -434,6 +443,7 @@ func (cfg *sharedConfig) setFromIniFile(profile string, file sharedConfigFile, e
ec2MetadataServiceEndpointModeKey, file.Filename, err)
}
updateString(&cfg.EC2IMDSEndpoint, section, ec2MetadataServiceEndpointKey)
updateBoolPtr(&cfg.EC2IMDSv1Disabled, section, ec2MetadataV1DisabledKey)
updateUseDualStackEndpoint(&cfg.UseDualStackEndpoint, section, useDualStackEndpoint)

View file

@ -125,6 +125,7 @@ var requiredSignedHeaders = rules{
"X-Amz-Copy-Source-Server-Side-Encryption-Customer-Algorithm": struct{}{},
"X-Amz-Copy-Source-Server-Side-Encryption-Customer-Key": struct{}{},
"X-Amz-Copy-Source-Server-Side-Encryption-Customer-Key-Md5": struct{}{},
"X-Amz-Expected-Bucket-Owner": struct{}{},
"X-Amz-Grant-Full-control": struct{}{},
"X-Amz-Grant-Read": struct{}{},
"X-Amz-Grant-Read-Acp": struct{}{},

View file

@ -5,4 +5,4 @@ package aws
const SDKName = "aws-sdk-go"
// SDKVersion is the version of this SDK
const SDKVersion = "1.45.27"
const SDKVersion = "1.47.9"

File diff suppressed because it is too large Load diff

View file

@ -92,7 +92,6 @@ import (
"strconv"
"sync"
"sync/atomic"
"syscall"
"time"
"github.com/golang/glog/internal/logsink"
@ -524,34 +523,6 @@ func fatalf(depth int, format string, args ...any) {
os.Exit(2) // Exit with the same code as the default SIGABRT handler.
}
// abortProcess attempts to kill the current process in a way that will dump the
// currently-running goroutines someplace useful (Coroner or stderr).
//
// It does this by sending SIGABRT to the current process. Unfortunately, the
// signal may or may not be delivered to the current thread; in order to do that
// portably, we would need to add a cgo dependency and call pthread_kill.
//
// If successful, abortProcess does not return.
func abortProcess() error {
p, err := os.FindProcess(os.Getpid())
if err != nil {
return err
}
if err := p.Signal(syscall.SIGABRT); err != nil {
return err
}
// Sent the signal. Now we wait for it to arrive and any SIGABRT handlers to
// run (and eventually terminate the process themselves).
//
// We could just "select{}" here, but there's an outside chance that would
// trigger the runtime's deadlock detector if there happen not to be any
// background goroutines running. So we'll sleep a while first to give
// the signal some time.
time.Sleep(10 * time.Second)
select {}
}
// Fatal logs to the FATAL, ERROR, WARNING, and INFO logs,
// including a stack trace of all running goroutines, then calls os.Exit(2).
// Arguments are handled in the manner of fmt.Print; a newline is appended if missing.

View file

@ -132,6 +132,11 @@ func create(tag string, t time.Time) (f *os.File, filename string, err error) {
symlink := filepath.Join(dir, link)
os.Remove(symlink) // ignore err
os.Symlink(name, symlink) // ignore err
if *logLink != "" {
lsymlink := filepath.Join(*logLink, link)
os.Remove(lsymlink) // ignore err
os.Symlink(fname, lsymlink) // ignore err
}
return f, fname, nil
}
lastErr = err
@ -153,8 +158,6 @@ var sinks struct {
}
func init() {
sinks.stderr.w = os.Stderr
// Register stderr first: that way if we crash during file-writing at least
// the log will have gone somewhere.
logsink.TextSinks = append(logsink.TextSinks, &sinks.stderr, &sinks.file)
@ -167,7 +170,7 @@ func init() {
// if they meet certain conditions.
type stderrSink struct {
mu sync.Mutex
w io.Writer
w io.Writer // if nil Emit uses os.Stderr directly
}
// Enabled implements logsink.Text.Enabled. It returns true if any of the
@ -182,8 +185,11 @@ func (s *stderrSink) Enabled(m *logsink.Meta) bool {
func (s *stderrSink) Emit(m *logsink.Meta, data []byte) (n int, err error) {
s.mu.Lock()
defer s.mu.Unlock()
dn, err := s.w.Write(data)
w := s.w
if w == nil {
w = os.Stderr
}
dn, err := w.Write(data)
n += dn
return n, err
}

39
vendor/github.com/golang/glog/glog_file_linux.go generated vendored Normal file
View file

@ -0,0 +1,39 @@
// Go support for leveled logs, analogous to https://github.com/google/glog.
//
// Copyright 2023 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build linux
package glog
import (
"errors"
"runtime"
"syscall"
)
// abortProcess attempts to kill the current process in a way that will dump the
// currently-running goroutines someplace useful (like stderr).
//
// It does this by sending SIGABRT to the current thread.
//
// If successful, abortProcess does not return.
func abortProcess() error {
runtime.LockOSThread()
if err := syscall.Tgkill(syscall.Getpid(), syscall.Gettid(), syscall.SIGABRT); err != nil {
return err
}
return errors.New("log: killed current thread with SIGABRT, but still running")
}

30
vendor/github.com/golang/glog/glog_file_other.go generated vendored Normal file
View file

@ -0,0 +1,30 @@
// Go support for leveled logs, analogous to https://github.com/google/glog.
//
// Copyright 2023 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !(unix || windows)
package glog
import (
"fmt"
"runtime"
)
// abortProcess returns an error on platforms that presumably don't support signals.
func abortProcess() error {
return fmt.Errorf("not sending SIGABRT (%s/%s does not support signals), falling back", runtime.GOOS, runtime.GOARCH)
}

53
vendor/github.com/golang/glog/glog_file_posix.go generated vendored Normal file
View file

@ -0,0 +1,53 @@
// Go support for leveled logs, analogous to https://github.com/google/glog.
//
// Copyright 2023 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build (unix || windows) && !linux
package glog
import (
"os"
"syscall"
"time"
)
// abortProcess attempts to kill the current process in a way that will dump the
// currently-running goroutines someplace useful (like stderr).
//
// It does this by sending SIGABRT to the current process. Unfortunately, the
// signal may or may not be delivered to the current thread; in order to do that
// portably, we would need to add a cgo dependency and call pthread_kill.
//
// If successful, abortProcess does not return.
func abortProcess() error {
p, err := os.FindProcess(os.Getpid())
if err != nil {
return err
}
if err := p.Signal(syscall.SIGABRT); err != nil {
return err
}
// Sent the signal. Now we wait for it to arrive and any SIGABRT handlers to
// run (and eventually terminate the process themselves).
//
// We could just "select{}" here, but there's an outside chance that would
// trigger the runtime's deadlock detector if there happen not to be any
// background goroutines running. So we'll sleep a while first to give
// the signal some time.
time.Sleep(10 * time.Second)
select {}
}

View file

@ -133,6 +133,11 @@ func (l *Level) Set(value string) error {
type vModuleFlag struct{ *verboseFlags }
func (f vModuleFlag) String() string {
// Do not panic on the zero value.
// https://groups.google.com/g/golang-nuts/c/Atlr8uAjn6U/m/iId17Td5BQAJ.
if f.verboseFlags == nil {
return ""
}
f.mu.Lock()
defer f.mu.Unlock()
@ -192,9 +197,7 @@ func (f *verboseFlags) levelForPC(pc uintptr) Level {
file, _ := fn.FileLine(pc)
// The file is something like /a/b/c/d.go. We want just the d for
// regular matches, /a/b/c/d for full matches.
if strings.HasSuffix(file, ".go") {
file = file[:len(file)-3]
}
file = strings.TrimSuffix(file, ".go")
full := file
if slash := strings.LastIndex(file, "/"); slash >= 0 {
file = file[slash+1:]

View file

@ -1,5 +1,16 @@
# Changelog
## [1.4.0](https://github.com/google/uuid/compare/v1.3.1...v1.4.0) (2023-10-26)
### Features
* UUIDs slice type with Strings() convenience method ([#133](https://github.com/google/uuid/issues/133)) ([cd5fbbd](https://github.com/google/uuid/commit/cd5fbbdd02f3e3467ac18940e07e062be1f864b4))
### Fixes
* Clarify that Parse's job is to parse but not necessarily validate strings. (Documents current behavior)
## [1.3.1](https://github.com/google/uuid/compare/v1.3.0...v1.3.1) (2023-08-18)

View file

@ -11,7 +11,7 @@ please explain why in the pull request description.
### Releasing
Commits that would precipitate a SemVer change, as desrcibed in the Conventional
Commits that would precipitate a SemVer change, as described in the Conventional
Commits Specification, will trigger [`release-please`](https://github.com/google-github-actions/release-please-action)
to create a release candidate pull request. Once submitted, `release-please`
will create a release.

View file

@ -56,11 +56,15 @@ func IsInvalidLengthError(err error) bool {
return ok
}
// Parse decodes s into a UUID or returns an error. Both the standard UUID
// forms of xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx and
// urn:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx are decoded as well as the
// Microsoft encoding {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx} and the raw hex
// encoding: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
// Parse decodes s into a UUID or returns an error if it cannot be parsed. Both
// the standard UUID forms defined in RFC 4122
// (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx and
// urn:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) are decoded. In addition,
// Parse accepts non-standard strings such as the raw hex encoding
// xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx and 38 byte "Microsoft style" encodings,
// e.g. {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}. Only the middle 36 bytes are
// examined in the latter case. Parse should not be used to validate strings as
// it parses non-standard encodings as indicated above.
func Parse(s string) (UUID, error) {
var uuid UUID
switch len(s) {
@ -294,3 +298,15 @@ func DisableRandPool() {
poolMu.Lock()
poolPos = randPoolSize
}
// UUIDs is a slice of UUID types.
type UUIDs []UUID
// Strings returns a string slice containing the string form of each UUID in uuids.
func (uuids UUIDs) Strings() []string {
var uuidStrs = make([]string, len(uuids))
for i, uuid := range uuids {
uuidStrs[i] = uuid.String()
}
return uuidStrs
}

View file

@ -1,3 +1,9 @@
## 0.7.5 (Nov 8, 2023)
BUG FIXES
- client: fixes an issue where the request body is not preserved on temporary redirects or re-established HTTP/2 connections [GH-207]
## 0.7.4 (Jun 6, 2023)
BUG FIXES

View file

@ -160,6 +160,20 @@ func (r *Request) SetBody(rawBody interface{}) error {
}
r.body = bodyReader
r.ContentLength = contentLength
if bodyReader != nil {
r.GetBody = func() (io.ReadCloser, error) {
body, err := bodyReader()
if err != nil {
return nil, err
}
if rc, ok := body.(io.ReadCloser); ok {
return rc, nil
}
return io.NopCloser(body), nil
}
} else {
r.GetBody = func() (io.ReadCloser, error) { return http.NoBody, nil }
}
return nil
}
@ -302,18 +316,19 @@ func NewRequest(method, url string, rawBody interface{}) (*Request, error) {
// The context controls the entire lifetime of a request and its response:
// obtaining a connection, sending the request, and reading the response headers and body.
func NewRequestWithContext(ctx context.Context, method, url string, rawBody interface{}) (*Request, error) {
bodyReader, contentLength, err := getBodyReaderAndContentLength(rawBody)
if err != nil {
return nil, err
}
httpReq, err := http.NewRequestWithContext(ctx, method, url, nil)
if err != nil {
return nil, err
}
httpReq.ContentLength = contentLength
return &Request{body: bodyReader, Request: httpReq}, nil
req := &Request{
Request: httpReq,
}
if err := req.SetBody(rawBody); err != nil {
return nil, err
}
return req, nil
}
// Logger interface allows to use other loggers than

View file

@ -1,5 +1,18 @@
# Changelog
## v4.11.3 - 2023-11-07
**Security**
* 'c.Attachment' and 'c.Inline' should escape filename in 'Content-Disposition' header to avoid 'Reflect File Download' vulnerability. [#2541](https://github.com/labstack/echo/pull/2541)
**Enhancements**
* Tests: refactor context tests to be separate functions [#2540](https://github.com/labstack/echo/pull/2540)
* Proxy middleware: reuse echo request context [#2537](https://github.com/labstack/echo/pull/2537)
* Mark unmarshallable yaml struct tags as ignored [#2536](https://github.com/labstack/echo/pull/2536)
## v4.11.2 - 2023-10-11
**Security**

View file

@ -1323,7 +1323,7 @@ func (b *ValueBinder) unixTime(sourceParam string, dest *time.Time, valueMustExi
case time.Second:
*dest = time.Unix(n, 0)
case time.Millisecond:
*dest = time.Unix(n/1e3, (n%1e3)*1e6) // TODO: time.UnixMilli(n) exists since Go1.17 switch to that when min version allows
*dest = time.UnixMilli(n)
case time.Nanosecond:
*dest = time.Unix(0, n)
}

View file

@ -584,8 +584,10 @@ func (c *context) Inline(file, name string) error {
return c.contentDisposition(file, name, "inline")
}
var quoteEscaper = strings.NewReplacer("\\", "\\\\", `"`, "\\\"")
func (c *context) contentDisposition(file, name, dispositionType string) error {
c.response.Header().Set(HeaderContentDisposition, fmt.Sprintf("%s; filename=%q", dispositionType, name))
c.response.Header().Set(HeaderContentDisposition, fmt.Sprintf(`%s; filename="%s"`, dispositionType, quoteEscaper.Replace(name)))
return c.File(file)
}

View file

@ -259,7 +259,7 @@ const (
const (
// Version of Echo
Version = "4.11.2"
Version = "4.11.3"
website = "https://echo.labstack.com"
// http://patorjk.com/software/taag/#p=display&f=Small%20Slant&t=Echo
banner = `

View file

@ -39,7 +39,7 @@ type (
// See https://blog.portswigger.net/2016/10/exploiting-cors-misconfigurations-for.html
//
// Optional.
AllowOriginFunc func(origin string) (bool, error) `yaml:"allow_origin_func"`
AllowOriginFunc func(origin string) (bool, error) `yaml:"-"`
// AllowMethods determines the value of the Access-Control-Allow-Methods
// response header. This header specified the list of methods allowed when

View file

@ -359,6 +359,10 @@ func ProxyWithConfig(config ProxyConfig) echo.MiddlewareFunc {
c.Set("_error", nil)
}
// This is needed for ProxyConfig.ModifyResponse and/or ProxyConfig.Transport to be able to process the Request
// that Balancer may have replaced with c.SetRequest.
req = c.Request()
// Proxy
switch {
case c.IsWebSocket():

View file

@ -27,7 +27,7 @@ type (
// Example:
// "^/old/[0.9]+/": "/new",
// "^/api/.+?/(.*)": "/v2/$1",
RegexRules map[*regexp.Regexp]string `yaml:"regex_rules"`
RegexRules map[*regexp.Regexp]string `yaml:"-"`
}
)

View file

@ -1,6 +1,7 @@
//go:build (darwin || freebsd || openbsd || netbsd || dragonfly || hurd) && !appengine
//go:build (darwin || freebsd || openbsd || netbsd || dragonfly || hurd) && !appengine && !tinygo
// +build darwin freebsd openbsd netbsd dragonfly hurd
// +build !appengine
// +build !tinygo
package isatty

View file

@ -1,5 +1,6 @@
//go:build appengine || js || nacl || wasm
// +build appengine js nacl wasm
//go:build (appengine || js || nacl || tinygo || wasm) && !windows
// +build appengine js nacl tinygo wasm
// +build !windows
package isatty

View file

@ -1,6 +1,7 @@
//go:build (linux || aix || zos) && !appengine
//go:build (linux || aix || zos) && !appengine && !tinygo
// +build linux aix zos
// +build !appengine
// +build !tinygo
package isatty

View file

@ -2,16 +2,32 @@ package pathpolicy
// MountpointPolicies is a set of default mountpoint policies used for filesystem customizations
var MountpointPolicies = NewPathPolicies(map[string]PathPolicy{
"/": {Exact: true},
"/boot": {Exact: true},
"/var": {},
"/opt": {},
"/srv": {},
"/usr": {},
"/app": {},
"/data": {},
"/home": {},
"/tmp": {},
"/": {},
// /etc must be on the root filesystem
"/etc": {Deny: true},
// NB: any mountpoints under /usr are not supported by systemd fstab
// generator in initram before the switch-root, so we don't allow them.
"/usr": {Exact: true},
// API filesystems
"/sys": {Deny: true},
"/proc": {Deny: true},
"/dev": {Deny: true},
"/run": {Deny: true},
// not allowed due to merged-usr
"/bin": {Deny: true},
"/sbin": {Deny: true},
"/lib": {Deny: true},
"/lib64": {Deny: true},
// used by ext filesystems
"/lost+found": {Deny: true},
// used by EFI
"/boot/efi": {Deny: true},
// used by systemd / ostree
"/sysroot": {Deny: true},
// symlink to ../run which is on tmpfs
"/var/run": {Deny: true},
// symlink to ../run/lock which is on tmpfs
"/var/lock": {Deny: true},
})
// CustomDirectoriesPolicies is a set of default policies for custom directories

View file

@ -65,12 +65,13 @@ func edgeRawImgType() imageType {
filename: "image.raw.xz",
compression: "xz",
mimeType: "application/xz",
image: edgeRawImage,
packageSets: nil,
defaultSize: 10 * common.GibiByte,
rpmOstree: true,
bootable: true,
bootISO: false,
image: edgeRawImage,
kernelOptions: "modprobe.blacklist=vc4",
buildPipelines: []string{"build"},
payloadPipelines: []string{"ostree-deployment", "image", "xz"},
exports: []string{"xz"},

View file

@ -8,6 +8,7 @@ import (
"github.com/osbuild/images/pkg/subscription"
)
// TODO: move these to the EC2 environment
const amiKernelOptions = "console=ttyS0,115200n8 console=tty0 net.ifnames=0 rd.blacklist=nouveau nvme_core.io_timeout=4294967295"
var (

View file

@ -69,6 +69,7 @@ var (
defaultImageConfig: &distro.ImageConfig{
Locale: common.ToPtr("en_US.UTF-8"),
},
kernelOptions: "modprobe.blacklist=vc4",
defaultSize: 10 * common.GibiByte,
rpmOstree: true,
bootable: true,
@ -145,6 +146,7 @@ var (
defaultImageConfig: &distro.ImageConfig{
Locale: common.ToPtr("en_US.UTF-8"),
},
kernelOptions: amiKernelOptions + " modprobe.blacklist=vc4",
defaultSize: 10 * common.GibiByte,
rpmOstree: true,
bootable: true,
@ -165,6 +167,7 @@ var (
defaultImageConfig: &distro.ImageConfig{
Locale: common.ToPtr("en_US.UTF-8"),
},
kernelOptions: "modprobe.blacklist=vc4",
defaultSize: 10 * common.GibiByte,
rpmOstree: true,
bootable: true,

View file

@ -405,7 +405,12 @@ func edgeRawImage(workload workload.Workload,
img.Users = users.UsersFromBP(customizations.GetUsers())
img.Groups = users.GroupsFromBP(customizations.GetGroups())
img.KernelOptionsAppend = []string{"modprobe.blacklist=vc4"}
// The kernel options defined on the image type are usually handled in
// osCustomiztions() but ostree images don't use OSCustomizations, so we
// handle them here separately.
if t.kernelOptions != "" {
img.KernelOptionsAppend = append(img.KernelOptionsAppend, t.kernelOptions)
}
img.Keyboard = "us"
img.Locale = "C.UTF-8"
if !common.VersionLessThan(t.arch.distro.osVersion, "9.2") || !t.arch.distro.isRHEL() {

View file

@ -155,10 +155,10 @@ func (p *AnacondaInstallerISOTree) serialize() osbuild.Pipeline {
pipeline.AddStage(osbuild.NewMkdirStage(&osbuild.MkdirStageOptions{
Paths: []osbuild.MkdirStagePath{
{
Path: "images",
Path: "/images",
},
{
Path: "images/pxeboot",
Path: "/images/pxeboot",
},
},
}))
@ -167,7 +167,7 @@ func (p *AnacondaInstallerISOTree) serialize() osbuild.Pipeline {
pipeline.AddStage(osbuild.NewMkdirStage(&osbuild.MkdirStageOptions{
Paths: []osbuild.MkdirStagePath{
{
Path: "LiveOS",
Path: "/LiveOS",
},
},
}))

View file

@ -98,10 +98,10 @@ func (p *CoreOSISOTree) serialize() osbuild.Pipeline {
pipeline.AddStage(osbuild.NewMkdirStage(&osbuild.MkdirStageOptions{
Paths: []osbuild.MkdirStagePath{
{
Path: "images",
Path: "/images",
},
{
Path: "images/pxeboot",
Path: "/images/pxeboot",
},
},
}))

View file

@ -30,12 +30,12 @@ func (p *ISORootfsImg) serialize() osbuild.Pipeline {
pipeline.AddStage(osbuild.NewMkdirStage(&osbuild.MkdirStageOptions{
Paths: []osbuild.MkdirStagePath{
{
Path: "LiveOS",
Path: "/LiveOS",
},
},
}))
pipeline.AddStage(osbuild.NewTruncateStage(&osbuild.TruncateStageOptions{
Filename: "LiveOS/rootfs.img",
Filename: "/LiveOS/rootfs.img",
Size: fmt.Sprintf("%d", p.Size),
}))

View file

@ -83,10 +83,24 @@ func (p *RawOSTreeImage) serialize() osbuild.Pipeline {
"ostree-tree": *osbuild.NewOSTreeCheckoutInput("org.osbuild.source", commitChecksum),
}
// Find the FS root mount name to use as the destination root
// for the target when copying the boot files.
var fsRootMntName string
for _, mnt := range *bootCopyMounts {
if mnt.Target == "/" {
fsRootMntName = mnt.Name
break
}
}
if fsRootMntName == "" {
panic("no mount found for the filesystem root")
}
for _, paths := range bootFiles {
bootCopyOptions.Paths = append(bootCopyOptions.Paths, osbuild.CopyStagePath{
From: fmt.Sprintf("input://ostree-tree/%s%s", commitChecksum, paths[0]),
To: fmt.Sprintf("mount://root%s", paths[1]),
To: fmt.Sprintf("mount://%s%s", fsRootMntName, paths[1]),
})
}

View file

@ -2,6 +2,7 @@ package osbuild
import (
"fmt"
"reflect"
"sort"
"github.com/osbuild/images/pkg/disk"
@ -64,10 +65,15 @@ func GenCopyFSTreeOptions(inputName, inputPipeline, filename string, pt *disk.Pa
devices := make(map[string]Device, len(pt.Partitions))
mounts := make([]Mount, 0, len(pt.Partitions))
var fsRootMntName string
genMounts := func(mnt disk.Mountable, path []disk.Entity) error {
stageDevices, name := getDevices(path, filename, false)
mountpoint := mnt.GetMountpoint()
if mountpoint == "/" {
fsRootMntName = name
}
var mount *Mount
t := mnt.GetFSType()
switch t {
@ -86,6 +92,13 @@ func GenCopyFSTreeOptions(inputName, inputPipeline, filename string, pt *disk.Pa
// update devices map with new elements from stageDevices
for devName := range stageDevices {
if existingDevice, exists := devices[devName]; exists {
// It is usual that the a device is generated twice for the same Entity e.g. LVM VG, which is OK.
// Therefore fail only if a device with the same name is generated for two different Entities.
if !reflect.DeepEqual(existingDevice, stageDevices[devName]) {
panic(fmt.Sprintf("the device name %q has been generated for two different devices", devName))
}
}
devices[devName] = stageDevices[devName]
}
return nil
@ -101,6 +114,10 @@ func GenCopyFSTreeOptions(inputName, inputPipeline, filename string, pt *disk.Pa
return mounts[i].Target < mounts[j].Target
})
if fsRootMntName == "" {
panic("no mount found for the filesystem root")
}
stageMounts := Mounts(mounts)
stageDevices := Devices(devices)
@ -108,7 +125,7 @@ func GenCopyFSTreeOptions(inputName, inputPipeline, filename string, pt *disk.Pa
Paths: []CopyStagePath{
{
From: fmt.Sprintf("input://%s/", inputName),
To: "mount://root/",
To: fmt.Sprintf("mount://%s/", fsRootMntName),
},
},
}

View file

@ -153,11 +153,11 @@ func deviceName(p disk.Entity) string {
switch payload := p.(type) {
case disk.Mountable:
return pathdot(payload.GetMountpoint())
return pathEscape(payload.GetMountpoint())
case *disk.LUKSContainer:
return "luks-" + payload.UUID[:4]
case *disk.LVMVolumeGroup:
return payload.Name + "vg"
return payload.Name
case *disk.LVMLogicalVolume:
return payload.Name
}
@ -206,12 +206,21 @@ func getDevices(path []disk.Entity, filename string, lockLoopback bool) (map[str
return do, parent
}
func pathdot(path string) string {
if path == "/" {
return "root"
// pathEscape implements similar path escaping as used by systemd-escape
// https://github.com/systemd/systemd/blob/c57ff6230e4e199d40f35a356e834ba99f3f8420/src/basic/unit-name.c#L389
func pathEscape(path string) string {
if len(path) == 0 || path == "/" {
return "-"
}
path = strings.TrimLeft(path, "/")
path = strings.Trim(path, "/")
return strings.ReplaceAll(path, "/", ".")
escapeChars := func(s, char string) string {
return strings.ReplaceAll(s, char, fmt.Sprintf("\\x%x", char[0]))
}
path = escapeChars(path, "\\")
path = escapeChars(path, "-")
return strings.ReplaceAll(path, "/", "-")
}

View file

@ -19,7 +19,7 @@ linters:
disable-all: true
enable:
#- bodyclose
- deadcode
# - deadcode ! deprecated since v1.49.0; replaced by 'unused'
#- depguard
#- dogsled
#- dupl
@ -51,12 +51,12 @@ linters:
#- rowserrcheck
#- scopelint
#- staticcheck
- structcheck
#- structcheck ! deprecated since v1.49.0; replaced by 'unused'
#- stylecheck
#- typecheck
- unconvert
#- unparam
#- unused
- varcheck
- unused
# - varcheck ! deprecated since v1.49.0; replaced by 'unused'
#- whitespace
fast: false

View file

@ -4,7 +4,7 @@ Cobra is a library for creating powerful modern CLI applications.
Cobra is used in many Go projects such as [Kubernetes](https://kubernetes.io/),
[Hugo](https://gohugo.io), and [GitHub CLI](https://github.com/cli/cli) to
name a few. [This list](./projects_using_cobra.md) contains a more extensive list of projects using Cobra.
name a few. [This list](site/content/projects_using_cobra.md) contains a more extensive list of projects using Cobra.
[![](https://img.shields.io/github/actions/workflow/status/spf13/cobra/test.yml?branch=main&longCache=true&label=Test&logo=github%20actions&logoColor=fff)](https://github.com/spf13/cobra/actions?query=workflow%3ATest)
[![Go Reference](https://pkg.go.dev/badge/github.com/spf13/cobra.svg)](https://pkg.go.dev/github.com/spf13/cobra)
@ -80,7 +80,7 @@ which maintains the same interface while adding POSIX compliance.
# Installing
Using Cobra is easy. First, use `go get` to install the latest version
of the library.
of the library.
```
go get -u github.com/spf13/cobra@latest
@ -105,8 +105,8 @@ go install github.com/spf13/cobra-cli@latest
For complete details on using the Cobra-CLI generator, please read [The Cobra Generator README](https://github.com/spf13/cobra-cli/blob/main/README.md)
For complete details on using the Cobra library, please read the [The Cobra User Guide](user_guide.md).
For complete details on using the Cobra library, please read the [The Cobra User Guide](site/content/user_guide.md).
# License
Cobra is released under the Apache 2.0 license. See [LICENSE.txt](https://github.com/spf13/cobra/blob/master/LICENSE.txt)
Cobra is released under the Apache 2.0 license. See [LICENSE.txt](LICENSE.txt)

View file

@ -17,6 +17,7 @@ package cobra
import (
"fmt"
"os"
"regexp"
"strings"
)
@ -29,6 +30,8 @@ const (
activeHelpGlobalDisable = "0"
)
var activeHelpEnvVarPrefixSubstRegexp = regexp.MustCompile(`[^A-Z0-9_]`)
// AppendActiveHelp adds the specified string to the specified array to be used as ActiveHelp.
// Such strings will be processed by the completion script and will be shown as ActiveHelp
// to the user.
@ -42,7 +45,7 @@ func AppendActiveHelp(compArray []string, activeHelpStr string) []string {
// GetActiveHelpConfig returns the value of the ActiveHelp environment variable
// <PROGRAM>_ACTIVE_HELP where <PROGRAM> is the name of the root command in upper
// case, with all - replaced by _.
// case, with all non-ASCII-alphanumeric characters replaced by `_`.
// It will always return "0" if the global environment variable COBRA_ACTIVE_HELP
// is set to "0".
func GetActiveHelpConfig(cmd *Command) string {
@ -55,9 +58,10 @@ func GetActiveHelpConfig(cmd *Command) string {
// activeHelpEnvVar returns the name of the program-specific ActiveHelp environment
// variable. It has the format <PROGRAM>_ACTIVE_HELP where <PROGRAM> is the name of the
// root command in upper case, with all - replaced by _.
// root command in upper case, with all non-ASCII-alphanumeric characters replaced by `_`.
func activeHelpEnvVar(name string) string {
// This format should not be changed: users will be using it explicitly.
activeHelpEnvVar := strings.ToUpper(fmt.Sprintf("%s%s", name, activeHelpEnvVarSuffix))
return strings.ReplaceAll(activeHelpEnvVar, "-", "_")
activeHelpEnvVar = activeHelpEnvVarPrefixSubstRegexp.ReplaceAllString(activeHelpEnvVar, "_")
return activeHelpEnvVar
}

View file

@ -1,157 +0,0 @@
# Active Help
Active Help is a framework provided by Cobra which allows a program to define messages (hints, warnings, etc) that will be printed during program usage. It aims to make it easier for your users to learn how to use your program. If configured by the program, Active Help is printed when the user triggers shell completion.
For example,
```
bash-5.1$ helm repo add [tab]
You must choose a name for the repo you are adding.
bash-5.1$ bin/helm package [tab]
Please specify the path to the chart to package
bash-5.1$ bin/helm package [tab][tab]
bin/ internal/ scripts/ pkg/ testdata/
```
**Hint**: A good place to use Active Help messages is when the normal completion system does not provide any suggestions. In such cases, Active Help nicely supplements the normal shell completions to guide the user in knowing what is expected by the program.
## Supported shells
Active Help is currently only supported for the following shells:
- Bash (using [bash completion V2](shell_completions.md#bash-completion-v2) only). Note that bash 4.4 or higher is required for the prompt to appear when an Active Help message is printed.
- Zsh
## Adding Active Help messages
As Active Help uses the shell completion system, the implementation of Active Help messages is done by enhancing custom dynamic completions. If you are not familiar with dynamic completions, please refer to [Shell Completions](shell_completions.md).
Adding Active Help is done through the use of the `cobra.AppendActiveHelp(...)` function, where the program repeatedly adds Active Help messages to the list of completions. Keep reading for details.
### Active Help for nouns
Adding Active Help when completing a noun is done within the `ValidArgsFunction(...)` of a command. Please notice the use of `cobra.AppendActiveHelp(...)` in the following example:
```go
cmd := &cobra.Command{
Use: "add [NAME] [URL]",
Short: "add a chart repository",
Args: require.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
return addRepo(args)
},
ValidArgsFunction: func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
var comps []string
if len(args) == 0 {
comps = cobra.AppendActiveHelp(comps, "You must choose a name for the repo you are adding")
} else if len(args) == 1 {
comps = cobra.AppendActiveHelp(comps, "You must specify the URL for the repo you are adding")
} else {
comps = cobra.AppendActiveHelp(comps, "This command does not take any more arguments")
}
return comps, cobra.ShellCompDirectiveNoFileComp
},
}
```
The example above defines the completions (none, in this specific example) as well as the Active Help messages for the `helm repo add` command. It yields the following behavior:
```
bash-5.1$ helm repo add [tab]
You must choose a name for the repo you are adding
bash-5.1$ helm repo add grafana [tab]
You must specify the URL for the repo you are adding
bash-5.1$ helm repo add grafana https://grafana.github.io/helm-charts [tab]
This command does not take any more arguments
```
**Hint**: As can be seen in the above example, a good place to use Active Help messages is when the normal completion system does not provide any suggestions. In such cases, Active Help nicely supplements the normal shell completions.
### Active Help for flags
Providing Active Help for flags is done in the same fashion as for nouns, but using the completion function registered for the flag. For example:
```go
_ = cmd.RegisterFlagCompletionFunc("version", func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
if len(args) != 2 {
return cobra.AppendActiveHelp(nil, "You must first specify the chart to install before the --version flag can be completed"), cobra.ShellCompDirectiveNoFileComp
}
return compVersionFlag(args[1], toComplete)
})
```
The example above prints an Active Help message when not enough information was given by the user to complete the `--version` flag.
```
bash-5.1$ bin/helm install myrelease --version 2.0.[tab]
You must first specify the chart to install before the --version flag can be completed
bash-5.1$ bin/helm install myrelease bitnami/solr --version 2.0.[tab][tab]
2.0.1 2.0.2 2.0.3
```
## User control of Active Help
You may want to allow your users to disable Active Help or choose between different levels of Active Help. It is entirely up to the program to define the type of configurability of Active Help that it wants to offer, if any.
Allowing to configure Active Help is entirely optional; you can use Active Help in your program without doing anything about Active Help configuration.
The way to configure Active Help is to use the program's Active Help environment
variable. That variable is named `<PROGRAM>_ACTIVE_HELP` where `<PROGRAM>` is the name of your
program in uppercase with any `-` replaced by an `_`. The variable should be set by the user to whatever
Active Help configuration values are supported by the program.
For example, say `helm` has chosen to support three levels for Active Help: `on`, `off`, `local`. Then a user
would set the desired behavior to `local` by doing `export HELM_ACTIVE_HELP=local` in their shell.
For simplicity, when in `cmd.ValidArgsFunction(...)` or a flag's completion function, the program should read the
Active Help configuration using the `cobra.GetActiveHelpConfig(cmd)` function and select what Active Help messages
should or should not be added (instead of reading the environment variable directly).
For example:
```go
ValidArgsFunction: func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
activeHelpLevel := cobra.GetActiveHelpConfig(cmd)
var comps []string
if len(args) == 0 {
if activeHelpLevel != "off" {
comps = cobra.AppendActiveHelp(comps, "You must choose a name for the repo you are adding")
}
} else if len(args) == 1 {
if activeHelpLevel != "off" {
comps = cobra.AppendActiveHelp(comps, "You must specify the URL for the repo you are adding")
}
} else {
if activeHelpLevel == "local" {
comps = cobra.AppendActiveHelp(comps, "This command does not take any more arguments")
}
}
return comps, cobra.ShellCompDirectiveNoFileComp
},
```
**Note 1**: If the `<PROGRAM>_ACTIVE_HELP` environment variable is set to the string "0", Cobra will automatically disable all Active Help output (even if some output was specified by the program using the `cobra.AppendActiveHelp(...)` function). Using "0" can simplify your code in situations where you want to blindly disable Active Help without having to call `cobra.GetActiveHelpConfig(cmd)` explicitly.
**Note 2**: If a user wants to disable Active Help for every single program based on Cobra, she can set the environment variable `COBRA_ACTIVE_HELP` to "0". In this case `cobra.GetActiveHelpConfig(cmd)` will return "0" no matter what the variable `<PROGRAM>_ACTIVE_HELP` is set to.
**Note 3**: If the user does not set `<PROGRAM>_ACTIVE_HELP` or `COBRA_ACTIVE_HELP` (which will be a common case), the default value for the Active Help configuration returned by `cobra.GetActiveHelpConfig(cmd)` will be the empty string.
## Active Help with Cobra's default completion command
Cobra provides a default `completion` command for programs that wish to use it.
When using the default `completion` command, Active Help is configurable in the same
fashion as described above using environment variables. You may wish to document this in more
details for your users.
## Debugging Active Help
Debugging your Active Help code is done in the same way as debugging your dynamic completion code, which is with Cobra's hidden `__complete` command. Please refer to [debugging shell completion](shell_completions.md#debugging) for details.
When debugging with the `__complete` command, if you want to specify different Active Help configurations, you should use the active help environment variable. That variable is named `<PROGRAM>_ACTIVE_HELP` where any `-` is replaced by an `_`. For example, we can test deactivating some Active Help as shown below:
```
$ HELM_ACTIVE_HELP=1 bin/helm __complete install wordpress bitnami/h<ENTER>
bitnami/haproxy
bitnami/harbor
_activeHelp_ WARNING: cannot re-use a name that is still in use
:0
Completion ended with directive: ShellCompDirectiveDefault
$ HELM_ACTIVE_HELP=0 bin/helm __complete install wordpress bitnami/h<ENTER>
bitnami/haproxy
bitnami/harbor
:0
Completion ended with directive: ShellCompDirectiveDefault
```

View file

@ -85,7 +85,7 @@ __%[1]s_handle_go_custom_completion()
local out requestComp lastParam lastChar comp directive args
# Prepare the command to request completions for the program.
# Calling ${words[0]} instead of directly %[1]s allows to handle aliases
# Calling ${words[0]} instead of directly %[1]s allows handling aliases
args=("${words[@]:1}")
# Disable ActiveHelp which is not supported for bash completion v1
requestComp="%[8]s=0 ${words[0]} %[2]s ${args[*]}"

View file

@ -1,93 +0,0 @@
# Generating Bash Completions For Your cobra.Command
Please refer to [Shell Completions](shell_completions.md) for details.
## Bash legacy dynamic completions
For backward compatibility, Cobra still supports its legacy dynamic completion solution (described below). Unlike the `ValidArgsFunction` solution, the legacy solution will only work for Bash shell-completion and not for other shells. This legacy solution can be used along-side `ValidArgsFunction` and `RegisterFlagCompletionFunc()`, as long as both solutions are not used for the same command. This provides a path to gradually migrate from the legacy solution to the new solution.
**Note**: Cobra's default `completion` command uses bash completion V2. If you are currently using Cobra's legacy dynamic completion solution, you should not use the default `completion` command but continue using your own.
The legacy solution allows you to inject bash functions into the bash completion script. Those bash functions are responsible for providing the completion choices for your own completions.
Some code that works in kubernetes:
```bash
const (
bash_completion_func = `__kubectl_parse_get()
{
local kubectl_output out
if kubectl_output=$(kubectl get --no-headers "$1" 2>/dev/null); then
out=($(echo "${kubectl_output}" | awk '{print $1}'))
COMPREPLY=( $( compgen -W "${out[*]}" -- "$cur" ) )
fi
}
__kubectl_get_resource()
{
if [[ ${#nouns[@]} -eq 0 ]]; then
return 1
fi
__kubectl_parse_get ${nouns[${#nouns[@]} -1]}
if [[ $? -eq 0 ]]; then
return 0
fi
}
__kubectl_custom_func() {
case ${last_command} in
kubectl_get | kubectl_describe | kubectl_delete | kubectl_stop)
__kubectl_get_resource
return
;;
*)
;;
esac
}
`)
```
And then I set that in my command definition:
```go
cmds := &cobra.Command{
Use: "kubectl",
Short: "kubectl controls the Kubernetes cluster manager",
Long: `kubectl controls the Kubernetes cluster manager.
Find more information at https://github.com/GoogleCloudPlatform/kubernetes.`,
Run: runHelp,
BashCompletionFunction: bash_completion_func,
}
```
The `BashCompletionFunction` option is really only valid/useful on the root command. Doing the above will cause `__kubectl_custom_func()` (`__<command-use>_custom_func()`) to be called when the built in processor was unable to find a solution. In the case of kubernetes a valid command might look something like `kubectl get pod [mypod]`. If you type `kubectl get pod [tab][tab]` the `__kubectl_customc_func()` will run because the cobra.Command only understood "kubectl" and "get." `__kubectl_custom_func()` will see that the cobra.Command is "kubectl_get" and will thus call another helper `__kubectl_get_resource()`. `__kubectl_get_resource` will look at the 'nouns' collected. In our example the only noun will be `pod`. So it will call `__kubectl_parse_get pod`. `__kubectl_parse_get` will actually call out to kubernetes and get any pods. It will then set `COMPREPLY` to valid pods!
Similarly, for flags:
```go
annotation := make(map[string][]string)
annotation[cobra.BashCompCustom] = []string{"__kubectl_get_namespaces"}
flag := &pflag.Flag{
Name: "namespace",
Usage: usage,
Annotations: annotation,
}
cmd.Flags().AddFlag(flag)
```
In addition add the `__kubectl_get_namespaces` implementation in the `BashCompletionFunction`
value, e.g.:
```bash
__kubectl_get_namespaces()
{
local template
template="{{ range .items }}{{ .metadata.name }} {{ end }}"
local kubectl_out
if kubectl_out=$(kubectl get -o template --template="${template}" namespace 2>/dev/null); then
COMPREPLY=( $( compgen -W "${kubectl_out}[*]" -- "$cur" ) )
fi
}
```

View file

@ -57,7 +57,7 @@ __%[1]s_get_completion_results() {
local requestComp lastParam lastChar args
# Prepare the command to request completions for the program.
# Calling ${words[0]} instead of directly %[1]s allows to handle aliases
# Calling ${words[0]} instead of directly %[1]s allows handling aliases
args=("${words[@]:1}")
requestComp="${words[0]} %[2]s ${args[*]}"

View file

@ -43,12 +43,13 @@ var initializers []func()
var finalizers []func()
const (
defaultPrefixMatching = false
defaultCommandSorting = true
defaultCaseInsensitive = false
defaultPrefixMatching = false
defaultCommandSorting = true
defaultCaseInsensitive = false
defaultTraverseRunHooks = false
)
// EnablePrefixMatching allows to set automatic prefix matching. Automatic prefix matching can be a dangerous thing
// EnablePrefixMatching allows setting automatic prefix matching. Automatic prefix matching can be a dangerous thing
// to automatically enable in CLI tools.
// Set this to true to enable it.
var EnablePrefixMatching = defaultPrefixMatching
@ -60,6 +61,10 @@ var EnableCommandSorting = defaultCommandSorting
// EnableCaseInsensitive allows case-insensitive commands names. (case sensitive by default)
var EnableCaseInsensitive = defaultCaseInsensitive
// EnableTraverseRunHooks executes persistent pre-run and post-run hooks from all parents.
// By default this is disabled, which means only the first run hook to be found is executed.
var EnableTraverseRunHooks = defaultTraverseRunHooks
// MousetrapHelpText enables an information splash screen on Windows
// if the CLI is started from explorer.exe.
// To disable the mousetrap, just set this variable to blank string ("").

View file

@ -30,7 +30,10 @@ import (
flag "github.com/spf13/pflag"
)
const FlagSetByCobraAnnotation = "cobra_annotation_flag_set_by_cobra"
const (
FlagSetByCobraAnnotation = "cobra_annotation_flag_set_by_cobra"
CommandDisplayNameAnnotation = "cobra_annotation_command_display_name"
)
// FParseErrWhitelist configures Flag parse errors to be ignored
type FParseErrWhitelist flag.ParseErrorsWhitelist
@ -99,7 +102,7 @@ type Command struct {
Deprecated string
// Annotations are key/value pairs that can be used by applications to identify or
// group commands.
// group commands or set special options.
Annotations map[string]string
// Version defines the version for this command. If this value is non-empty and the command does not
@ -115,6 +118,8 @@ type Command struct {
// * PostRun()
// * PersistentPostRun()
// All functions get the same args, the arguments after the command name.
// The *PreRun and *PostRun functions will only be executed if the Run function of the current
// command has been declared.
//
// PersistentPreRun: children of this command will inherit and execute.
PersistentPreRun func(cmd *Command, args []string)
@ -181,6 +186,9 @@ type Command struct {
// versionTemplate is the version template defined by user.
versionTemplate string
// errPrefix is the error message prefix defined by user.
errPrefix string
// inReader is a reader defined by the user that replaces stdin
inReader io.Reader
// outWriter is a writer defined by the user that replaces stdout
@ -346,6 +354,11 @@ func (c *Command) SetVersionTemplate(s string) {
c.versionTemplate = s
}
// SetErrPrefix sets error message prefix to be used. Application can use it to set custom prefix.
func (c *Command) SetErrPrefix(s string) {
c.errPrefix = s
}
// SetGlobalNormalizationFunc sets a normalization function to all flag sets and also to child commands.
// The user should not have a cyclic dependency on commands.
func (c *Command) SetGlobalNormalizationFunc(n func(f *flag.FlagSet, name string) flag.NormalizedName) {
@ -595,6 +608,18 @@ func (c *Command) VersionTemplate() string {
`
}
// ErrPrefix return error message prefix for the command
func (c *Command) ErrPrefix() string {
if c.errPrefix != "" {
return c.errPrefix
}
if c.HasParent() {
return c.parent.ErrPrefix()
}
return "Error:"
}
func hasNoOptDefVal(name string, fs *flag.FlagSet) bool {
flag := fs.Lookup(name)
if flag == nil {
@ -752,7 +777,9 @@ func (c *Command) findNext(next string) *Command {
}
if len(matches) == 1 {
return matches[0]
// Temporarily disable gosec G602, which produces a false positive.
// See https://github.com/securego/gosec/issues/1005.
return matches[0] // #nosec G602
}
return nil
@ -910,15 +937,31 @@ func (c *Command) execute(a []string) (err error) {
return err
}
parents := make([]*Command, 0, 5)
for p := c; p != nil; p = p.Parent() {
if EnableTraverseRunHooks {
// When EnableTraverseRunHooks is set:
// - Execute all persistent pre-runs from the root parent till this command.
// - Execute all persistent post-runs from this command till the root parent.
parents = append([]*Command{p}, parents...)
} else {
// Otherwise, execute only the first found persistent hook.
parents = append(parents, p)
}
}
for _, p := range parents {
if p.PersistentPreRunE != nil {
if err := p.PersistentPreRunE(c, argWoFlags); err != nil {
return err
}
break
if !EnableTraverseRunHooks {
break
}
} else if p.PersistentPreRun != nil {
p.PersistentPreRun(c, argWoFlags)
break
if !EnableTraverseRunHooks {
break
}
}
}
if c.PreRunE != nil {
@ -955,10 +998,14 @@ func (c *Command) execute(a []string) (err error) {
if err := p.PersistentPostRunE(c, argWoFlags); err != nil {
return err
}
break
if !EnableTraverseRunHooks {
break
}
} else if p.PersistentPostRun != nil {
p.PersistentPostRun(c, argWoFlags)
break
if !EnableTraverseRunHooks {
break
}
}
}
@ -1048,7 +1095,7 @@ func (c *Command) ExecuteC() (cmd *Command, err error) {
c = cmd
}
if !c.SilenceErrors {
c.PrintErrln("Error:", err.Error())
c.PrintErrln(c.ErrPrefix(), err.Error())
c.PrintErrf("Run '%v --help' for usage.\n", c.CommandPath())
}
return c, err
@ -1077,7 +1124,7 @@ func (c *Command) ExecuteC() (cmd *Command, err error) {
// If root command has SilenceErrors flagged,
// all subcommands should respect it
if !cmd.SilenceErrors && !c.SilenceErrors {
c.PrintErrln("Error:", err.Error())
c.PrintErrln(cmd.ErrPrefix(), err.Error())
}
// If root command has SilenceUsage flagged,
@ -1380,6 +1427,9 @@ func (c *Command) CommandPath() string {
if c.HasParent() {
return c.Parent().CommandPath() + " " + c.Name()
}
if displayName, ok := c.Annotations[CommandDisplayNameAnnotation]; ok {
return displayName
}
return c.Name()
}
@ -1402,6 +1452,7 @@ func (c *Command) UseLine() string {
// DebugFlags used to determine which flags have been assigned to which commands
// and which persist.
// nolint:goconst
func (c *Command) DebugFlags() {
c.Println("DebugFlags called on", c.Name())
var debugflags func(*Command)

View file

@ -145,6 +145,20 @@ func (c *Command) RegisterFlagCompletionFunc(flagName string, f func(cmd *Comman
return nil
}
// GetFlagCompletionFunc returns the completion function for the given flag of the command, if available.
func (c *Command) GetFlagCompletionFunc(flagName string) (func(*Command, []string, string) ([]string, ShellCompDirective), bool) {
flag := c.Flag(flagName)
if flag == nil {
return nil, false
}
flagCompletionMutex.RLock()
defer flagCompletionMutex.RUnlock()
completionFunc, exists := flagCompletionFunctions[flag]
return completionFunc, exists
}
// Returns a string listing the different directive enabled in the specified parameter
func (d ShellCompDirective) string() string {
var directives []string
@ -283,9 +297,13 @@ func (c *Command) getCompletions(args []string) (*Command, []string, ShellCompDi
// These flags are normally added when `execute()` is called on `finalCmd`,
// however, when doing completion, we don't call `finalCmd.execute()`.
// Let's add the --help and --version flag ourselves.
finalCmd.InitDefaultHelpFlag()
finalCmd.InitDefaultVersionFlag()
// Let's add the --help and --version flag ourselves but only if the finalCmd
// has not disabled flag parsing; if flag parsing is disabled, it is up to the
// finalCmd itself to handle the completion of *all* flags.
if !finalCmd.DisableFlagParsing {
finalCmd.InitDefaultHelpFlag()
finalCmd.InitDefaultVersionFlag()
}
// Check if we are doing flag value completion before parsing the flags.
// This is important because if we are completing a flag value, we need to also
@ -389,6 +407,11 @@ func (c *Command) getCompletions(args []string) (*Command, []string, ShellCompDi
finalCmd.InheritedFlags().VisitAll(func(flag *pflag.Flag) {
doCompleteFlags(flag)
})
// Try to complete non-inherited flags even if DisableFlagParsing==true.
// This allows programs to tell Cobra about flags for completion even
// if the actual parsing of flags is not done by Cobra.
// For instance, Helm uses this to provide flag name completion for
// some of its plugins.
finalCmd.NonInheritedFlags().VisitAll(func(flag *pflag.Flag) {
doCompleteFlags(flag)
})

View file

@ -113,7 +113,7 @@ function __%[1]s_clear_perform_completion_once_result
__%[1]s_debug ""
__%[1]s_debug "========= clearing previously set __%[1]s_perform_completion_once_result variable =========="
set --erase __%[1]s_perform_completion_once_result
__%[1]s_debug "Succesfully erased the variable __%[1]s_perform_completion_once_result"
__%[1]s_debug "Successfully erased the variable __%[1]s_perform_completion_once_result"
end
function __%[1]s_requires_order_preservation

View file

@ -1,4 +0,0 @@
## Generating Fish Completions For Your cobra.Command
Please refer to [Shell Completions](shell_completions.md) for details.

View file

@ -24,6 +24,7 @@ import (
const (
requiredAsGroup = "cobra_annotation_required_if_others_set"
oneRequired = "cobra_annotation_one_required"
mutuallyExclusive = "cobra_annotation_mutually_exclusive"
)
@ -43,6 +44,22 @@ func (c *Command) MarkFlagsRequiredTogether(flagNames ...string) {
}
}
// MarkFlagsOneRequired marks the given flags with annotations so that Cobra errors
// if the command is invoked without at least one flag from the given set of flags.
func (c *Command) MarkFlagsOneRequired(flagNames ...string) {
c.mergePersistentFlags()
for _, v := range flagNames {
f := c.Flags().Lookup(v)
if f == nil {
panic(fmt.Sprintf("Failed to find flag %q and mark it as being in a one-required flag group", v))
}
if err := c.Flags().SetAnnotation(v, oneRequired, append(f.Annotations[oneRequired], strings.Join(flagNames, " "))); err != nil {
// Only errs if the flag isn't found.
panic(err)
}
}
}
// MarkFlagsMutuallyExclusive marks the given flags with annotations so that Cobra errors
// if the command is invoked with more than one flag from the given set of flags.
func (c *Command) MarkFlagsMutuallyExclusive(flagNames ...string) {
@ -59,7 +76,7 @@ func (c *Command) MarkFlagsMutuallyExclusive(flagNames ...string) {
}
}
// ValidateFlagGroups validates the mutuallyExclusive/requiredAsGroup logic and returns the
// ValidateFlagGroups validates the mutuallyExclusive/oneRequired/requiredAsGroup logic and returns the
// first error encountered.
func (c *Command) ValidateFlagGroups() error {
if c.DisableFlagParsing {
@ -71,15 +88,20 @@ func (c *Command) ValidateFlagGroups() error {
// groupStatus format is the list of flags as a unique ID,
// then a map of each flag name and whether it is set or not.
groupStatus := map[string]map[string]bool{}
oneRequiredGroupStatus := map[string]map[string]bool{}
mutuallyExclusiveGroupStatus := map[string]map[string]bool{}
flags.VisitAll(func(pflag *flag.Flag) {
processFlagForGroupAnnotation(flags, pflag, requiredAsGroup, groupStatus)
processFlagForGroupAnnotation(flags, pflag, oneRequired, oneRequiredGroupStatus)
processFlagForGroupAnnotation(flags, pflag, mutuallyExclusive, mutuallyExclusiveGroupStatus)
})
if err := validateRequiredFlagGroups(groupStatus); err != nil {
return err
}
if err := validateOneRequiredFlagGroups(oneRequiredGroupStatus); err != nil {
return err
}
if err := validateExclusiveFlagGroups(mutuallyExclusiveGroupStatus); err != nil {
return err
}
@ -142,6 +164,27 @@ func validateRequiredFlagGroups(data map[string]map[string]bool) error {
return nil
}
func validateOneRequiredFlagGroups(data map[string]map[string]bool) error {
keys := sortedKeys(data)
for _, flagList := range keys {
flagnameAndStatus := data[flagList]
var set []string
for flagname, isSet := range flagnameAndStatus {
if isSet {
set = append(set, flagname)
}
}
if len(set) >= 1 {
continue
}
// Sort values, so they can be tested/scripted against consistently.
sort.Strings(set)
return fmt.Errorf("at least one of the flags in the group [%v] is required", flagList)
}
return nil
}
func validateExclusiveFlagGroups(data map[string]map[string]bool) error {
keys := sortedKeys(data)
for _, flagList := range keys {
@ -176,6 +219,7 @@ func sortedKeys(m map[string]map[string]bool) []string {
// enforceFlagGroupsForCompletion will do the following:
// - when a flag in a group is present, other flags in the group will be marked required
// - when none of the flags in a one-required group are present, all flags in the group will be marked required
// - when a flag in a mutually exclusive group is present, other flags in the group will be marked as hidden
// This allows the standard completion logic to behave appropriately for flag groups
func (c *Command) enforceFlagGroupsForCompletion() {
@ -185,9 +229,11 @@ func (c *Command) enforceFlagGroupsForCompletion() {
flags := c.Flags()
groupStatus := map[string]map[string]bool{}
oneRequiredGroupStatus := map[string]map[string]bool{}
mutuallyExclusiveGroupStatus := map[string]map[string]bool{}
c.Flags().VisitAll(func(pflag *flag.Flag) {
processFlagForGroupAnnotation(flags, pflag, requiredAsGroup, groupStatus)
processFlagForGroupAnnotation(flags, pflag, oneRequired, oneRequiredGroupStatus)
processFlagForGroupAnnotation(flags, pflag, mutuallyExclusive, mutuallyExclusiveGroupStatus)
})
@ -204,6 +250,26 @@ func (c *Command) enforceFlagGroupsForCompletion() {
}
}
// If none of the flags of a one-required group are present, we make all the flags
// of that group required so that the shell completion suggests them automatically
for flagList, flagnameAndStatus := range oneRequiredGroupStatus {
set := 0
for _, isSet := range flagnameAndStatus {
if isSet {
set++
}
}
// None of the flags of the group are set, mark all flags in the group
// as required
if set == 0 {
for _, fName := range strings.Split(flagList, " ") {
_ = c.MarkFlagRequired(fName)
}
}
}
// If a flag that is mutually exclusive to others is present, we hide the other
// flags of that group so the shell completion does not suggest them
for flagList, flagnameAndStatus := range mutuallyExclusiveGroupStatus {

View file

@ -47,7 +47,7 @@ filter __%[1]s_escapeStringWithSpecialChars {
`+" $_ -replace '\\s|#|@|\\$|;|,|''|\\{|\\}|\\(|\\)|\"|`|\\||<|>|&','`$&'"+`
}
[scriptblock]$__%[2]sCompleterBlock = {
[scriptblock]${__%[2]sCompleterBlock} = {
param(
$WordToComplete,
$CommandAst,
@ -122,7 +122,7 @@ filter __%[1]s_escapeStringWithSpecialChars {
__%[1]s_debug "Calling $RequestComp"
# First disable ActiveHelp which is not supported for Powershell
$env:%[10]s=0
${env:%[10]s}=0
#call the command store the output in $out and redirect stderr and stdout to null
# $Out is an array contains each line per element
@ -279,7 +279,7 @@ filter __%[1]s_escapeStringWithSpecialChars {
}
}
Register-ArgumentCompleter -CommandName '%[1]s' -ScriptBlock $__%[2]sCompleterBlock
Register-ArgumentCompleter -CommandName '%[1]s' -ScriptBlock ${__%[2]sCompleterBlock}
`, name, nameForVar, compCmd,
ShellCompDirectiveError, ShellCompDirectiveNoSpace, ShellCompDirectiveNoFileComp,
ShellCompDirectiveFilterFileExt, ShellCompDirectiveFilterDirs, ShellCompDirectiveKeepOrder, activeHelpEnvVar(name)))

View file

@ -1,3 +0,0 @@
# Generating PowerShell Completions For Your Own cobra.Command
Please refer to [Shell Completions](shell_completions.md#powershell-completions) for details.

View file

@ -1,64 +0,0 @@
## Projects using Cobra
- [Allero](https://github.com/allero-io/allero)
- [Arewefastyet](https://benchmark.vitess.io)
- [Arduino CLI](https://github.com/arduino/arduino-cli)
- [Bleve](https://blevesearch.com/)
- [Cilium](https://cilium.io/)
- [CloudQuery](https://github.com/cloudquery/cloudquery)
- [CockroachDB](https://www.cockroachlabs.com/)
- [Constellation](https://github.com/edgelesssys/constellation)
- [Cosmos SDK](https://github.com/cosmos/cosmos-sdk)
- [Datree](https://github.com/datreeio/datree)
- [Delve](https://github.com/derekparker/delve)
- [Docker (distribution)](https://github.com/docker/distribution)
- [Etcd](https://etcd.io/)
- [Gardener](https://github.com/gardener/gardenctl)
- [Giant Swarm's gsctl](https://github.com/giantswarm/gsctl)
- [Git Bump](https://github.com/erdaltsksn/git-bump)
- [GitHub CLI](https://github.com/cli/cli)
- [GitHub Labeler](https://github.com/erdaltsksn/gh-label)
- [Golangci-lint](https://golangci-lint.run)
- [GopherJS](https://github.com/gopherjs/gopherjs)
- [GoReleaser](https://goreleaser.com)
- [Helm](https://helm.sh)
- [Hugo](https://gohugo.io)
- [Infracost](https://github.com/infracost/infracost)
- [Istio](https://istio.io)
- [Kool](https://github.com/kool-dev/kool)
- [Kubernetes](https://kubernetes.io/)
- [Kubescape](https://github.com/kubescape/kubescape)
- [KubeVirt](https://github.com/kubevirt/kubevirt)
- [Linkerd](https://linkerd.io/)
- [Mattermost-server](https://github.com/mattermost/mattermost-server)
- [Mercure](https://mercure.rocks/)
- [Meroxa CLI](https://github.com/meroxa/cli)
- [Metal Stack CLI](https://github.com/metal-stack/metalctl)
- [Moby (former Docker)](https://github.com/moby/moby)
- [Moldy](https://github.com/Moldy-Community/moldy)
- [Multi-gitter](https://github.com/lindell/multi-gitter)
- [Nanobox](https://github.com/nanobox-io/nanobox)/[Nanopack](https://github.com/nanopack)
- [nFPM](https://nfpm.goreleaser.com)
- [Okteto](https://github.com/okteto/okteto)
- [OpenShift](https://www.openshift.com/)
- [Ory Hydra](https://github.com/ory/hydra)
- [Ory Kratos](https://github.com/ory/kratos)
- [Pixie](https://github.com/pixie-io/pixie)
- [Polygon Edge](https://github.com/0xPolygon/polygon-edge)
- [Pouch](https://github.com/alibaba/pouch)
- [ProjectAtomic (enterprise)](https://www.projectatomic.io/)
- [Prototool](https://github.com/uber/prototool)
- [Pulumi](https://www.pulumi.com)
- [QRcp](https://github.com/claudiodangelis/qrcp)
- [Random](https://github.com/erdaltsksn/random)
- [Rclone](https://rclone.org/)
- [Scaleway CLI](https://github.com/scaleway/scaleway-cli)
- [Sia](https://github.com/SiaFoundation/siad)
- [Skaffold](https://skaffold.dev/)
- [Tendermint](https://github.com/tendermint/tendermint)
- [Twitch CLI](https://github.com/twitchdev/twitch-cli)
- [UpCloud CLI (`upctl`)](https://github.com/UpCloudLtd/upcloud-cli)
- [Vitess](https://vitess.io)
- VMware's [Tanzu Community Edition](https://github.com/vmware-tanzu/community-edition) & [Tanzu Framework](https://github.com/vmware-tanzu/tanzu-framework)
- [Werf](https://werf.io/)
- [ZITADEL](https://github.com/zitadel/zitadel)

View file

@ -1,576 +0,0 @@
# Generating shell completions
Cobra can generate shell completions for multiple shells.
The currently supported shells are:
- Bash
- Zsh
- fish
- PowerShell
Cobra will automatically provide your program with a fully functional `completion` command,
similarly to how it provides the `help` command.
## Creating your own completion command
If you do not wish to use the default `completion` command, you can choose to
provide your own, which will take precedence over the default one. (This also provides
backwards-compatibility with programs that already have their own `completion` command.)
If you are using the `cobra-cli` generator,
which can be found at [spf13/cobra-cli](https://github.com/spf13/cobra-cli),
you can create a completion command by running
```bash
cobra-cli add completion
```
and then modifying the generated `cmd/completion.go` file to look something like this
(writing the shell script to stdout allows the most flexible use):
```go
var completionCmd = &cobra.Command{
Use: "completion [bash|zsh|fish|powershell]",
Short: "Generate completion script",
Long: fmt.Sprintf(`To load completions:
Bash:
$ source <(%[1]s completion bash)
# To load completions for each session, execute once:
# Linux:
$ %[1]s completion bash > /etc/bash_completion.d/%[1]s
# macOS:
$ %[1]s completion bash > $(brew --prefix)/etc/bash_completion.d/%[1]s
Zsh:
# If shell completion is not already enabled in your environment,
# you will need to enable it. You can execute the following once:
$ echo "autoload -U compinit; compinit" >> ~/.zshrc
# To load completions for each session, execute once:
$ %[1]s completion zsh > "${fpath[1]}/_%[1]s"
# You will need to start a new shell for this setup to take effect.
fish:
$ %[1]s completion fish | source
# To load completions for each session, execute once:
$ %[1]s completion fish > ~/.config/fish/completions/%[1]s.fish
PowerShell:
PS> %[1]s completion powershell | Out-String | Invoke-Expression
# To load completions for every new session, run:
PS> %[1]s completion powershell > %[1]s.ps1
# and source this file from your PowerShell profile.
`,cmd.Root().Name()),
DisableFlagsInUseLine: true,
ValidArgs: []string{"bash", "zsh", "fish", "powershell"},
Args: cobra.MatchAll(cobra.ExactArgs(1), cobra.OnlyValidArgs),
Run: func(cmd *cobra.Command, args []string) {
switch args[0] {
case "bash":
cmd.Root().GenBashCompletion(os.Stdout)
case "zsh":
cmd.Root().GenZshCompletion(os.Stdout)
case "fish":
cmd.Root().GenFishCompletion(os.Stdout, true)
case "powershell":
cmd.Root().GenPowerShellCompletionWithDesc(os.Stdout)
}
},
}
```
**Note:** The cobra generator may include messages printed to stdout, for example, if the config file is loaded; this will break the auto-completion script so must be removed.
## Adapting the default completion command
Cobra provides a few options for the default `completion` command. To configure such options you must set
the `CompletionOptions` field on the *root* command.
To tell Cobra *not* to provide the default `completion` command:
```
rootCmd.CompletionOptions.DisableDefaultCmd = true
```
To tell Cobra to mark the default `completion` command as *hidden*:
```
rootCmd.CompletionOptions.HiddenDefaultCmd = true
```
To tell Cobra *not* to provide the user with the `--no-descriptions` flag to the completion sub-commands:
```
rootCmd.CompletionOptions.DisableNoDescFlag = true
```
To tell Cobra to completely disable descriptions for completions:
```
rootCmd.CompletionOptions.DisableDescriptions = true
```
# Customizing completions
The generated completion scripts will automatically handle completing commands and flags. However, you can make your completions much more powerful by providing information to complete your program's nouns and flag values.
## Completion of nouns
### Static completion of nouns
Cobra allows you to provide a pre-defined list of completion choices for your nouns using the `ValidArgs` field.
For example, if you want `kubectl get [tab][tab]` to show a list of valid "nouns" you have to set them.
Some simplified code from `kubectl get` looks like:
```go
validArgs = []string{ "pod", "node", "service", "replicationcontroller" }
cmd := &cobra.Command{
Use: "get [(-o|--output=)json|yaml|template|...] (RESOURCE [NAME] | RESOURCE/NAME ...)",
Short: "Display one or many resources",
Long: get_long,
Example: get_example,
Run: func(cmd *cobra.Command, args []string) {
cobra.CheckErr(RunGet(f, out, cmd, args))
},
ValidArgs: validArgs,
}
```
Notice we put the `ValidArgs` field on the `get` sub-command. Doing so will give results like:
```bash
$ kubectl get [tab][tab]
node pod replicationcontroller service
```
#### Aliases for nouns
If your nouns have aliases, you can define them alongside `ValidArgs` using `ArgAliases`:
```go
argAliases = []string { "pods", "nodes", "services", "svc", "replicationcontrollers", "rc" }
cmd := &cobra.Command{
...
ValidArgs: validArgs,
ArgAliases: argAliases
}
```
The aliases are shown to the user on tab completion only if no completions were found within sub-commands or `ValidArgs`.
### Dynamic completion of nouns
In some cases it is not possible to provide a list of completions in advance. Instead, the list of completions must be determined at execution-time. In a similar fashion as for static completions, you can use the `ValidArgsFunction` field to provide a Go function that Cobra will execute when it needs the list of completion choices for the nouns of a command. Note that either `ValidArgs` or `ValidArgsFunction` can be used for a single cobra command, but not both.
Simplified code from `helm status` looks like:
```go
cmd := &cobra.Command{
Use: "status RELEASE_NAME",
Short: "Display the status of the named release",
Long: status_long,
RunE: func(cmd *cobra.Command, args []string) {
RunGet(args[0])
},
ValidArgsFunction: func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
if len(args) != 0 {
return nil, cobra.ShellCompDirectiveNoFileComp
}
return getReleasesFromCluster(toComplete), cobra.ShellCompDirectiveNoFileComp
},
}
```
Where `getReleasesFromCluster()` is a Go function that obtains the list of current Helm releases running on the Kubernetes cluster.
Notice we put the `ValidArgsFunction` on the `status` sub-command. Let's assume the Helm releases on the cluster are: `harbor`, `notary`, `rook` and `thanos` then this dynamic completion will give results like:
```bash
$ helm status [tab][tab]
harbor notary rook thanos
```
You may have noticed the use of `cobra.ShellCompDirective`. These directives are bit fields allowing to control some shell completion behaviors for your particular completion. You can combine them with the bit-or operator such as `cobra.ShellCompDirectiveNoSpace | cobra.ShellCompDirectiveNoFileComp`
```go
// Indicates that the shell will perform its default behavior after completions
// have been provided (this implies none of the other directives).
ShellCompDirectiveDefault
// Indicates an error occurred and completions should be ignored.
ShellCompDirectiveError
// Indicates that the shell should not add a space after the completion,
// even if there is a single completion provided.
ShellCompDirectiveNoSpace
// Indicates that the shell should not provide file completion even when
// no completion is provided.
ShellCompDirectiveNoFileComp
// Indicates that the returned completions should be used as file extension filters.
// For example, to complete only files of the form *.json or *.yaml:
// return []string{"yaml", "json"}, ShellCompDirectiveFilterFileExt
// For flags, using MarkFlagFilename() and MarkPersistentFlagFilename()
// is a shortcut to using this directive explicitly.
//
ShellCompDirectiveFilterFileExt
// Indicates that only directory names should be provided in file completion.
// For example:
// return nil, ShellCompDirectiveFilterDirs
// For flags, using MarkFlagDirname() is a shortcut to using this directive explicitly.
//
// To request directory names within another directory, the returned completions
// should specify a single directory name within which to search. For example,
// to complete directories within "themes/":
// return []string{"themes"}, ShellCompDirectiveFilterDirs
//
ShellCompDirectiveFilterDirs
// ShellCompDirectiveKeepOrder indicates that the shell should preserve the order
// in which the completions are provided
ShellCompDirectiveKeepOrder
```
***Note***: When using the `ValidArgsFunction`, Cobra will call your registered function after having parsed all flags and arguments provided in the command-line. You therefore don't need to do this parsing yourself. For example, when a user calls `helm status --namespace my-rook-ns [tab][tab]`, Cobra will call your registered `ValidArgsFunction` after having parsed the `--namespace` flag, as it would have done when calling the `RunE` function.
#### Debugging
Cobra achieves dynamic completion through the use of a hidden command called by the completion script. To debug your Go completion code, you can call this hidden command directly:
```bash
$ helm __complete status har<ENTER>
harbor
:4
Completion ended with directive: ShellCompDirectiveNoFileComp # This is on stderr
```
***Important:*** If the noun to complete is empty (when the user has not yet typed any letters of that noun), you must pass an empty parameter to the `__complete` command:
```bash
$ helm __complete status ""<ENTER>
harbor
notary
rook
thanos
:4
Completion ended with directive: ShellCompDirectiveNoFileComp # This is on stderr
```
Calling the `__complete` command directly allows you to run the Go debugger to troubleshoot your code. You can also add printouts to your code; Cobra provides the following functions to use for printouts in Go completion code:
```go
// Prints to the completion script debug file (if BASH_COMP_DEBUG_FILE
// is set to a file path) and optionally prints to stderr.
cobra.CompDebug(msg string, printToStdErr bool) {
cobra.CompDebugln(msg string, printToStdErr bool)
// Prints to the completion script debug file (if BASH_COMP_DEBUG_FILE
// is set to a file path) and to stderr.
cobra.CompError(msg string)
cobra.CompErrorln(msg string)
```
***Important:*** You should **not** leave traces that print directly to stdout in your completion code as they will be interpreted as completion choices by the completion script. Instead, use the cobra-provided debugging traces functions mentioned above.
## Completions for flags
### Mark flags as required
Most of the time completions will only show sub-commands. But if a flag is required to make a sub-command work, you probably want it to show up when the user types [tab][tab]. You can mark a flag as 'Required' like so:
```go
cmd.MarkFlagRequired("pod")
cmd.MarkFlagRequired("container")
```
and you'll get something like
```bash
$ kubectl exec [tab][tab]
-c --container= -p --pod=
```
### Specify dynamic flag completion
As for nouns, Cobra provides a way of defining dynamic completion of flags. To provide a Go function that Cobra will execute when it needs the list of completion choices for a flag, you must register the function using the `command.RegisterFlagCompletionFunc()` function.
```go
flagName := "output"
cmd.RegisterFlagCompletionFunc(flagName, func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
return []string{"json", "table", "yaml"}, cobra.ShellCompDirectiveDefault
})
```
Notice that calling `RegisterFlagCompletionFunc()` is done through the `command` with which the flag is associated. In our example this dynamic completion will give results like so:
```bash
$ helm status --output [tab][tab]
json table yaml
```
#### Debugging
You can also easily debug your Go completion code for flags:
```bash
$ helm __complete status --output ""
json
table
yaml
:4
Completion ended with directive: ShellCompDirectiveNoFileComp # This is on stderr
```
***Important:*** You should **not** leave traces that print to stdout in your completion code as they will be interpreted as completion choices by the completion script. Instead, use the cobra-provided debugging traces functions mentioned further above.
### Specify valid filename extensions for flags that take a filename
To limit completions of flag values to file names with certain extensions you can either use the different `MarkFlagFilename()` functions or a combination of `RegisterFlagCompletionFunc()` and `ShellCompDirectiveFilterFileExt`, like so:
```go
flagName := "output"
cmd.MarkFlagFilename(flagName, "yaml", "json")
```
or
```go
flagName := "output"
cmd.RegisterFlagCompletionFunc(flagName, func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
return []string{"yaml", "json"}, ShellCompDirectiveFilterFileExt})
```
### Limit flag completions to directory names
To limit completions of flag values to directory names you can either use the `MarkFlagDirname()` functions or a combination of `RegisterFlagCompletionFunc()` and `ShellCompDirectiveFilterDirs`, like so:
```go
flagName := "output"
cmd.MarkFlagDirname(flagName)
```
or
```go
flagName := "output"
cmd.RegisterFlagCompletionFunc(flagName, func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
return nil, cobra.ShellCompDirectiveFilterDirs
})
```
To limit completions of flag values to directory names *within another directory* you can use a combination of `RegisterFlagCompletionFunc()` and `ShellCompDirectiveFilterDirs` like so:
```go
flagName := "output"
cmd.RegisterFlagCompletionFunc(flagName, func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
return []string{"themes"}, cobra.ShellCompDirectiveFilterDirs
})
```
### Descriptions for completions
Cobra provides support for completion descriptions. Such descriptions are supported for each shell
(however, for bash, it is only available in the [completion V2 version](#bash-completion-v2)).
For commands and flags, Cobra will provide the descriptions automatically, based on usage information.
For example, using zsh:
```
$ helm s[tab]
search -- search for a keyword in charts
show -- show information of a chart
status -- displays the status of the named release
```
while using fish:
```
$ helm s[tab]
search (search for a keyword in charts) show (show information of a chart) status (displays the status of the named release)
```
Cobra allows you to add descriptions to your own completions. Simply add the description text after each completion, following a `\t` separator. This technique applies to completions returned by `ValidArgs`, `ValidArgsFunction` and `RegisterFlagCompletionFunc()`. For example:
```go
ValidArgsFunction: func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
return []string{"harbor\tAn image registry", "thanos\tLong-term metrics"}, cobra.ShellCompDirectiveNoFileComp
}}
```
or
```go
ValidArgs: []string{"bash\tCompletions for bash", "zsh\tCompletions for zsh"}
```
If you don't want to show descriptions in the completions, you can add `--no-descriptions` to the default `completion` command to disable them, like:
```bash
$ source <(helm completion bash)
$ helm completion [tab][tab]
bash (generate autocompletion script for bash) powershell (generate autocompletion script for powershell)
fish (generate autocompletion script for fish) zsh (generate autocompletion script for zsh)
$ source <(helm completion bash --no-descriptions)
$ helm completion [tab][tab]
bash fish powershell zsh
```
## Bash completions
### Dependencies
The bash completion script generated by Cobra requires the `bash_completion` package. You should update the help text of your completion command to show how to install the `bash_completion` package ([Kubectl docs](https://kubernetes.io/docs/tasks/tools/install-kubectl/#enabling-shell-autocompletion))
### Aliases
You can also configure `bash` aliases for your program and they will also support completions.
```bash
alias aliasname=origcommand
complete -o default -F __start_origcommand aliasname
# and now when you run `aliasname` completion will make
# suggestions as it did for `origcommand`.
$ aliasname <tab><tab>
completion firstcommand secondcommand
```
### Bash legacy dynamic completions
For backward compatibility, Cobra still supports its bash legacy dynamic completion solution.
Please refer to [Bash Completions](bash_completions.md) for details.
### Bash completion V2
Cobra provides two versions for bash completion. The original bash completion (which started it all!) can be used by calling
`GenBashCompletion()` or `GenBashCompletionFile()`.
A new V2 bash completion version is also available. This version can be used by calling `GenBashCompletionV2()` or
`GenBashCompletionFileV2()`. The V2 version does **not** support the legacy dynamic completion
(see [Bash Completions](bash_completions.md)) but instead works only with the Go dynamic completion
solution described in this document.
Unless your program already uses the legacy dynamic completion solution, it is recommended that you use the bash
completion V2 solution which provides the following extra features:
- Supports completion descriptions (like the other shells)
- Small completion script of less than 300 lines (v1 generates scripts of thousands of lines; `kubectl` for example has a bash v1 completion script of over 13K lines)
- Streamlined user experience thanks to a completion behavior aligned with the other shells
`Bash` completion V2 supports descriptions for completions. When calling `GenBashCompletionV2()` or `GenBashCompletionFileV2()`
you must provide these functions with a parameter indicating if the completions should be annotated with a description; Cobra
will provide the description automatically based on usage information. You can choose to make this option configurable by
your users.
```
# With descriptions
$ helm s[tab][tab]
search (search for a keyword in charts) status (display the status of the named release)
show (show information of a chart)
# Without descriptions
$ helm s[tab][tab]
search show status
```
**Note**: Cobra's default `completion` command uses bash completion V2. If for some reason you need to use bash completion V1, you will need to implement your own `completion` command.
## Zsh completions
Cobra supports native zsh completion generated from the root `cobra.Command`.
The generated completion script should be put somewhere in your `$fpath` and be named
`_<yourProgram>`. You will need to start a new shell for the completions to become available.
Zsh supports descriptions for completions. Cobra will provide the description automatically,
based on usage information. Cobra provides a way to completely disable such descriptions by
using `GenZshCompletionNoDesc()` or `GenZshCompletionFileNoDesc()`. You can choose to make
this a configurable option to your users.
```
# With descriptions
$ helm s[tab]
search -- search for a keyword in charts
show -- show information of a chart
status -- displays the status of the named release
# Without descriptions
$ helm s[tab]
search show status
```
*Note*: Because of backward-compatibility requirements, we were forced to have a different API to disable completion descriptions between `zsh` and `fish`.
### Limitations
* Custom completions implemented in Bash scripting (legacy) are not supported and will be ignored for `zsh` (including the use of the `BashCompCustom` flag annotation).
* You should instead use `ValidArgsFunction` and `RegisterFlagCompletionFunc()` which are portable to the different shells (`bash`, `zsh`, `fish`, `powershell`).
* The function `MarkFlagCustom()` is not supported and will be ignored for `zsh`.
* You should instead use `RegisterFlagCompletionFunc()`.
### Zsh completions standardization
Cobra 1.1 standardized its zsh completion support to align it with its other shell completions. Although the API was kept backward-compatible, some small changes in behavior were introduced.
Please refer to [Zsh Completions](zsh_completions.md) for details.
## fish completions
Cobra supports native fish completions generated from the root `cobra.Command`. You can use the `command.GenFishCompletion()` or `command.GenFishCompletionFile()` functions. You must provide these functions with a parameter indicating if the completions should be annotated with a description; Cobra will provide the description automatically based on usage information. You can choose to make this option configurable by your users.
```
# With descriptions
$ helm s[tab]
search (search for a keyword in charts) show (show information of a chart) status (displays the status of the named release)
# Without descriptions
$ helm s[tab]
search show status
```
*Note*: Because of backward-compatibility requirements, we were forced to have a different API to disable completion descriptions between `zsh` and `fish`.
### Limitations
* Custom completions implemented in bash scripting (legacy) are not supported and will be ignored for `fish` (including the use of the `BashCompCustom` flag annotation).
* You should instead use `ValidArgsFunction` and `RegisterFlagCompletionFunc()` which are portable to the different shells (`bash`, `zsh`, `fish`, `powershell`).
* The function `MarkFlagCustom()` is not supported and will be ignored for `fish`.
* You should instead use `RegisterFlagCompletionFunc()`.
* The following flag completion annotations are not supported and will be ignored for `fish`:
* `BashCompFilenameExt` (filtering by file extension)
* `BashCompSubdirsInDir` (filtering by directory)
* The functions corresponding to the above annotations are consequently not supported and will be ignored for `fish`:
* `MarkFlagFilename()` and `MarkPersistentFlagFilename()` (filtering by file extension)
* `MarkFlagDirname()` and `MarkPersistentFlagDirname()` (filtering by directory)
* Similarly, the following completion directives are not supported and will be ignored for `fish`:
* `ShellCompDirectiveFilterFileExt` (filtering by file extension)
* `ShellCompDirectiveFilterDirs` (filtering by directory)
## PowerShell completions
Cobra supports native PowerShell completions generated from the root `cobra.Command`. You can use the `command.GenPowerShellCompletion()` or `command.GenPowerShellCompletionFile()` functions. To include descriptions use `command.GenPowerShellCompletionWithDesc()` and `command.GenPowerShellCompletionFileWithDesc()`. Cobra will provide the description automatically based on usage information. You can choose to make this option configurable by your users.
The script is designed to support all three PowerShell completion modes:
* TabCompleteNext (default windows style - on each key press the next option is displayed)
* Complete (works like bash)
* MenuComplete (works like zsh)
You set the mode with `Set-PSReadLineKeyHandler -Key Tab -Function <mode>`. Descriptions are only displayed when using the `Complete` or `MenuComplete` mode.
Users need PowerShell version 5.0 or above, which comes with Windows 10 and can be downloaded separately for Windows 7 or 8.1. They can then write the completions to a file and source this file from their PowerShell profile, which is referenced by the `$Profile` environment variable. See `Get-Help about_Profiles` for more info about PowerShell profiles.
```
# With descriptions and Mode 'Complete'
$ helm s[tab]
search (search for a keyword in charts) show (show information of a chart) status (displays the status of the named release)
# With descriptions and Mode 'MenuComplete' The description of the current selected value will be displayed below the suggestions.
$ helm s[tab]
search show status
search for a keyword in charts
# Without descriptions
$ helm s[tab]
search show status
```
### Aliases
You can also configure `powershell` aliases for your program and they will also support completions.
```
$ sal aliasname origcommand
$ Register-ArgumentCompleter -CommandName 'aliasname' -ScriptBlock $__origcommandCompleterBlock
# and now when you run `aliasname` completion will make
# suggestions as it did for `origcommand`.
$ aliasname <tab>
completion firstcommand secondcommand
```
The name of the completer block variable is of the form `$__<programName>CompleterBlock` where every `-` and `:` in the program name have been replaced with `_`, to respect powershell naming syntax.
### Limitations
* Custom completions implemented in bash scripting (legacy) are not supported and will be ignored for `powershell` (including the use of the `BashCompCustom` flag annotation).
* You should instead use `ValidArgsFunction` and `RegisterFlagCompletionFunc()` which are portable to the different shells (`bash`, `zsh`, `fish`, `powershell`).
* The function `MarkFlagCustom()` is not supported and will be ignored for `powershell`.
* You should instead use `RegisterFlagCompletionFunc()`.
* The following flag completion annotations are not supported and will be ignored for `powershell`:
* `BashCompFilenameExt` (filtering by file extension)
* `BashCompSubdirsInDir` (filtering by directory)
* The functions corresponding to the above annotations are consequently not supported and will be ignored for `powershell`:
* `MarkFlagFilename()` and `MarkPersistentFlagFilename()` (filtering by file extension)
* `MarkFlagDirname()` and `MarkPersistentFlagDirname()` (filtering by directory)
* Similarly, the following completion directives are not supported and will be ignored for `powershell`:
* `ShellCompDirectiveFilterFileExt` (filtering by file extension)
* `ShellCompDirectiveFilterDirs` (filtering by directory)

View file

@ -1,726 +0,0 @@
# User Guide
While you are welcome to provide your own organization, typically a Cobra-based
application will follow the following organizational structure:
```
▾ appName/
▾ cmd/
add.go
your.go
commands.go
here.go
main.go
```
In a Cobra app, typically the main.go file is very bare. It serves one purpose: initializing Cobra.
```go
package main
import (
"{pathToYourApp}/cmd"
)
func main() {
cmd.Execute()
}
```
## Using the Cobra Generator
Cobra-CLI is its own program that will create your application and add any
commands you want. It's the easiest way to incorporate Cobra into your application.
For complete details on using the Cobra generator, please refer to [The Cobra-CLI Generator README](https://github.com/spf13/cobra-cli/blob/main/README.md)
## Using the Cobra Library
To manually implement Cobra you need to create a bare main.go file and a rootCmd file.
You will optionally provide additional commands as you see fit.
### Create rootCmd
Cobra doesn't require any special constructors. Simply create your commands.
Ideally you place this in app/cmd/root.go:
```go
var rootCmd = &cobra.Command{
Use: "hugo",
Short: "Hugo is a very fast static site generator",
Long: `A Fast and Flexible Static Site Generator built with
love by spf13 and friends in Go.
Complete documentation is available at https://gohugo.io/documentation/`,
Run: func(cmd *cobra.Command, args []string) {
// Do Stuff Here
},
}
func Execute() {
if err := rootCmd.Execute(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
```
You will additionally define flags and handle configuration in your init() function.
For example cmd/root.go:
```go
package cmd
import (
"fmt"
"os"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
var (
// Used for flags.
cfgFile string
userLicense string
rootCmd = &cobra.Command{
Use: "cobra-cli",
Short: "A generator for Cobra based Applications",
Long: `Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.`,
}
)
// Execute executes the root command.
func Execute() error {
return rootCmd.Execute()
}
func init() {
cobra.OnInitialize(initConfig)
rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is $HOME/.cobra.yaml)")
rootCmd.PersistentFlags().StringP("author", "a", "YOUR NAME", "author name for copyright attribution")
rootCmd.PersistentFlags().StringVarP(&userLicense, "license", "l", "", "name of license for the project")
rootCmd.PersistentFlags().Bool("viper", true, "use Viper for configuration")
viper.BindPFlag("author", rootCmd.PersistentFlags().Lookup("author"))
viper.BindPFlag("useViper", rootCmd.PersistentFlags().Lookup("viper"))
viper.SetDefault("author", "NAME HERE <EMAIL ADDRESS>")
viper.SetDefault("license", "apache")
rootCmd.AddCommand(addCmd)
rootCmd.AddCommand(initCmd)
}
func initConfig() {
if cfgFile != "" {
// Use config file from the flag.
viper.SetConfigFile(cfgFile)
} else {
// Find home directory.
home, err := os.UserHomeDir()
cobra.CheckErr(err)
// Search config in home directory with name ".cobra" (without extension).
viper.AddConfigPath(home)
viper.SetConfigType("yaml")
viper.SetConfigName(".cobra")
}
viper.AutomaticEnv()
if err := viper.ReadInConfig(); err == nil {
fmt.Println("Using config file:", viper.ConfigFileUsed())
}
}
```
### Create your main.go
With the root command you need to have your main function execute it.
Execute should be run on the root for clarity, though it can be called on any command.
In a Cobra app, typically the main.go file is very bare. It serves one purpose: to initialize Cobra.
```go
package main
import (
"{pathToYourApp}/cmd"
)
func main() {
cmd.Execute()
}
```
### Create additional commands
Additional commands can be defined and typically are each given their own file
inside of the cmd/ directory.
If you wanted to create a version command you would create cmd/version.go and
populate it with the following:
```go
package cmd
import (
"fmt"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(versionCmd)
}
var versionCmd = &cobra.Command{
Use: "version",
Short: "Print the version number of Hugo",
Long: `All software has versions. This is Hugo's`,
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Hugo Static Site Generator v0.9 -- HEAD")
},
}
```
### Organizing subcommands
A command may have subcommands which in turn may have other subcommands. This is achieved by using
`AddCommand`. In some cases, especially in larger applications, each subcommand may be defined in
its own go package.
The suggested approach is for the parent command to use `AddCommand` to add its most immediate
subcommands. For example, consider the following directory structure:
```text
├── cmd
│   ├── root.go
│   └── sub1
│   ├── sub1.go
│   └── sub2
│   ├── leafA.go
│   ├── leafB.go
│   └── sub2.go
└── main.go
```
In this case:
* The `init` function of `root.go` adds the command defined in `sub1.go` to the root command.
* The `init` function of `sub1.go` adds the command defined in `sub2.go` to the sub1 command.
* The `init` function of `sub2.go` adds the commands defined in `leafA.go` and `leafB.go` to the
sub2 command.
This approach ensures the subcommands are always included at compile time while avoiding cyclic
references.
### Returning and handling errors
If you wish to return an error to the caller of a command, `RunE` can be used.
```go
package cmd
import (
"fmt"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(tryCmd)
}
var tryCmd = &cobra.Command{
Use: "try",
Short: "Try and possibly fail at something",
RunE: func(cmd *cobra.Command, args []string) error {
if err := someFunc(); err != nil {
return err
}
return nil
},
}
```
The error can then be caught at the execute function call.
## Working with Flags
Flags provide modifiers to control how the action command operates.
### Assign flags to a command
Since the flags are defined and used in different locations, we need to
define a variable outside with the correct scope to assign the flag to
work with.
```go
var Verbose bool
var Source string
```
There are two different approaches to assign a flag.
### Persistent Flags
A flag can be 'persistent', meaning that this flag will be available to the
command it's assigned to as well as every command under that command. For
global flags, assign a flag as a persistent flag on the root.
```go
rootCmd.PersistentFlags().BoolVarP(&Verbose, "verbose", "v", false, "verbose output")
```
### Local Flags
A flag can also be assigned locally, which will only apply to that specific command.
```go
localCmd.Flags().StringVarP(&Source, "source", "s", "", "Source directory to read from")
```
### Local Flag on Parent Commands
By default, Cobra only parses local flags on the target command, and any local flags on
parent commands are ignored. By enabling `Command.TraverseChildren`, Cobra will
parse local flags on each command before executing the target command.
```go
command := cobra.Command{
Use: "print [OPTIONS] [COMMANDS]",
TraverseChildren: true,
}
```
### Bind Flags with Config
You can also bind your flags with [viper](https://github.com/spf13/viper):
```go
var author string
func init() {
rootCmd.PersistentFlags().StringVar(&author, "author", "YOUR NAME", "Author name for copyright attribution")
viper.BindPFlag("author", rootCmd.PersistentFlags().Lookup("author"))
}
```
In this example, the persistent flag `author` is bound with `viper`.
**Note**: the variable `author` will not be set to the value from config,
when the `--author` flag is provided by user.
More in [viper documentation](https://github.com/spf13/viper#working-with-flags).
### Required flags
Flags are optional by default. If instead you wish your command to report an error
when a flag has not been set, mark it as required:
```go
rootCmd.Flags().StringVarP(&Region, "region", "r", "", "AWS region (required)")
rootCmd.MarkFlagRequired("region")
```
Or, for persistent flags:
```go
rootCmd.PersistentFlags().StringVarP(&Region, "region", "r", "", "AWS region (required)")
rootCmd.MarkPersistentFlagRequired("region")
```
### Flag Groups
If you have different flags that must be provided together (e.g. if they provide the `--username` flag they MUST provide the `--password` flag as well) then
Cobra can enforce that requirement:
```go
rootCmd.Flags().StringVarP(&u, "username", "u", "", "Username (required if password is set)")
rootCmd.Flags().StringVarP(&pw, "password", "p", "", "Password (required if username is set)")
rootCmd.MarkFlagsRequiredTogether("username", "password")
```
You can also prevent different flags from being provided together if they represent mutually
exclusive options such as specifying an output format as either `--json` or `--yaml` but never both:
```go
rootCmd.Flags().BoolVar(&ofJson, "json", false, "Output in JSON")
rootCmd.Flags().BoolVar(&ofYaml, "yaml", false, "Output in YAML")
rootCmd.MarkFlagsMutuallyExclusive("json", "yaml")
```
In both of these cases:
- both local and persistent flags can be used
- **NOTE:** the group is only enforced on commands where every flag is defined
- a flag may appear in multiple groups
- a group may contain any number of flags
## Positional and Custom Arguments
Validation of positional arguments can be specified using the `Args` field of `Command`.
The following validators are built in:
- Number of arguments:
- `NoArgs` - report an error if there are any positional args.
- `ArbitraryArgs` - accept any number of args.
- `MinimumNArgs(int)` - report an error if less than N positional args are provided.
- `MaximumNArgs(int)` - report an error if more than N positional args are provided.
- `ExactArgs(int)` - report an error if there are not exactly N positional args.
- `RangeArgs(min, max)` - report an error if the number of args is not between `min` and `max`.
- Content of the arguments:
- `OnlyValidArgs` - report an error if there are any positional args not specified in the `ValidArgs` field of `Command`, which can optionally be set to a list of valid values for positional args.
If `Args` is undefined or `nil`, it defaults to `ArbitraryArgs`.
Moreover, `MatchAll(pargs ...PositionalArgs)` enables combining existing checks with arbitrary other checks.
For instance, if you want to report an error if there are not exactly N positional args OR if there are any positional
args that are not in the `ValidArgs` field of `Command`, you can call `MatchAll` on `ExactArgs` and `OnlyValidArgs`, as
shown below:
```go
var cmd = &cobra.Command{
Short: "hello",
Args: cobra.MatchAll(cobra.ExactArgs(2), cobra.OnlyValidArgs),
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Hello, World!")
},
}
```
It is possible to set any custom validator that satisfies `func(cmd *cobra.Command, args []string) error`.
For example:
```go
var cmd = &cobra.Command{
Short: "hello",
Args: func(cmd *cobra.Command, args []string) error {
// Optionally run one of the validators provided by cobra
if err := cobra.MinimumNArgs(1)(cmd, args); err != nil {
return err
}
// Run the custom validation logic
if myapp.IsValidColor(args[0]) {
return nil
}
return fmt.Errorf("invalid color specified: %s", args[0])
},
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Hello, World!")
},
}
```
## Example
In the example below, we have defined three commands. Two are at the top level
and one (cmdTimes) is a child of one of the top commands. In this case the root
is not executable, meaning that a subcommand is required. This is accomplished
by not providing a 'Run' for the 'rootCmd'.
We have only defined one flag for a single command.
More documentation about flags is available at https://github.com/spf13/pflag
```go
package main
import (
"fmt"
"strings"
"github.com/spf13/cobra"
)
func main() {
var echoTimes int
var cmdPrint = &cobra.Command{
Use: "print [string to print]",
Short: "Print anything to the screen",
Long: `print is for printing anything back to the screen.
For many years people have printed back to the screen.`,
Args: cobra.MinimumNArgs(1),
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Print: " + strings.Join(args, " "))
},
}
var cmdEcho = &cobra.Command{
Use: "echo [string to echo]",
Short: "Echo anything to the screen",
Long: `echo is for echoing anything back.
Echo works a lot like print, except it has a child command.`,
Args: cobra.MinimumNArgs(1),
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Echo: " + strings.Join(args, " "))
},
}
var cmdTimes = &cobra.Command{
Use: "times [string to echo]",
Short: "Echo anything to the screen more times",
Long: `echo things multiple times back to the user by providing
a count and a string.`,
Args: cobra.MinimumNArgs(1),
Run: func(cmd *cobra.Command, args []string) {
for i := 0; i < echoTimes; i++ {
fmt.Println("Echo: " + strings.Join(args, " "))
}
},
}
cmdTimes.Flags().IntVarP(&echoTimes, "times", "t", 1, "times to echo the input")
var rootCmd = &cobra.Command{Use: "app"}
rootCmd.AddCommand(cmdPrint, cmdEcho)
cmdEcho.AddCommand(cmdTimes)
rootCmd.Execute()
}
```
For a more complete example of a larger application, please checkout [Hugo](https://gohugo.io/).
## Help Command
Cobra automatically adds a help command to your application when you have subcommands.
This will be called when a user runs 'app help'. Additionally, help will also
support all other commands as input. Say, for instance, you have a command called
'create' without any additional configuration; Cobra will work when 'app help
create' is called. Every command will automatically have the '--help' flag added.
### Example
The following output is automatically generated by Cobra. Nothing beyond the
command and flag definitions are needed.
$ cobra-cli help
Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.
Usage:
cobra-cli [command]
Available Commands:
add Add a command to a Cobra Application
completion Generate the autocompletion script for the specified shell
help Help about any command
init Initialize a Cobra Application
Flags:
-a, --author string author name for copyright attribution (default "YOUR NAME")
--config string config file (default is $HOME/.cobra.yaml)
-h, --help help for cobra-cli
-l, --license string name of license for the project
--viper use Viper for configuration
Use "cobra-cli [command] --help" for more information about a command.
Help is just a command like any other. There is no special logic or behavior
around it. In fact, you can provide your own if you want.
### Grouping commands in help
Cobra supports grouping of available commands in the help output. To group commands, each group must be explicitly
defined using `AddGroup()` on the parent command. Then a subcommand can be added to a group using the `GroupID` element
of that subcommand. The groups will appear in the help output in the same order as they are defined using different
calls to `AddGroup()`. If you use the generated `help` or `completion` commands, you can set their group ids using
`SetHelpCommandGroupId()` and `SetCompletionCommandGroupId()` on the root command, respectively.
### Defining your own help
You can provide your own Help command or your own template for the default command to use
with the following functions:
```go
cmd.SetHelpCommand(cmd *Command)
cmd.SetHelpFunc(f func(*Command, []string))
cmd.SetHelpTemplate(s string)
```
The latter two will also apply to any children commands.
## Usage Message
When the user provides an invalid flag or invalid command, Cobra responds by
showing the user the 'usage'.
### Example
You may recognize this from the help above. That's because the default help
embeds the usage as part of its output.
$ cobra-cli --invalid
Error: unknown flag: --invalid
Usage:
cobra-cli [command]
Available Commands:
add Add a command to a Cobra Application
completion Generate the autocompletion script for the specified shell
help Help about any command
init Initialize a Cobra Application
Flags:
-a, --author string author name for copyright attribution (default "YOUR NAME")
--config string config file (default is $HOME/.cobra.yaml)
-h, --help help for cobra-cli
-l, --license string name of license for the project
--viper use Viper for configuration
Use "cobra [command] --help" for more information about a command.
### Defining your own usage
You can provide your own usage function or template for Cobra to use.
Like help, the function and template are overridable through public methods:
```go
cmd.SetUsageFunc(f func(*Command) error)
cmd.SetUsageTemplate(s string)
```
## Version Flag
Cobra adds a top-level '--version' flag if the Version field is set on the root command.
Running an application with the '--version' flag will print the version to stdout using
the version template. The template can be customized using the
`cmd.SetVersionTemplate(s string)` function.
## PreRun and PostRun Hooks
It is possible to run functions before or after the main `Run` function of your command. The `PersistentPreRun` and `PreRun` functions will be executed before `Run`. `PersistentPostRun` and `PostRun` will be executed after `Run`. The `Persistent*Run` functions will be inherited by children if they do not declare their own. These functions are run in the following order:
- `PersistentPreRun`
- `PreRun`
- `Run`
- `PostRun`
- `PersistentPostRun`
An example of two commands which use all of these features is below. When the subcommand is executed, it will run the root command's `PersistentPreRun` but not the root command's `PersistentPostRun`:
```go
package main
import (
"fmt"
"github.com/spf13/cobra"
)
func main() {
var rootCmd = &cobra.Command{
Use: "root [sub]",
Short: "My root command",
PersistentPreRun: func(cmd *cobra.Command, args []string) {
fmt.Printf("Inside rootCmd PersistentPreRun with args: %v\n", args)
},
PreRun: func(cmd *cobra.Command, args []string) {
fmt.Printf("Inside rootCmd PreRun with args: %v\n", args)
},
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("Inside rootCmd Run with args: %v\n", args)
},
PostRun: func(cmd *cobra.Command, args []string) {
fmt.Printf("Inside rootCmd PostRun with args: %v\n", args)
},
PersistentPostRun: func(cmd *cobra.Command, args []string) {
fmt.Printf("Inside rootCmd PersistentPostRun with args: %v\n", args)
},
}
var subCmd = &cobra.Command{
Use: "sub [no options!]",
Short: "My subcommand",
PreRun: func(cmd *cobra.Command, args []string) {
fmt.Printf("Inside subCmd PreRun with args: %v\n", args)
},
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("Inside subCmd Run with args: %v\n", args)
},
PostRun: func(cmd *cobra.Command, args []string) {
fmt.Printf("Inside subCmd PostRun with args: %v\n", args)
},
PersistentPostRun: func(cmd *cobra.Command, args []string) {
fmt.Printf("Inside subCmd PersistentPostRun with args: %v\n", args)
},
}
rootCmd.AddCommand(subCmd)
rootCmd.SetArgs([]string{""})
rootCmd.Execute()
fmt.Println()
rootCmd.SetArgs([]string{"sub", "arg1", "arg2"})
rootCmd.Execute()
}
```
Output:
```
Inside rootCmd PersistentPreRun with args: []
Inside rootCmd PreRun with args: []
Inside rootCmd Run with args: []
Inside rootCmd PostRun with args: []
Inside rootCmd PersistentPostRun with args: []
Inside rootCmd PersistentPreRun with args: [arg1 arg2]
Inside subCmd PreRun with args: [arg1 arg2]
Inside subCmd Run with args: [arg1 arg2]
Inside subCmd PostRun with args: [arg1 arg2]
Inside subCmd PersistentPostRun with args: [arg1 arg2]
```
## Suggestions when "unknown command" happens
Cobra will print automatic suggestions when "unknown command" errors happen. This allows Cobra to behave similarly to the `git` command when a typo happens. For example:
```
$ hugo srever
Error: unknown command "srever" for "hugo"
Did you mean this?
server
Run 'hugo --help' for usage.
```
Suggestions are automatically generated based on existing subcommands and use an implementation of [Levenshtein distance](https://en.wikipedia.org/wiki/Levenshtein_distance). Every registered command that matches a minimum distance of 2 (ignoring case) will be displayed as a suggestion.
If you need to disable suggestions or tweak the string distance in your command, use:
```go
command.DisableSuggestions = true
```
or
```go
command.SuggestionsMinimumDistance = 1
```
You can also explicitly set names for which a given command will be suggested using the `SuggestFor` attribute. This allows suggestions for strings that are not close in terms of string distance, but make sense in your set of commands but for which
you don't want aliases. Example:
```
$ kubectl remove
Error: unknown command "remove" for "kubectl"
Did you mean this?
delete
Run 'kubectl help' for usage.
```
## Generating documentation for your command
Cobra can generate documentation based on subcommands, flags, etc. Read more about it in the [docs generation documentation](doc/README.md).
## Generating shell completions
Cobra can generate a shell-completion file for the following shells: bash, zsh, fish, PowerShell. If you add more information to your commands, these completions can be amazingly powerful and flexible. Read more about it in [Shell Completions](shell_completions.md).
## Providing Active Help
Cobra makes use of the shell-completion system to define a framework allowing you to provide Active Help to your users. Active Help are messages (hints, warnings, etc) printed as the program is being used. Read more about it in [Active Help](active_help.md).

View file

@ -1,48 +0,0 @@
## Generating Zsh Completion For Your cobra.Command
Please refer to [Shell Completions](shell_completions.md) for details.
## Zsh completions standardization
Cobra 1.1 standardized its zsh completion support to align it with its other shell completions. Although the API was kept backwards-compatible, some small changes in behavior were introduced.
### Deprecation summary
See further below for more details on these deprecations.
* `cmd.MarkZshCompPositionalArgumentFile(pos, []string{})` is no longer needed. It is therefore **deprecated** and silently ignored.
* `cmd.MarkZshCompPositionalArgumentFile(pos, glob[])` is **deprecated** and silently ignored.
* Instead use `ValidArgsFunction` with `ShellCompDirectiveFilterFileExt`.
* `cmd.MarkZshCompPositionalArgumentWords()` is **deprecated** and silently ignored.
* Instead use `ValidArgsFunction`.
### Behavioral changes
**Noun completion**
|Old behavior|New behavior|
|---|---|
|No file completion by default (opposite of bash)|File completion by default; use `ValidArgsFunction` with `ShellCompDirectiveNoFileComp` to turn off file completion on a per-argument basis|
|Completion of flag names without the `-` prefix having been typed|Flag names are only completed if the user has typed the first `-`|
`cmd.MarkZshCompPositionalArgumentFile(pos, []string{})` used to turn on file completion on a per-argument position basis|File completion for all arguments by default; `cmd.MarkZshCompPositionalArgumentFile()` is **deprecated** and silently ignored|
|`cmd.MarkZshCompPositionalArgumentFile(pos, glob[])` used to turn on file completion **with glob filtering** on a per-argument position basis (zsh-specific)|`cmd.MarkZshCompPositionalArgumentFile()` is **deprecated** and silently ignored; use `ValidArgsFunction` with `ShellCompDirectiveFilterFileExt` for file **extension** filtering (not full glob filtering)|
|`cmd.MarkZshCompPositionalArgumentWords(pos, words[])` used to provide completion choices on a per-argument position basis (zsh-specific)|`cmd.MarkZshCompPositionalArgumentWords()` is **deprecated** and silently ignored; use `ValidArgsFunction` to achieve the same behavior|
**Flag-value completion**
|Old behavior|New behavior|
|---|---|
|No file completion by default (opposite of bash)|File completion by default; use `RegisterFlagCompletionFunc()` with `ShellCompDirectiveNoFileComp` to turn off file completion|
|`cmd.MarkFlagFilename(flag, []string{})` and similar used to turn on file completion|File completion by default; `cmd.MarkFlagFilename(flag, []string{})` no longer needed in this context and silently ignored|
|`cmd.MarkFlagFilename(flag, glob[])` used to turn on file completion **with glob filtering** (syntax of `[]string{"*.yaml", "*.yml"}` incompatible with bash)|Will continue to work, however, support for bash syntax is added and should be used instead so as to work for all shells (`[]string{"yaml", "yml"}`)|
|`cmd.MarkFlagDirname(flag)` only completes directories (zsh-specific)|Has been added for all shells|
|Completion of a flag name does not repeat, unless flag is of type `*Array` or `*Slice` (not supported by bash)|Retained for `zsh` and added to `fish`|
|Completion of a flag name does not provide the `=` form (unlike bash)|Retained for `zsh` and added to `fish`|
**Improvements**
* Custom completion support (`ValidArgsFunction` and `RegisterFlagCompletionFunc()`)
* File completion by default if no other completions found
* Handling of required flags
* File extension filtering no longer mutually exclusive with bash usage
* Completion of directory names *within* another directory
* Support for `=` form of flags

View file

@ -21,7 +21,6 @@ import (
"flag"
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"sort"
@ -195,7 +194,7 @@ func Run(args []string) int {
}
fs := flag.NewFlagSet("", flag.ContinueOnError)
fs.SetOutput(ioutil.Discard)
fs.SetOutput(io.Discard)
ctx := context.Background()

View file

@ -24,7 +24,6 @@ import (
"flag"
"fmt"
"io"
"io/ioutil"
"net/url"
"os"
"path"
@ -63,7 +62,7 @@ func (f *ArchiveFlag) ReadOvf(fpath string) ([]byte, error) {
}
defer r.Close()
return ioutil.ReadAll(r)
return io.ReadAll(r)
}
func (f *ArchiveFlag) ReadEnvelope(data []byte) (*ovf.Envelope, error) {

View file

@ -5,7 +5,7 @@ Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
@ -17,6 +17,7 @@ limitations under the License.
package importx
import (
"bytes"
"context"
"errors"
"flag"
@ -47,6 +48,7 @@ type ovfx struct {
Name string
VerifyManifest bool
Hidden bool
Client *vim25.Client
Datacenter *object.Datacenter
@ -77,6 +79,7 @@ func (cmd *ovfx) Register(ctx context.Context, f *flag.FlagSet) {
f.StringVar(&cmd.Name, "name", "", "Name to use for new entity")
f.BoolVar(&cmd.VerifyManifest, "m", false, "Verify checksum of uploaded files against manifest (.mf)")
f.BoolVar(&cmd.Hidden, "hidden", false, "Enable hidden properties")
}
func (cmd *ovfx) Process(ctx context.Context) error {
@ -243,6 +246,13 @@ func (cmd *ovfx) Import(fpath string) (*types.ManagedObjectReference, error) {
if e.VirtualSystem.Name != nil {
name = *e.VirtualSystem.Name
}
if cmd.Hidden {
// TODO: userConfigurable is optional and defaults to false, so we should *add* userConfigurable=true
// if not set for a Property. But, there'd be a bunch more work involved to preserve other data in doing
// a complete xml.Marshal of the .ovf
o = bytes.ReplaceAll(o, []byte(`userConfigurable="false"`), []byte(`userConfigurable="true"`))
}
}
// Override name from options if specified

View file

@ -5,7 +5,7 @@ Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
@ -59,6 +59,8 @@ type spec struct {
*ArchiveFlag
*flags.ClientFlag
*flags.OutputFlag
hidden bool
}
func init() {
@ -73,6 +75,8 @@ func (cmd *spec) Register(ctx context.Context, f *flag.FlagSet) {
cmd.OutputFlag, ctx = flags.NewOutputFlag(ctx)
cmd.OutputFlag.Register(ctx, f)
f.BoolVar(&cmd.hidden, "hidden", false, "Enable hidden properties")
}
func (cmd *spec) Process(ctx context.Context) error {
@ -147,7 +151,10 @@ func (cmd *spec) Map(e *ovf.Envelope) (res []Property) {
for _, p := range e.VirtualSystem.Product {
for i, v := range p.Property {
if v.UserConfigurable == nil || !*v.UserConfigurable {
if v.UserConfigurable == nil {
continue
}
if !*v.UserConfigurable && !cmd.hidden {
continue
}
@ -161,17 +168,8 @@ func (cmd *spec) Map(e *ovf.Envelope) (res []Property) {
d = strings.Title(d)
}
// From OVF spec, section 9.5.1:
// key-value-env = [class-value "."] key-value-prod ["." instance-value]
k := v.Key
if p.Class != nil {
k = fmt.Sprintf("%s.%s", *p.Class, k)
}
if p.Instance != nil {
k = fmt.Sprintf("%s.%s", k, *p.Instance)
}
np := Property{KeyValue: KeyValue{Key: p.Key(v), Value: d}}
np := Property{KeyValue: KeyValue{Key: k, Value: d}}
if cmd.Verbose() {
np.Spec = &p.Property[i]
}

View file

@ -20,7 +20,7 @@ import (
"context"
"flag"
"fmt"
"io/ioutil"
"os"
"reflect"
"strings"
@ -58,7 +58,7 @@ func (e *extraConfigFile) Set(v string) error {
var fileContents = ""
if len(r[1]) > 0 {
contents, err := ioutil.ReadFile(r[1])
contents, err := os.ReadFile(r[1])
if err != nil {
return fmt.Errorf("failed to parse extraConfigFile '%s': %w", v, err)
}

View file

@ -21,5 +21,5 @@ const (
ClientName = "govmomi"
// ClientVersion is the version of this SDK
ClientVersion = "0.32.0"
ClientVersion = "0.33.0"
)

View file

@ -1,11 +1,11 @@
/*
Copyright (c) 2015 VMware, Inc. All Rights Reserved.
Copyright (c) 2015-2023 VMware, Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
@ -16,6 +16,10 @@ limitations under the License.
package ovf
import (
"fmt"
)
type Envelope struct {
References []File `xml:"References>File"`
@ -61,6 +65,7 @@ type Content struct {
type Section struct {
Required *bool `xml:"required,attr"`
Info string `xml:"Info"`
Category string `xml:"Category"`
}
type AnnotationSection struct {
@ -85,6 +90,20 @@ type ProductSection struct {
Property []Property `xml:"Property"`
}
func (p ProductSection) Key(prop Property) string {
// From OVF spec, section 9.5.1:
// key-value-env = [class-value "."] key-value-prod ["." instance-value]
k := prop.Key
if p.Class != nil {
k = fmt.Sprintf("%s.%s", *p.Class, k)
}
if p.Instance != nil {
k = fmt.Sprintf("%s.%s", k, *p.Instance)
}
return k
}
type Property struct {
Key string `xml:"key,attr"`
Type string `xml:"type,attr"`

View file

@ -285,3 +285,31 @@ func (c *Client) GetProfileNameByID(ctx context.Context, profileID string) (stri
}
return "", fmt.Errorf("no pbm profile found with id: %q", profileID)
}
func (c *Client) QueryAssociatedProfile(ctx context.Context, entity types.PbmServerObjectRef) ([]types.PbmProfileId, error) {
req := types.PbmQueryAssociatedProfile{
This: c.ServiceContent.ProfileManager,
Entity: entity,
}
res, err := methods.PbmQueryAssociatedProfile(ctx, c, &req)
if err != nil {
return nil, err
}
return res.Returnval, nil
}
func (c *Client) QueryAssociatedProfiles(ctx context.Context, entities []types.PbmServerObjectRef) ([]types.PbmQueryProfileResult, error) {
req := types.PbmQueryAssociatedProfiles{
This: c.ServiceContent.ProfileManager,
Entities: entities,
}
res, err := methods.PbmQueryAssociatedProfiles(ctx, c, &req)
if err != nil {
return nil, err
}
return res.Returnval, nil
}

View file

@ -21,7 +21,6 @@ import (
"crypto/sha1"
"encoding/json"
"fmt"
"io/ioutil"
"net/url"
"os"
"os/user"
@ -187,7 +186,7 @@ func localTicket(ctx context.Context, m *session.Manager) (*url.Userinfo, error)
return nil, err
}
password, err := ioutil.ReadFile(ticket.PasswordFilePath)
password, err := os.ReadFile(ticket.PasswordFilePath)
if err != nil {
return nil, err
}

View file

@ -18,7 +18,6 @@ package session
import (
"context"
"io/ioutil"
"net/url"
"os"
"strings"
@ -47,7 +46,7 @@ func Secret(value string) (string, error) {
if len(value) == 0 {
return value, nil
}
contents, err := ioutil.ReadFile(value)
contents, err := os.ReadFile(value)
if err != nil {
if os.IsPermission(err) {
return "", err

View file

@ -22,7 +22,6 @@ import (
"encoding/json"
"fmt"
"io"
"io/ioutil"
"net/http"
"net/url"
"strings"
@ -32,6 +31,7 @@ import (
"github.com/vmware/govmomi/vapi/internal"
"github.com/vmware/govmomi/vim25"
"github.com/vmware/govmomi/vim25/soap"
"github.com/vmware/govmomi/vim25/types"
)
// Client extends soap.Client to support JSON encoding, while inheriting security features, debug tracing and session persistence.
@ -180,6 +180,11 @@ func (c *Client) Do(ctx context.Context, req *http.Request, resBody interface{})
}
}
// OperationID (see soap.Client.soapRoundTrip)
if id, ok := ctx.Value(types.ID{}).(string); ok {
req.Header.Add("X-Request-ID", id)
}
if headers, ok := ctx.Value(headersContext{}).(http.Header); ok {
for k, v := range headers {
for _, v := range v {
@ -195,7 +200,7 @@ func (c *Client) Do(ctx context.Context, req *http.Request, resBody interface{})
case http.StatusNoContent:
case http.StatusBadRequest:
// TODO: structured error types
detail, err := ioutil.ReadAll(res.Body)
detail, err := io.ReadAll(res.Body)
if err != nil {
return err
}

View file

@ -34,29 +34,41 @@ func NewListView(c *vim25.Client, ref types.ManagedObjectReference) *ListView {
}
}
func (v ListView) Add(ctx context.Context, refs []types.ManagedObjectReference) error {
func (v ListView) Add(ctx context.Context, refs []types.ManagedObjectReference) ([]types.ManagedObjectReference, error) {
req := types.ModifyListView{
This: v.Reference(),
Add: refs,
}
_, err := methods.ModifyListView(ctx, v.Client(), &req)
return err
res, err := methods.ModifyListView(ctx, v.Client(), &req)
if err != nil {
return nil, err
}
return res.Returnval, nil
}
func (v ListView) Remove(ctx context.Context, refs []types.ManagedObjectReference) error {
func (v ListView) Remove(ctx context.Context, refs []types.ManagedObjectReference) ([]types.ManagedObjectReference, error) {
req := types.ModifyListView{
This: v.Reference(),
Remove: refs,
}
_, err := methods.ModifyListView(ctx, v.Client(), &req)
return err
res, err := methods.ModifyListView(ctx, v.Client(), &req)
if err != nil {
return nil, err
}
return res.Returnval, nil
}
func (v ListView) Reset(ctx context.Context, refs []types.ManagedObjectReference) error {
func (v ListView) Reset(ctx context.Context, refs []types.ManagedObjectReference) ([]types.ManagedObjectReference, error) {
req := types.ResetListView{
This: v.Reference(),
Obj: refs,
}
_, err := methods.ResetListView(ctx, v.Client(), &req)
return err
res, err := methods.ResetListView(ctx, v.Client(), &req)
if err != nil {
return nil, err
}
return res.Returnval, nil
}

View file

@ -73,7 +73,7 @@ func (v TaskView) Collect(ctx context.Context, f func([]types.TaskInfo)) error {
tasks = change.Val.(types.ArrayOfManagedObjectReference).ManagedObjectReference
if len(tasks) != 0 {
reset = func() {
_ = v.Reset(ctx, tasks)
_, _ = v.Reset(ctx, tasks)
// Remember any tasks we've reported as complete already,
// to avoid reporting multiple times when Reset is triggered.
@ -113,7 +113,7 @@ func (v TaskView) Collect(ctx context.Context, f func([]types.TaskInfo)) error {
if reset != nil {
reset()
} else if len(prune) != 0 {
_ = v.Remove(ctx, prune)
_, _ = v.Remove(ctx, prune)
}
if len(tasks) != 0 && len(infos) == 0 {

View file

@ -27,7 +27,6 @@ import (
"errors"
"fmt"
"io"
"io/ioutil"
"log"
"net"
"net/http"
@ -267,7 +266,7 @@ func (c *Client) SetRootCAs(pemPaths string) error {
pool := x509.NewCertPool()
for _, name := range filepath.SplitList(pemPaths) {
pem, err := ioutil.ReadFile(filepath.Clean(name))
pem, err := os.ReadFile(filepath.Clean(name))
if err != nil {
return err
}
@ -319,6 +318,20 @@ func (c *Client) Thumbprint(host string) string {
return c.hosts[host]
}
// KnownThumbprint checks whether the provided thumbprint is known to this client.
func (c *Client) KnownThumbprint(tp string) bool {
c.hostsMu.Lock()
defer c.hostsMu.Unlock()
for _, v := range c.hosts {
if v == tp {
return true
}
}
return false
}
// LoadThumbprints from file with the give name.
// If name is empty or name does not exist this function will return nil.
func (c *Client) LoadThumbprints(file string) error {

View file

@ -3,7 +3,6 @@
// license that can be found in the LICENSE file.
//go:build go1.12
// +build go1.12
package acme

View file

@ -3,7 +3,6 @@
// license that can be found in the LICENSE file.
//go:build gc && !purego
// +build gc,!purego
package chacha20

View file

@ -3,7 +3,6 @@
// license that can be found in the LICENSE file.
//go:build gc && !purego
// +build gc,!purego
#include "textflag.h"

View file

@ -3,7 +3,6 @@
// license that can be found in the LICENSE file.
//go:build (!arm64 && !s390x && !ppc64le) || !gc || purego
// +build !arm64,!s390x,!ppc64le !gc purego
package chacha20

View file

@ -3,7 +3,6 @@
// license that can be found in the LICENSE file.
//go:build gc && !purego
// +build gc,!purego
package chacha20

View file

@ -20,7 +20,6 @@
// due to the calling conventions and initialization of constants.
//go:build gc && !purego
// +build gc,!purego
#include "textflag.h"

View file

@ -3,7 +3,6 @@
// license that can be found in the LICENSE file.
//go:build gc && !purego
// +build gc,!purego
package chacha20

View file

@ -3,7 +3,6 @@
// license that can be found in the LICENSE file.
//go:build gc && !purego
// +build gc,!purego
#include "go_asm.h"
#include "textflag.h"

View file

@ -3,7 +3,6 @@
// license that can be found in the LICENSE file.
//go:build gc && !purego
// +build gc,!purego
package chacha20poly1305

View file

@ -5,7 +5,6 @@
// This file was originally from https://golang.org/cl/24717 by Vlad Krasnov of CloudFlare.
//go:build gc && !purego
// +build gc,!purego
#include "textflag.h"
// General register allocation
@ -184,11 +183,31 @@ GLOBL ·andMask<>(SB), (NOPTR+RODATA), $240
#define shiftD1Right BYTE $0x66; BYTE $0x45; BYTE $0x0f; BYTE $0x3a; BYTE $0x0f; BYTE $0xd2; BYTE $0x04 // PALIGNR $4, X10, X10
#define shiftD2Right BYTE $0x66; BYTE $0x45; BYTE $0x0f; BYTE $0x3a; BYTE $0x0f; BYTE $0xdb; BYTE $0x04 // PALIGNR $4, X11, X11
#define shiftD3Right BYTE $0x66; BYTE $0x45; BYTE $0x0f; BYTE $0x3a; BYTE $0x0f; BYTE $0xff; BYTE $0x04 // PALIGNR $4, X15, X15
// Some macros
// ROL rotates the uint32s in register R left by N bits, using temporary T.
#define ROL(N, R, T) \
MOVO R, T; PSLLL $(N), T; PSRLL $(32-(N)), R; PXOR T, R
// ROL16 rotates the uint32s in register R left by 16, using temporary T if needed.
#ifdef GOAMD64_v2
#define ROL16(R, T) PSHUFB ·rol16<>(SB), R
#else
#define ROL16(R, T) ROL(16, R, T)
#endif
// ROL8 rotates the uint32s in register R left by 8, using temporary T if needed.
#ifdef GOAMD64_v2
#define ROL8(R, T) PSHUFB ·rol8<>(SB), R
#else
#define ROL8(R, T) ROL(8, R, T)
#endif
#define chachaQR(A, B, C, D, T) \
PADDD B, A; PXOR A, D; PSHUFB ·rol16<>(SB), D \
PADDD B, A; PXOR A, D; ROL16(D, T) \
PADDD D, C; PXOR C, B; MOVO B, T; PSLLL $12, T; PSRLL $20, B; PXOR T, B \
PADDD B, A; PXOR A, D; PSHUFB ·rol8<>(SB), D \
PADDD B, A; PXOR A, D; ROL8(D, T) \
PADDD D, C; PXOR C, B; MOVO B, T; PSLLL $7, T; PSRLL $25, B; PXOR T, B
#define chachaQR_AVX2(A, B, C, D, T) \

Some files were not shown because too many files have changed in this diff Show more