5a9d8c792b
V2 is compliant with api.openshift.com design guidelines. Errors are predefined, have codes, and are queryable. All requests have an operationId set: a unique identifier which is sortable by time. This is added to the response in case of an error. All returned objects have the href, id, and kind field set.
234 lines
8.8 KiB
Markdown
234 lines
8.8 KiB
Markdown
# ksuid [](https://goreportcard.com/report/github.com/segmentio/ksuid) [](https://godoc.org/github.com/segmentio/ksuid) [](https://circleci.com/gh/segmentio/ksuid.svg?style=shield)
|
|
|
|
ksuid is an efficient, comprehensive, battle-tested Go library for
|
|
generating and parsing a specific kind of globally unique identifier
|
|
called a *KSUID*. This library serves as its reference implementation.
|
|
|
|
## Install
|
|
```sh
|
|
go get -u github.com/segmentio/ksuid
|
|
```
|
|
|
|
## What is a KSUID?
|
|
|
|
KSUID is for K-Sortable Unique IDentifier. It is a kind of globally
|
|
unique identifier similar to a [RFC 4122 UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier), built from the ground-up to be "naturally"
|
|
sorted by generation timestamp without any special type-aware logic.
|
|
|
|
In short, running a set of KSUIDs through the UNIX `sort` command will result
|
|
in a list ordered by generation time.
|
|
|
|
## Why use KSUIDs?
|
|
|
|
There are numerous methods for generating unique identifiers, so why KSUID?
|
|
|
|
1. Naturally ordered by generation time
|
|
2. Collision-free, coordination-free, dependency-free
|
|
3. Highly portable representations
|
|
|
|
Even if only one of these properties are important to you, KSUID is a great
|
|
choice! :) Many projects chose to use KSUIDs *just* because the text
|
|
representation is copy-and-paste friendly.
|
|
|
|
### 1. Naturally Ordered By Generation Time
|
|
|
|
Unlike the more ubiquitous UUIDv4, a KSUID contains a timestamp component
|
|
that allows them to be loosely sorted by generation time. This is not a strong
|
|
guarantee (an invariant) as it depends on wall clocks, but is still incredibly
|
|
useful in practice. Both the binary and text representations will sort by
|
|
creation time without any special sorting logic.
|
|
|
|
### 2. Collision-free, Coordination-free, Dependency-free
|
|
|
|
While RFC 4122 UUIDv1s *do* include a time component, there aren't enough
|
|
bytes of randomness to provide strong protection against collisions
|
|
(duplicates). With such a low amount of entropy, it is feasible for a
|
|
malicious party to guess generated IDs, creating a problem for systems whose
|
|
security is, implicitly or explicitly, sensitive to an adversary guessing
|
|
identifiers.
|
|
|
|
To fit into a 64-bit number space, [Snowflake IDs](https://blog.twitter.com/2010/announcing-snowflake)
|
|
and its derivatives require coordination to avoid collisions, which
|
|
significantly increases the deployment complexity and operational burden.
|
|
|
|
A KSUID includes 128 bits of pseudorandom data ("entropy"). This number space
|
|
is 64 times larger than the 122 bits used by the well-accepted RFC 4122 UUIDv4
|
|
standard. The additional timestamp component can be considered "bonus entropy"
|
|
which further decreases the probability of collisions, to the point of physical
|
|
infeasibility in any practical implementation.
|
|
|
|
### Highly Portable Representations
|
|
|
|
The text *and* binary representations are lexicographically sortable, which
|
|
allows them to be dropped into systems which do not natively support KSUIDs
|
|
and retain their time-ordered property.
|
|
|
|
The text representation is an alphanumeric base62 encoding, so it "fits"
|
|
anywhere alphanumeric strings are accepted. No delimiters are used, so
|
|
stringified KSUIDs won't be inadvertently truncated or tokenized when
|
|
interpreted by software that is designed for human-readable text, a common
|
|
problem for the text representation of RFC 4122 UUIDs.
|
|
|
|
## How do KSUIDs work?
|
|
|
|
Binary KSUIDs are 20-bytes: a 32-bit unsigned integer UTC timestamp and
|
|
a 128-bit randomly generated payload. The timestamp uses big-endian
|
|
encoding, to support lexicographic sorting. The timestamp epoch is adjusted
|
|
to March 5th, 2014, providing over 100 years of life. The payload is
|
|
generated by a cryptographically-strong pseudorandom number generator.
|
|
|
|
The text representation is always 27 characters, encoded in alphanumeric
|
|
base62 that will lexicographically sort by timestamp.
|
|
|
|
## High Performance
|
|
|
|
This library is designed to be used in code paths that are performance
|
|
critical. Its code has been tuned to eliminate all non-essential
|
|
overhead. The `KSUID` type is derived from a fixed-size array, which
|
|
eliminates the additional reference chasing and allocation involved in
|
|
a variable-width type.
|
|
|
|
The API provides an interface for use in code paths which are sensitive
|
|
to allocation. For example, the `Append` method can be used to parse the
|
|
text representation and replace the contents of a `KSUID` value
|
|
without additional heap allocation.
|
|
|
|
All public package level "pure" functions are concurrency-safe, protected
|
|
by a global mutex. For hot loops that generate a large amount of KSUIDs
|
|
from a single Goroutine, the `Sequence` type is provided to elide the
|
|
potential contention.
|
|
|
|
By default, out of an abundance of caution, the cryptographically-secure
|
|
PRNG is used to generate the random bits of a KSUID. This can be relaxed
|
|
in extremely performance-critical code using the included `FastRander`
|
|
type. `FastRander` uses the standard PRNG with a seed generated by the
|
|
cryptographically-secure PRNG.
|
|
|
|
*_NOTE:_ While there is no evidence that `FastRander` will increase the
|
|
probability of a collision, it shouldn't be used in scenarios where
|
|
uniqueness is important to security, as there is an increased chance
|
|
the generated IDs can be predicted by an adversary.*
|
|
|
|
## Battle Tested
|
|
|
|
This code has been used in production at Segment for several years,
|
|
across a diverse array of projects. Trillions upon trillions of
|
|
KSUIDs have been generated in some of Segment's most
|
|
performance-critical, large-scale distributed systems.
|
|
|
|
## Plays Well With Others
|
|
|
|
Designed to be integrated with other libraries, the `KSUID` type
|
|
implements many standard library interfaces, including:
|
|
|
|
* `Stringer`
|
|
* `database/sql.Scanner` and `database/sql/driver.Valuer`
|
|
* `encoding.BinaryMarshal` and `encoding.BinaryUnmarshal`
|
|
* `encoding.TextMarshal` and `encoding.TextUnmarshal`
|
|
(`encoding/json` friendly!)
|
|
|
|
## Command Line Tool
|
|
|
|
This package comes with a command-line tool `ksuid`, useful for
|
|
generating KSUIDs as well as inspecting the internal components of
|
|
existing KSUIDs. Machine-friendly output is provided for scripting
|
|
use cases.
|
|
|
|
Given a Go build environment, it can be installed with the command:
|
|
|
|
```sh
|
|
$ go install github.com/segmentio/ksuid/cmd/ksuid
|
|
```
|
|
|
|
## CLI Usage Examples
|
|
|
|
### Generate a KSUID
|
|
|
|
```sh
|
|
$ ksuid
|
|
0ujsswThIGTUYm2K8FjOOfXtY1K
|
|
```
|
|
|
|
### Generate 4 KSUIDs
|
|
|
|
```sh
|
|
$ ksuid -n 4
|
|
0ujsszwN8NRY24YaXiTIE2VWDTS
|
|
0ujsswThIGTUYm2K8FjOOfXtY1K
|
|
0ujssxh0cECutqzMgbtXSGnjorm
|
|
0ujsszgFvbiEr7CDgE3z8MAUPFt
|
|
```
|
|
|
|
### Inspect the components of a KSUID
|
|
|
|
```sh
|
|
$ ksuid -f inspect 0ujtsYcgvSTl8PAuAdqWYSMnLOv
|
|
|
|
REPRESENTATION:
|
|
|
|
String: 0ujtsYcgvSTl8PAuAdqWYSMnLOv
|
|
Raw: 0669F7EFB5A1CD34B5F99D1154FB6853345C9735
|
|
|
|
COMPONENTS:
|
|
|
|
Time: 2017-10-09 21:00:47 -0700 PDT
|
|
Timestamp: 107608047
|
|
Payload: B5A1CD34B5F99D1154FB6853345C9735
|
|
```
|
|
|
|
### Generate a KSUID and inspect its components
|
|
|
|
```sh
|
|
$ ksuid -f inspect
|
|
|
|
REPRESENTATION:
|
|
|
|
String: 0ujzPyRiIAffKhBux4PvQdDqMHY
|
|
Raw: 066A029C73FC1AA3B2446246D6E89FCD909E8FE8
|
|
|
|
COMPONENTS:
|
|
|
|
Time: 2017-10-09 21:46:20 -0700 PDT
|
|
Timestamp: 107610780
|
|
Payload: 73FC1AA3B2446246D6E89FCD909E8FE8
|
|
|
|
```
|
|
|
|
### Inspect a KSUID with template formatted inspection output
|
|
|
|
```sh
|
|
$ ksuid -f template -t '{{ .Time }}: {{ .Payload }}' 0ujtsYcgvSTl8PAuAdqWYSMnLOv
|
|
2017-10-09 21:00:47 -0700 PDT: B5A1CD34B5F99D1154FB6853345C9735
|
|
```
|
|
|
|
### Inspect multiple KSUIDs with template formatted output
|
|
|
|
```sh
|
|
$ ksuid -f template -t '{{ .Time }}: {{ .Payload }}' $(ksuid -n 4)
|
|
2017-10-09 21:05:37 -0700 PDT: 304102BC687E087CC3A811F21D113CCF
|
|
2017-10-09 21:05:37 -0700 PDT: EAF0B240A9BFA55E079D887120D962F0
|
|
2017-10-09 21:05:37 -0700 PDT: DF0761769909ABB0C7BB9D66F79FC041
|
|
2017-10-09 21:05:37 -0700 PDT: 1A8F0E3D0BDEB84A5FAD702876F46543
|
|
```
|
|
|
|
### Generate KSUIDs and output JSON using template formatting
|
|
|
|
```sh
|
|
$ ksuid -f template -t '{ "timestamp": "{{ .Timestamp }}", "payload": "{{ .Payload }}", "ksuid": "{{.String}}"}' -n 4
|
|
{ "timestamp": "107611700", "payload": "9850EEEC191BF4FF26F99315CE43B0C8", "ksuid": "0uk1Hbc9dQ9pxyTqJ93IUrfhdGq"}
|
|
{ "timestamp": "107611700", "payload": "CC55072555316F45B8CA2D2979D3ED0A", "ksuid": "0uk1HdCJ6hUZKDgcxhpJwUl5ZEI"}
|
|
{ "timestamp": "107611700", "payload": "BA1C205D6177F0992D15EE606AE32238", "ksuid": "0uk1HcdvF0p8C20KtTfdRSB9XIm"}
|
|
{ "timestamp": "107611700", "payload": "67517BA309EA62AE7991B27BB6F2FCAC", "ksuid": "0uk1Ha7hGJ1Q9Xbnkt0yZgNwg3g"}
|
|
```
|
|
|
|
## Implementations for other languages
|
|
|
|
- Python: [svix-ksuid](https://github.com/svixhq/python-ksuid/)
|
|
- Ruby: [ksuid-ruby](https://github.com/michaelherold/ksuid-ruby)
|
|
- Java: [ksuid](https://github.com/ksuid/ksuid)
|
|
- Rust: [rksuid](https://github.com/nharring/rksuid)
|
|
- dotNet: [Ksuid.Net](https://github.com/JoyMoe/Ksuid.Net)
|
|
|
|
## License
|
|
|
|
ksuid source code is available under an MIT [License](/LICENSE.md).
|