Version 5.22 introduced a new option to /etc/containers/policy.json called
keyPaths, see
https://github.com/containers/image/pull/1609
EL9 immediately took advantage of this new feature and started using it, see
04645c4a84
This quickly became an issue in our code: The go library (containers/image)
parses the configuration file very strictly and refuses to create a client
when policy.json with an unknown key is present on the filesystem. As we
used 5.21.1 that doesn't know the new key, our unit tests started to
failing when containers-common was present.
Reproducer:
podman run --pull=always --rm -it centos:stream9
dnf install -y dnf-plugins-core
dnf config-manager --set-enabled crb
dnf install -y gpgme-devel libassuan-devel krb5-devel golang git-core
git clone https://github.com/osbuild/osbuild-composer
cd osbuild-composer
# install the new containers-common and run the test
dnf install -y https://kojihub.stream.centos.org/kojifiles/packages/containers-common/1/44.el9/x86_64/containers-common-1-44.el9.x86_64.rpm
go test -count 1 ./...
# this returns:
--- FAIL: TestClientResolve (0.00s)
client_test.go:31:
Error Trace: client_test.go:31
Error: Received unexpected error:
Unknown key "keyPaths"
invalid policy in "/etc/containers/policy.json"
github.com/containers/image/v5/signature.NewPolicyFromFile
/osbuild-composer/vendor/github.com/containers/image/v5/signature/policy_config.go:88
github.com/osbuild/osbuild-composer/internal/container.NewClient
/osbuild-composer/internal/container/client.go:123
github.com/osbuild/osbuild-composer/internal/container_test.TestClientResolve
/osbuild-composer/internal/container/client_test.go:29
testing.tRunner
/usr/lib/golang/src/testing/testing.go:1439
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1571
Test: TestClientResolve
client_test.go:32:
Error Trace: client_test.go:32
Error: Expected value not to be nil.
Test: TestClientResolve
When run with an older containers-common, it succeeds:
dnf install -y https://kojihub.stream.centos.org/kojifiles/packages/containers-common/1/40.el9/x86_64/containers-common-1-40.el9.x86_64.rpm
go test -count 1 ./...
PASS
To sum it up, I had to upgrade github.com/containers/image/v5 to v5.22.0.
Unfortunately, this wasn't so simple, see
go get github.com/containers/image/v5@latest
go: github.com/containers/image/v5@v5.22.0 requires
github.com/letsencrypt/boulder@v0.0.0-20220331220046-b23ab962616e requires
github.com/honeycombio/beeline-go@v1.1.1 requires
github.com/gobuffalo/pop/v5@v5.3.1 requires
github.com/mattn/go-sqlite3@v2.0.3+incompatible: reading github.com/mattn/go-sqlite3/go.mod at revision v2.0.3: unknown revision v2.0.3
It turns out that github.com/mattn/go-sqlite3@v2.0.3+incompatible has been
recently retracted https://github.com/mattn/go-sqlite3/pull/998 and this
broke a ton of packages depending on it. I was able to fix it by adding
exclude github.com/mattn/go-sqlite3 v2.0.3+incompatible
to our go.mod, see
https://github.com/mattn/go-sqlite3/issues/975#issuecomment-955661657
After adding it,
go get github.com/containers/image/v5@latest
succeeded and tools/prepare-source.sh took care of the rest.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
276 lines
8.1 KiB
Go
276 lines
8.1 KiB
Go
package toml
|
||
|
||
import (
|
||
"fmt"
|
||
"strings"
|
||
)
|
||
|
||
// ParseError is returned when there is an error parsing the TOML syntax.
|
||
//
|
||
// For example invalid syntax, duplicate keys, etc.
|
||
//
|
||
// In addition to the error message itself, you can also print detailed location
|
||
// information with context by using ErrorWithPosition():
|
||
//
|
||
// toml: error: Key 'fruit' was already created and cannot be used as an array.
|
||
//
|
||
// At line 4, column 2-7:
|
||
//
|
||
// 2 | fruit = []
|
||
// 3 |
|
||
// 4 | [[fruit]] # Not allowed
|
||
// ^^^^^
|
||
//
|
||
// Furthermore, the ErrorWithUsage() can be used to print the above with some
|
||
// more detailed usage guidance:
|
||
//
|
||
// toml: error: newlines not allowed within inline tables
|
||
//
|
||
// At line 1, column 18:
|
||
//
|
||
// 1 | x = [{ key = 42 #
|
||
// ^
|
||
//
|
||
// Error help:
|
||
//
|
||
// Inline tables must always be on a single line:
|
||
//
|
||
// table = {key = 42, second = 43}
|
||
//
|
||
// It is invalid to split them over multiple lines like so:
|
||
//
|
||
// # INVALID
|
||
// table = {
|
||
// key = 42,
|
||
// second = 43
|
||
// }
|
||
//
|
||
// Use regular for this:
|
||
//
|
||
// [table]
|
||
// key = 42
|
||
// second = 43
|
||
type ParseError struct {
|
||
Message string // Short technical message.
|
||
Usage string // Longer message with usage guidance; may be blank.
|
||
Position Position // Position of the error
|
||
LastKey string // Last parsed key, may be blank.
|
||
Line int // Line the error occurred. Deprecated: use Position.
|
||
|
||
err error
|
||
input string
|
||
}
|
||
|
||
// Position of an error.
|
||
type Position struct {
|
||
Line int // Line number, starting at 1.
|
||
Start int // Start of error, as byte offset starting at 0.
|
||
Len int // Lenght in bytes.
|
||
}
|
||
|
||
func (pe ParseError) Error() string {
|
||
msg := pe.Message
|
||
if msg == "" { // Error from errorf()
|
||
msg = pe.err.Error()
|
||
}
|
||
|
||
if pe.LastKey == "" {
|
||
return fmt.Sprintf("toml: line %d: %s", pe.Position.Line, msg)
|
||
}
|
||
return fmt.Sprintf("toml: line %d (last key %q): %s",
|
||
pe.Position.Line, pe.LastKey, msg)
|
||
}
|
||
|
||
// ErrorWithUsage() returns the error with detailed location context.
|
||
//
|
||
// See the documentation on ParseError.
|
||
func (pe ParseError) ErrorWithPosition() string {
|
||
if pe.input == "" { // Should never happen, but just in case.
|
||
return pe.Error()
|
||
}
|
||
|
||
var (
|
||
lines = strings.Split(pe.input, "\n")
|
||
col = pe.column(lines)
|
||
b = new(strings.Builder)
|
||
)
|
||
|
||
msg := pe.Message
|
||
if msg == "" {
|
||
msg = pe.err.Error()
|
||
}
|
||
|
||
// TODO: don't show control characters as literals? This may not show up
|
||
// well everywhere.
|
||
|
||
if pe.Position.Len == 1 {
|
||
fmt.Fprintf(b, "toml: error: %s\n\nAt line %d, column %d:\n\n",
|
||
msg, pe.Position.Line, col+1)
|
||
} else {
|
||
fmt.Fprintf(b, "toml: error: %s\n\nAt line %d, column %d-%d:\n\n",
|
||
msg, pe.Position.Line, col, col+pe.Position.Len)
|
||
}
|
||
if pe.Position.Line > 2 {
|
||
fmt.Fprintf(b, "% 7d | %s\n", pe.Position.Line-2, lines[pe.Position.Line-3])
|
||
}
|
||
if pe.Position.Line > 1 {
|
||
fmt.Fprintf(b, "% 7d | %s\n", pe.Position.Line-1, lines[pe.Position.Line-2])
|
||
}
|
||
fmt.Fprintf(b, "% 7d | %s\n", pe.Position.Line, lines[pe.Position.Line-1])
|
||
fmt.Fprintf(b, "% 10s%s%s\n", "", strings.Repeat(" ", col), strings.Repeat("^", pe.Position.Len))
|
||
return b.String()
|
||
}
|
||
|
||
// ErrorWithUsage() returns the error with detailed location context and usage
|
||
// guidance.
|
||
//
|
||
// See the documentation on ParseError.
|
||
func (pe ParseError) ErrorWithUsage() string {
|
||
m := pe.ErrorWithPosition()
|
||
if u, ok := pe.err.(interface{ Usage() string }); ok && u.Usage() != "" {
|
||
lines := strings.Split(strings.TrimSpace(u.Usage()), "\n")
|
||
for i := range lines {
|
||
if lines[i] != "" {
|
||
lines[i] = " " + lines[i]
|
||
}
|
||
}
|
||
return m + "Error help:\n\n" + strings.Join(lines, "\n") + "\n"
|
||
}
|
||
return m
|
||
}
|
||
|
||
func (pe ParseError) column(lines []string) int {
|
||
var pos, col int
|
||
for i := range lines {
|
||
ll := len(lines[i]) + 1 // +1 for the removed newline
|
||
if pos+ll >= pe.Position.Start {
|
||
col = pe.Position.Start - pos
|
||
if col < 0 { // Should never happen, but just in case.
|
||
col = 0
|
||
}
|
||
break
|
||
}
|
||
pos += ll
|
||
}
|
||
|
||
return col
|
||
}
|
||
|
||
type (
|
||
errLexControl struct{ r rune }
|
||
errLexEscape struct{ r rune }
|
||
errLexUTF8 struct{ b byte }
|
||
errLexInvalidNum struct{ v string }
|
||
errLexInvalidDate struct{ v string }
|
||
errLexInlineTableNL struct{}
|
||
errLexStringNL struct{}
|
||
errParseRange struct {
|
||
i interface{} // int or float
|
||
size string // "int64", "uint16", etc.
|
||
}
|
||
errParseDuration struct{ d string }
|
||
)
|
||
|
||
func (e errLexControl) Error() string {
|
||
return fmt.Sprintf("TOML files cannot contain control characters: '0x%02x'", e.r)
|
||
}
|
||
func (e errLexControl) Usage() string { return "" }
|
||
|
||
func (e errLexEscape) Error() string { return fmt.Sprintf(`invalid escape in string '\%c'`, e.r) }
|
||
func (e errLexEscape) Usage() string { return usageEscape }
|
||
func (e errLexUTF8) Error() string { return fmt.Sprintf("invalid UTF-8 byte: 0x%02x", e.b) }
|
||
func (e errLexUTF8) Usage() string { return "" }
|
||
func (e errLexInvalidNum) Error() string { return fmt.Sprintf("invalid number: %q", e.v) }
|
||
func (e errLexInvalidNum) Usage() string { return "" }
|
||
func (e errLexInvalidDate) Error() string { return fmt.Sprintf("invalid date: %q", e.v) }
|
||
func (e errLexInvalidDate) Usage() string { return "" }
|
||
func (e errLexInlineTableNL) Error() string { return "newlines not allowed within inline tables" }
|
||
func (e errLexInlineTableNL) Usage() string { return usageInlineNewline }
|
||
func (e errLexStringNL) Error() string { return "strings cannot contain newlines" }
|
||
func (e errLexStringNL) Usage() string { return usageStringNewline }
|
||
func (e errParseRange) Error() string { return fmt.Sprintf("%v is out of range for %s", e.i, e.size) }
|
||
func (e errParseRange) Usage() string { return usageIntOverflow }
|
||
func (e errParseDuration) Error() string { return fmt.Sprintf("invalid duration: %q", e.d) }
|
||
func (e errParseDuration) Usage() string { return usageDuration }
|
||
|
||
const usageEscape = `
|
||
A '\' inside a "-delimited string is interpreted as an escape character.
|
||
|
||
The following escape sequences are supported:
|
||
\b, \t, \n, \f, \r, \", \\, \uXXXX, and \UXXXXXXXX
|
||
|
||
To prevent a '\' from being recognized as an escape character, use either:
|
||
|
||
- a ' or '''-delimited string; escape characters aren't processed in them; or
|
||
- write two backslashes to get a single backslash: '\\'.
|
||
|
||
If you're trying to add a Windows path (e.g. "C:\Users\martin") then using '/'
|
||
instead of '\' will usually also work: "C:/Users/martin".
|
||
`
|
||
|
||
const usageInlineNewline = `
|
||
Inline tables must always be on a single line:
|
||
|
||
table = {key = 42, second = 43}
|
||
|
||
It is invalid to split them over multiple lines like so:
|
||
|
||
# INVALID
|
||
table = {
|
||
key = 42,
|
||
second = 43
|
||
}
|
||
|
||
Use regular for this:
|
||
|
||
[table]
|
||
key = 42
|
||
second = 43
|
||
`
|
||
|
||
const usageStringNewline = `
|
||
Strings must always be on a single line, and cannot span more than one line:
|
||
|
||
# INVALID
|
||
string = "Hello,
|
||
world!"
|
||
|
||
Instead use """ or ''' to split strings over multiple lines:
|
||
|
||
string = """Hello,
|
||
world!"""
|
||
`
|
||
|
||
const usageIntOverflow = `
|
||
This number is too large; this may be an error in the TOML, but it can also be a
|
||
bug in the program that uses too small of an integer.
|
||
|
||
The maximum and minimum values are:
|
||
|
||
size │ lowest │ highest
|
||
───────┼────────────────┼──────────
|
||
int8 │ -128 │ 127
|
||
int16 │ -32,768 │ 32,767
|
||
int32 │ -2,147,483,648 │ 2,147,483,647
|
||
int64 │ -9.2 × 10¹⁷ │ 9.2 × 10¹⁷
|
||
uint8 │ 0 │ 255
|
||
uint16 │ 0 │ 65535
|
||
uint32 │ 0 │ 4294967295
|
||
uint64 │ 0 │ 1.8 × 10¹⁸
|
||
|
||
int refers to int32 on 32-bit systems and int64 on 64-bit systems.
|
||
`
|
||
|
||
const usageDuration = `
|
||
A duration must be as "number<unit>", without any spaces. Valid units are:
|
||
|
||
ns nanoseconds (billionth of a second)
|
||
us, µs microseconds (millionth of a second)
|
||
ms milliseconds (thousands of a second)
|
||
s seconds
|
||
m minutes
|
||
h hours
|
||
|
||
You can combine multiple units; for example "5m10s" for 5 minutes and 10
|
||
seconds.
|
||
`
|