Commit graph

5 commits

Author SHA1 Message Date
Lars Karlitski
12b5c6aaa4 sources: bump maximum message size to 64k
These messages contain certificate data, which is quite large.

We should probably use streaming sockets in the future.
2020-01-09 23:55:43 +01:00
Lars Karlitski
e123715bc6 osbuild: introduce secrets
Add a new command line option `--secrets`, which accepts a JSON file
that is structured similarly to a source file. It is should contain data
that is necessary to fetch content, but shouldn't appear in any logs.
2020-01-09 23:55:43 +01:00
Lars Karlitski
02ad4e3810 sources: fail gracefully when a source returns invalid JSON
Include the actual output of the source to help debugging.
2020-01-09 23:55:43 +01:00
Lars Karlitski
b9b2f99123 osbuild: create API sockets in the thread they're used in
This might (hopefully) fix a race in destructing the asyncio.EventLoop
that's used in all API classes, which leads to warnings about unhandled
exceptions on CI.

This also puts their creation closer to where the client-side sockets
are created.
2019-12-25 17:48:26 +01:00
Lars Karlitski
510e2b1e94 osbuild: introduce sources
Pipelines encode which source content they need in the form of
repository metadata checksums (or rpm checksums). In addition, they
encode where they fetch that source content from in the form of URLs.
This is overly specific and doesn't have to be in the pipeline's hash:
the checksum is enough to specify an image.

In practice, this precluded using alternative ways of getting at source
packages, such as local mirrors, which could speed up development.

Introduce a new osbuild API: sources. With it, a stage can query for a
way to fetch source content based on checksums.

The first such source is `org.osbuild.dnf`, which returns repository
configuration for a metadata checksum. Note that the dnf stage continues
to verify that the content it received matches the checksum it expects.

Sources are implemented as programs, living in a `sources` directory.
They are run on the host (i.e., uncontained) right now. Each source gets
passed options, which are taken from a new command line argument to
osbuild, and an array of checksums for which to return content.

This API is only available to stages right now.
2019-12-23 01:12:38 +01:00