Commit graph

20 commits

Author SHA1 Message Date
Achilleas Koutsou
bacba210b0 rhel84: new image type: rhel-edge-installer
New image type that generates a Boot ISO.  The ISO contains a RHEL Edge
commit and an installer.  On Boot, it sets up a new RHEL Edge system
with the commit.

The RHEL Edge commit (ostree commit) is downloaded during build from a
URL that should be supplied with the compose request.  The commit's hash
and URL need to be added to the Sources list in the Manifest.

Unlike other types, the new image type defines its own "build" package
set that is added to the distro and arch build package lists.
2021-03-17 18:12:17 +00:00
Achilleas Koutsou
541cbab0f6 distro/rhel84: new imageType implementation
imageTypeS2 implements the distro.ImageType interface buts generates a
Manifest matching the new osbuild v2 schema.

Two new image types are added to the rhel84 distro (x84_64 and aarch64)
for generating OCI containers contain an Edge (ostree) commit and, when
run, start a web serer to serve the commit.

The image type uses the new PackageSets map to define packages (and
excludes) for the image.  The old methods (Packages() and
BuildPackages()) are implemented for compatibility with the old
workflow.
The image also defines an extra package set for the container that will
serve the package: "httpd" (and its dependencies).

The distro.ImageType interface has a new method: Exports()
It should return a list of names or IDs of artefacts that should be
exported from osbuild when the job is complete.
For the old image types, this is simply set to "assembler".
2021-03-17 18:12:17 +00:00
Brian C. Lane
ec19003b70 store: Fix copy failure when initializing blueprints commit list
copy() will not append to a slice, so if the length is 0 nothing will be
copied. Initialize the slice with the length instead of the capacity.

This fixes a problem with blueprint commits vanishing when the server is
restarted.

This commit also includes a new test in json_test for the
newCommitsFromV0() function.
2020-08-02 11:21:20 +01:00
Peter Robinson
f32770fed6 distro/rhel8: add RHEL for Edge
Initial definition for RHEL for edge package set on x86.
This is based marginally on the upstream Fedora IoT package set
but slimmed down and adjusted for RHEL package naming.

Based on Tom's patch.

Signed-off-by: Tom Gundersen <teg@jklm.no>
Signed-off-by: Peter Robinson <pbrobinson@gmail.com>
2020-06-11 20:44:17 +02:00
Tom Gundersen
0417c6d8bb distro: make the osbuild package internal to the distros
Rather than Manifest() returning an osbuild.Manifest object, introduce a
new distro.Manifest object which represents it as an opaque, JSON
serializable object. This new type has the following properties:

1) its serialization is compatible with the input to osbuild,
2) any valid osbuild input can be deserialized into it, and
3) marshalling and unmarshaling to and from JSON is lossless.

This means that even as we change the subset of valid osbulid manifests
that we support, we can still load any previous state from disk, and it
will continue to work just as before, even though we can no longer
deserialize it into our internal notion of osbuild.Manifest.

This fixes the underlying problem of which #685 was a symptom.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-03 00:30:01 +02:00
Tom Gundersen
4aced4e749 store/ImageBuild: make Manifest a required property
Any valid ImageBuild must contain a Manifest, so don't allow this to be
nil, simplifying the code a bit in the process.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-03 00:30:01 +02:00
Tom Gundersen
d606c5195b store/json: marshal manifests as raw JSON
The manifests we support marshalling/unmarshalling in composer are a
subset of the ones supported by osbuild. osbuild needs to preserve
backwards compatibility, but composer only needs to be concerned with
producing valid manifests at a given time.

A problem with this is that when we marshal our manifests to disk, and
read them out with a future version of composer, we would have to make
sure that the serialization is compatible, or unmarshalling the entire
store will fail. This is what happened when commit 5938276d3b
changed the way we marshal the file source.

Improve on this by only dropping those composes from the store on
unmarshal, whose manifests we are no longer able to unmarshal.

In future commits we will drop that requirement too.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-03 00:30:01 +02:00
Tom Gundersen
b0cfec767a distro/f32: add iot-commit image type
This produces a Fedora IoT commit tarball, based on
https://github.com/gicmo/ostree-osbuild-demo and the fedora 32 specific
configuration found at
https://pagure.io/fedora-iot/ostree/c/465c8a2c1d1c0cf4b762477af87fb1d82c8ee002?branch=master

The usbguard package is excluded due to some SELinux failure that
remains to be resolved.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-25 22:37:29 +02:00
Tom Gundersen
a22cd78eb3 store/json: log on dropping composes on unmarshal
If an invalid compose is encountered, then we drop it. Make sure to log
if a logger object is provided.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
f8db2e28e1 store/json: move commits backwards compatibility
By the same logic as the previous patch.

This exposed an apparent bug, where we are sorting by the version field
of the blueprint in the change objects, but these were never
(un)marshalled, so would always be set to zero in the past.

Remove the no-op sorting for now, as it never had an effect, and
put a note whether we should instantiate the blueprint in the change
object somehow.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
15a0e78c15 store/json: move queue status backwards compatibility
Apply the backwards compatibility where the ImageBuild struct is
assembled. This should make the code simpler to reason about, by
reducing the structs we produce and later fix up.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
cdc4b1bd53 store/json: add some basic tests
In the process clarify the code a bit to make it easter to
test/understand.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
fba2af5f51 store: drop support for multi-image composes
The store only serves the weldr API, and that hard-codes the assumption
of only one image build per compose all over the place. Move this
assumption into the json serialization handler.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
e28fca493c store/json: split newComposesFromV0()
Not a functional change, just for readability.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
594858ab2c store/json: simplify ImageType handling
Serialize to a plain string, and do the translation for compatibility
from string to string.

This removes a lot of code, and if we change the store format, we should
just skip the conversion altogether.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
2c3790f20c common/ImageType: move to the store package
This is now only used for marshalling in the store, so keep the logic
where it is used.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
2fe4450620 store/compose/ImageType: use distro.ImageType objects
This reduces the amount of resolving and error checking we have to do.
This exposed a bug in weldr's ComposeEntry type, which will be fixed in
a follow-up commit.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
5ba7e21a72 store/compose: rename 'Id' to 'ID' to make linter happy
Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
df7a0fec22 store: merge the compose package into the store
The types exposed by the compose package are only used in the store API,
so move them there where they belong.

Making the ownership of the types clear, rather than having them live in
a package for themselves, will make it clearer how the types can be
modified in follow-up commits.

Also remove the JSON annotations, as these types are no longer used for
serialization.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
27e52a9755 store: use dedicated types for (un)marshaling
Distinguish between the types used at runtimes and the types used for
(un)marshaling. Also make the types private to the store package.

This should allow us to reduce the interdependencies between the
packages and more easily change things without accidentally breaking
backwards compatibility.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00