Commit graph

4912 commits

Author SHA1 Message Date
Achilleas Koutsou
785f9cb7dd manifest: remove private osTreeCommit type
Use the ostree.CommitSpec type instead and return it directly from the
pipeline attributes when needed.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
d32dff14b4 image: make read only sysroot optional for OSTreeRawImage 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
4ae8304bd2 image: make ostree commit mandatory in the OSTreeRawImage
Make the ostree commit spec mandatory in the OSTreeRawImage by adding it
to the constructor.

Use the ostree.CommitSpec to specify parameters in the OSTreeRawImage
ImageKind and the OSTreeDeployment Pipeline.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
5f98d7082c manifest: make ostree commit spec mandatory in the OSTreeInstaller
Make the ostree commit spec mandatory in the OSTreeInstaller ImageKind.
The installer image type is not just for ostree types so make the ostree
parameters optional for the ISOTree Pipeline.

Use the ostree.CommitSpec to specify commits parameters.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
711fc697e4 manifest: minor docstring spacing fix 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
99952afbd4 Separate ostree ref from parent spec in Pipelines and ImageKinds
In the OS pipeline, the parent configuration was used to detect if the
pipeline's setup was meant for an ostree commit or not.  Also, the
pipeline used a new type to specify the ostree parameters.

- Use the ostree.CommitSpec for the parent configuration.
- Add a new attribute, OSTreeRef, that defines the ref for the ostree
  commit being built.  An empty string indicates that the tree is not
  for an ostree commit.

Additionally, in the ImageKind configurations for the ostree archive and
container, separate the ostree ref from the parent spec, make the parent
spec optional (pointer) and the ostree ref mandatory, by requiring it in
the constructor of the ImageKind.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
390ae15eaa distro: replace ostree.RequestParams with new OSTreeImageOptions
Instead of using the ostree.RequestParams in the OSTReeImageOptions,
define a new struct specific to ImageOptions for the ostree parameters.
This is almost identical to the new ostree.CommitSpec but the meaning of
the parameters changes based on image type and it would not be clear if
the CommitSpec was used in all cases.  For example, the parameters of
the new OSTreeImageOptions do not always refer to the same commit.  The
URL and Checksum may point to a parent commit to be pulled in to base
the new commit on, while the Ref refers to the new commit that will be
built (which may have a different ref from the parent).

The ostree.ResolveParams() function now returns two strings, the
resolved ref, which is replaced by the defaultRef if it's not specified
in the request, and the resolved parent checksum if a URL is specified.
The URL does not need to be returned since it's always the same as the
one specified in the request.
The function has been rewritten to make the logic more clear.
The docstring for the function has been rewritten to cover all use cases
and error conditions.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
c6b999f178 ostree: rename CommitSource to CommitSpec and add Ref
The CommitSource was used to specify the source URL and checksum of a
commit for use in manifest sources.  Renaming to CommitSpec and adding a
Ref parameter generalises the type so that we can use it to specify
commits in various situations.  This is building towards separating when
ostree parameters are used for fetching a commit, fetching a parent
commit, and building one.

The CommitSpec is (very roughly) analogous to the rpmmd.PackageSpec.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
938bc7bafd ostree: small docstring clarification 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
a6d3ad3959 spec: bump osbuild dep to >= 69
osbuild 69 fixed a bug in the ostree.config stage that we now require
for setting the ior-raw-image [1].

The version hasn't been released yet, so pin osbuild version that
provides the fix for all distros.

[1] 8a7b6d382d
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
10820a8675 test: update Fedora IoT Installer manifests 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
6bd15c19a3 manifest: use size from partition table for efiboot image 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
3ee5622ad7 manifest: fix typos and docstrings for OS pipeline
Fix the NewOS() docstring to mention only the function args.
2022-10-11 10:00:22 +02:00
dependabot[bot]
10a981cfcb build(deps): bump github.com/aws/aws-sdk-go from 1.44.111 to 1.44.112
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.111 to 1.44.112.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.111...v1.44.112)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-06 12:16:29 +02:00
dependabot[bot]
f02067bf0c build(deps): bump github.com/aws/aws-sdk-go from 1.44.108 to 1.44.111
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.108 to 1.44.111.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.108...v1.44.111)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-05 13:53:56 +02:00
dependabot[bot]
b7418d6bba build(deps): bump github.com/labstack/gommon from 0.3.1 to 0.4.0
Bumps [github.com/labstack/gommon](https://github.com/labstack/gommon) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/labstack/gommon/releases)
- [Commits](https://github.com/labstack/gommon/compare/v0.3.1...v0.4.0)

---
updated-dependencies:
- dependency-name: github.com/labstack/gommon
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-05 13:00:08 +02:00
schutzbot
9a7027ef77 Post release version bump
[skip ci]
2022-10-05 08:32:10 +00:00
Ondřej Budai
5bda62fff9 cloudapi: add iot-installer
This was forgotten in previous commits that fixed this image type.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-10-05 10:29:05 +02:00
Diaa Sami
5ffb9e693e tools/appsre: remove monit setup code & scripts
Since it doesn't not work since we moved workers to app-sre
2022-10-04 16:26:08 +02:00
Diaa Sami
420097ae63 tools/appsre: remove leftover code from before 2022-10-04 16:26:08 +02:00
Juan Abia
34b1be2bf4 test: get correct CIV tag in azure.sh
In order to run azure.sh inside CIV CI, we need to use the appropiate
tag. This means getting `prod` tag when running in osbuild-composer and
the pr tag when running in cloud-image-val
2022-10-04 15:54:38 +02:00
Xiaofeng Wang
989c64040c test: Add Fedora 36 and 37 and remove 35 test for ostree.sh 2022-10-03 12:21:39 +02:00
Brian C. Lane
d9cfbea399 weldr: Add test for blueprint with empty version commit message 2022-09-30 17:42:07 +02:00
Brian C. Lane
b209bcec55 fedora: blueprint must have a name 2022-09-30 17:42:07 +02:00
Brian C. Lane
f164bfb272 cloudapi: blueprint must have a name 2022-09-30 17:42:07 +02:00
Brian C. Lane
d42d5fa17f blueprints: Fix commit message when version is empty
With an empty or missing version number the commit message would not
include the version (which is set to 0.0.0 by calling Initialize).  This
adds a call to Initialize() in the API code before constructing the
commit message. It also moves the check for non-empty blueprint name
into the Initialize call where it belongs.
2022-09-30 17:42:07 +02:00
Sanne Raymaekers
8fcbeaadb3 dbjobqueue: Backoff after listener error
The connection can be briefly interrupted for short periods of time,
which is fine. To avoid spamming the logs with error messages, backoff
for half a second.
2022-09-30 16:38:43 +02:00
Ondřej Budai
f25dca793d packer: remove Fedora 35
Our workers already run on Fedora 36 so there's no need to build F35 anymore.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-30 14:52:24 +02:00
Sanne Raymaekers
a132113452 worker: log error details on job failure
Currently errors like clienterror 28 ("at least one target failed") have
all the relevant information in the details, don't omit these details
from the worker logs.
2022-09-30 12:10:07 +02:00
dependabot[bot]
c6a960fca8 build(deps): bump google.golang.org/api from 0.97.0 to 0.98.0
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.97.0 to 0.98.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.97.0...v0.98.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-30 12:07:03 +02:00
dependabot[bot]
79cef73227 build(deps): bump github.com/aws/aws-sdk-go from 1.44.104 to 1.44.108
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.104 to 1.44.108.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.104...v1.44.108)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-30 12:06:16 +02:00
Brian C. Lane
b1461c91bb store: Fix TagBlueprint
This fixes a bug where tagging a blueprint would overwrite the blueprint
in the blueprintChanges storage with an older version of the blueprint.
It also adds checks to make sure the blueprint is present in the
blueprintsCommits and blueprintsChanges maps before trying to reference
them -- just in case something in the store gets out of sync.
2022-09-30 11:49:52 +02:00
Brian C. Lane
6ebe871710 tests: Fix TestTagBlueprint to fail properly
TagBlueprint has a bug, it replaces the tagged blueprint with the last
untagged blueprint. The test wasn't testing for this, so correct that
before fixing the actual bug.
2022-09-30 11:49:52 +02:00
Xiaofeng Wang
cc95d731bc Pin osbuild to commit "stages: fix ostree config stage"
This commit fixed /sysroot mount as rw issue
2022-09-29 18:09:38 +02:00
Xiaofeng Wang
8e8704e03b test: Fix CS9 UEFI installation failure issue 2022-09-29 18:09:38 +02:00
Achilleas Koutsou
e01fa00da1 test: update iot-raw-image manifests 2022-09-29 18:09:38 +02:00
Achilleas Koutsou
6f89e9d499 fedora: add "rw" to kernel options for iot-raw-image
Needed when /sysroot is mounted read-only to keep other (stateful)
parts of the filesystem (/var and /etc) writeable.
2022-09-29 18:09:38 +02:00
Achilleas Koutsou
445de61a80 manifest: move ostree config stage after the deployment
Configure ostree after it's deployed and add the ostree mount.
This fixes sysroot not being mounted 'ro' in the final image when it's
booted.
2022-09-29 18:09:38 +02:00
Xiaofeng Wang
9eab01874a test: Fix UEFI parameters error
Fix error "loader attribute 'readonly' cannot be specified when
firmware autoselection is enabled"
2022-09-29 18:09:38 +02:00
Xiaofeng Wang
6e74ae33bf test: /sysroot should be mounted as ro since Fedora 37
Test script should be updated to work with this change
2022-09-29 18:09:38 +02:00
Achilleas Koutsou
9f3da4511e CI: enable ostree-ng (with installer) on F36 and F37 2022-09-29 18:09:38 +02:00
Achilleas Koutsou
42d1c72d4d test: update Fedora iot-installer manifests 2022-09-29 18:09:38 +02:00
Achilleas Koutsou
aec3ef4161 image: update ostree installer to use new pipelines
Fedora iot-installer now uses the new ostree installer pipelines that no
longer use the bootiso.mono stage.
2022-09-29 18:09:38 +02:00
Achilleas Koutsou
254a3b16d4 manifest: use internal ostree package struct in bootiso-tree 2022-09-29 18:09:38 +02:00
Achilleas Koutsou
14f608de0a osbuild: use internal users package structs in stages
Don't pass blueprint Users and Groups options all the way down to the
osbuild stage bindings.  Instead, convert them to the internal
users.User and users.Group structs.
Ideally we would do this even higher up in the code path, before
reaching the distro, but this is the first step towards that.
2022-09-29 18:09:38 +02:00
Achilleas Koutsou
5bf4b2ab98 manifest: delete bootiso.mono option helper function 2022-09-29 18:09:38 +02:00
Achilleas Koutsou
a1bfcfe91c manifest: update bootiso-tree pipeline to use new stages
- Use newer granular stages to build the bootiso tree rather than the
  deprecated bootiso.mono stage.
- Adjust the ISOTree struct properties: add the ones needed by the new
  stages and remove any properties that have been moved to the new
  pipelines.
2022-09-29 18:09:38 +02:00
Achilleas Koutsou
c549742c02 manifest: new pipeline for creating an EFI boot tree
Contains a single stage that creates an boot filesystem tree, that can
be consumed to create an efiboot.img.
2022-09-29 18:09:38 +02:00
Achilleas Koutsou
d27bdac369 manifest: new pipeline for building ISO rootfs.img 2022-09-29 18:09:38 +02:00
Achilleas Koutsou
c0fcbfc5c2 osbuild: add isolinux stage
Add support for the org.osbuild.isolinux stage.
2022-09-29 18:09:38 +02:00