Commit graph

2411 commits

Author SHA1 Message Date
Brian C. Lane
8c78ff8904 test: Add a test for a complete blueprint's TOML output 2022-10-31 15:24:15 +01:00
Tomáš Hozza
a001da966a distro: add support for RHEL-9.2
Add relevant repo definitions and generate image test manifests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-31 08:07:09 +01:00
Tomáš Hozza
fc95dac86a distro: add support for RHEL-8.8
Add relevant repo definitions and generate image test manifests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-31 08:07:09 +01:00
Sanne Raymaekers
e94ea7c995 internal/worker: add rhsm to ostree resolve job 2022-10-28 16:14:30 +02:00
Sanne Raymaekers
c865286fb9 internal/ostree: add consumer certs when resolving ostree ref 2022-10-28 16:14:30 +02:00
Sanne Raymaekers
632f272573 internal/rhsm: add consumer secrets to Subscriptions
The consumer key/cert is used to uniquely identify a system against a
candlepin instances. They're useful for any Red Hat (ostree) content
which requires (cert) authentication.
2022-10-28 16:14:30 +02:00
Sanne Raymaekers
500341a25f internal/osbuild: contenturl and rhsm secrets for ostree sources 2022-10-28 16:14:30 +02:00
Tomáš Hozza
417a941a1e internal/target/azure_image: mark Location as optional
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-27 19:33:43 +02:00
Tomáš Hozza
1af01ad135 cloudapi: make location optional in Azure Upload Options
Providing the `location` is no longer required for Azure Upload Options.
If it is not provided, the implementation determines the location from
the provided Resource Group. This will make the API nicer for any
client, since they won't need to provide redundant information.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-27 19:33:43 +02:00
Tomáš Hozza
4df3b0ca03 internal/upload/azure: make location optional in various methods
Make the `location` argument optional (can be now empty "") in
`RegisterImage()` and `CreateStorageAccount()` methods.

If the provided `location` argument is an empty string, then the location
is determined from the provided Resource Group instead.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-27 19:33:43 +02:00
Tomáš Hozza
641f7a7d29 internal/upload/azure: add method for getting resource group location
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-27 19:33:43 +02:00
Ondřej Budai
6291eddbf2 koji: put artifacts uploaded to koji under a second level directory
Koji clean-up doesn't delete the top-level directories. As we named every
top-level directory different, they were just cumulating on kojihub, which
is obviously wrong.

This commit changes that behaviour to put all the temporary directories under
a new osbuild-cg top-level one. This way, osbuild-cg won't ve ever cleaned,
whereas osbuild-cg/osbuild-composer-koji-{UUID} will be, which is exactly
what we want.

Closes: https://github.com/osbuild/osbuild-composer/issues/3064
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-10-25 14:48:13 +02:00
Sanne Raymaekers
620536fd61 internal/cloudapi: add ostree options for all otree image types
b01792d9dd broke this behaviour. All
ostree image types should have an ostree resolve job.
2022-10-21 22:32:24 +02:00
Sanne Raymaekers
8fdd158799 cloudapi/v2: use the ostree resolve job to resolve ostree refs 2022-10-19 18:14:10 +02:00
Sanne Raymaekers
ebeb339f96 osbuild-worker: add ostree resolve job
This job resolves an ostree ref. Similar to the depsolve and container
resolve jobs, this should be a dependency of a manifest job.
2022-10-19 18:14:10 +02:00
Sanne Raymaekers
b01792d9dd internal/ostree: offload using default ostree ref to caller
If params.Ref is an empty string, it's set to the distro's default
ref. The only difference here is that the default ref also gets
verified.

It makes splitting out resolving ostree refs to a new job easier.

In the weldr and cloud apis, ostree.ResolveParams always got executed,
also for non-ostree image types. Make it more explicit by only resolving
if the image type is actually an ostree image.
2022-10-19 18:14:10 +02:00
Tomáš Hozza
ae04c56c32 distro/rhel9: adjust RHSM-related config on EC2 9.1+ images
The EC2 images starting with 9.1 should:
 - not configure RHSM using osbuild
 - install `redhat-cloud-client-configuration` package which ships the
   RHSM configuration.

Regenerate affected image manifests.

Related to COMPOSER-1805

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-19 17:07:36 +02:00
Tomáš Hozza
b65bb9491f distro/rhel8: adjust RHSM-related config on EC2 8.7+ images
The EC2 images starting with 8.7 should:
 - not configure RHSM using osbuild
 - install `redhat-cloud-client-configuration` package which ships the
   RHSM configuration.

Regenerate affected image manifests

Related to COMPOSER-1804.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-19 17:07:36 +02:00
Achilleas Koutsou
abdddc6e55 osbuild: fix gpgkeys json key in ostree source 2022-10-12 03:07:40 +02:00
Tomáš Hozza
dc476671e4 weldr/upload/gcp: make Object optional
Previously, it was expected from the user to provide the Object name
when uploading image to GCP. The object name does not matter much,
because the object is deleted once image import finishes. Make
the specification of the object name optional and generate it if not
provided.

Adjust the GCP Weldr test case to not provide the Object name when
uploading the image.

The user can still provide the Object name if needed.
2022-10-11 13:23:18 +02:00
Tomáš Hozza
809107cd45 cloudapi: make Bucket optional in GCPUploadOptions
GCP Bucket to use can be now configured in the worker configuration.
Make the `Bucket` optional in the Cloud API when uploading image to GCP.

Adjust the Cloud API test case to configure GCP Bucket on the worker and
not provide it in the API request.
2022-10-11 13:23:18 +02:00
Tomáš Hozza
ad34043087 internal/target/gcp: make Bucket optional
The Bucket can now be set also in the worker configuration.
2022-10-11 13:23:18 +02:00
Tomáš Hozza
13f0894094 worker/aws: don't generate object key in worker
There is a desire to make the worker as "dumb" as possible. Therefore it
is not desired to generate the AWS object key names in the worker if it
was not provided in the job.

Modify the worker code to not generate the AWS object key in any case
and instead set an error in case the object key was not provided.

Modify Weldr API implementation to generate the object key, if it was
not provided by the user. This is consistent with Cloud API
implementation.
2022-10-11 13:23:18 +02:00
Achilleas Koutsou
7ebe3f550c distro/rhel: set ostree sysroot.readonly to false
We used to always set the sysroot.readonly setting to true, but this
never worked because of a bug in osbuild [1].
The bug is now fixed and the RHEL and CentOS edge-raw images are crated
with sysroot.readonly = true, and the images aren't booting.

Fixing the option to false.  This changes the manifests, but not the
generated images because of the change in osbuild.

If sysroot is meant to be readonly, we will change it in a future
update.

[1] https://github.com/osbuild/osbuild/pull/1129
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
97dddd24a5 fedora: remove BIOS boot for IoT raw images 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
e6c7815d41 fedora: use product and os version for discinfo release field
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
1324f7fb8c osbuild: simplify copy stage test 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
547f7a66b3 fedora: set sysroot read-only only for Fedora 37+ iot-raw-image 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
785f9cb7dd manifest: remove private osTreeCommit type
Use the ostree.CommitSpec type instead and return it directly from the
pipeline attributes when needed.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
d32dff14b4 image: make read only sysroot optional for OSTreeRawImage 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
4ae8304bd2 image: make ostree commit mandatory in the OSTreeRawImage
Make the ostree commit spec mandatory in the OSTreeRawImage by adding it
to the constructor.

Use the ostree.CommitSpec to specify parameters in the OSTreeRawImage
ImageKind and the OSTreeDeployment Pipeline.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
5f98d7082c manifest: make ostree commit spec mandatory in the OSTreeInstaller
Make the ostree commit spec mandatory in the OSTreeInstaller ImageKind.
The installer image type is not just for ostree types so make the ostree
parameters optional for the ISOTree Pipeline.

Use the ostree.CommitSpec to specify commits parameters.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
711fc697e4 manifest: minor docstring spacing fix 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
99952afbd4 Separate ostree ref from parent spec in Pipelines and ImageKinds
In the OS pipeline, the parent configuration was used to detect if the
pipeline's setup was meant for an ostree commit or not.  Also, the
pipeline used a new type to specify the ostree parameters.

- Use the ostree.CommitSpec for the parent configuration.
- Add a new attribute, OSTreeRef, that defines the ref for the ostree
  commit being built.  An empty string indicates that the tree is not
  for an ostree commit.

Additionally, in the ImageKind configurations for the ostree archive and
container, separate the ostree ref from the parent spec, make the parent
spec optional (pointer) and the ostree ref mandatory, by requiring it in
the constructor of the ImageKind.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
390ae15eaa distro: replace ostree.RequestParams with new OSTreeImageOptions
Instead of using the ostree.RequestParams in the OSTReeImageOptions,
define a new struct specific to ImageOptions for the ostree parameters.
This is almost identical to the new ostree.CommitSpec but the meaning of
the parameters changes based on image type and it would not be clear if
the CommitSpec was used in all cases.  For example, the parameters of
the new OSTreeImageOptions do not always refer to the same commit.  The
URL and Checksum may point to a parent commit to be pulled in to base
the new commit on, while the Ref refers to the new commit that will be
built (which may have a different ref from the parent).

The ostree.ResolveParams() function now returns two strings, the
resolved ref, which is replaced by the defaultRef if it's not specified
in the request, and the resolved parent checksum if a URL is specified.
The URL does not need to be returned since it's always the same as the
one specified in the request.
The function has been rewritten to make the logic more clear.
The docstring for the function has been rewritten to cover all use cases
and error conditions.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
c6b999f178 ostree: rename CommitSource to CommitSpec and add Ref
The CommitSource was used to specify the source URL and checksum of a
commit for use in manifest sources.  Renaming to CommitSpec and adding a
Ref parameter generalises the type so that we can use it to specify
commits in various situations.  This is building towards separating when
ostree parameters are used for fetching a commit, fetching a parent
commit, and building one.

The CommitSpec is (very roughly) analogous to the rpmmd.PackageSpec.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
938bc7bafd ostree: small docstring clarification 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
6bd15c19a3 manifest: use size from partition table for efiboot image 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
3ee5622ad7 manifest: fix typos and docstrings for OS pipeline
Fix the NewOS() docstring to mention only the function args.
2022-10-11 10:00:22 +02:00
Ondřej Budai
5bda62fff9 cloudapi: add iot-installer
This was forgotten in previous commits that fixed this image type.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-10-05 10:29:05 +02:00
Brian C. Lane
d9cfbea399 weldr: Add test for blueprint with empty version commit message 2022-09-30 17:42:07 +02:00
Brian C. Lane
b209bcec55 fedora: blueprint must have a name 2022-09-30 17:42:07 +02:00
Brian C. Lane
f164bfb272 cloudapi: blueprint must have a name 2022-09-30 17:42:07 +02:00
Brian C. Lane
d42d5fa17f blueprints: Fix commit message when version is empty
With an empty or missing version number the commit message would not
include the version (which is set to 0.0.0 by calling Initialize).  This
adds a call to Initialize() in the API code before constructing the
commit message. It also moves the check for non-empty blueprint name
into the Initialize call where it belongs.
2022-09-30 17:42:07 +02:00
Brian C. Lane
b1461c91bb store: Fix TagBlueprint
This fixes a bug where tagging a blueprint would overwrite the blueprint
in the blueprintChanges storage with an older version of the blueprint.
It also adds checks to make sure the blueprint is present in the
blueprintsCommits and blueprintsChanges maps before trying to reference
them -- just in case something in the store gets out of sync.
2022-09-30 11:49:52 +02:00
Brian C. Lane
6ebe871710 tests: Fix TestTagBlueprint to fail properly
TagBlueprint has a bug, it replaces the tagged blueprint with the last
untagged blueprint. The test wasn't testing for this, so correct that
before fixing the actual bug.
2022-09-30 11:49:52 +02:00
Achilleas Koutsou
6f89e9d499 fedora: add "rw" to kernel options for iot-raw-image
Needed when /sysroot is mounted read-only to keep other (stateful)
parts of the filesystem (/var and /etc) writeable.
2022-09-29 18:09:38 +02:00
Achilleas Koutsou
445de61a80 manifest: move ostree config stage after the deployment
Configure ostree after it's deployed and add the ostree mount.
This fixes sysroot not being mounted 'ro' in the final image when it's
booted.
2022-09-29 18:09:38 +02:00
Achilleas Koutsou
aec3ef4161 image: update ostree installer to use new pipelines
Fedora iot-installer now uses the new ostree installer pipelines that no
longer use the bootiso.mono stage.
2022-09-29 18:09:38 +02:00
Achilleas Koutsou
254a3b16d4 manifest: use internal ostree package struct in bootiso-tree 2022-09-29 18:09:38 +02:00