Commit graph

3533 commits

Author SHA1 Message Date
Sanne Raymaekers
e9daacd1dd internal/cloudapi: Allow bp.Customizations being nil
Customizations being nil isn't the same as an empty object. The
edge-installer types require customizations equalling nil.

This partially reverts bb486bcccb.
2022-02-23 11:24:27 +00:00
Xiaofeng Wang
05cbbf0cc5 test: Skip rollback test only on UEFI VM of ostree-rebase.sh 2022-02-23 11:08:24 +01:00
Xiaofeng Wang
45ee6c297f test: Remove ANSIBLE_STDOUT_CALLBACK=yaml in ostree-rebase.sh 2022-02-23 11:08:24 +01:00
Xiaofeng Wang
bc9d1431bc test: Add OSTree rebase test for OSTree parent ref support
To run ostree-rebase.sh, like RHEL 8.6 and RHEL 9, the CS8 VM
needs "large" flavor for more CPU and memory resource.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
129536dad0 distro/rhel86,rhel90: don't write keys for ostree commits in user stage
Writing the key to the user home directory has no effect for ostree
commits.  Instead we write them using a fist-boot service.
In certain situations (e.g., when building an upgrade commit against an
existing parent), the user's home directory might not exist during the
build and the user key creation fails in the users stage, so let's
remove it entirely to avoid the issue.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
be382ecef8 test/ostree: update ostree compose request to match new requirements
Specifying a parent commit ID without a URL isn't allowed now.  The
upgrade commit is built without a parent.  The OS upgrade should still
work without it.
Getting the ID of the first commit is not necessary now.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
459a07d101 cloudapi/v2: update tests to match new ostree parameter rules
- Create test servers that serve a commit ref for ostree compose
  requests.
- Adapt test cases t new rules.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
3839a7d0dc cloudapi/v2: catch ParameterComboError from ostree param validation 2022-02-23 11:08:24 +01:00
Achilleas Koutsou
9e0af86775 ostree: rename errors and introduce ParameterComboError
Shorter and clearer error types.
- InvalidParameterError is now RefError.  It is returned when a ref (the
  new ref or the parent ref) is invalid.
- The ResolveRefError is not renamed.  It is returned when a ref can't
  be resolved (e.g., ref not found at URL).

New error type ParameterComboError.  It is returned when a parent ref is
specified but no URL, which is an invalid parameter combination.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
36d8cd1dd2 ostree: move test server to separate package
Make it reusable for other packages
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
ef513329a1 weldr: update tests to match new ostree parameter rules
- Create test servers that serve a commit ref for ostree compose
  requests.
- Adapt test cases to new rules.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
6e42ff7b44 weldr: reformat Compose test cases
Easier to read and modify if the structs are broken up.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
135cd684fc ostree: new ostree parameter rules
All parameters can now be specified at the same time.
See https://github.com/osbuild/osbuild-composer/issues/2131 for a
description of how the parameters are meant to interact.

In brief, the only invalid parameter combination is specifying a Parent
without a URL.
All other error conditions are for malformed URLs, invalid characters in
a ref, or error responses from the server.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
134a84ac23 cloudapi/v2: test ostree url + parent error conditions
Supplying both a URL and a parent ID (regardless of ref) is currently an
error, just like in the existing Weldr API.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
bfaf0c073b cloudapi/v2: support specifying parent ref for ostree requests 2022-02-23 11:08:24 +01:00
Achilleas Koutsou
24b70837c1 cloudapi/v2: validate ostree params using common validation function
Use the ostree package error types to keep the existing distinction
between Ref- and URL-related errors.
Introduce a new error condition for a general InvalidOSTreeParams
failure.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
c68850fa26 ostree: return cusotm errors from validation functions
Errors implement the error interface, so we can return them as generic
errors. Callers can test the type only if they want to determine the
failure type.

The error types that can be returned from each function are documented
in the docstring.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
30e127d1b6 ostree: custom errors for ostree parameter validation 2022-02-23 11:08:24 +01:00
Achilleas Koutsou
7f69d56fc9 ostree: simplify VerifyRef() function
Return value of condition immediately
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
923850ff1b cloudapi/v2: test ostree params and error conditions 2022-02-23 11:08:24 +01:00
Achilleas Koutsou
f885de4c68 distro: remove OSTreeImageOptions type and use ostree.RequestParams 2022-02-23 11:08:24 +01:00
Achilleas Koutsou
102c322ec9 ostree: simplify ResolveParams arguments
No need to pass the entire image type. We just need the default ref.
This removes the distro package dependency from the ostree package,
which we will need so distro can use the ostree types and functions.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
b92754b135 ostree: rename request parameters type
It's in the ostree package, so it doesn't need to have OSTree in the
name.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
db345d75dd ostree: include ref in error message when invalid 2022-02-23 11:08:24 +01:00
Achilleas Koutsou
417d913439 weldr: use new ResolveOSTreeParams function 2022-02-23 11:08:24 +01:00
Achilleas Koutsou
c60c54aa43 ostree: ostree parameter validation and resolve function
Validates the ostree parameters and resolves the parent ID.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
4d31c8e69d weldr: change all OSTree related error IDs to OSTreeOptionsError
These will all be reported from the same validation function.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
26468315ab weldr: expand ostree parameter test cases
Adding three more combinations that weren't covered by previous tests:
- Supplying ref, parent, and URL: should result in an error
- Supplying ref and parent: OK
- Supplying parent, and URL: same as first case (ref gets default value
  from image type)

Added default OSTreeRef() to test image type to cover the cases where
the ref isn't specified but affects the validation.

Separated and commented the test cases.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
01b2b15ebc weldr: reuse api for all test cases in TestCompose 2022-02-23 11:08:24 +01:00
Alexander Todorov
646fbe93c7 Revert "tests: Conditionally enable osbuild-dnf-json-tests"
Current version is 45.

This reverts commit 966692be11.
2022-02-22 21:57:30 +01:00
He Yi
f24e3d93e3 Update team_ssh_keys.txt 2022-02-22 21:55:56 +01:00
He Yi
866d2aabe7 Update team_ssh_keys.txt 2022-02-22 21:55:56 +01:00
Gianluca Zuccarelli
8e8d99336f templates/worker: fix depsolve error rate
The depsolve error rate had the incorrect query
and was returning the error rate for the build
jobs. This has now been fixed.
2022-02-22 19:55:14 +00:00
Ondřej Budai
2263b821a7 worker/osbuild-koji: fix double-reporting of osbuild-koji job status
This was causing:

Feb 21 06:08:06 ip-10-30-18-43.us-east-1.aws.redhat.com osbuild-worker[20798]: time="2022-02-21T06:08:06Z" level=warning msg="Error reporting job result: error setting job status: 404 — Token not found (IMAGE-BUILDER-WORKER-5)"

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-22 19:27:17 +00:00
Ondřej Budai
0fa7c96549 Revert "mockbuild: temporarily pin RHEL 9 compose to an older one"
This reverts commit 10c0891fe0.
2022-02-22 19:26:42 +00:00
Xiaofeng Wang
8051ddced5 test: Remove "ansible-galaxy collection install"
Since "ANSIBLE_STDOUT_CALLBACK=yaml" has been removed from all
ostree test script, "ansible-galaxy collection install" should be
removed as well
2022-02-22 19:25:48 +00:00
Sanne Raymaekers
7133dc7943 Schutzfile: Pin centos-9's osbuild commit 2022-02-22 19:24:12 +00:00
Christian Kellner
34924c7af4 distro: use new GenImageKernelOptions in ostreeDeployPipeline
The new `GenImageKernelOptions` will retrun all needed kernel options
that are derived from the partition table. The only case - for now -
is if the partition table contains a LUKS decice, where the uuid of it
needs to be passed to the on the kernel command line so that it will
get unlocked.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Christian Kellner
2c6e726c94 disk: add new GenImageKernelOptions helper
This creates the needed kernel options for an image, if any,
based on the partition table.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Christian Kellner
54a4616a7d distro: remove unused sfdiskStageOptions
This was replaced by the previous switch to `GenImage*Stages`.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Christian Kellner
4a8cf5e4c4 distro: switch liveImagePipelines to use GenImage*Stages
Use the new `GenImage{Prepare,Finish}Stages` in all liveImagePipelines
methods. This should allow us to handle partition tables with LUKS2 and
LVM2 transparently. Since none of the existing layouts contain any of
those, nothing should currently change.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Christian Kellner
85e2a53606 osbuild2: add new GenImage{Prepare,Finish}Stages
Add new helper functions that will return the necessary stages to
prepare a disk image, i.e. take care of creating the raw image
file, partitioning it, creating the devices on it (LUKS2, LVM2),
and formatting all the file systems: `GenImageStagesPrepare`.
Additionally, some partition layouts require some post-processing,
e.g. LVM2 where the volume group needs to be renamed "offline".
For this the `GenImageFinishStages` was added.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Christian Kellner
670b936e6b osbuild2: new GenDeviceFinishStages method
This should be called at the end of the pipeline that creates an
image, to add stages that are needed to finish up the image.
Currently the only stage that will be added is the `lvm2.metadata`
stage in the case the partition layout contains LVM2.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Christian Kellner
3c6e34aeb3 osbuild2: LVM2Metadata: only validate CreationTime if not empty
It is ok to not pass the creation time to osbuild, i.e. it can be
empty. If so, don't validate it.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Christian Kellner
2314ca37d7 osbuild2: NewLVM2MetadataStage takes map of devices
Instead of just passing in a single device, pass in a `Devices`
type, i.e. a map of devices, since the LVM can be located on a
LUKS2 container which is located inside a partition so multiple
devices are needed to access it.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Christian Kellner
d48d6f22e1 osbuild2: GenCopyFSTreeOptions takes filename not device
The only thing we needed from the passed in device was the filename,
so just pass in that directly.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Christian Kellner
a2e1fcc451 osbuild2: basic checks for GenDeviceCreationStages
Verify that we get the necessary "luks2.format" and "lvm2.create"
stages for a partition table that has LVM2 on top of LUKS2.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Achilleas Koutsou
c347373c90 disk: basic partition table creation tests
Add a few test partition tables with well known configurations, like
LVM, LVM on top of LVM and Btrfs in addition to the plain layout that
is currently used in most partition tables. Add very basic checks,
such as that `NewPartitionTable` works and we have approximately the
correct size.

Co-Authored-By: Christian Kellner <christian@kellner.me>
2022-02-22 19:23:41 +00:00
Christian Kellner
d1b1e32b6e disk: Partition.Payload is now an Entity
Partition.Payload now supports every type of disk.Entity which enables
creating PartitionTables with LUKS, LVM, and Btrfs. \o/

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00
Christian Kellner
206e030f2c disk: guard against nil pointer dereferencing
Add guards protecting against dereferencing the pointer receiver of
method calls.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-22 19:23:41 +00:00