Commit graph

4803 commits

Author SHA1 Message Date
Achilleas Koutsou
a505defad3 manifest: lock the root password for OSTreeDeployment 2022-09-13 16:06:19 +01:00
Achilleas Koutsou
fbe8c1cb39 manifest: use new ostree mount in fstab stage for commit deployment
Use the new stage mount instead of the old stage-specific options.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
1de55c73e6 ostree: define type for handling ostree remote config
A new struct in ostree can be used to define configuration options for
the ostree remote of an image.  So far remotes were always set up with
the remote URL used to pull the commit.  Now we support setting a
different remote with extra configuration options.

This is used by the fedora-iot-raw-image to set up the remote
configuration of the final image, separately from the source of the
commit.

Test manifests updated.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
0386d68db4 manifest: add ostree mount to stages in OSTreeDeployment
Stages that modify files in the ostree deployment need the sysroot
bindmount for any changes to take effect.
Adding the mount to the keymap and locale stages since they need to
modify files in /etc.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
0750049739 osbuild: update org.osbuild.ostree.remotes stage
New options added to stage in osbuild [0].

[0] https://github.com/osbuild/osbuild/pull/1097
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
0ff1c3241c osbuild: add mount ostree helper method to Stage
A convenience method that adds an ostree deployment mount to any Stage.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
4a68df45c7 osbuild: add ostree.deployment mount type
New mount type added: org.osbuild.ostree.deployment.

Adding this to a stage will setup all needed bind mounts so that a given
`tree` will look like an active OSTree deployment, very much as OSTree
does during early boot.

This is often necessary when making changes to files in /etc for an
ostree image.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
7246f8fdd5 osbuild: fix base Mount struct properties
Source and Target are not required -> omitempty
Options is of type MountOptions which is an interface, so we don't need
to make it a pointer.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
76031d4104 manifest: add OSTreeDeployment config options
Adding support for config options to OSTreeDeployment that are required
by the IoT raw image:
- Kernel command line options
- Keyboard layout
- Locale

Test manifests updated.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
ce7b251b54 platform: fix error message for invalid image format 2022-09-13 16:06:19 +01:00
Achilleas Koutsou
21d2c78f20 manifest: remove s390x case from OSTreeDeployment
Not supported.  No need to switch.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
901b3a5da9 fedora: add platform packages to RawOSTreeImage build root
Include the platform packages when getting the build packages for the
RawOSTreeImage.
rpm-ostree is explicitly added for this image type.
dracut-config-generic and efibootmgr are temporarily added here, but we
should define a platform that includes them instead (some cleanup
required in general).
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
b85fe8109c manifest: add the platform configuration to the RawOSTreeImage 2022-09-13 16:06:19 +01:00
Achilleas Koutsou
c38c7ad126 disk: add packages for LUKS containers
Add the packages required for LUKS containers to the build root when
getting the list of packages from the PartitionTable.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
eb731e031d test: generate new manifests
Add new image type to format request map and generate manifests.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
f32e653315 store: trim compat string map
The map is meant to maintain compatibility with changes in image type
names.  Most entries however map to themselves.  It's redundant to keep
updating this map with every new image type name and map it to itself.
Instead, check the map for a compat string and if one doesn't exist,
return the original.

Conversely, when the inverse lookup is performed, first check if the
compat string is a valid image type name and only iterate the map if
it's not.

Now we don't need to update the map every time we introduce a new image
type, unless we need to map an old name to a new one.
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
c6843db6b4 fedora: add the iot-raw-image to x86 and aarch64
Add the new image type to the list in each architecture and update
tests.

Ignore ostree raw images in Kernel count test in distro_test_common:

Edge and IoT raw images don't need a kernel specified in their OS
pipeline.  The kernel (and the OS in general, including all packages)
come from the commit that is pulled and deployed in the image.

This test passes on RHEL (for edge-raw-image types) because the
blueprint defaults to returning the main kernel, but this isn't
necessary and is likely to change in the near future.

Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
74a196e427 image: complete OSTreeRawImage.InstantiateManifest()
Create all the necessary pipelines for the OSTreeRawImage and return the
artifact from the XZ pipeline.

Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
f39b3bf6d3 manifest: add xz pipeline for compressing ostree images
The XZ pipeline is added explicitly for ostree raw images for now.
It should be changed to support compressing general raw images as well.

Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
2683474519 manifest: add RawOSTreeImage
The RawOSTreeImage pipeline is the OSTree-equivalent to the RawImage.
The serialization is very similar between the two (almost identical),
but the two Pipelines depend on different Tree implementations.

Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
a8a2191670 manifest: implement OSTreeDeployment pipeline
New pipeline for creating an ostree commit deployment.
Heavily based on the ostreeDeployPipeline from RHEL 9.x.

Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
526ea0541c distro/fedora: return OSTreeRawImage from iotRawImage
Some basic information and ostree options copied to image.

Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
5d61ba5d39 image: add OSTreeRawImage ImageKind
InstantiateManifest is a stub.  No implementation yet.

Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
280d27a1c8 distro/fedora: add iot partition tables
Based on the official images.
Converted to GPT (from DOS).
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
660444fed1 distro/fedora: introduce fedora-iot-raw-image
Image definition is a stub.  No implementation yet and not added to
distro image type list.

Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Ondřej Budai
e2a298723e ci: enable fedora 37 tests
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Ondřej Budai
a8fe0d2620 distro/fedora: add support for Fedora 37 and 38
The most interesting change is the removal of smc-meera-fonts in 37. As
suggested, rit-meera-new-fonts is used instead.

Existing F35 and F36 manifests updated with package changes.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Ondřej Budai
ff8af88a24 distro/fedora: deduplicate Fedora distribution structs
These are just super-simple to construct using a small helper.

It would be great if we can make `distro.go` totally version-agnostic but
that's something for the future.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Achilleas Koutsou
c88d0255da test/diff-manifest: fetch base branch from GitHub
Distro version branches aren't synced to GitLab, so we will need to
fetch them from GitHub directly.
This is required for any PR made against any branch other than main.
2022-09-13 10:57:16 +02:00
Xiaofeng Wang
05503c58c2 test: Remove OCP4 test
Since api.ocp.ci.centos.org is going away:
https://lists.centos.org/pipermail/ci-users/2022-June/004547.html.

After discussion, OCP4 test will be removed here, but RHEL for Edge
downstream test will keep OCP4 case
2022-09-12 19:14:13 +02:00
Jakub Rusz
63a57ed383 CI: Introduce x86_64 rules
We have recently added testing on aarch64 but not all of the test are
supported on that architecture. Adding specific rules for all and x86_64
arches to simplify which test jobs run on what arches.
2022-09-12 13:02:41 +02:00
Jakub Rusz
47d0703cb6 test/upgrade: install vdo package
This package is required for the upgrade process.
2022-09-12 13:02:41 +02:00
Lukas Zapletal
a8afca4634 Introduce logging adapter for jobqueue 2022-09-09 16:27:38 +02:00
Lukas Zapletal
c43b4992fe Add IntelliJ IDEA/GoLand to .gitignore 2022-09-09 16:27:38 +02:00
schutzbot
001b8339c5 schutzfile: Update snapshots to 20220906
Updates all snapshots bar centos-stream-9, as tests using libvirt are
broken with the newer snapshots.
2022-09-09 12:59:01 +02:00
Diaa Sami
6df494f810 tools/appsre: disable packer parallel builds 2022-09-09 12:08:29 +02:00
Diaa Sami
819a63e50e templates/packer: reasonable aws_polling limits for rhel AWS builds 2022-09-09 12:08:29 +02:00
Diaa Sami
46d36a0e73 Revert "appsre: disable aarch64 AMI creation until issue is resolved"
This reverts commit 84f46eebdb.
2022-09-09 12:08:29 +02:00
Alexander Todorov
647acfa2f4 Adjust AWS EC2 type in tests depending on CPU arch 2022-09-09 10:49:41 +03:00
Alexander Todorov
769ffc617d tests: Execute vmware testing only on x86_64 2022-09-09 10:49:41 +03:00
Alexander Todorov
3b16145ec9 tests: Terraform RPM available only on x86_64
according to
https://www.hashicorp.com/blog/announcing-the-hashicorp-linux-repository
https://discuss.hashicorp.com/t/rhel-8-aarch64-rpms/24244/3

and
https://github.com/hashicorp/packer/issues/11516

Don't execute azure.sh if the runner is not x86_64
2022-09-09 10:49:41 +03:00
Alexander Todorov
c4f8947e73 Enable integration testing on aarch64
NOTE: yaml can merge maps but not sequences
2022-09-09 10:49:41 +03:00
dependabot[bot]
5088e40795 build(deps): bump github.com/aws/aws-sdk-go from 1.44.44 to 1.44.93
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.44 to 1.44.93.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.44...v1.44.93)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-08 14:16:34 +02:00
Juan Abia
810f3fe0e9 test: change CIV tag
We were using `latest` as tag, this can be dangerous as it's the default
tag, an anyone can accidentally update it. Using `prod` is safer.

Also use dev container image if the test script is running in CIV CI.
2022-09-07 17:59:50 +02:00
Diaa Sami
84f46eebdb appsre: disable aarch64 AMI creation until issue is resolved
after merging of PR #2718, generation of AMIs has been failing with 'ResourceNotReady: exceeded wait attempts'.
issue tracked in #2961
2022-09-07 12:28:40 +02:00
schutzbot
a8d3e07577 Post release version bump
[skip ci]
2022-09-07 08:30:20 +00:00
Tomas Hozza
351bb69d2b distro: use reflection in ImageConfig.InheritFrom()
As it turned out, people make mistakes and forget to write some parts of
code, unless a unit test screams at them. This is true for the
`InheritFrom()` method, which is not handling all members of the
`ImageConfig` structure.

Use reflection, instead of inheriting from each specific hard-coded
structure member. This will make the implementation future-proof in case
the `ImageConfig` structure is extended with additional members.
2022-09-06 14:43:24 +02:00
Tomas Hozza
c8382f1654 distro: use pointers for basic types in ImageConfig
Using basic types as values in the `ImageConfig` structure makes it
impossible to distinguish if the empty value for the type was set
intentionally or if it is just the value the variable was initialized
to. This is very bad especially for `bool` type.

While working on unifying `vhd` and `azure-rhui` image types I found
out, that some newly added variables in the `ImageConfig` structure
were forgotten in the `InheritFrom()` method. This makes it impossible
to inherit their values from a parent configuration. This is however
required for the unification of `vhd` and `azure-rhui` image types. As
described above, it would be impossible to decide whether a `bool` value
should be inherited from the parent configuration or not. The only
solution is to use a pointer to the type. For consistency, use pointer
for all basic types.

Adjust distro implementations accordingly.
2022-09-06 14:43:24 +02:00
Sanne Raymaekers
ab3bd7d94f templates/packer: Increase aws timeouts for rhel-8-aarch64
This job is failing with "ResourceNotReady: exceeded wait attempts".

https://www.packer.io/plugins/builders/amazon#resourcenotready-error
2022-09-05 14:39:12 +02:00
Diaa Sami
a22b0c229a appsre-ansible: subscribe before upgrading packages 2022-09-05 12:08:57 +02:00