Commit graph

156 commits

Author SHA1 Message Date
Simon de Vlieger
258f450021 iot: change the copy stages
Split this into two separate copystages so that we are not overriding
types.
2023-04-18 21:21:09 +02:00
Simon de Vlieger
b0fce3bfd1 platform: add the aarch64_iot platform
This platform copies more files into `/boot` which are necessary to be
able to boot IoT on some single board computers.

We also immediately set this on the `Aarch64_IoT` platform which needs
u-boot to be placed in the `/boot`.

This closes #3312.
2023-04-18 21:21:09 +02:00
Simon de Vlieger
8041563a36 manifest: image-tree -> ostree-deployment
Describes much better what the pipeline actually does.
2023-04-18 21:21:09 +02:00
Simon de Vlieger
66345cf54c manifest: commit_deployment -> ostree_deployment 2023-04-18 21:21:09 +02:00
Achilleas Koutsou
d41a74475e osbuild/skopeo: helper function for files input
Add a helper function that collects all the manifest list digests from a
list of container specs and returns a FilesInput to be used with the
stage.

Use the function in the OS pipeline when adding containers.  The
manifests input to the stage constructor will be empty if there are no
manifest lists in the container specs.
2023-04-17 18:30:41 +02:00
Achilleas Koutsou
c95072d686 osbuild: add manifest-lists input to skopeo stage
The skopeo stage in osbuild supports an second optional set of inputs
called `manifest-lists`.  This is an array of files, i.e.,
`org.osbuild.files` type input.

To support this we need a new type for the skopeo stage inputs that can
encompass both input types, images and manifest-lists.
2023-04-17 18:30:41 +02:00
Achilleas Koutsou
dfae9fbefb osbuild/skopeo: reverse the constructor args
Put the path (which becomes an option) first and the inputs second.
This is more in line with other stage constructors.
2023-04-17 18:30:41 +02:00
Achilleas Koutsou
816d23988c manifest: add support for shell vars in OSCustomizations
ShellVars are added as a nested map of the form
filename->key->value.  A stage is generated and added if any are
defined.
2023-03-30 18:40:12 +02:00
Tomáš Hozza
dc637d9a57 Manifest: always set kernel options in grub2 stage
It turned out that when we stopped setting the kernel options in grubenv
file, we also stopped setting them in /etc/default/grub under
`GRUB_CMDLINE_LINUX`. This file is used by grub2-mkconfig when
generating grub configuration.

10_linux script executed by grub2-mkconfig recently started to overwrite
the /etc/kernel/cmdline, if its timestamp is older than the timestamp of
/etc/default/grub [1]. As a result, all kernel options were wiped out from
/etc/kernel/cmdline.

Make sure that we always set the `KernelOptions` in the grub2 stage
options, even if the `WriteCmdLine` is set to `false`.

In addition, unify the way we concatenate kernel options set in the
grub2 stage options. Some pipeline implementations were previously using
space, other were using comma. Space is now used everywhere.

Regenerate all affected image manifests.

[1] https://src.fedoraproject.org/rpms/grub2/c/fc76aed5333f56dd05400521a35b944a5df52ebc

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-23 16:32:34 +01:00
Antonio Murdaca
414424cd30 simplified-installer: enable isolinux
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2023-03-15 12:35:50 +01:00
Tomáš Hozza
f8b39cd34e manifest/os: fix SUPPRESSED_ERROR issue reported by Coverity
Fix the following error reported by Coverity:
```
Defect type: SUPPRESSED_ERROR
1. osbuild-composer-75/_build/src/github.com/osbuild/osbuild-composer/internal/manifest/os.go:546: suppressed_error: The error returned by the function "GetVerStrFromPackageSpecList(p.packageSpecs, "dracut-config-rescue")" is not explicitly checked.
2. osbuild-composer-75/_build/src/github.com/osbuild/osbuild-composer/internal/manifest/os.go:546: remediation: Explicitly check the return error.
   544|   				}
   545|
   546|-> 				rescueVer, _ := rpmmd.GetVerStrFromPackageSpecList(p.packageSpecs, "dracut-config-rescue")
   547|   				hasRescue := rescueVer != ""
   548|   				bootloader = osbuild.NewGrub2LegacyStage(
```

Change the code to check if the called function returned an error or
not. If not error is returned, then the function returns a package
version string.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-01 20:20:52 +01:00
Antonio Murdaca
ed4d12fe33 ignition: enable systemd firstboot condition through kargs
This is a workaround to make the systemd believe it's firstboot
when ignition runs on real firstboot.
Right now, since we ship /etc/machine-id, systemd thinks it's not firstboot
and ignition depends on it to run on the real firstboot to enable services from presets.
Since this only applies to artifacts with ignition and changing machineid-compat at
commit creation time may have undesiderable effect, we're doing it here as a stopgap.
We may revisit this in the future.
This patch also pins the version of osbuild because it depends on a fix
for the ignition stage.

Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-28 18:35:45 +01:00
Achilleas Koutsou
f11589367e internal: FIXME notes for container package set
Tracing the package set and repository journey, I found that the
container package defined on the image type is never really used.

Added notes to fix later.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
c7e0b69704 manifest: add only necessary repositories to each pipeline
Each pipeline constructor receives the full list of repositories and
filters the list based on its own internal name.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
a66003e513 manifest: helper function for collecting pipeline repos
Function that filters a list of repositories to return only the global
repos and any that define a given pipeline or package set name in their
list of PackageSets.
2023-02-23 16:22:42 +01:00
Tomáš Hozza
b3f695db72 manifest/os: support directories and files customization
Extend the `OSCustomizations` with a list of custom files and
directories, that should be created in the image. If any `Files` or
`Directories` are specified in the `OSCustomizations`, the appropriate
osbuild stages will be added to the `os` pipeline. In addition to that,
any custom files data will be returned by the `getInline()` method of
the `os` pipeline.

This customization can't be yet used by users, because the translation
from BP customization to the `OSCustomizations` is missing.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
b36c03f20c manifest/os: move the systemd services stage more to the end
Move the handling of the list of enabled and disabled systemd services
more to the end of the `os` pipeline, just before the SELinux stage.
This has no functional effect on produced images, but it will make it
nicer once the handling of the custom files and directories will be
added to the pipeline. Specifically it should be added right before the
services stage to allow enabling custom service files, but after all
other configurations that are applied to the image.

Regenerate all manifests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Brian C. Lane
6846f59ff4 manifest: Add tests for subscriptions
There are 3 different ways for subscriptions to be setup, test the
pipeline to make sure it includes the correct commands, and check the
package set chain to make sure it includes the packages needed to
support the selected subscription mode.
2023-02-09 09:41:15 +01:00
Brian C. Lane
b0e388d115 cloudapi: Add subscription option for rhc
When rhc is selected it will install the required packages, register
using rhc and always enable insights.

When rhc is not selected it will use subscription manager for
registration, and optionally enable insights. Also installing required
packages.
2023-02-09 09:41:15 +01:00
Antonio Murdaca
b3f1b0edc8 ignition: drop embedded provisioning url
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-07 13:30:22 +01:00
Achilleas Koutsou
2f96cc7051 manifest: fix dracut modules and drivers for Anaconda installers
When the image definitions were updated to the new framework, I failed
to update the dracut modules for Anaconda installers to match the
existing ones.

The changes in the manifest are at commit
c4af0a1886.
The nvdimm module and the additional drivers were removed.

The nvdimm module in particular is required for http boot but should
only be specified for RHEL 9.  In RHEL 8 it is part of the default set
of modules.

See 02bb7a0b4f and
dc95382ba3 for the original commits that
introduced these changes.
2023-01-31 11:03:26 +01:00
Tomáš Hozza
dd59ce6a16 osbuild: rework XZ stage inputs
The `FilesInputs` was since the beginning an XZ-specific implementation
of the input, but it was implemented in the `files_input.go` in a false
hope that it could be used as a generic stage inputs by any stages. It
turned out that various stages require different implementation of
its input. Specifically there is usually a stage-specific key, which has
assigned a common input type. For XZ stage, the key is `file`.

Remove `FilesInputs` and instead implement `XzStageInputs` which is now
accepted by the XZ stage.

Fix all affected pipeline implementations that use XZ stage.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-30 11:24:08 +01:00
Tomáš Hozza
413538a18e osbuild: rework files input
Rework files input implementation to support all reference types
supported by the input schema. Also implement helper functions to
generate supported reference types. In some cases, the reference
supports e.g. referencing multiple pipelines in the stage inputs, but
this is currently not implemented, since no pipeline in composer uses
it.

Rework the files input unit tests to cover the new functionality.

Adjust all code affected by the changes made to files input.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-30 11:24:08 +01:00
Achilleas Koutsou
450c44aaff manifest: include yumconfig options in OS customizations
YUMConfig is supported in ImageConfig for RHEL 7.  We now copy the
options over to OSCustomizations and create the stage when necessary.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
d2114d0f9e manifest: support force-relabel for SELinux in image config
The RHEL 7 images need to enable the force_autorelabel option for the
SELinux stage in osbuild.  This option should almost never be used but
it was added specifically for RHEL 7.  With the rewrite to the new
definitions and the sharing of pipeline code between all distros, we
need to add support to all stages of the pipeline generation to be able
to enable it.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
822571e28e manifest: support specifying force_size for VPC
The RHEL 7 vpc subformat in qemu does not support force_size so we need
to be able to disable it.  The parameter in all parts is defined as a
pointer because the default value is 'true'.  Not specifying it will
keep the option in the osbuild stage as 'nil', falling back to 'true' in
osbuild.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
b39d802155 manifest: add support for selecting grub2.legacy
Older OS versions (RHEL 7) with older versions of grub2 don't support
BLS entries.  Setting NoBLS to true configures the bootloader with
traditional menu entries through the grub2.legacy osbuild stage.  This
requires specifying extra information for the OS to the pipeline:
version, product, and nick.
2023-01-25 20:37:12 +01:00
Achilleas Koutsou
83fc8218d6 manifest: make the partition tool for raw images configurable
Add the partition tool as an option on the Raw pipeline.  Set it to the
old value (sfdisk) by default.

Expose the option up through the liveImage image kind so that the
distribution can set it if needed.
For RHEL 7, set it to sgdisk.
2023-01-25 20:37:12 +01:00
Tomáš Hozza
8d0f2d7e79 osbuild/mkdir: make Mode a pointer
The default value for the `os.FileMode` is zero, but the actual default
value used by the stage if no value is specified in the options is
`0777`. By using the pointer, we'll allow one to specify `0000`
permissions as a value which won't be omitted from the stage options.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-19 11:15:33 +01:00
Tomáš Hozza
30c3ea791e osbuild/mkdir: rename Path struct to MkdirStagePath
The plain `Path` name was a bit unfortunate, since it was specific to
the `mkdir` stage, but it was used outside of the `osbuild` package as
`osbuild.Path` which was making a wrong impression of it being a generic
path structure. This is not true.

Rename the structure to contain the stage name.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-19 11:15:33 +01:00
Achilleas Koutsou
0b263d5478 distro/rhel8: update qcow2 and openstack to new definitions
Copy osCustomizations() and liveImage() functions from RHEL 9.
Delete unused functions.
2023-01-18 11:50:35 +01:00
Antonio Murdaca
1672fb1cf6 wire ignition bp customization to simplified-installer and raw image
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-01-17 14:02:02 +01:00
Antonio Murdaca
cca0e773f6 support ignition in edge simplified-installer and raw-image
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
Signed-off-by: Irene Diez <idiez@redhat.com>
Co-authored-by: Irene Diez <idiez@redhat.com>
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-01-17 14:02:02 +01:00
Ondřej Budai
b997142db0 common: merge all *ToPtr methods to one generic ToPtr
After introducing Go 1.18 to a project, it's required by law to convert at
least one method to a generic one.

Everyone hates IntToPtr, StringToPtr, BoolToPtr and Uint64ToPtr, so let's
convert them to the ultimate generic ToPtr one.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-01-09 14:03:18 +01:00
Irene Diez
76728818c9 internal/distro/rhel9: handle ignition.embedded for edge
Signed-off-by: Irene Diez <idiez@redhat.com>
2022-12-15 11:34:17 +00:00
Achilleas Koutsou
a0528d2e53 osbuild: remove 'users' boolean argument from NewAnacondaStageOptions()
Drop the argument altogether.
2022-12-13 20:33:27 +00:00
Achilleas Koutsou
a63fe649cb manifest: don't use the users flag in NewAnacondaStageOptions()
Before we only had one optional kickstart module we would enable that
was Users, so we just had a single boolean for adding it.  Now that we
often need to add multiple modules, keep the default ones in the
NewAnacondaStageOptions() function, but only have a single argument for
adding additional, optional modules.  If the user module needs to be
enabled, add it to the list of additional ones.
2022-12-13 20:33:27 +00:00
Achilleas Koutsou
6bfa3dd366 image: add ISOLinux stage only for x86_64
The previous fix (9e66ee13e7) only fixed
half the problem with this bug.  The ISOLinux stage was still added to
the iso-tree pipeline for Anaconda builds.

This is now removed.
2022-12-13 21:14:55 +01:00
Achilleas Koutsou
eaf3dc2ecc manifest: always add inst.stage2 kernel option for Anaconda
The bootiso.mono stage in osbuild that we used until recently adds the
inst.stage2 option unconditionally [1] whereas the current grub2.iso
stage that we use now doesn't.

[1] 8511add169/stages/org.osbuild.bootiso.mono (L369)
2022-12-13 07:49:32 +00:00
Achilleas Koutsou
19ec3be6bd manifest: return inline FDO cert from CoreOSInstaller pipeline
The CoreOSInstaller pipeline supports adding inline data to the tree for
the FDO cert.  This needs to be returned by the pipeline object via the
getInline() method to attach the necessary data to the Sources array in
the manifest.
2022-12-11 12:37:37 +00:00
Achilleas Koutsou
bc4583eb81 manifest: new COI bootiso-tree pipeline
New bootiso-tree pipeline specific to the CoreOS Installer.

This is very similar to the Anaconda bootiso-tree pipeline.  Usually we
generalise these pipelines to support both installers.  The bootiso-tree
is different enough that generalising the one for Anaconda would make it
harder to work with.
2022-12-11 12:37:37 +00:00
Achilleas Koutsou
bd448edad0 manifest: support adding per-distro dracut modules to installers
Different distros and image types have different dracut modules
available and enabled.  Set these at the distro level and push them down
through the appropriate manifests, pipelines, and stages.

Added to both anaconda and coreos installers.
2022-12-11 12:37:37 +00:00
Achilleas Koutsou
9511c76063 manifest: use options from fdo package instead of blueprint
Use the internal representation of the FDO options in on the
CoreOSInstaller pipeline instead of the blueprint.
2022-12-11 12:37:37 +00:00
Achilleas Koutsou
edce65eabd manifest: move kernel option handling outside EFIBootTree
The kernel arguments for the EFIBootTree depend on the type of ISO we're
building, the payload, and the location of the kickstart file if any.
Instead of setting multiple fields on the object, most of which would be
mutually exclusive, set the kernel arguments from the image manifest
generator function which has all the information.
2022-12-11 12:37:37 +00:00
Achilleas Koutsou
fae824d574 manifest: rename ISOTree to AnacondaISOTree
The ISOTree struct and pipeline generator is specific to Anaconda.
Rename it for clarity and to differentiate from any other ISO tree
pipeline.
2022-12-11 12:37:37 +00:00
Achilleas Koutsou
3d1f283dab manifest: add any pipeline to ISO pipeline
Support adding any pipeline as the tree for the ISO.

The existing ISOTree (bootiso-tree) pipeline is specific to Anaconda.
This change will allow the ISO (bootiso) pipeline to be created with any
pipeline, so we can add an ISO tree that is specific to the CoreOS
Installer.
2022-12-11 12:37:37 +00:00
Achilleas Koutsou
115c0717c4 manifest: remove Anaconda pipeline from EFIBootTree
Remove dependence of EFIBootTree pipeline (efiboot-tree) on an Anaconda
pipeline.  The Anaconda pipeline was used to get the product and version
strings, so we pass those in instead.
This way, we an use the pipeline without needing an Anaconda pipeline.
The EFI tree shouldn't depend on an installer (or any) pipeline.
2022-12-11 12:37:37 +00:00
Achilleas Koutsou
2b73a3624c manifest: add any pipeline to ISO rootfs
Support adding any pipeline as the source of the ISO rootfs.
This will support both Anaconda and COI, or anything else we might need
in the future.
2022-12-11 12:37:37 +00:00
Achilleas Koutsou
c8710e407c manifest: new CoreOS Installer pipeline
New CoreOS Installer (coi-tree) pipeline.
Based on the existing pipeline in rhel9/pipelines.
2022-12-11 12:37:37 +00:00
Sayan Paul
6f56bee611 build:add journald stage to thel 8/9 pipeline
Adds the new journald stage to rhel 8/9
rpmostree commit pipeline.This will add
the dropin file `10-persitent.conf` with
the storage option set as persistent.

Co-authored-by: Sayan Paul <paul.sayan@gmail.com>
Co-authored-by: Irene Diez <idiez@redhat.com>
Signed-off-by: Sayan Paul <saypaul@redhat.com>
2022-12-06 12:15:09 +01:00