Pass through all repos to the initalizeManifest() function. Each
pipeline will then select which repositories it needs based on the
PackageSets field of each repository.
Before, we only passed global repos down to the manifest generators and
pipeline-specific repositories would only be used if they were attached
to package sets and were handled explicitly by a pipeline generator.
The repositories of the "blueprint" package set are explicitly added to
the workload and returned by the "os" pipeline.
The repositories of the "installer" package set are explicitly added to
the "anaconda-tree" pipeline.
If a repository was specified for any other pipeline, for example
"build", the repositories for the that package set were never added to
the pipeline.
Fixes#3290
Match the key for the OS packages to the name of the OS pipeline ("os").
We will use this key to identify package sets that are returned from the
OS pipeline since the manifest returns package sets indexed by each
pipeline's name.
Add a default policy for custom directories and files to constrain what
users can do. The intention is to ensure that directories and files can
be created only in `/etc` and also that none of the important
configuration files can be overwritten by this customization.
Add the policy validation to all distro implementation.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Hook up the custom BP directories and files implementation with OS
pipeline implementation. The user-provided values are now set in the OS
customizations structure and will be used by the OS pipeline generator
when adding stages to the pipeline.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Move the `CheckMountpoints()` implementation to `blueprint` package,
since it does not operate on any data structures from the `disk`.
Move the default mountpoint allow list policy definition to the
`pathpolicy` package.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The host distro object was identical to the regular distro objects for a
while now. The constructors in the registry have been aliases to the
base constructors for a long time.
- Deleted all HostDistro constructors from the distributions.
- Changed the supported distro list to only contain base constructor
functions.
- The host distro in the distro registry is a copy of the base distro
that matches the host and does not call a separate constructor.
After introducing Go 1.18 to a project, it's required by law to convert at
least one method to a generic one.
Everyone hates IntToPtr, StringToPtr, BoolToPtr and Uint64ToPtr, so let's
convert them to the ultimate generic ToPtr one.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Fedora 35 is going EOL on Tue 2022-12-13. At the time of writing this commit
message, that's the next day. As we do releases on Wednesdays, the next
release will never find its way to F35 and thus, there's no point in keeping
support for it.
Let's delete everything that relates to Fedora 35. If there's something that
cannot be deleted (e.g. CI containers based on F35), let's upgrade it to F37.
TestCrossArchDepsolve now uses CentOS Stream 8 because RHEL 8.4 cannot read
F37 repository metadata. This is a similar issue to
https://bugzilla.redhat.com/show_bug.cgi?id=2004853 . Basically, newer
repositories can be only read by libmodulemd >= 2.11.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Before we only had one optional kickstart module we would enable that
was Users, so we just had a single boolean for adding it. Now that we
often need to add multiple modules, keep the default ones in the
NewAnacondaStageOptions() function, but only have a single argument for
adding additional, optional modules. If the user module needs to be
enabled, add it to the list of additional ones.
Make edge raw images (both the raw image type and the same in the edge
simplified installer) mount the /sysroot as read-only. This was already
done in Fedora 37+ (547f7a66b3).
Copied commit message from 6f89e9d499 to a
comment in all places where the accompanying "rw" kernel option is set
explaining the requirement, since the option is counter-intuitive.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
Add support for specifying the squashfs compression method for ISOTree
pipelines from the caller.
Build Fedora ISOs with lz4 compression and RHEL with xz.
- Add a list of image type names to skip with explanations.
- Run the test in two configurations: empty blueprint and blueprint with
named kernel (kernel, kernel-debug).
- Count only unique kernel names. Specifying the same kernel twice
isn't an issue. We might change this in the future, but for now the
kernel selection logic is a bit messy.
- Temporarily skip the image-installer until we settle on a common rule
for both Fedora and RHEL.
Similar to 194351e681 and
9ae47b6843cace3c9bef55d40a33fa11d8de391a.
Manifest initialisation during package collection uses options and
customizations to determine build root packages. Just like adding a
fake ostree checksum and an unresolved container list, we also need to
add the ostree ref, which is used to determine if a pipeline is being
generated for an ostree-based image type.
This fixes manifest generation when an ostree commit or container is
being built with an embedded container through the cloud API. Without
this change, the python toml module is not installed in the build root
and the container storage stage fails.
Adding support for container embedding.
The containers need to be specified in the image function (imageFunc)
arguments and when specified, propagate down to the OS pipeline
generator to add the necessary stages.
Support is added for RHEL 9.x and Fedora.
Requires a temporary container spec array with the info from the
blueprint for the first initialization of the manifest that's needed
when collecting required packages.
This should be simplified in the future.
OSCustomizations defined firewall options as blueprint firewall
customizations. Changed it to instead be osbuild firewall stage
options.
This is swapping out one messy thing for another, but at least now we're
consistent in our mess.
The org.osbuild.chrony stage was extended to allow additional directives
alongside time servers [1]. The old Timeservers string slice was kept
for backwards compatibility. Removing support for it in
osbuild-composer makes working with the stage's options simpler. Using
the new struct slice Servers field and only specifying a Hostname for
each element is equivalent to the old behaviour, so no functionality is
lost.
This simplifies the chrony stage since no validation is required
anymore.
It also simplifies the propagation of configuration options through the
pipeline generation code which doesn't need to check for both types of
stage options.
[1] https://github.com/osbuild/osbuild/pull/692
This adds the `fedora-image-installer` and
`fedora-image-installer-preview` images.
The image installer type installs anaconda-webui on Fedora >= 38 to use
the new UI. It also writes its setting to
`/usr/share/anaconda/interactive-defaults.ks` as the current
anaconda-webui has not yet been tested in kickstart mode.
To do so manifest.Anaconda was expanded to take a (subset) of options
for a KickstartStage which is will write into interactive-defaults.ks.
And to take a list of additional modules to enable, so we can set up
Anaconda with all default modules.
F37 no longer ships sil-scheherazade-fonts, but
sil-scheherazade-new-fonts instead. Let's change this. The repos for
test manifests must have been updated in order to get the new package.
Co-authored-by: Ondřej Budai <ondrej@budai.cz>
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Previously, this just happened silently and let to extremely odd errors. Let's
just print the error to simplify debugging the next time.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
8fdd158799 modified the Cloud API to resolve
ostree commits using a separate job. This change caused the API handler
to call PackageSets without any ostree options (because they are not resolved
yet).
Unfortunately, the new implementation of PackageSets initializes the manifest.
The initialization checks the options and if the type is iot-installer and
it doesn't have the fetch checksum for IoT, it just returns an error.
To work around this (we need an initialized manifest to create the chains),
this commit just gives the initialization method a dummy checksum. The ostree
options currently don't have any effect on the package sets, so this should
be fine.
In order to make this workaround at least slightly sane, a warning is printed,
there's a new test just for this behaviour and a long comment to remember to
delete these lines.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Make the ostree commit spec mandatory in the OSTreeRawImage by adding it
to the constructor.
Use the ostree.CommitSpec to specify parameters in the OSTreeRawImage
ImageKind and the OSTreeDeployment Pipeline.
Make the ostree commit spec mandatory in the OSTreeInstaller ImageKind.
The installer image type is not just for ostree types so make the ostree
parameters optional for the ISOTree Pipeline.
Use the ostree.CommitSpec to specify commits parameters.
In the OS pipeline, the parent configuration was used to detect if the
pipeline's setup was meant for an ostree commit or not. Also, the
pipeline used a new type to specify the ostree parameters.
- Use the ostree.CommitSpec for the parent configuration.
- Add a new attribute, OSTreeRef, that defines the ref for the ostree
commit being built. An empty string indicates that the tree is not
for an ostree commit.
Additionally, in the ImageKind configurations for the ostree archive and
container, separate the ostree ref from the parent spec, make the parent
spec optional (pointer) and the ostree ref mandatory, by requiring it in
the constructor of the ImageKind.
Instead of using the ostree.RequestParams in the OSTReeImageOptions,
define a new struct specific to ImageOptions for the ostree parameters.
This is almost identical to the new ostree.CommitSpec but the meaning of
the parameters changes based on image type and it would not be clear if
the CommitSpec was used in all cases. For example, the parameters of
the new OSTreeImageOptions do not always refer to the same commit. The
URL and Checksum may point to a parent commit to be pulled in to base
the new commit on, while the Ref refers to the new commit that will be
built (which may have a different ref from the parent).
The ostree.ResolveParams() function now returns two strings, the
resolved ref, which is replaced by the defaultRef if it's not specified
in the request, and the resolved parent checksum if a URL is specified.
The URL does not need to be returned since it's always the same as the
one specified in the request.
The function has been rewritten to make the logic more clear.
The docstring for the function has been rewritten to cover all use cases
and error conditions.
Don't pass blueprint Users and Groups options all the way down to the
osbuild stage bindings. Instead, convert them to the internal
users.User and users.Group structs.
Ideally we would do this even higher up in the code path, before
reaching the distro, but this is the first step towards that.
Remove the 'fedora' prefix from the canonical name for fedora-iot image
types. Make the previous names aliases.
This has little functional change since we're simply swapping the
canonical name with an existing alias.
Don't redefine the storage unit multiples in each distro, but use the
constants defined in the `common` package. This will make it easier to
split related image type definitions into separate files.
A new struct in ostree can be used to define configuration options for
the ostree remote of an image. So far remotes were always set up with
the remote URL used to pull the commit. Now we support setting a
different remote with extra configuration options.
This is used by the fedora-iot-raw-image to set up the remote
configuration of the final image, separately from the source of the
commit.
Test manifests updated.
Adding support for config options to OSTreeDeployment that are required
by the IoT raw image:
- Kernel command line options
- Keyboard layout
- Locale
Test manifests updated.
Include the platform packages when getting the build packages for the
RawOSTreeImage.
rpm-ostree is explicitly added for this image type.
dracut-config-generic and efibootmgr are temporarily added here, but we
should define a platform that includes them instead (some cleanup
required in general).
Add the new image type to the list in each architecture and update
tests.
Ignore ostree raw images in Kernel count test in distro_test_common:
Edge and IoT raw images don't need a kernel specified in their OS
pipeline. The kernel (and the OS in general, including all packages)
come from the commit that is pulled and deployed in the image.
This test passes on RHEL (for edge-raw-image types) because the
blueprint defaults to returning the main kernel, but this isn't
necessary and is likely to change in the near future.
Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
