Commit graph

165 commits

Author SHA1 Message Date
Tom Gundersen
cc677dea3c repositories: distinguish between RHEL8 beta and GA
We need the same RPMs to work equally well on a host running a beta
release (pulling beta content) as on a machine running GA (pulling GA
content). Detect this at run-time and point at the right repository.

Testing this is a bit hairy as we are building 8.3 images, but obviously
there is currently no 8.3 content at the GA URLs.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-08-19 15:06:21 +02:00
Major Hayden
3daa751095 distro/rhel8: increase default image size to 4GB
RHEL 8.3 requires more than 2GB to install on aarch64/ppc64le.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-27 10:27:15 -05:00
Martin Sehnoutka
3fc03503a5 distro/f32: manually relabel cp in buildroot with install_t
By labeling `cp` with `system_u:object_r:install_exec_t:s0` we allow it
to copy labels unknown to the host.

See also corresponding commit in osbuild:
e80130a830
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
636b761b6b distro/f32: add SELinux RPM to build pipeline
This patch adds SELinux stage to build pipeline for f32 just like
we do for RHEL8. This helps with issues like this:
https://github.com/osbuild/osbuild-composer/issues/798
2020-07-22 10:47:46 +01:00
Martin Sehnoutka
60269abdff distro: print more information in case the test fails
This helps when looking for the issue because the previous input did not
mention which specific qcow2 test case failed.
2020-07-22 10:47:46 +01:00
chloenayon
ae3fb5e1c5 distro: enable cloud-init services for openstack and qcow2 images for fedora
Previously, cloud-init was installed on qcow and openstack images but was not
enabled, preventing a user from logging in to these images. Cloud-init, cloud-final,
cloud-init-local and cloud-config services are enabled to make cloud-init work
and to maintain consistency with official fedora 31 and fedora 32 images.

The image tests have been updated to reflect these changes.

Fixes #698
2020-07-18 11:57:05 +02:00
Lars Karlitski
f38d55da88 distro/tests: use test case filename as test name
These tests used the image type as test name, which is ambiguous. Use
the file name for the test case instead.
2020-07-18 10:52:11 +01:00
Major Hayden
0f32f35756 📦 Use raw image format for AWS
Although the vhdx format is smaller than raw, AWS often has some issues
booting instances from an imported vhdx snapshot. The raw format seems
to boot consistently at AWS.

Fixes #754.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-02 13:11:11 -05:00
Tom Gundersen
8c7d8a442b osbuild: pass UID/GID as int
These were passed as strings, which is not what osbuild expects.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-29 16:04:38 +02:00
Tom Gundersen
50d469fe45 distro: replace BasePackages() with Packages()
Rather than getting a set of base packages from the ImageType, and then
appending the requested packages from the blueprint, pass the blueprint
into the new Packages() function, and return the full set of packages to
be depsolved.

This allows us to also append packages based on other customizations
too, and use that to append chrony when the timezone is set. This
matches the behavior anaconda had, and there was a TODO item to do this,
which had been overlooked.

Fixes #787.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-29 16:04:38 +02:00
Tom Gundersen
d31e3ebb65 distro: add groups before users
If a user is added to a group, then the group must be added first.
2020-06-29 16:04:38 +02:00
Christian Kellner
408a704669 distro/{rhel8, fedora32}: no fstab for OSTree
How exactly the final file system layout looks like is determined
by the installer (anaconda) and thus can not be known at commit
creation time. Thus creating an /etc/fstab file is unnecessary and
the information in it probably wrong. The file wont be used though
because it will be overwritten during the installation process.
2020-06-21 16:16:14 +02:00
Martin Sehnoutka
1e5c31a7ad distro/rhel8: introduce s390x support
This patch introduces support for s390x architecture. It has some
specific requirements like using kernel-cmdline stage before the RPM
stage because RPM scriptlets require these values to be present in the
system already. It also uses zipl stage and bootloader as opposed to all
the other architectures.
2020-06-11 21:50:09 +02:00
Peter Robinson
f32770fed6 distro/rhel8: add RHEL for Edge
Initial definition for RHEL for edge package set on x86.
This is based marginally on the upstream Fedora IoT package set
but slimmed down and adjusted for RHEL package naming.

Based on Tom's patch.

Signed-off-by: Tom Gundersen <teg@jklm.no>
Signed-off-by: Peter Robinson <pbrobinson@gmail.com>
2020-06-11 20:44:17 +02:00
Tom Gundersen
ab276bdb77 tools/test-case-generator: add more options
Allow individual test-cases or sub-sets of test-cases to be generated
more easily.

We allow explicit skipping of image-info generation (and hence the
osbuild run), and also individual image types to be specified.

Also drop distros and image types that are no longer supported.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-10 19:25:54 +02:00
Tom Gundersen
cbf19d1319 distro/rhel8: label the build root
We require the right SELinux labels to be set on some of the tools we
use, make sure the policy is installed and the labels applied in the
buildroot, in the same way as in the target image.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-10 19:25:54 +02:00
Tom Gundersen
5c3f4193e5 osbuild/rpm: set check_gpg per-package
Rather than passing an array of checksums, pass an array of objects
with package properties. One is the `checksum`, which is required, and
in addition to that we now pass the boolean check_gpg, which indicates
that that RPM should have its GPG signature verified.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-10 19:25:54 +02:00
Tom Gundersen
83a63aaf17 distro/rhel8: add tar image type for all architectures
This is a very minimal image type, which is not expected to do anything
with an empty blueprint. It is documented as a way to install RHEL for
Satellite.

This fixes #720.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-08 13:32:59 +02:00
Martin Sehnoutka
385df479f3 distro/rhe8: introduce ppc64le qcow2 image type
This patch introduces a qcow2 image type for RHEL8 distro and associated
test case.
2020-06-06 16:56:39 +02:00
Martin Sehnoutka
7979a705b6 osbuild: filesystem is not required member of qemu assembler
The input options of qemu assembler do not require filesystem for every
partition. This patch makes it optional and omits the object in case it
is empty.

osbuild schema definition:
162797f206/assemblers/org.osbuild.qemu (L63)
2020-06-05 20:09:20 +02:00
Martin Sehnoutka
673e967b2d osbuild: Change QEMUPartition "Type" to match osbuild
Partition type, as defined in osbuild qemu assembler, is of type string
because it is either UUID or identifier. See here for details:
162797f206/assemblers/org.osbuild.qemu (L88)

Change type of the "Type" struct member to match the definition in
osbuild. This patch is needed for ppc64le support.
2020-06-05 20:09:20 +02:00
Martin Sehnoutka
65bc635c30 osbuild: change legacy type from bool to string
This work is based on previous PRs, namely:
https://github.com/osbuild/osbuild-composer/pull/501
and
https://github.com/osbuild/osbuild/pull/327

The problem here is that we used to treat legacy as a boolean before we
started introducing support for alternative architectures, but now we
need to specify exact strings for the grub2 stage, for example for
ppc64le the legacy parameter looks like this:

```
"legacy": "powerpc-ieee1275"
```

This patch will allow us to introduce support for ppc64le and fix
associated issues:
https://github.com/osbuild/osbuild-composer/issues/693
2020-06-03 12:29:47 +02:00
Tom Gundersen
0417c6d8bb distro: make the osbuild package internal to the distros
Rather than Manifest() returning an osbuild.Manifest object, introduce a
new distro.Manifest object which represents it as an opaque, JSON
serializable object. This new type has the following properties:

1) its serialization is compatible with the input to osbuild,
2) any valid osbuild input can be deserialized into it, and
3) marshalling and unmarshaling to and from JSON is lossless.

This means that even as we change the subset of valid osbulid manifests
that we support, we can still load any previous state from disk, and it
will continue to work just as before, even though we can no longer
deserialize it into our internal notion of osbuild.Manifest.

This fixes the underlying problem of which #685 was a symptom.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-03 00:30:01 +02:00
Tom Gundersen
59355c6c91 distro/rhel8: add preliminary support for ppc64le and s390x
This adds placeholder architecture structs, but no image types are
hooked up.

This will allow osbuild-composer to run on these architectures, and
serve the weldr API, but without support for building any images.

Image type support will be added in the future.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-28 07:34:48 +02:00
Tom Gundersen
a13076a97b distro/rhel8: refactor to be based on fedora-32
This is not a functional change, all produced manifests are as before.
This is in preparation for adding support for more architectures in a
follow-up patch.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-28 07:34:48 +02:00
Tom Gundersen
97b365618a distro/fedora32: minor style fixes
Rename the `Fedora32` type to simply `distribution`, to avoid the
stutter. Move `New()` to the bottom of the file, which is the only
non-generic part. Also make the linter happy.

No functional change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-28 07:34:48 +02:00
Tom Gundersen
48079b3a4d rpmmd/RepoConf: rename Id to Name
This is how it is used in the rest of the code, as a name to represent
the repository in the weldr API. Rename to match its use, and avoid
confusion with the ID passed to dnf-json, which is not the same.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-28 00:23:54 +02:00
Jacob Kozol
5938276d3b osbuild: update files source for rhsm
osbuild manifest's urls should now contain a url and any secrets. The
secrets should contain a name.
2020-05-28 00:23:54 +02:00
Tom Gundersen
e66368b099 distro: drop fedora-30
We never shipped into fedora-30 and it is now EOL.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-27 16:20:35 +02:00
Tom Gundersen
e2c14b9fe6 distro: drop legacy image types
We do not properly test, and do not have properly defined use-cases for
the ext4-filesystem, partitioned-disk, nor tar image types. Drop them to
focus on delivering the things we car properly test.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-27 16:20:35 +02:00
Tom Gundersen
b0cfec767a distro/f32: add iot-commit image type
This produces a Fedora IoT commit tarball, based on
https://github.com/gicmo/ostree-osbuild-demo and the fedora 32 specific
configuration found at
https://pagure.io/fedora-iot/ostree/c/465c8a2c1d1c0cf4b762477af87fb1d82c8ee002?branch=master

The usbguard package is excluded due to some SELinux failure that
remains to be resolved.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-25 22:37:29 +02:00
Tom Gundersen
77ea9d67bf distro/f32: pass ImageOptions to assemblers
Prepare for assemblers needing more image-type specific options.

This is not a functional change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-25 22:37:29 +02:00
Tom Gundersen
0b3702cb19 distro/Manifest: generalize the size argument
When generating an osbuild manifest for an image type, we take a
customizations struct, which specifies the image-type-independent
customizations to apply. We also take the size argument, which is
specific to the image build and not part of the blueprint.

Introduce a new argument ImageOptions, which for now just wraps the size
argument. These options are specific to the image build/type, and
therefore does not belong with the other customizations.

For now this is a non-functional change, but follow-up commits will
introduce more types of image options.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-25 22:37:29 +02:00
Tom Gundersen
cc231e0ed1 distro/rhel-8: merge all the RHEL-8 minor releases into one
As it turns out, the default expectation is not to distinguish between
these. We will now produce whatever is the most recent minor release by
default, and image tests will still be pinned at a given snapshot to be
reproducible.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-21 17:57:28 +02:00
Tom Gundersen
bcf0bae2a7 distro/test: make TestArch and TestImageType public
No need to statically query these values from tests.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
eb658f5495 distro: add Distro() and Arch() accessors
These return the parent object of the Arch and ImageType, respectively.

Not a functional change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
c22797ffdb distro/test: rename to test_distro
Make the package easier to use from elsewhere, `test` is too generic.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Jiri Kortus
cdcd09dd67 [tests] use assert.ElementsMatch instead of reflect.DeepEqual in distro_test
The asserts should do the same job and also make sure that the two
compared lists are the same at the same time.

Refs: #442
2020-05-14 00:34:30 +02:00
Jiri Kortus
b24957134b Improve coverage for distro packages - RHEL-8.3
Refs: #442
2020-05-13 20:32:05 +02:00
Jiri Kortus
04a0862c9a Improve coverage for distro packages - RHEL-8.2
Refs: #442
2020-05-13 20:32:05 +02:00
Jiri Kortus
9315ccb88e Assert existing pipelines in TestDistro_Manifest
Without the assertion some tests can just pass without actually doing
anything if no matching pipeline jsons exist.

Refs: #442
2020-05-08 10:12:28 +03:00
Jiri Kortus
3cea1f27cf Fix paths in TestDistro_Manifest
It turned out the paths contained more ../s than appropriate.

Refs: #442
2020-05-08 10:12:28 +03:00
Jiri Kortus
50df564caf Improve coverage for distro packages - F32
Refs: #442
2020-04-30 11:26:27 +02:00
Jiri Kortus
72192a0f7c Improve coverage for distro packages - F31
Refs: #442
2020-04-30 11:26:27 +02:00
Jiri Kortus
0150adae7a Use assert.NoError in F30 distro test
Refs: #442
2020-04-30 10:58:47 +02:00
Ondřej Budai
83a3a8d821 change the format of the ami image type to vhdx
Prior this commit the ami image type produced raw.xz images. This was bad for
two reasons:

- The upload was broken because AWS doesn't support tar.xz format
- XZ compression is terribly slow

This commit changes the format to vhdx, which is supported by AWS and also
quite quick. See https://github.com/osbuild/osbuild-composer/issues/257
why vhdx was chosen.

Fixes #257
2020-04-24 15:27:39 +02:00
Jiri Kortus
ae5924d29a Improve coverage for distro packages - F30
Refs: #442
2020-04-24 11:16:39 +02:00
Jiri Kortus
46e230212b Make TestDistro_Manifest more versatile
Move TestDistro_Manifest to a separate package and make it parametric,
so that it can be used to test specific distros.

Refs: #442
2020-04-24 11:16:39 +02:00
Jiri Kortus
c4d40b4b0f Fedora31 - fix Size() for vhd
Refs: #442
2020-04-24 08:30:02 +02:00
Major Hayden
1d743f048a 🐣 Add initial RHEL 8.3 support
The osbuild changes were made in osbuild/osbuild#341.

Signed-off-by: Major Hayden <major@redhat.com>
2020-04-22 15:25:59 +02:00