Commit graph

139 commits

Author SHA1 Message Date
Tom Gundersen
48079b3a4d rpmmd/RepoConf: rename Id to Name
This is how it is used in the rest of the code, as a name to represent
the repository in the weldr API. Rename to match its use, and avoid
confusion with the ID passed to dnf-json, which is not the same.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-28 00:23:54 +02:00
Jacob Kozol
5938276d3b osbuild: update files source for rhsm
osbuild manifest's urls should now contain a url and any secrets. The
secrets should contain a name.
2020-05-28 00:23:54 +02:00
Tom Gundersen
e66368b099 distro: drop fedora-30
We never shipped into fedora-30 and it is now EOL.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-27 16:20:35 +02:00
Tom Gundersen
e2c14b9fe6 distro: drop legacy image types
We do not properly test, and do not have properly defined use-cases for
the ext4-filesystem, partitioned-disk, nor tar image types. Drop them to
focus on delivering the things we car properly test.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-27 16:20:35 +02:00
Tom Gundersen
b0cfec767a distro/f32: add iot-commit image type
This produces a Fedora IoT commit tarball, based on
https://github.com/gicmo/ostree-osbuild-demo and the fedora 32 specific
configuration found at
https://pagure.io/fedora-iot/ostree/c/465c8a2c1d1c0cf4b762477af87fb1d82c8ee002?branch=master

The usbguard package is excluded due to some SELinux failure that
remains to be resolved.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-25 22:37:29 +02:00
Tom Gundersen
77ea9d67bf distro/f32: pass ImageOptions to assemblers
Prepare for assemblers needing more image-type specific options.

This is not a functional change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-25 22:37:29 +02:00
Tom Gundersen
0b3702cb19 distro/Manifest: generalize the size argument
When generating an osbuild manifest for an image type, we take a
customizations struct, which specifies the image-type-independent
customizations to apply. We also take the size argument, which is
specific to the image build and not part of the blueprint.

Introduce a new argument ImageOptions, which for now just wraps the size
argument. These options are specific to the image build/type, and
therefore does not belong with the other customizations.

For now this is a non-functional change, but follow-up commits will
introduce more types of image options.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-25 22:37:29 +02:00
Tom Gundersen
cc231e0ed1 distro/rhel-8: merge all the RHEL-8 minor releases into one
As it turns out, the default expectation is not to distinguish between
these. We will now produce whatever is the most recent minor release by
default, and image tests will still be pinned at a given snapshot to be
reproducible.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-21 17:57:28 +02:00
Tom Gundersen
bcf0bae2a7 distro/test: make TestArch and TestImageType public
No need to statically query these values from tests.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
eb658f5495 distro: add Distro() and Arch() accessors
These return the parent object of the Arch and ImageType, respectively.

Not a functional change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
c22797ffdb distro/test: rename to test_distro
Make the package easier to use from elsewhere, `test` is too generic.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Jiri Kortus
cdcd09dd67 [tests] use assert.ElementsMatch instead of reflect.DeepEqual in distro_test
The asserts should do the same job and also make sure that the two
compared lists are the same at the same time.

Refs: #442
2020-05-14 00:34:30 +02:00
Jiri Kortus
b24957134b Improve coverage for distro packages - RHEL-8.3
Refs: #442
2020-05-13 20:32:05 +02:00
Jiri Kortus
04a0862c9a Improve coverage for distro packages - RHEL-8.2
Refs: #442
2020-05-13 20:32:05 +02:00
Jiri Kortus
9315ccb88e Assert existing pipelines in TestDistro_Manifest
Without the assertion some tests can just pass without actually doing
anything if no matching pipeline jsons exist.

Refs: #442
2020-05-08 10:12:28 +03:00
Jiri Kortus
3cea1f27cf Fix paths in TestDistro_Manifest
It turned out the paths contained more ../s than appropriate.

Refs: #442
2020-05-08 10:12:28 +03:00
Jiri Kortus
50df564caf Improve coverage for distro packages - F32
Refs: #442
2020-04-30 11:26:27 +02:00
Jiri Kortus
72192a0f7c Improve coverage for distro packages - F31
Refs: #442
2020-04-30 11:26:27 +02:00
Jiri Kortus
0150adae7a Use assert.NoError in F30 distro test
Refs: #442
2020-04-30 10:58:47 +02:00
Ondřej Budai
83a3a8d821 change the format of the ami image type to vhdx
Prior this commit the ami image type produced raw.xz images. This was bad for
two reasons:

- The upload was broken because AWS doesn't support tar.xz format
- XZ compression is terribly slow

This commit changes the format to vhdx, which is supported by AWS and also
quite quick. See https://github.com/osbuild/osbuild-composer/issues/257
why vhdx was chosen.

Fixes #257
2020-04-24 15:27:39 +02:00
Jiri Kortus
ae5924d29a Improve coverage for distro packages - F30
Refs: #442
2020-04-24 11:16:39 +02:00
Jiri Kortus
46e230212b Make TestDistro_Manifest more versatile
Move TestDistro_Manifest to a separate package and make it parametric,
so that it can be used to test specific distros.

Refs: #442
2020-04-24 11:16:39 +02:00
Jiri Kortus
c4d40b4b0f Fedora31 - fix Size() for vhd
Refs: #442
2020-04-24 08:30:02 +02:00
Major Hayden
1d743f048a 🐣 Add initial RHEL 8.3 support
The osbuild changes were made in osbuild/osbuild#341.

Signed-off-by: Major Hayden <major@redhat.com>
2020-04-22 15:25:59 +02:00
Alexander Todorov
869bb2afd7 tests: Use the require/assert package and replace cmp.Diff 2020-04-19 18:20:57 +02:00
Martin Sehnoutka
439b005514 distro: harcode partuuids
We need to make partition uuids stable to be able to run image-info
tests on images using gpt. For example all aarch64 images use gpt.

Also change the type of the (part) UUID to string because that's what
the other UUIDs use and it is easier to work with.
2020-04-15 15:22:57 +02:00
Martin Sehnoutka
ae294f967d distro: don't include grub2 stage in ext4 images
There is no point in having the grub2 stage in pipelines for image types
that are not bootable. The current version is probably a result of
previous refactoring where the member variable was named `IncludeFSTab`.

Moving the grub2 stage into the conditional branch should also fix test
generation on aarch64.

Finally it is necessary to regenerate test cases for non-bootable image
types.
2020-04-11 13:10:13 +02:00
Ondřej Budai
bbd71efeff distro/rhel: remove dracut-config-generic from build packages
The build pipeline doesn't run dracut nor install kernel, the package is
therefore useless in it.
2020-04-01 12:29:59 +02:00
Ondřej Budai
c48728a049 distro/rhel: move dracut-config-generic to bootloader packages
This change makes the dracut-config-generic installation in par with
Fedora. Also, by having it in bootloader packages, it is not installed in
non-bootable images, where it provides no value.
2020-04-01 12:29:59 +02:00
Ondřej Budai
f14224ede1 distro/fedora: add dracut-config-generic to bootloader packages
Dracut is unfortunately very host-dependant by default. The package
dracut-config-generic forces it use a generic configuration instead of a
configuration generated from the host environment.

This change should make the image generation more reproducible. For example
it was not possible to boot ami images built on Travis on AWS prior this
commit.

Also, the tests were re-generated in this commit.
2020-04-01 12:29:59 +02:00
Martin Sehnoutka
92145822d9 distro: rename ListArchs to ListArches
In PR#395 we discussed the spelling of archs vs. arches and we agreed to
use arches. This patch only renames the public method `ListArchs`in the
`Distro` interface.
2020-03-30 10:41:02 +02:00
Tom Gundersen
7825132ae2 test/cases: include repository information
This means that the unit tests no longer need to load the
repositories from the git repo, and in a follow-up, osbuild-composer
won't need to either.

By splitting the repositories used for testing from the system
repositories available through the weldr API we are able to extend
the system repositories without affecting the reproducibility of
the tests.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-29 16:08:36 +02:00
Tom Gundersen
a949843f1e test/cases: rename output-format to image-type
Bring this in line with our naming convention.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-29 16:08:36 +02:00
Martin Sehnoutka
db4b6e71ed distro_test: add test that covers BuildPackages
The main reason here is not to make sure we have the right set of build
packages, but rather to make sure the pointer hierarchy (distro->arch->
image type) works as expected.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
91f572e961 fedora30,32: merge fedora3XImageType into imageType struct
This patch uses the same approach as fedora31 few commits ago. It moves
the arch pointer to the imageType struct and sets it while creating the
distro->arch->image type hierarchy in the setImageType function. The
rest of the patch is just renaming.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
ff065cc7d0 fedora30,32: merge fedora3XArch structure into arch
It uses the same approach as fedora31 few commits ago. The pointer to
distro is moved to "arch" and "arch" now contains a map of allowed
image types for each arch.

Also include setArches and setImageTypes helper functions to ease the
creation of pointers from the structures to a parent structure while
creating the distro->arches->image types hierarchy.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
507194772b fedora31: create setImageTypes and setArches helpers
Previously we had to type the name of the architecture and image type
multiple times (in the definition, when inserting it into each parent
structure etc.) and it would get even worse because we would have to do
the same for all distros. These helper functions make it less error
prone by copying the name in the function automatically.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
adae7110af fedora31: merge fedora31ImageType with imageType
The former one was just a wrapper to the second one. In this patch I
removed the fedora31ImageType struct and instead moved the pointer to
the imageType struct. The previous solution always created a new wrapper
and returned it to the called. This new solution creates a shallow copy
of imageType, assignes arch field, and then returns the result to the
caller. It should achieve the same without the additional wrapper.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
473fa3b341 fedora31: merge fedora31Arch struct into arch
It was unnecessary to have these separate any more.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
6d2d259e57 distro: remove unused FilenameFromType() function
This information is now provided only when an architecture is specified,
so it is necessary to first obtain object implementing the Arch interface
then object implementing the ImageType interface and then you can get
the filename and mime type.

Tests are changed accordingly to the new API.
2020-03-28 00:21:31 +01:00
Lars Karlitski
ad1a12f7aa tree-wide: use uuid.MustParse() when possible
According to the new guidelines in docs/errors.md.
2020-03-25 10:22:16 +01:00
Martin Sehnoutka
e1e5c1bb0b distro: include xz in distro build packages
xz compression is used for the ami and tar output types, it should be in
the buildroot. This fixes Weldr integration test (/cmd/osbuild-tests) on
ARM architecture (aarch64).

The fact that it worked on x86_64 was just a happy coincidence because
we require grub2-pc which in turn requires dracut which requires xz. We
should not rely on these implicit dependencies because we need xz
unconditionaly, therefore adding it to build packages for all platforms.
2020-03-24 20:45:30 +01:00
Tom Gundersen
f903601ec4 distro: add ListArchs() to Distro interface
Similar to ListImageTypes() add a helper to enumerate the supported
architectures.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 15:46:57 +01:00
Tom Gundersen
87b0bb6e5d distro: mass cleanup
Delete unused methods and make types and fields private where
possible. Some code is moved around, but apart from that there
is no change in behavior.

The naming of the distros were moved back into the distro
packages as the common types now only had one user, and this
allowed us to drop some redundant error checking.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 14:13:03 +01:00
Jiri Kortus
f5cf649511 Fix typo in RootFilesystemUUID
RootFilesystemUUDI -> RootFilesystemUUID
2020-03-24 11:01:20 +01:00
Tom Gundersen
d642fd2c69 distro: drop redundant parts of the interfaces
Usage of these have now been entirely replaced with the newly
introduced interfaces. The individual distros should be cleaned up
and dead code removed, buth that is left for a future patch.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
7fe3421afb distro/test: use new interfaces
Test in terms of the newly introduced interfaces. This should not
have any functional effect.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
5de98458e5 ditro/registry: index by name
This is purely internal, and there is no benefit to translating
to a tag, a string identifier will do just fine.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
bbd8dc338d distro: introduce Arch and ImageType interfaces
Objects implementing these interfaces will represent the
architecture support for a given distro and the image type
support for a given architecture distro combination, respectively.

The idea is to always resolve to these objects early, and drop
the equilavent methods from the distro interface. This means that
we convert our input strings to real objects once, and then never
have to verify their correctness again.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00
Tom Gundersen
ff0ec01eb6 distro: drop NewDefaultRegistry()
Require the caller to pass in the required distros explicitly. This
would allow us to easily add distros in osbuild-pipeline and tests
before exposing them in composer itself, for instance.

This means there is no longer a dependency from the distro package
to each of the individual distros, so the distros are now able
to depend on the distro packag for types and interfaces.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-23 15:43:55 +01:00