Commit graph

3365 commits

Author SHA1 Message Date
Lars Karlitski
5bce59b979 worker: do not get the distro from the host
Add a `Distro` field to both `Job` structures and send that to the
worker.
2019-12-15 22:05:31 +01:00
Lars Karlitski
8c4ee795f7 distro: add Distro.Name()
Right now, there is no way to get at the name from a Distro instance.
We will need this to include the distro's name in the job we pass to the
worker, for instance.
2019-12-15 22:05:31 +01:00
Tom Gundersen
ba32f58ac8 distro/rhel82/qcow2: bring package set in line with official guest image
Also hook up the tast case to verify that the image boots.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-15 21:41:33 +01:00
Tom Gundersen
5e41526d16 distro/rhel82: add aarch64 support
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
b18c66f9fb distro/rhel82: add UEFI support
This follows the style used in fedor-30.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
55cc82d545 distro/fedora-30: add aarch64 support
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
db300cb481 distro/fedora-30: add UEFI support
On architectures that require EFI, we must create the ESP partition
and use a GPT partition table. We must also install either the UEFI
or the legacy version of GRUB2 in the image.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
8c1f194fc1 distro/fedora-30: set the module_platform_id
Make sure we don't fallback to looking at /etc/os-release.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
18971bd8ef pipeline/stage/grub2: support explicit legacy/uefi API
This allows legacy bootloader and UEFI bootloaders to be
enabled/disabled independently of each other.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
7e4c4e4f6d pipeline/stage/grub2: make the boot uuid optional
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
4c892d6d32 pipeline/assembler/qemu: rework partition table handling
Move to the new options format, allowing more flexible partition
tables. The pipeline changes, but the result should be the same.

This requires a yet-to-be-released version of osbuild.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
dbdac222a5 pipeline/stage/fstab: represent UUID as a string
These may not be strictly speaking well-formed UUID. Some filesystems
use other kinds of identifiers.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
506b012f08 distro: take architecture as an argument
Allow bootloader specific packages to be defined per architecture,
and allow repositories to depend on the architecture.

This does not altert he pipelines we produce, part from the ami
image now contains the grub2-pc package, rather than the grub2
package. This should make no difference.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
b034117e62 distro: sort the packages passed to the dnf stage
This should not have any effect apart from hiding any internal changes
to the way we compile the lists.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
6e2731deb8 distro/rhel82: fix the repositories
Fix these while we develop so we are guaranteed all tests run against
the same packages.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
e3be97990b distro/rhel82: include dracut-config-generic
We must make sure not to depend on the modules loaded in the host
kernel. This makes the initrd the same between different hosts.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
85ff8bcec2 distro/rhel82: exclude the timedatex pacakge for now
As for other output types, there is an SELinux bug that makes this
uninstallable.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
5b9d85dece distro/rhel82/vhd: drop stale comments
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Lars Karlitski
b28705d4b4 jobqueue: copy outputs from the right location
d55abdfd75 disabled caching the osbuild
store by moving it to /var/tmp. Fix the directory we're copying the
outputs from as well.
2019-12-13 12:38:36 +01:00
Lars Karlitski
b60bb1d011 distro: pass compression argument to the tar assembler 2019-12-13 12:21:47 +01:00
Lars Karlitski
57aea4160d weldr: return version as string from api/status
This is what lorax-composer does and composer-cli expects.
2019-12-13 00:14:40 +01:00
Lars Karlitski
42f46a5992 distro/rhel82: make openstack's package list constistent with lorax 2019-12-13 00:14:07 +01:00
Martin Sehnoutka
b7b60363dc Modify RHEL8.2 VHD image to boot in Azure 2019-12-12 13:52:21 +01:00
Tom Gundersen
d074395474 distro/rhel-8.2: update VHD images to be closer to lorax
The current one was taken from fedora and that failed both due to an
SELinux bug in RHEL8.2, but also because the list of packages was
wrong for the distro.

We don't enable WAAgent, which we probably should. And there are a few
other config tweaks to cloud-init and WAAgent that we skip too.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-12 13:52:21 +01:00
Lars Karlitski
305b2dda5c rpmmd: add IgnoreSSL key to RepoConfig
This is the opposite of dnf.conf's `sslverify`, because go's default for
booleans is always false. This is error prone: we'd like to default to
true.
2019-12-12 01:27:04 +00:00
Lars Karlitski
2b42612336 distro: insert custom sources into pipeline
This is an additional argument on `distro.Pipeline` right now. In the
future, we probably want this per-blueprint.
2019-12-11 21:19:32 +00:00
Lars Karlitski
d9ad5d7062 weldr: add toml output to projects/source/info 2019-12-11 18:53:10 +00:00
Lars Karlitski
72e1da47ab weldr: make projects/source/new accept text/x-toml
composer-cli sends sources as toml if the original format is toml.
2019-12-11 18:53:10 +00:00
Tom Gundersen
d55abdfd75 osbuild-worker: disable the osbuild cache
We still need to figure out the garbage collection, so for now don't
enable the cache.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-11 15:23:24 +01:00
Tom Gundersen
d33fc5f010 composer: add provisional multi-arch support
The pipeline generation now takes the architecture as an argument.
Currently only x86_64 is supported. The architecture is detected
at start-up, and passed down to each pipeline translation.

For osbuild-pipeline we now requrie the architecture to be passed
in.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-11 15:23:24 +01:00
Lars Karlitski
dcc9cdedee distro: rename output of qcow2 image to disk.qcow2
Integration tests currently expect that and they need to be able to run
against lorax-composer (which outputs disk.qcow2) and osbuild-composer.
2019-12-10 20:21:47 +00:00
Lars Karlitski
ea70d9dfc5 blueprint: don't set omitempty for toml groups and modules
The integration tests expect the empty arrays to be there.
2019-12-10 20:21:47 +00:00
Lars Karlitski
0718766fbc weldr: fix reply on compose/info route
composer-cli requires `dependencies.packages` to exist. We can still
leave it empty for now, because it doesn't look at the contents.
2019-12-10 20:21:47 +00:00
Lars Karlitski
b7dac20c74 weldr: add test for blueprint/depsolve route 2019-12-10 20:38:22 +01:00
Lars Karlitski
280a7c3950 distro/rhel82: use latest nightly
Now that osbuild-composer is inserting repository checksums dynamically,
we can use a URL that points to a changing repository.
2019-12-10 20:38:22 +01:00
Lars Karlitski
d3a0b788a2 distro: set the repository checksum dynamically
Instead of having a static repository checksum, set it dynamically from
the metadata that osbuild-composer last saw. This is implemented in
dnf-json, which returns the checksums for each repository on every call.

This enables the use of repositories that change over time, such as
fedora-updates. Note that the osbuild pipeline will break when such a
repository changes. This is intentional: pipelines have to be
reproducible.
2019-12-10 20:38:22 +01:00
Lars Karlitski
75218ad2d9 distro/fedora30: don't include variables in repo URL
Replace those variables with their contents everywhere the repository
URLs are used is error-prone.
2019-12-10 20:38:22 +01:00
Ondřej Budai
6bbc89d5f3 api: implement /compose/delete route 2019-12-10 01:14:36 +01:00
Ondřej Budai
6d15833e4e store: introduce state directory
Prior to this commit outputs directory used by local target was owned by root.
This made impossible for osbuild-composer to delete images. (osbuild-composer
doesn't run as root).

This commit introduces state directory in which osbuild-composer creates
outputs directory. Because this directory is owned by osbuild-composer, it's
able to delete files inside.
2019-12-10 01:14:36 +01:00
Ondřej Budai
4e62f181fa api: implement /compose/logs route
The implementation is just a stub returning always the same tar archive.
The ability to return actual logs will be implemented in the future - osbuild
isn't currently returning any logs.
2019-12-08 00:21:30 +01:00
Ondřej Budai
d7f81b36e6 api: implement /compose/info route
Not everything is implemented in this route but enough to pass the tests.
2019-12-06 10:42:19 +01:00
Ondřej Budai
ad7650384e api: use committed blueprint in POST /compose
lorax-composer apparently doesn't use blueprint from workspace to create a new
compose. This commit implements the same behaviour.
2019-12-05 13:08:50 +01:00
Ondřej Budai
24faab8c18 store: clean up blueprint API
Prior to this commit blueprint getters looked like C-style API with output
parameters. This commit refactors them to more conventional multiple return
values API.
2019-12-05 13:08:50 +01:00
Ondřej Budai
f89a9671be store: add image struct into compose struct
As a part of f4991cb1 ComposeEntry struct was removed from store package.
This change made sense because this struct is connected more with API than
with store - store uses its own Compose struct. In addition, converters
between Compose and ComposeEntry were added. Unfortunately, ComposeEntry
contains ImageSize which was not stored in Compose but retrieved from store
using GetImage method. This made those converters dependent on the store,
which was messy.

To solve this issue this commit adds image struct into Compose struct.
The content of image struct is generated on the worker side - when the worker
sets the compose status to FINISHED, it also sends Image struct with detailed
information about the result.
2019-12-05 09:48:21 +01:00
Lars Karlitski
b2880cacc6 weldr: don't indent toml output
Be consistent with lorax-composer.

The integration tests do a string compare on received toml files.
2019-12-04 19:15:33 +01:00
Lars Karlitski
b1c65bb671 blueprint: tag all struct fields for toml
This is repetitive, but there doesn't seem to be another way to export
lower-case field names.
2019-12-04 19:15:33 +01:00
Lars Karlitski
94bfcc518c weldr: bump blueprint version
According to the same rules as lorax-compposer:

  https://weldr.io/lorax/lorax-composer.html#blueprints
2019-12-04 18:35:18 +01:00
Ondřej Budai
b6307cafd5 api: fix compose routes returning null instead of empty array
Cockpit-composer really doesn't like that
2019-12-03 11:49:35 +01:00
Ondřej Budai
dca7ab2847 api: handle blueprint depsolve error 2019-12-03 11:49:27 +01:00
Lars Karlitski
20bb60f3fd weldr: add support for toml blueprints 2019-12-03 01:27:00 +01:00