Commit graph

775 commits

Author SHA1 Message Date
Tom Gundersen
51f1691ab6 test/cases/rhel82: add image info to the test cases
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-17 14:19:51 +01:00
Tom Gundersen
93b13e3fa7 blueprints: fix group handling
When group names are passed on to dnf, they must be prefixed with an
ampersand, or they are treated as a regular package, potentially
causing the build to fail.

Add a testcase to verify this behavior.

This resolves rhbz#1784035.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-17 09:24:15 +01:00
Ondřej Budai
66885bc9d6 spec: fix build
Commit 04012641 broke the rpm build by introducing bad package name
to be build. This commit fixes it.
2019-12-17 08:54:05 +01:00
Lars Karlitski
74812cad85 5 2019-12-16 01:49:09 +01:00
Lars Karlitski
73641ea904 distro: rename openstack and vmdk images to match lorax 2019-12-16 01:32:41 +01:00
Lars Karlitski
5f8f417f65 worker: wrap errors, adding descriptions
It's hard to figure out where an error came from otherwise.
2019-12-15 22:05:31 +01:00
Lars Karlitski
df99e8b359 worker: log errors that are not returned to composer 2019-12-15 22:05:31 +01:00
Lars Karlitski
b1c5ef2ad4 worker: store osbuild logs in result.json for local builds 2019-12-15 22:05:31 +01:00
Lars Karlitski
5bce59b979 worker: do not get the distro from the host
Add a `Distro` field to both `Job` structures and send that to the
worker.
2019-12-15 22:05:31 +01:00
Lars Karlitski
dff8cb56be worker: don't call log.Fatalf() unconditionally
Oops.
2019-12-15 22:05:31 +01:00
Lars Karlitski
8c4ee795f7 distro: add Distro.Name()
Right now, there is no way to get at the name from a Distro instance.
We will need this to include the distro's name in the job we pass to the
worker, for instance.
2019-12-15 22:05:31 +01:00
Tom Gundersen
ba32f58ac8 distro/rhel82/qcow2: bring package set in line with official guest image
Also hook up the tast case to verify that the image boots.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-15 21:41:33 +01:00
Lars Karlitski
d73e9817b1 submodule: bump to current osbuild master 2019-12-15 20:57:40 +01:00
Tom Gundersen
5e41526d16 distro/rhel82: add aarch64 support
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
b18c66f9fb distro/rhel82: add UEFI support
This follows the style used in fedor-30.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
55cc82d545 distro/fedora-30: add aarch64 support
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
db300cb481 distro/fedora-30: add UEFI support
On architectures that require EFI, we must create the ESP partition
and use a GPT partition table. We must also install either the UEFI
or the legacy version of GRUB2 in the image.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
8c1f194fc1 distro/fedora-30: set the module_platform_id
Make sure we don't fallback to looking at /etc/os-release.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
18971bd8ef pipeline/stage/grub2: support explicit legacy/uefi API
This allows legacy bootloader and UEFI bootloaders to be
enabled/disabled independently of each other.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
7e4c4e4f6d pipeline/stage/grub2: make the boot uuid optional
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
4c892d6d32 pipeline/assembler/qemu: rework partition table handling
Move to the new options format, allowing more flexible partition
tables. The pipeline changes, but the result should be the same.

This requires a yet-to-be-released version of osbuild.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
dbdac222a5 pipeline/stage/fstab: represent UUID as a string
These may not be strictly speaking well-formed UUID. Some filesystems
use other kinds of identifiers.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
506b012f08 distro: take architecture as an argument
Allow bootloader specific packages to be defined per architecture,
and allow repositories to depend on the architecture.

This does not altert he pipelines we produce, part from the ami
image now contains the grub2-pc package, rather than the grub2
package. This should make no difference.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
b034117e62 distro: sort the packages passed to the dnf stage
This should not have any effect apart from hiding any internal changes
to the way we compile the lists.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
caf72fd97f test/cases/rhel82/qcow2: include a user and the ssh daemon
We don't currently boot-test this, but at least it should work
in principle with these changes.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
9f05e0b7c4 test/run: print more helpful output
Make sure we catch expected errors and fail gracefully. Also, make
sure the output id is printed on successufl osbuild run so it can
be introspected from outside the test suite.

Handle the case where osbuild suceeds but does not return an
output_id.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
028adf375a test/cases/rhel82: add basic test-cases
These are used to verify that our pipeline generation is stable, and
that the piplines can generate images that boot.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
6e2731deb8 distro/rhel82: fix the repositories
Fix these while we develop so we are guaranteed all tests run against
the same packages.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
e3be97990b distro/rhel82: include dracut-config-generic
We must make sure not to depend on the modules loaded in the host
kernel. This makes the initrd the same between different hosts.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
85ff8bcec2 distro/rhel82: exclude the timedatex pacakge for now
As for other output types, there is an SELinux bug that makes this
uninstallable.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
5b9d85dece distro/rhel82/vhd: drop stale comments
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
8255d6ff9c test/run: fix ssh test
This was broken in a rework. The correct status was printed, but
the test failures were not passed on.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
9b7ad82773 test/run/ssh: keep trying if the '--wait' switch does not work
When we get an ssh connection, before the image is fully booted,
systemctl is-system-started returnse "starting", treat this as a
failed connection, and keep retrying.

Some distros may not support the wait switch correctly.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
b9e20c4c06 service: adjust the IO and CPU scheduling classes
Image generation is very much a batch job, latency is not important
and because it is so CPU and IO intensive, it really affects the
overall responsiveness of the system if left on the default settings.

Set scheduling to idle/batched for the worker. The composer service
is still left in the normal scheduling classes as its latency is
important for the UI.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-13 20:13:15 +01:00
Lars Karlitski
b28705d4b4 jobqueue: copy outputs from the right location
d55abdfd75 disabled caching the osbuild
store by moving it to /var/tmp. Fix the directory we're copying the
outputs from as well.
2019-12-13 12:38:36 +01:00
Lars Karlitski
b60bb1d011 distro: pass compression argument to the tar assembler 2019-12-13 12:21:47 +01:00
Lars Karlitski
57aea4160d weldr: return version as string from api/status
This is what lorax-composer does and composer-cli expects.
2019-12-13 00:14:40 +01:00
Lars Karlitski
42f46a5992 distro/rhel82: make openstack's package list constistent with lorax 2019-12-13 00:14:07 +01:00
Tom Gundersen
cce58cae8f submodule: update to current master
This contains new API for getting multi-arch to work. We expect more additions
before the next release, but this way we can get things working in parallel.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-12 18:16:01 +01:00
Martin Sehnoutka
b7b60363dc Modify RHEL8.2 VHD image to boot in Azure 2019-12-12 13:52:21 +01:00
Tom Gundersen
d074395474 distro/rhel-8.2: update VHD images to be closer to lorax
The current one was taken from fedora and that failed both due to an
SELinux bug in RHEL8.2, but also because the list of packages was
wrong for the distro.

We don't enable WAAgent, which we probably should. And there are a few
other config tweaks to cloud-init and WAAgent that we skip too.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-12 13:52:21 +01:00
Lars Karlitski
305b2dda5c rpmmd: add IgnoreSSL key to RepoConfig
This is the opposite of dnf.conf's `sslverify`, because go's default for
booleans is always false. This is error prone: we'd like to default to
true.
2019-12-12 01:27:04 +00:00
Lars Karlitski
2b42612336 distro: insert custom sources into pipeline
This is an additional argument on `distro.Pipeline` right now. In the
future, we probably want this per-blueprint.
2019-12-11 21:19:32 +00:00
Lars Karlitski
d9ad5d7062 weldr: add toml output to projects/source/info 2019-12-11 18:53:10 +00:00
Lars Karlitski
72e1da47ab weldr: make projects/source/new accept text/x-toml
composer-cli sends sources as toml if the original format is toml.
2019-12-11 18:53:10 +00:00
Tom Gundersen
7036c157fb travis: disable PPC tests
These seemed not to get triggered (no available workers?), so disable
for now. We do want PPC support, but it is currently our last
priority.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-11 15:23:24 +01:00
Tom Gundersen
04012641ba spec: don't ship devel tools
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-11 15:23:24 +01:00
Tom Gundersen
d55abdfd75 osbuild-worker: disable the osbuild cache
We still need to figure out the garbage collection, so for now don't
enable the cache.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-11 15:23:24 +01:00
Tom Gundersen
22d57ff1a6 service: don't pull down the socket when the service goes down
The intention is for the sockets always to stay around, even when
the service is restarted. This way the worker can always connect.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-11 15:23:24 +01:00
Tom Gundersen
df73a9bfad service: make sure the worker pulls in the socket
The worker cannot be started unless the socket is available, so make
this a hard requirement.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-11 15:23:24 +01:00