Commit graph

20 commits

Author SHA1 Message Date
Lars Karlitski
1c5adfc682 job: use image.raw.xz filename
RHEL and Fedora used different output formats (.ami vs .raw.xz). The job
package assumed `image.ami`, which failed for RHEL.

Change Fedora to use image.raw.xz as well. This makes it consistent, but
we should also get the filename from the distro at some point.
2019-12-01 00:05:17 +01:00
Lars Karlitski
5dad3bfc8e worker: pass build environment to osbuild
Detect it from the host using the distro package.
2019-11-29 00:46:05 +01:00
Lars Karlitski
85e6182bdc distro: don't fall back to fedora-30
Make osbuild-composer use FromHost() directly. Everywhere else needs to
specify the distro explicitly.

Also don't panic when a distro doesn't exist. Instead, return nil. Make
sure all callers check for that.
2019-11-29 00:46:05 +01:00
Lars Karlitski
14ebed46da distro: register distros explicitly
Automatically registering on `init()` is clever, but a bit too magical
and easy to get wrong, because every binary must include all distros
somewhere.

Flip this inside out: distros now have a `New()`, which returns
something that implements the `Distro` interface. The distro package
explicitly creates all of them.

This means that distros cannot import distro itself anymore, because go
forbids import cycles. This only affected `InvalidOutputFormatError`.
Return a generic error for now.
2019-11-29 00:46:05 +01:00
Lars Karlitski
c0516420cc distro/rhel82: move ami output closer to official ec2 image
This includes adding (and excluding) the right packages, setting the right
systemd boot target, fixing kernel options, changing output type to
.raw.xz, and setting the size of the image to 6GB.
2019-11-29 00:46:05 +01:00
Tom Gundersen
553edef01d distro/f30: rework in the same way as RHEL8.2
The produced pipelines are unchanged.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-11-28 01:24:05 +01:00
Lars Karlitski
da311f13eb distro: add rhel82
This takes a different approach to outputs and customizations, which is
much shorter and duplicates less code.

This uses links to internal repositories for now, because 8.2 hasn't
been released yet.

Add a `distro` flag to `osbuild-pipeline`.
2019-11-27 17:08:51 +01:00
Lars Karlitski
4cf94e2629 rpmmd: add checksum to repoconfig
We should always specify a checksum when describing a repository to pull
content from. For now, fedora-30 duplicated the checksum. The idea is
that we can have it in one place at some point.
2019-11-27 17:08:51 +01:00
Lars Karlitski
7b54f5cfdc pipeline: support osbuild runners
osbuild has a concept of runners now: scripts that set up a build
environment. Update the osbuild submodule to latest master, change
`Pipeline` to to the new buildroot description format, and use the
`org.osbuild.fedora30` runner from the fedora30 distro.
2019-11-27 02:47:36 +01:00
Tom Gundersen
382d4e2118 distro/f30/ami: make the images boot on aws
This makes the image in line whith the ones produced by lorax.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-11-26 22:00:04 +01:00
Tom Gundersen
de93ddc757 distro/f30: rework customizations
This slightly changes the customizations logic. We now make sure
that each stage is appended exactly once.

customizations.go are now responsible only for the things that are
completely generic, and not per-ouput-type. helpers.go contain more
high-level helpers that combine customziations and per-output-type
defaults.

This does not change the behaviour, though some pipelines are slightly
reordered to make them consistent.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-11-26 22:00:04 +01:00
Tom Gundersen
c6e73e65a5 distro/f30: drop liveiso support
This never worked, either here or in lorax. Drop it so it is not
shown in the UI.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-11-26 22:00:04 +01:00
Tom Gundersen
7f5c869cd2 upload/aws: add a sample AWS upload client
This commandline tools uploads a file to S3, as a proof of concept.

All options are mandatory. Credentials are only read from the
commandline and not from the environment or configuration files.

The next step is to add support for importing from S3 to EC2,
currently the images we produce cannot be imported as-is, so this
requires more research.

To try this out: create an S3 bucket, get your credentials and
call the tool, passing any value as `key`. Note that if the key
already exists, it will be overwritten.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-11-26 01:05:34 +01:00
Martin Sehnoutka
4910cd18e3 change image size to fulfill Azure requirements
the hardcoded image size is now aligned to 1MB, because Azure requires
it: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/create-upload-generic

once we make the size configurable, we should make sure the alignment
stays the same

fix test cases to reflect new image size
2019-11-25 17:06:32 +01:00
Martin Sehnoutka
ea73597c5d use vpc type in call to qemu-img for vhd images
vhd images currently use qcow2 format, because osbuild does not support
vpc yet, but it should land soon in related PR
2019-11-25 17:06:32 +01:00
Tom Gundersen
283e89d0f4 distros/f30/ami: image type should be RAW
This was accidentally left as QCOW2.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-11-24 17:43:00 +01:00
Lars Karlitski
b911d0b928 weldr: get repositories from distro
Make distros export repository information and use those in the weldr
API. This means that repos are only specified once and that the API
returns the right packages when we allow different distros.
2019-11-18 12:22:10 +01:00
Martin Sehnoutka
77c2ab0e1c Introduce local boot test case for QCOW2
The tests works by executing osbuild with predefined pipeline. Then the
image boots and the testing script creates SSH connection to the running
VM. If everything goes fine `systemctl is-system-running` is executed
with result `running` and the test case passed.

The JSON definition of the test case contains also a blueprint that
should generate the desired pipeline, but it didn't work for me, so I'm
including it for future use from the golang unit tests.
2019-11-11 15:47:01 +01:00
Lars Karlitski
f9cbc8593f distro: add DetectHost
lorax-composer uses the host's repositories for building images. This is
prone to accidental configuration errors and duplicates functionality
(adding custom repositories via the source API is much more explicit).

However, blueprints don't specify the distribution they're based on.
This is something they should do in the future to enable cross-distro
image builds. Until then, read `/etc/os-release` to detect the host OS
and use that as the base distro for a blueprint.

Detection works by concatenating the ID field with a `-` and the
VERSION_ID field. This mandates how additional distributions should
register themselves to the `distro` package.

If composer cannot build the detected distro, fall back to fedora-30.

Use this detection everywhere that fedora-30 was hard-coded before.
2019-11-10 17:23:14 +01:00
Lars Karlitski
b33ed9e5d2 blueprint: move pipeline generation into its own package
Introduce the `distro` package, which contains an interface for OS
implementations. Its main purpose is to convert a blueprint to a
distro-specific pipeline.

Also introduce the `distro/fedora30` package. It is the first
implementation of the distro interface. Most of its code has been copied
with minimal modifications from the blueprint package.

The `blueprint` package is now back to serving a single purpose:
representing a weldr blueprint. It does not depend on the `pipeline`
package anymore.

Change osbuild-composer and osbuild-pipeline to use the new API,
hard-coding "fedora-30". This looks a bit weird now, but is the same
behavior as before.

All test cases now also take an "distro" key in the "compose" object.
2019-11-07 17:13:20 +01:00