We've been effectively using the basearch of the host, making the stage
non-reproducible: if the same pipeline was run on machines with
different architectures, it would produce different results. However,
pipelines producing different outputs must be different. Thus, this
patch includes the basearch in the pipeline.
In principle, this allows cross-arch builds. dnf should be the only
stage running binaries from the target tree. This is not yet tested.
The locale stage now cannot be used to set the keymap. Use the keymap
stage instead. Also, the stage was refactored to look like keymap and
timezone stages just to be consistent (systemd-firstboot is now used).
We use chroot connection type to "connect" to the target filesystem
where ansible should run the playbook. However, the target is not booted
system, it's just an image of not-yet-booted one. Unfortunately, many
ansible modules cannot be used inside not-booted system. Also, the core
principle of osbuild is to never boot the currently built image.
Therefore we decided to remove the ansible stage.
If ansible is needed in the future, there is a possibility to add a new
ansible stage, which would run the playbook during the first boot.
Adds a new systemd unit to the image that will be pulled in by default,
run a given command, forward the output to a virtio serial port and
shutdown the machine.
We add a sample that uses this to verify that systemd conciders the
machine successfully booted. A simple way to run this test from the
commandline is to use
`$ socat UNIX-LISTEN:qemu.sock -`
to listen for either `running` for success or `degraded` or
`maintenance` for failure.
The image should then be booted using something like
`$ qemu-kvm -m 1024 -nographic -monitor none -serial none -chardev socket,path=qemu.sock,id=char0 -device virtio-serial -device virtserialport,chardev=char0,id=test0 -snapshot base.qcow2`
Signed-off-by: Tom Gundersen <teg@jklm.no>
This gives shell access into the image on a given tty. Useful for
testing and debugging, while minimally affecting the image.
Note that this must never be used in production, as it allows root
access without a password.
For instance this could be used to verify that an image was fully
booted:
```
[teg@teg-x270 osbuild]$ qemu-kvm -m 1024 -nographic -serial mon:stdio -snapshot base.qcow2
sh-5.0# systemctl is-system-running --wait
running
```
Signed-off-by: Tom Gundersen <teg@jklm.no>
Move the decision whether the root fs should be mounted ro or rw
into the pipeline configuration.
Update the pipelines accordingly.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Default to True, which is what dnf defaults to, but allow it to be
overridden in the pipeline. Whether this option should be used should
be a distro policy, but for now we just want it to get images compatible
with the official fedora ones.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We don't want non-functional configuration in the pipelne, we want to
restrict ourselves to options that changes the final image.
Signed-off-by: Tom Gundersen <teg@jklm.no>
grub2-mkrelpath uses /proc/self/mountinfo to find the source of the file
system it is installed to. This breaks in a container.
Add org.osbuild.fix-bls which goes through /boot/loader/entries and
fixes paths by removing anything before /boot.
This stage allows to add or modify users. For now, this includes all
fields available in passwd, setting auxiliary groups, and setting an ssh
key.
Based on a patch by Martin Sehnoutka <msehnout@redhat.com>.
Travis uses Ubuntu, which does not ship dnf, so introduce a yum
stage that allows us to test actual generation of trees on Travis.
We use this to generate a tree containing the tools necessary to
create abritrary Fedora-based build images in the future. We base
this on Fedora 27, as that is the last version that is installable
using yum rather than dnf.
In the future, once we support pipelines with nested build-images,
rather than just using the host OS as the build image, this will
allow us to bootstrap arbitrary pipelines on Travis.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This way the assemblers/stages are valid in isolation, even without
osbulid installed system-wide. This would be needed to have this work
when --libdir is not the system-wide one, as the library would
otherwise not be in sys.path.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Since we no longer use grub2-mkconfig, but write static configuration
we can drop most of the helpers.
The partitin table id was never used in the first place. We use
filesystem UUIDs, not partition UUIDs to name our root/boot partitions.
Signed-off-by: Tom Gundersen <teg@jklm.no>
configparser writes strings with quotes and lists with enclosing
brackets, both of which may not appear in dnf.conf.
dnf.conf(5) defines the format loosely.
This removes the possibility of passing in arbitrary input data. We
now restrict ourselves to explicitly specified files/directories or
a base tree given by its pipeline id.
This drops the tar/tree stages/assemblers, as the tree/untree ones
are implicit in osbuild, and if we wish to also support compressed
trees, then we should add that to osbuild core as an option.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Afetr discussions with Javier Martinez Canillas and Alberto Ruiz
from the grub team, it was clear that we want move to a static
configuration shipped in the grub rpm in the future. For now,
we use our own, but the aim is to use the same as upstream.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We only need the filesystem with the correct fs-UUID to chroot into,
there is no need to set up a whole partition table.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This is for the sake of debuggability, but I figure dnf is the most
complex of our tools, so instrumenting that a bit makes sense.
The defaults are "install" and "info", as before.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We cannot use the tool from within the tree unless we chroot
(library versions ay be wring etc). If possible we want to
always use tools from the build image, in order to avoid forcing
the contenst of the target tree, so do that instead.
Finally, add a missing include.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We want to copy the contents of a directory to another, the correct
syntax for that is `cp -a src/. dst`. I was not aware of this beauty,
so the previous patch simulated the functionality in python code.
Signed-off-by: Tom Gundersen <teg@jklm.no>
In the simplest case we don't need to make an image, a filesystem tree
will do. Also support using such a tree as input for further pipelines
through the untree stage.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The stage populates the tree with grub2 configuration. The API and
semantics is the way we want it, but internally this is a massive
hack.
GRUB2 is only able to run grub2-mkconfig on the image it wants to
configure. The reason is that it will inspect / and /boot to detect
the existing UUIDs and filesystems to use, despite this being
information we already know. In principle, the tool does support
passing this is, but due to several bugs that functionality does
not work.
We therefore create the image we want, copy over the tree, run
grub2-mkconfig in this image, then copy it back over the tree. The
end result is that the files /etc/defaults/grub,
/boot/grub2/grub.cfg and /boot/grub2/grubev are added to the tree.
The alternative would be to do what tools typically do, and just
run grub2-mkconfig on the final image at the time it is being
assembled. We want to avoid this in order to fully split filesystem
tree generation from image assembly. This way we can better control
and verify what ends up on the filesystem which should help with
reprobucibility and reuse of filesystem trees. Above all though,
we want to make sure that we can actually place some guarantees
on what each stage of the image building process actually does,
allowing us to argue about and change it without worrying about
arbitrary fallout.
Signed-off-by: Tom Gundersen <teg@jklm.no>
All stages must be able to handle an input_dir argument, as we now
either pass it to all or none for agiven run. Simply set it to
'None' if it is not provided.
Signed-off-by: Tom Gundersen <teg@jklm.no>
