Some dnf packages introduce random seed file. If we leave in the tree
it would mean all systems running from the created image would use
the same random seed. This can be potentially dangerous, therefore we
just remove the generated random seed from our images.
Normally, machine ID is generated randomly when dnf installs @Core
group. Unfortunately this isn't helping us with reproducibility
of images.
This commit introduces the concept of fake machine ID. Before dnf
command is run in dnf stage, we set the machine ID to a fake one.
This ensures all the scriptlets requiring machine ID have predictable
outputs.
For example GRUB uses machine-id in file names inside
/boot/loader/entries. With fixed machine ID the file names are always
the same and totally predictable.
Require "checksum" option for each repository, which contains the
checksum of the `repodata/repomd.xml` file. This file (indirectly)
contains checksums for all packages.
Verify that the metadata dnf downloaded to install packages matches that
checksum. This way, this stage will give an error when a reposiory
changed between putting together the pipeline and running it.
This is similar to the previous commit for the dnf stage.
Don't pass through arbitrary options. This means that pipeline repo
objects don't have the same options as yum repo files anymore:
1. Hard code repo name to repo id. The name has no influence on the
resulting image and should thus not appear in a pipeline.
2. Set gpgcheck=1 when gpgkey is given. It defaults to false, which
means that all sample and test pipelines didn't verify packages. It
would have failed anyway, because the container doesn't have the key
referenced in /etc. Change all gpgkeys to refer to the key id and import
them manually.
3. Don't allow lists for baseurl and gpgkey. We can add that if we need
it at some point.
Also be less verbose.
Don't pass through arbitrary options. This means that pipeline repo
objects don't have the same options as dnf repo files anymore:
1. Hard code repo name to repo id. The name has no influence on the
resulting image and should thus not appear in a pipeline.
2. Set gpgcheck=1 when gpgkey is given. It defaults to false, which
means that all sample and test pipelines didn't verify packages. It
would have failed anyway, because the container doesn't have the key
referenced in /etc. Change all gpgkeys to refer to the key id and import
them manually.
3. Don't allow lists for baseurl and gpgkey. We can add that if we need
it at some point.
We've been effectively using the basearch of the host, making the stage
non-reproducible: if the same pipeline was run on machines with
different architectures, it would produce different results. However,
pipelines producing different outputs must be different. Thus, this
patch includes the basearch in the pipeline.
In principle, this allows cross-arch builds. dnf should be the only
stage running binaries from the target tree. This is not yet tested.
The locale stage now cannot be used to set the keymap. Use the keymap
stage instead. Also, the stage was refactored to look like keymap and
timezone stages just to be consistent (systemd-firstboot is now used).
We use chroot connection type to "connect" to the target filesystem
where ansible should run the playbook. However, the target is not booted
system, it's just an image of not-yet-booted one. Unfortunately, many
ansible modules cannot be used inside not-booted system. Also, the core
principle of osbuild is to never boot the currently built image.
Therefore we decided to remove the ansible stage.
If ansible is needed in the future, there is a possibility to add a new
ansible stage, which would run the playbook during the first boot.
Adds a new systemd unit to the image that will be pulled in by default,
run a given command, forward the output to a virtio serial port and
shutdown the machine.
We add a sample that uses this to verify that systemd conciders the
machine successfully booted. A simple way to run this test from the
commandline is to use
`$ socat UNIX-LISTEN:qemu.sock -`
to listen for either `running` for success or `degraded` or
`maintenance` for failure.
The image should then be booted using something like
`$ qemu-kvm -m 1024 -nographic -monitor none -serial none -chardev socket,path=qemu.sock,id=char0 -device virtio-serial -device virtserialport,chardev=char0,id=test0 -snapshot base.qcow2`
Signed-off-by: Tom Gundersen <teg@jklm.no>
This gives shell access into the image on a given tty. Useful for
testing and debugging, while minimally affecting the image.
Note that this must never be used in production, as it allows root
access without a password.
For instance this could be used to verify that an image was fully
booted:
```
[teg@teg-x270 osbuild]$ qemu-kvm -m 1024 -nographic -serial mon:stdio -snapshot base.qcow2
sh-5.0# systemctl is-system-running --wait
running
```
Signed-off-by: Tom Gundersen <teg@jklm.no>
Move the decision whether the root fs should be mounted ro or rw
into the pipeline configuration.
Update the pipelines accordingly.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Default to True, which is what dnf defaults to, but allow it to be
overridden in the pipeline. Whether this option should be used should
be a distro policy, but for now we just want it to get images compatible
with the official fedora ones.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We don't want non-functional configuration in the pipelne, we want to
restrict ourselves to options that changes the final image.
Signed-off-by: Tom Gundersen <teg@jklm.no>
grub2-mkrelpath uses /proc/self/mountinfo to find the source of the file
system it is installed to. This breaks in a container.
Add org.osbuild.fix-bls which goes through /boot/loader/entries and
fixes paths by removing anything before /boot.
This stage allows to add or modify users. For now, this includes all
fields available in passwd, setting auxiliary groups, and setting an ssh
key.
Based on a patch by Martin Sehnoutka <msehnout@redhat.com>.
Travis uses Ubuntu, which does not ship dnf, so introduce a yum
stage that allows us to test actual generation of trees on Travis.
We use this to generate a tree containing the tools necessary to
create abritrary Fedora-based build images in the future. We base
this on Fedora 27, as that is the last version that is installable
using yum rather than dnf.
In the future, once we support pipelines with nested build-images,
rather than just using the host OS as the build image, this will
allow us to bootstrap arbitrary pipelines on Travis.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This way the assemblers/stages are valid in isolation, even without
osbulid installed system-wide. This would be needed to have this work
when --libdir is not the system-wide one, as the library would
otherwise not be in sys.path.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Since we no longer use grub2-mkconfig, but write static configuration
we can drop most of the helpers.
The partitin table id was never used in the first place. We use
filesystem UUIDs, not partition UUIDs to name our root/boot partitions.
Signed-off-by: Tom Gundersen <teg@jklm.no>
configparser writes strings with quotes and lists with enclosing
brackets, both of which may not appear in dnf.conf.
dnf.conf(5) defines the format loosely.
This removes the possibility of passing in arbitrary input data. We
now restrict ourselves to explicitly specified files/directories or
a base tree given by its pipeline id.
This drops the tar/tree stages/assemblers, as the tree/untree ones
are implicit in osbuild, and if we wish to also support compressed
trees, then we should add that to osbuild core as an option.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Afetr discussions with Javier Martinez Canillas and Alberto Ruiz
from the grub team, it was clear that we want move to a static
configuration shipped in the grub rpm in the future. For now,
we use our own, but the aim is to use the same as upstream.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We only need the filesystem with the correct fs-UUID to chroot into,
there is no need to set up a whole partition table.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This is for the sake of debuggability, but I figure dnf is the most
complex of our tools, so instrumenting that a bit makes sense.
The defaults are "install" and "info", as before.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We cannot use the tool from within the tree unless we chroot
(library versions ay be wring etc). If possible we want to
always use tools from the build image, in order to avoid forcing
the contenst of the target tree, so do that instead.
Finally, add a missing include.
Signed-off-by: Tom Gundersen <teg@jklm.no>
