This patch adds support for Opentelemetry. If
OTEL_EXPORTER_OTLP_ENDPOINT env variable is defined, it will send traces
there. Otherwise there is no change.
The whole compose is wrapped in a single span. Nested under that are
spans for operations that involve a remote server.
* Talking to CTS
* Sending API requests to Koji
* Any git repo clone
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We can have a compose with unsigned packages.
By the time the next compose is generated, the packages could have been
signed. However, the new compose would still reuse the ISO with unsigned
copies.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Running xorriso to modify an ISO image can update content of included
images such as images/eltorito.img, unless we explicitly update the
image, which is undesirable (https://pagure.io/pungi/issue/1647).
However, when the file is changed, the checksum changes and .treeinfo no
longer matches.
This patch implements a workaround: once the DVD is written, it looks
for incorrect checksums, recalculates them and updates the .treeinfo on
the DVD. Since only the checksum is changing and the size of the file
remains the same, this seems to help fix the issue.
An additional step for implanting MD5 is needed again, as that gets
erased by the workaround.
JIRA: RHELCMP-13664
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Pungi can use either genisoimage or xorriso to create ISOs.
It also needed isoinfo utility for querying volume ID from the ISO
image. However, the utility is part of the genisoimage suite of tools.
On systems that no longer provide genisoimage, the image would be
successfully generate with xorriso, but then pungi would fail to extract
the volume id leading to metadata with missing values.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Use a different approach for building DVDs when xorriso is enabled.
The default of using genisoimage is not changed at all. When the config
option is set to use xorriso, the actual execution is different between
bootable and non-bootable images.
The non-bootable images are still created by running xorrisofs (which is
a compatibility tool with same UI as genisoimage). Since the image is
not bootable, there should be no problems with boot options.
For bootable images, Pungi will instead take the boot.iso generated by
Lorax, and use xorriso to inject all the extra files into the image.
The shell script that used to invoke all the commands to build the ISO
now runs the `xorriso` command in interactive mode and feeds another
file into it. The new file contains the xorriso commands to add the
required files to the image.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Checking start of the line is not sufficient for extra_isos that have
the variants in separate directories.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It was checked in a condition together with the configuration value, and
only ever explicitly used with the same value.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch allows Pungi to reuse ISO image created in previous compose
if a list of assumptions proves to hold:
* If image is bootable, buildinstall phase must have been reused too.
* Compose configuration must have not changed (except for a few
whitelisted options).
* Volume ID of the ISO much not have changed.
* No RPM on the ISO must have changed.
The ISO also contains other files. Changes in extra files and product ID
certificates should be visible in configuration (the SHA will differ).
Similarly any repodata configuration would be reflected in
configuration.
JIRA: RHELCMP-5969
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
A new configuration *createiso_use_xorrisofs* is added to determine
which tool to use for creating ISOs.
By default, createiso_use_xorrisofs = False and genisoimage will be used.
When set to True, xorrisofs will be used.
JIRA: RHELCMP-2875
Fixes: https://pagure.io/pungi/issue/1130
Signed-off-by: Haibo Lin <hlin@redhat.com>
When `link_type = "symlink"` is used, the packages are in fact symlinks
to /mnt/koji. When graft points file is generated, the paths in this graft
points file point to symlinks and therefore symlinks are copied into the
generated ISO file instead of real files.
In this commit, the code to generate the graft points file is changed
so it resolves the symlink to real file stored on /mnt/koji. To make
this code safer, it does such resolving only in case the symlink points
outside of `compose.paths.compose.topdir()`. Therefore you can still
generate ISO file with symlink pointing to file stored within the ISO
file itself, although this is not done currently afaik.
The main reason for this is to be able to generate ISO files even
without hardlinks (which would need read-write access on /mnt/koji)
and without copying all the packages from /mnt/koji to local storage.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
The `runroot_method` now accepts `dict` value with phase name as a key
and runroot method as a value. For backward compatibility, the `str`
value is still supported.
The new `global_runroot_method` option has been added which defines
the runroot method in case it is not set in `dict` in the `runroot_method`.
This commit allows running `createiso` phase locally while keeping the other
phases in Koji.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Hardlink command can be installed in /usr/sbin, where it is not visible
to non-priviledged users. They can still run it, but don't have it in
their PATH.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Even if we want to break hardlinks from Koji volume, there may be files
that can be hardlinked and save some space on the media.
JIRA: COMPOSE-3482
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This adds new `Runroot` class and new `runroot_method` option which makes
it possible to choose between two currently available runroot methods:
- Local
- Koji
The main goal of this commit is to make it possible to add new runroot
methods in the future and this is the first step in that direction.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
The default is the original behaviour. On F30+ a new option should be
added to config to make it work.
Over time as users move to this option (which requires a new enough
version of lorax), the default should be switched and then the option
removed.
Resolves: https://pagure.io/pungi/issue/1126
Merges: https://pagure.io/pungi/pull-request/1128
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch allows the configuration to express maximum expected size for
ISOs created in createiso and extra_isos phases. If the image is larger
than this limit, a warning is emitted in test phase. The compose itself
is not affected in any way.
JIRA: COMPOSE-2824
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The function loads existing treeinfo, removes reference to boot.iso and
adds [media] section. This is the basic tweak that should happen for all
ISOs. Additional changes depend on the actual content.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The main variant is already available, this patch adds information about
additional variants that are included in the image.
JIRA: COMPOSE-2917
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the configuration specifically excludes a variant from buildinstall,
but does not also disable ISO creation, we should just create a
non-bootable ISO instead of reporting a warning.
JIRA: COMPOSE-2887
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
On some systems we need to use python3, in other places it can be
another version. Instead of guessing, let's look at shebang line in
lorax executable and use the same.
JIRA: COMPOSE-2852
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the data needs to be split into multiple ISOs, we need to make sure
the paths are unique for each image. Otherwise all files will be copied
into the same directory, and once the first image is finished, the whole
staging dir is deleting. That obviously breaks the tasks that are still
in progress.
JIRA: COMPOSE-2610
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If a file has multiple hard links, genisoimage will put the wrong number
on the ISO. This patch can work around it by copying hard-linked files
into a temporary staging directory.
JIRA: COMPOSE-2610
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This way some parts of the code will be reusable. This should have no
effects on the outcome, the tests still pass without any changes needed.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the ISO is meant to be bootable but lorax fails, there's no point in
creating the ISO as it will not behave as expected.
Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1574585
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
To be more precise, new createrepo parameter "workers" was customized and
new default value was set. This fixes issue #752.
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
Instead use the definition from python-six. Once we drop Py 2 support
completely, we'll just swap underscores with dots.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When splitting is not possible, we should not print a huge number as
free space on single media. This patch also adds more information so
that the numbers can actually be connected to variant and architecture.
The list of ignored files will only be printed now if not empty.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
