=======
Plugins
=======

Following plugins are available in default koji installation.

Runroot
=======

Plugin for running any command in buildroot. It has three parts as most of the
others (hub, builder and CLI).

Builder
-------
You enable plugin by editing ``/etc/kojid.conf`` by adding ``plugin = runroot``
there. Plugin itself has separate configuration file on each builder located at
``/etc/kojid/plugins/runroot.conf`` There is a sample configuration file
with option descriptions installed.

Hub
---
On the hub side ``Plugins = runroot_hub`` needs to be added to
``/etc/koji-hub/hub.conf``. Note, that by default policy runroot tasks are
assigned to ``runroot`` channel. As this is a plugin, we don't create it
automatically. There are three options - create channel when adding first builder
there via ``koji add-host-to-channel --new hostname runroot`` or by changing the
default channel policy according to :doc:`defining_hub_policies`. Last option is
to use ``--channel-override`` option in CLI to drive task to channel of choice.

CLI
---
CLI is looking for available plugins every run, so it if it is installed, you'll
see new command ``runroot`` with options described in its help. No config
options are needed to enable it.

Save Failed Tree
================

In some cases developers want to investigate exact environment in which their
build failed. Reconstructing this environment via mock needn't end with
exactly same structure (due to builder settings, etc.). In such case this
plugin can be used to retrieve tarball with complete mock tree.

Additional feature is that some paths from buildroot can be left out from
tarball. Feature can be configured via
`/etc/kojid/plugins/save_failed_tree.conf` file. Currently only field
filters.paths is used and it consists of globs (standard python's fnmatch is
used) separated by whitespaces.

.. code-block:: ini

  [filters]
  paths = /etc/*.keytab /tmp/secret_data

.. warning::
  For security reasons, currently all ``/tmp/krb5cc*`` and ``/etc/*.keytab``
  files are removed from tarball. If we found some other dangerous pieces,
  they can be added to this blacklist.

Special task method is created for achieving this which is called
``SaveFailedTree``. This task can be created via CLI:
``koji save-failed-tree <taskID>``. Additional options are:

.. option:: --full

   directs koji to create tarball with complete tree.

.. option:: --nowait

   exit immediately after creating task

.. option:: --quiet

   don't print any information to output

After task finishes, one can find the tarball on relevant task web page (URL
will be printed to stdout until ``--quiet`` is used.

Plugin allow to save trees only for tasks defined in config
``/etc/koji-hub/plugins/save_failed_tree.conf``. Option
``allowed_methods`` contains list of comma-delimited names of tasks. Default
configuration contains line: ``allowed_methods = buildArch``. Anybody
is allowed to create this type of task (and download tarball).

.. warning::
  Don't forget that this type of task can generate huge amount of data, so use
  it wisely.

TODO
----
 * Separate volume/directory on hub
 * garbage collector + policy for retaining generated tarballs

Sidetag
=======

Sidetag plugin is originally work of Mikolaj Izdebski and was pulled into base
koji due to easier integration with rest of the code.

It is used for managing `sidetags` which are light-weight short-lived build tags
for developer's use. Sidetag creation is governed by hub's policy.

Hub
---

Example for `/etc/koji-hub/hub.conf`:

.. code-block:: ini

    PluginPath = /usr/lib/koji-hub-plugins
    Plugins = sidetag_hub

    [policy]
    sidetag =
        # allow maximum of 10 sidetags per user for f30-build tag
        tag f30-build && compare number_of_tags <= 10 :: allow
        # forbid everything else
        all :: deny

    package_list =
        # allow blocking for owners in their sidetags
        match action block && is_sidetag_owner :: allow
        all :: deny

There are two special policy tests ``is_sidetag`` and ``is_sidetag_owner`` with
expectable behaviour. ``is_sidetag_owner`` can handle optional
``tag``/``fromtag``/``both`` keywords which specify data to be tested. Default
is testing ``tag`` in policy data, ``fromtag`` can test  this field (e.g. in
``untagBuild`` case) and ``both`` fails if any of the involved tags is not owned
by sidetag owner.

Now Sidetag Koji plugin should be installed.  To verify that, run
`koji list-api` command -- it should now display `createSideTag`
as one of available API calls.

Plugin has also its own configuration file
``/etc/koji-hub/plugins/sidetag.conf`` which contains following options:

.. glossary::
   remove_empty = off
       If this is set, sidetag is automatically deleted when
       last package is untagged from there.

   allowed_suffixes =
       List of strings delimited by commas. These suffixes are then allowed to
       be requested via ``createSideTag``

   name_template = {basetag}s-side-{tag_id}d
       Python string template to be used for generation of sidetag name. It needs
       to contain both basetag/tag_id placeholders.

Parent tag can define following extras:

.. glossary::
   sidetag_debuginfo_allowed = 0
       If set to 1/True, debuginfo repo could be created (``with_debuginfo`` can
       be set for tag by sidetag owner via ``--debuginfo`` option)

   sidetag_rpm_macros_allowed = None
       This option can be set to space-separated fnmatch masks which allows user
       to override specific rpm macros.
       (``koji edit-sidetag <tag> --rpm-macro <macro>``).

CLI
---

For convenient handling, also CLI part is provided. Typical session would look
like:

.. code-block:: shell

   $ koji add-sidetag f30-build --wait
   f30-build-side-123456
   Successfully waited 1:36 for a new f30-build-side-123456 repo

   $ koji remove-sidetag f30-build-side-123456

API
---
And in scripts, you can use following calls:

.. code-block:: python

    import koji
    ks = koji.ClientSession('https://koji.fedoraproject.org/kojihub')
    ks.gssapi_login()
    ks.createSideTag('f30-build')

.. _protonmsg-config:

Proton messaging
================

The ``protonmsg`` plugin for the hub will, if enabled, send a wide range of
messages about Koji activity to the configured amqps message brokers.
Most callback events on the hub are translated into messages.

In order to enable this plugin, you must:

* add ``protonmsg`` to the ``Plugins`` setting in ``/etc/koji-hub/hub.conf``

* provide a configuration file for the plugin at
  ``/etc/koji-hub/plugins/protonmsg.conf``

The configuration file is ini-style format with three sections: broker,
queue and message.
The ``[broker]`` section defines how the plugin connects to the message bus.
The following fields are understood:

* ``urls`` -- a space separated list of amqps urls. Additional urls are
  treated as fallbacks. The plugin will send to the first one that accepts
  the message
* ``cert`` -- the combined client cert and key file for authenticating koji to
  the broker.
* ``cacert`` -- the CA certificate to verify the broker server TLS connection
* ``topic_prefix`` -- Koji uses this string as a prefix for all message
  topics. For example, if you choose ``topic://koji``, then Koji
  will publish messages on ``topic://koji.package.add`` when an user runs
  ``kojidev add-pkg`` etc. Use ``topic://`` prefixes for ActiveMQ brokers,
  ``/topic/`` for RabbitMQ brokers.
* ``connect_timeout`` -- the number of seconds to wait for a connection before
  timing out
* ``send_timeout`` -- the number of seconds to wait while sending a message
  before timing out

The ``[message]`` section sets parameters for how messages are formed.

* ``extra_limit`` -- the maximum allowed size for ``build.extra`` fields that
  appear in messages. If the ``build.extra`` field is longer (in terms of 
  json-encoded length), then it will be omitted. The default value is ``0``
  which means no limit.
* ``ttl`` -- The time to live to set for messages. Measured in seconds

The ``[queue]`` section controls how (or if) the plugin will use the database
to queue messages when they cannot be immediately sent.
The following fields are understood:

* ``enabled`` -- if true, then the feature is enabled
* ``batch_size`` -- the maximum number of queued messages to send at one time
* ``max_age`` -- the age (in hours) at which old messages in the queue are discarded

It is important to note that the database queue is only a fallback mechanism.
The plugin will always attempt to send messages as they are issued.
Messages are only placed in the database queue when they cannot be immediately
sent on the bus (e.g. if the amqps server is offline).

Admins should consider the balance between the ``batch_size`` and
``extra_limit`` options, as both can affect the total amount of data that the
plugin could attempt to send during a single call.


Image builds using Kiwi
=======================

Plugin for creating images via `kiwi <http://osinside.github.io/kiwi/>`_
project. Minimal supported version of kiwi is ``kiwi-9.24.2``.

All three parts (cli/hub/builder) needs to be installed. There is currently no
configuration except allowing the plugins (name is 'kiwi' for all components).

Builders don't need to have any specific library installed (kiwi
invocation/usage is only in buildroots not on builder itself). (Temporarily
``python3-kiwi`` needs to be installed on builder for kojid to be able to parse
kiwi output. It will be changed to json in next version and this requirement
will be dropped.)

``image`` channel is the default one and ``channel`` policy can be used to
request other channel for this type of tasks as usual.

Buildtag needs to be configured by adding special group ``kiwi`` which should
contain at least ``kiwi-cli``, potentially ``jing`` for better description files
validation and any ``kiwi-systemdeps-*`` packages for requested image types. So,
most simple configuration will look like:

.. code-block:: shell

   $ koji add-group kiwi-build-tag kiwi-build
   $ koji add-group-pkg kiwi-build-tag kiwi-build kiwi-cli kiwi-systemdeps

Another thing we need to ensure is that we're building in chroot and not in
container.

.. code-block:: shell

   $ koji edit-tag kiwi-build-tag -x mock.new_chroot=False

Calling the build itself is a matter of simple CLI call:

.. code-block: shell

   $ koji kiwi-build kiwi-target git+https://my.git/image-descriptions#master my_image_path

Selecting other than default kiwi profile can be done by ``--kiwi-profile``
option. Similarly to other image tasks, alternative architecture failures can be
ignored for successful build by ``--can-fail`` option. ``--arch`` can be used to
limit build tag architectures.

There are some limitation to used kiwi configuration:

 * ``include`` node can use only ``this://`` protocol. Other types like ``file://``
   or ``https://`` could reach out of the repo preventing reproducible build.
 * All repositories from description (and included files) are removed and replaced
   by buildroot repo and other repositories specified by ``--repo`` option.

Driver Update Disks building
============================

**This is just a tech-preview. API/usage can drastically change in upcoming
releases**

Plugin for creating Driver Update Disks with ``xorrisofs``.

All three parts (cli/hub/builder) needs to be installed. There is currently no
configuration except allowing the plugins (name is 'dud' for all components).

Builders don't need to have any specific library installed (xorrisofs
invocation/usage is only in buildroots not on builder itself).

Buildtag needs to be configured by adding special group ``dud-build`` which should contain
the following packages:

.. code-block:: shell


   $ koji add-group dud-build-tag dud-build
   $ koji add-group-pkg dud-build-tag dud-build xorriso
   $ koji add-group-pkg dud-build-tag dud-build createrepo_c
   $ koji add-group-pkg dud-build-tag dud-build dnf
   $ koji add-group-pkg dud-build-tag dud-build dnf-plugins-core

Another thing we need to ensure is that we're building in chroot and not in
container.

.. code-block:: shell

   $ koji edit-tag dud-build-tag -x mock.new_chroot=False

Calling the build itself is a matter of simple CLI call:

.. code-block: shell

   $ koji dud-build dud-target --scmurl=git+https://my.git/image-descriptions#master myamazingdud 1 package1 package2

The command options allows to bring all the package dependencies into the DUD
ISO with ``--alldeps``. ``--scmurl`` allows to include non-RPM related content
inside the produced ISO.

Similarly to other image tasks, alternative architecture failures can be
ignored for successful build by ``--can-fail`` option. ``--arch`` can be used
to limit build tag architectures.

tag2distrepo
============

Koji plugin to automatically regenerate distrepos on tag operations

It uses the following options on a tag to control behaviour:

- ``tag2distrepo.enabled``: set to "true" to enable automatic distrepos
- ``tag2distrepo.keys``: set to a space-separated list of keys to use for distrepos

Following parameters correspond to relevant ``brew dist-repo`` options.

- ``tag2distrepo.inherit``: follow inheritance (default: False)
- ``tag2distrepo.latest``: use only latest tagged builds (default: False)
- ``tag2distrepo.split_debuginfo``: separate directory for debuginfo default: False

The tag must have at least one arch configured on it.

Installing plugin on Koji Hub
-----------------------------

1. Edit the following settings in ``/etc/koji-hub/hub.conf`` to enable the plugin:

.. code-block:: ini

         PluginPath = /usr/lib/koji-hub-plugins
         Plugins = tag2distrepo

2. Reload Apache

.. code-block:: shell

        $ systemctl reload httpd

Example usage
-------------

Here is an example of enabling the plugin on an "f33-infra" tag. Create the tag and ensure it has
at least one arch and a package list (direct or inherited) so we can tag packages into it.

.. code-block:: shell

    $ koji add-tag f33-infra --arches=x86_64
    $ koji add-pkg --owner kdreyer f33-infra bash

Set the extra options on the tag so the plugin will generate the repository:

.. code-block:: shell

    $ koji edit-tag -x tag2distrepo.enabled=True -x tag2distrepo.keys=47dd8ef9 f33-infra

Tag a new build to trigger the plugin:

.. code-block:: shell

    $ koji tag f33-infra bash-5.0.17-2.fc33

The hub will immediately queue a new distRepo task, using the tagBuild task host as the distRepo
task owner. When the distRepo task completes, you can find the new repository under the ``topurl``
for your Koji instance.

To confirm that the tag has the correct options set, use the `koji taginfo` command:

.. code-block:: shell

    $ koji taginfo f33-infra
    Tag: f33-infra [18680]
    Arches: x86_64
    Tag options:
      tag2distrepo.enabled : 'true'
      tag2distrepo.keys : '47dd8ef9'

To disable the plugin for the same tag:

.. code-block:: shell

    $ koji edit-tag -r tag2distrepo.enabled -r tag2distrepo.keys f33-infra

Using Multiple Keys
-------------------

If you want to create a repository that contains builds signed by more than one key, list your
desired key IDs ordered by preference.

For example:

.. code-block:: shell

    $ koji edit-tag coreos-pool -x tag2distrepo.keys="45719a39 9867c58f 38ab71f4 5323552a"

For each RPM in the tag, Koji will use the first signed copy that it finds. In other words,
Koji will try the first key (`45719a39`), and if Koji does not have the first key's signature
for that RPM, then it will try the second key (`9867c58f`), third key (`38ab71f4`), and so on.


SCM policy
==========

This plugin adds additional policy check after content is checked out from SCM.
New policy is simply named ``scm``.

Data which can be checked there contains ``build_tag``, ``method``,
``scratch``, and ``branches`` fields. Especially ``branches`` is the reason -
policy can e.g. check if reference being built is part of any allowed branch
and e.g. not random commit which can disappear later. Two new policy tests are
part of the plugin ``match_any`` and ``match_all`` which tests the list
against glob. So, in this case any (or all respectively) branch must pass the
glob test.


Example policy:

::

  scm =
     # anything can be built as a scratch build
     bool scratch :: allow

     # regular build must be present at lease on one branch
     match_all branches * !! deny Source ref must be contained in a branch

     # Combination of method, scm and repo
     method buildContainer && buildtag container-test-* && match scm_host git.example.com && match scm_repository /containers/* :: allow

     # deny any other buildContainer task
     method buildContainer :: deny Only specific buildContainer tasks can be executed

     # allow anything else
     all :: allow

Builder
-------

Plugin is simply activated by adding it as ``plugin = scmpolicy`` to
``/etc/kojid.conf``. No other configuration is required.