particle-os/debian-koji
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

PR#4025: docs: mock's configuration

Browse source 
Merges #4025
https://pagure.io/koji/pull-request/4025

Fixes: #4019
https://pagure.io/koji/issue/4019
RLIMITS settings for nspawn
This commit is contained in:
Tomas Kopecek 2024-03-19 11:10:03 +01:00
commit 91f410f8fb
6 changed files with 204 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/source/hub_conf.rst
View file

@ -233,6 +233,8 @@ Notification options
For more on notifications in Koji, see :ref:`notification-basics`
.. _hub_resource_limits:
Resource limits
^^^^^^^^^^^^^^^
If configured, the following resource limits are applied by the hub at startup.

3
docs/source/index.rst
View file

@ -11,7 +11,7 @@ Koji
====
Koji is the software that builds `RPM packages for the Fedora project`_. It
uses `Mock`_ to create chroot environments to perform builds. To download the
uses :doc:`Mock <mock_config>` to create chroot environments to perform builds. To download the
source code, report bugs, join the mailing list etc., see the `Koji project
website`_.
@ -43,6 +43,7 @@ Contents
database_howto
kojid_conf
hub_conf
mock_config
using_the_koji_build_system
setting_rpm_macros
profiles

73
docs/source/kojid_conf.rst
View file

@ -1,11 +1,15 @@
kojid.conf Options
------------------
kojid.conf is standard .ini-like configuration file. Its main section is
called ``[kojid]`` and contains following options. They are split by function
to section but can occur anywhere in ``[kojid]`` section in config file.
General
^^^^^^^
``/etc/kojid/kojid.conf`` is a standard .ini-like configuration file. Its main
section is named ``[kojid]`` and can contain the following options.
For readability, we have broken them into categories in this document,
but they should all go under the ``[kojid]`` block of the config file.
General Options
^^^^^^^^^^^^^^^
These options affect overall kojid behavior:
.. glossary::
chroot_tmpdir=/chroot_tmpdir
The temporary directory in buildroot. It is advised to not use ``/tmp``
@ -31,7 +35,7 @@ General
buildroot_final_cleanup_delay
buildroot_final_cleanup_delay=86400
Time after which buildroot (pre-cleand after
Time after which buildroot (pre-cleaned after
``buildroot_basic_cleanup_delay``) is deleted completely. (1 day in
seconds)
@ -96,8 +100,10 @@ General
workdir=/tmp/koji
The directory root for temporary storage on builder.
Building
^^^^^^^^
Build Options
^^^^^^^^^^^^^
The following options affect various build processes in kojid:
.. glossary::
allowed_scms=scm.example.com:/cvs/example git.example.org:/example svn.example.org:/users/\*:no
Controls which source control systems the builder will accept. It is a
@ -192,8 +198,10 @@ Building
the image. Default value is compromise between speed and resource usage.
Only one option (not space-separated) is allowed here for now.
RPM Builds
^^^^^^^^^^
RPM Build Options
^^^^^^^^^^^^^^^^^
The following options affect rpms builds:
.. glossary::
distribution=Koji
The distribution to use in rpm headers. Value is propagated via macros
@ -214,8 +222,27 @@ RPM Builds
The vendor to use in rpm headers. Value is propagated via macros to
rpmbuild.
Mock
^^^^
.. _rlimits_kojid:
General RLIMIT options
^^^^^^^^^^^^^^^^^^^^^^
Kojid accepts the same rlimit options as the hub.
.. glossary::
RLIMIT_*
Same resource limit options as in the :ref:`hub config
<hub_resource_limits>`. Note, that when ``mock.new_chroot`` is set, those
are not propagated to the mock chroot. To apply rlimits to mock in
such cases, you can set them separately in
``/etc/mock/site-defaults.cfg``. See :doc:`mock_config` for more information.
.. _mock_kojid_conf:
Mock Options
^^^^^^^^^^^^
The following options affect how Koji uses Mock:
.. glossary::
mockdir=/var/lib/mock
The directory root for mock.
@ -234,20 +261,26 @@ Mock
yum_proxy=None
Address of proxy server which will be passed via mock to yum.
Notifications
^^^^^^^^^^^^^
rlimits
The ``RLIMIT_*`` options can affect mock's behaviour and are described :ref:`here
<rlimits_kojid>`.
Notification Options
^^^^^^^^^^^^^^^^^^^^
.. glossary::
admin_emails=''
Space-separated list of addresses for sending logs.
from_addr=Koji Build System <buildsys@example.com>
from_addr=Koji Build System <buildsys\@example.com>
The From address used when sending email notifications.
smtphost=example.com
The mail host to use for sending email notifications.
Kerberos Authentication
^^^^^^^^^^^^^^^^^^^^^^^
Kerberos Options
^^^^^^^^^^^^^^^^
The following options control Kerberos authentication:
.. glossary::
ccache=/var/tmp/kojid.ccache
Credentials cache used for krbV login.
@ -260,8 +293,10 @@ Kerberos Authentication
Location of the keytab.
SSL Authentication
^^^^^^^^^^^^^^^^^^
SSL Options
^^^^^^^^^^^
The following options control SSL authentication:
.. glossary::
ca=''
noop, obsoleted, will be removed soon.

131
docs/source/mock_config.rst Normal file
View file

@ -0,0 +1,131 @@
Interaction with Mock
---------------------
`Mock`_ is crucial component for building in Koji. Most build types use mock to
prepare a fresh, isolated environment for building content.
For further reading on the mock tool itself,
see the `mock documentation <https://rpm-software-management.github.io/mock/>`_.
Mock config files
=================
For each buildroot Koji prepares a configuration file for mock to use.
The content of the file depends in part on some administrator's decisions
and can be tweaked by options in the build tag.
These files are managed by kojid and stored under ``/etc/mock/koji`` on
the build host.
Since Koji 1.34.1, builds will save the mock configuration file as a log named
``mock_config.log`` and store it alongside other logs from the build.
The contents of this file are generated by Koji and are affected by:
* :ref:`kojid configuration <mock_kojid_conf>`
* the build tag
* the build arch
* the repo used for the build
* build tag options, see :ref:`tuning-mock-per-tag`
* the controlling build task (e.g. buildArch for rpms)
You can generate mock configs locally in the same way that kojid does.
See :ref:`cli_mock_config`
Site defaults
=============
In addition to the mock configuration file for the buildroot, the mock command
also consults ``/etc/mock/site-defaults.cfg`` on the build host.
You can use this file to set global defaults on your builders.
The following settings are recommended for use with Koji:
- ``use_bootstrap = False`` - Bootstrap is a) not needed in most cases b)
slows down the build c) e.g. in combination with next option could lead to
unreproducible build. While this feature is pretty useful in some cases,
it should be turned off as default and could be selectively turned on for
specific buildroots per tag configuration.
- ``use_bootstrap_image`` - As noted before, this is generally unsafe
option, especially if used with image tags which are expected to be moved
(e.g. "latest"). Note, that to use this option (even via tag's
configuration), you need to explicitly allow it in :doc:`kojid.conf
<kojid_conf>` via ``mock_bootstrap_image = True``.
- ``nspawn_args`` - if you want to apply rlimit settings for mock with nspawn
isolation, you will need to use this option. See :ref:`rlimits_mock` below.
- ``macros`` - some rpm macros could be defined every time for every build
based on builder itself. Typical usecase is to limit usage of CPUs on
given builder by setting ``config_opts['macros']['%_smp_ncpus_max'] = 2``.
- ``plugin_conf`` - Various plugins could be enabled and configured to be
available in build environment. Example is `rpmautospec`_ which allows
various ``%auto*`` macros in spec files.
.. _rlimits_mock:
RLIMIT settings
^^^^^^^^^^^^^^^
When mock uses simple isolation (aka old-chroot), the
:ref:`RLIMIT_* settings <rlimits_kojid>` from ``kojid.conf`` will propagate
to mock.
However, when mock uses nspawn isolation (aka new-chroot, controlled by the
``mock.new_chroot`` tag extra option), these limits are not propagated automatically.
To apply such limits to the mock nspawn chroots, you can add them to the
``nspawn_args`` option in ``site-defaults.cfg``.
For example, you might set:
::
config_opts['nspawn_args'] = ['--rlimit=RLIMIT_NOFILE=16384:16384']
If your system uses a mix of mock isolation settings, then you will likely want
to apply rlimit settings in *both* ``kojid.conf`` and ``site-defaults.cfg`` for
consistency.
.. _cli_mock_config:
Local mock configs
==================
The ``koji mock-config`` command can be used to generate a mock config file locally
in the same way that the builders do. These can be useful for local testing or debugging.
::
$ koji mock-config --help
Usage: koji mock-config [options]
(Specify the --help global option for a list of other help options)
Options:
-h, --help show this help message and exit
-a ARCH, --arch=ARCH Specify the arch
-n NAME, --name=NAME Specify the name for the buildroot
--tag=TAG Create a mock config for a tag
--target=TARGET Create a mock config for a build target
--task=TASK Duplicate the mock config of a previous task
--latest use the latest redirect url
--buildroot=BUILDROOT
Duplicate the mock config for the specified buildroot
id
--mockdir=DIR Specify mockdir
--topdir=DIR Specify topdir, topdir tops the topurl
--topurl=URL URL under which Koji files are accessible, when topdir
is specified, topdir tops the topurl
--distribution=DISTRIBUTION
Change the distribution macro
--yum-proxy=YUM_PROXY
Specify a yum proxy
-o FILE Output to a file
.. _Mock: https://rpm-software-management.github.io/mock/
.. _rpmautospec: https://docs.pagure.org/fedora-infra.rpmautospec/

3
docs/source/server_howto.rst
View file

@ -1054,9 +1054,12 @@ Configuration Files
* ``/etc/kojid/kojid.conf`` - Koji Daemon Configuration
* ``/etc/sysconfig/kojid`` - Koji Daemon Switches
* ``/etc/mock/site-defaults.cfg`` - Mock's base configuration file
All options for `kojid.conf` are described :doc:`here <kojid_conf>`.
Mock-tuning options are described :doc:`here <mock_config>`.
Install kojid
-------------

14
docs/source/using_the_koji_build_system.rst
View file

@ -390,11 +390,17 @@ environment follows:
* ``mock.package_manager`` - If this is set, it will override mock's default
package manager. Typically used with ``yum`` or ``dnf`` values.
* ``mock.new_chroot`` - 0/1 value. If it is set, ``--new-chroot`` or
`--old-chroot` option is appended to any mock call. If it is not set,
mock's default behavior is used.
`--old-chroot` option is appended to any mock call. If it is not set, mock's
default behavior is used. Note, that in the case when new chroot (nspawn) is
used, some of the options in ``kojid.conf`` will not be propagated to chroot
env properly, typically ``RLIMIT_*``, see :ref:`kojid.conf's options
<rlimits_kojid>`.
* ``mock.releasever`` - When doing cross-compiles it may be necessary
to explicitly set the ``releasever`` to be used.
* ``mock.use_bootstrap`` - 0/1 value. If it is set, ``--bootstrap-chroot``
is appended to the mock init call. This tells mock to build in two stages,
using chroot rpm for creating the build chroot. If it is not set, mock's
@ -402,6 +408,7 @@ environment follows:
<https://github.com/rpm-software-management/mock/wiki/Feature-bootstrap>`_.
Note, that it is not turn on by default by koji, as it is often not needed and
it consumes additional resources (larger buildroot, downloading more data).
* ``mock.bootstrap_image`` - set to name of image, which can builder's podman
download (e.g. ``fedora:32``). See mock's `doc
<https://github.com/rpm-software-management/mock/wiki/Feature-container-for-bootstrap>`_
@ -470,6 +477,9 @@ do:
koji edit-tag dnf-fedora-tag -x rpm.env.CC=clang
Note, that mock's behaviour is always impacted by builder's default values in
``/etc/mock/site-defaults.cfg``. Check :doc:`mock_config` for details.
Using Koji to control tasks
^^^^^^^^^^^^^^^^^^^^^^^^^^^