188 lines
5.8 KiB
ReStructuredText
188 lines
5.8 KiB
ReStructuredText
Storage Volumes
|
|
===============
|
|
|
|
|
|
Introduction
|
|
------------
|
|
|
|
Since version 1.7.0, Koji has had the ability to store builds on
|
|
multiple volumes.
|
|
|
|
Each build in Koji has a volume field that indicates which volume it is
|
|
stored on. The default volume is named ``DEFAULT`` and corresponds to the
|
|
original paths under ``/mnt/koji`` that predate this feature.
|
|
|
|
Additional volumes correspond to paths under
|
|
``/mnt/koji/vol/NAME`` (where NAME is the name of the volume). All builds
|
|
associated with such a volume will be stored under this directory.
|
|
The expectation is that this directory will map to a different file system.
|
|
|
|
|
|
.. hint::
|
|
| If ``koji-1.13.0-1`` is on the DEFAULT volume, its path will be:
|
|
| /mnt/koji/packages/koji/1.13.0/1
|
|
|
|
| If ``koji-1.13.0-1`` is on the volume named "test", its path will be:
|
|
| /mnt/koji/vol/test/packages/koji/1.13.0/1
|
|
|
|
|
|
Constructing the path
|
|
---------------------
|
|
|
|
If you are using the Koji python api, things should **just work**.
|
|
|
|
::
|
|
|
|
>>> import koji
|
|
>>> mykoji = koji.get_profile_module('mykoji')
|
|
>>> opts = mykoji.grab_session_options(mykoji.config)
|
|
>>> session = mykoji.ClientSession(mykoji.config.server, opts)
|
|
>>> binfo = session.getBuild('test-1-1')
|
|
>>> binfo['volume_name']
|
|
'DEFAULT'
|
|
>>> mykoji.pathinfo.build(binfo)
|
|
'/mnt/koji/packages/test/1/1'
|
|
>>> binfo = session.getBuild('fake-1.0-21')
|
|
>>> binfo['volume_name']
|
|
'vol3'
|
|
>>> mykoji.pathinfo.build(binfo)
|
|
'/mnt/koji/vol/vol3/packages/fake/1.0/21'
|
|
|
|
If you are constructing these paths yourself, then you may need to do
|
|
a little work.
|
|
|
|
* Look for volume information in build data. Hub calls that return build
|
|
data include ``volume_name`` and ``volume_id`` fields in their return.
|
|
* If the volume is ``DEFAULT``, then the path is the "normal" path.
|
|
* Otherwise, you need to insert ``/vol/<volume_name>`` after the
|
|
top directory (normally /mnt/koji).
|
|
|
|
|
|
Symlinks on default volume
|
|
--------------------------
|
|
|
|
For backwards compatibility, Koji maintains symlinks on the default volume
|
|
for builds on other volumes.
|
|
|
|
::
|
|
|
|
$ file /mnt/koji/packages/fake/1.0/21
|
|
/mnt/koji/packages/fake/1.0/21: symbolic link to ../../../vol/vol3/packages/fake/1.0/21
|
|
|
|
|
|
Adding a new volume
|
|
-------------------
|
|
|
|
The new volume directory should initially contain a packages/
|
|
subdirectory, and the permissions should be the same as the default
|
|
packages directory.
|
|
|
|
Assuming you do use a mount for a vol/NAME directory, you will want to
|
|
ensure that the same mounts are created on all systems that interface with
|
|
``/mnt/koji``, such as builders that run createrepo tasks, hosts running
|
|
kojira or similar maintenance, and any hosts that rely on the topdir option
|
|
rather than the topurl option.
|
|
|
|
Once you have the directory set up, you can tell Koji about it by
|
|
running ``koji add-volume NAME``. This call will fail if the hub can't find
|
|
the directory.
|
|
|
|
Moving builds onto other volumes
|
|
--------------------------------
|
|
|
|
By default, all builds live on the DEFAULT volume.
|
|
|
|
An admin can move a build to a different volume by using the
|
|
``koji set-build-volume`` command, or by using the underlying
|
|
``changeBuildVolume`` api call.
|
|
|
|
Moving a build across volumes will cause kojira to trigger repo
|
|
regenerations, if appropriate. When the new volume is not DEFAULT, Koji will
|
|
create a relative symlink to the new build directory on the default
|
|
volume. Moving builds across volumes may immediately break repos (until
|
|
the regen occurs), so use caution.
|
|
|
|
Consider the following example:
|
|
|
|
::
|
|
|
|
# mypkg-1.1-20 initially on default volume
|
|
$ file /mnt/koji/packages/mypkg/1.1/20
|
|
/mnt/koji/packages/mypkg/1.1/20: directory
|
|
|
|
# move it to test volume
|
|
$ koji set-build-volume test mypkg-1.1-20
|
|
$ file /mnt/koji/vol/test/packages/mypkg/1.1/20
|
|
/mnt/koji/vol/test/packages/mypkg/1.1/20: directory
|
|
|
|
# original location is now a symlink
|
|
$ file /mnt/koji/packages/mypkg/1.1/20
|
|
/mnt/koji/packages/mypkg/1.1/20: symbolic link to ../../../vol/test/packages/mypkg/1.1/20
|
|
|
|
|
|
Using the volume in policy checks
|
|
---------------------------------
|
|
|
|
Policies involving builds (e.g. gc policy, tag policy), can test a
|
|
build's volume with the ``volume`` test. This is a pattern match
|
|
test against the volume name.
|
|
|
|
Setting a volume policy
|
|
-----------------------
|
|
|
|
The Koji 1.14.0 release adds the ability to set a volume policy on the hub.
|
|
This policy is used at import time to determine which volume the build should
|
|
be assigned to. This provides a systematic way to distribute builds
|
|
across multiple volumes without manual intervention.
|
|
|
|
There is relatively limited data available to the volume policy. Tests that are
|
|
expected to work include:
|
|
|
|
- user based tests (the user performing the build or running the import)
|
|
- package based tests (e.g. ``is_new_package`` or ``package``)
|
|
- cg match tests
|
|
- the buildtag test
|
|
|
|
The action value for the volume policy should be simply the name of the volume
|
|
to use.
|
|
|
|
The default volume policy is ``all :: DEFAULT``.
|
|
|
|
If the volume policy contains errors, or does not return a result, then the
|
|
DEFAULT volume is used.
|
|
|
|
For more information about Koji policies see:
|
|
:doc:`Defining hub policies <defining_hub_policies>`
|
|
|
|
|
|
CLI commands
|
|
------------
|
|
|
|
``add-volume``
|
|
adds a new volume (directory must already be set up)
|
|
``list-volumes``
|
|
prints a list of known volumes
|
|
``set-build-volume``
|
|
moves a build to different volume
|
|
|
|
|
|
API calls
|
|
---------
|
|
|
|
``addVolume(name, strict=True)``
|
|
Add a new storage volume in the database
|
|
|
|
``applyVolumePolicy(build, strict=False)``
|
|
Apply the volume policy to a given build
|
|
|
|
``changeBuildVolume(build, volume, strict=True)``
|
|
Move a build to a different storage volume
|
|
|
|
``getVolume(volume, strict=False)``
|
|
Lookup the given volume
|
|
|
|
``listVolumes()``
|
|
List storage volumes
|
|
|
|
``removeVolume(volume)``
|
|
Remove unused storage volume from the database
|