321 lines
13 KiB
HTML
321 lines
13 KiB
HTML
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
<head>
|
|
<title>Koji HOWTO</title>
|
|
</head>
|
|
<body>
|
|
<h1>Introduction</h1>
|
|
|
|
Koji is a system for building and tracking RPMs. It was designed with the following
|
|
features in mind:
|
|
|
|
<p>
|
|
<b>Security</b>
|
|
<ul>
|
|
<li>New buildroot for each build</li>
|
|
<li>nfs is used (mostly) read-only</li>
|
|
</ul>
|
|
|
|
<b>Leverage other software</b>
|
|
<ul>
|
|
<li>Uses Yum and Mock open-source components</li>
|
|
<li>XML-RPC APIs for easy integration with other tools</li>
|
|
</ul>
|
|
|
|
<b>Flexibility</b>
|
|
<ul>
|
|
<li>rich data model</li>
|
|
<li>active code base</li>
|
|
</ul>
|
|
|
|
<b>Usability</b>
|
|
<ul>
|
|
<li>Web interface with Kerberos authentication</li>
|
|
<li>Thin, portable client</li>
|
|
<li>Users can create local buildroots</li>
|
|
</ul>
|
|
|
|
<b>Reproducibility</b>
|
|
<ul>
|
|
<li>Buildroot contents are tracked in the database</li>
|
|
<li>Versioned data</li>
|
|
</ul>
|
|
|
|
|
|
<p> This HOWTO document covers the basic tasks that a developer needs to be
|
|
able to accomplish with Koji.
|
|
</p>
|
|
|
|
<h1>Getting started</h1>
|
|
|
|
<h2>The web interface</h2>
|
|
<p>The primary interface for viewing Koji data is a web application. Most of the interface
|
|
is read-only, but if you are logged in (see below) and have sufficient privileges there
|
|
are some actions that can be performed though the web. For example:
|
|
<ul>
|
|
<li>Cancel a build</li>
|
|
<li>Resubmit a failed task</li>
|
|
</ul>
|
|
Those with admin privileges will find additional actions, such as:
|
|
<ul>
|
|
<li>Create/Edit/Delete a tag</li>
|
|
<li>Create/Edit/Delete a target</li>
|
|
<li>Enable/Disable a build host</li>
|
|
</ul>
|
|
|
|
<p>The web site utilizes Kerberos authentication. In order to log in you will
|
|
need a valid Kerberos ticket and your web browser will need to be configured to send the
|
|
Kerberos information to the server.
|
|
|
|
<p>In Firefox or Mozilla, you will need to use the about:config page to set a few parameters.
|
|
Use the search term 'negotiate' to filter the list. Change
|
|
network.negotiate-auth.trusted-uris to the domain you want to authenticate against,
|
|
e.g .example.com. You can leave network.negotiate-auth.delegation-uris blank, as it
|
|
enables Kerberos ticket passing, which is not required. If you do not see those two
|
|
config options listed, your version of Firefox or Mozilla may be too old to support
|
|
Negotiate authentication, and you should consider upgrading.
|
|
|
|
<p>In order to obtain a Kerberos ticket, use the kinit command.
|
|
|
|
|
|
<h2>Installing the Koji cli</h2>
|
|
<p>There is a single point of entry for most operations. The command is
|
|
called 'koji' and is included in the main koji package.
|
|
|
|
<p>Repos/webpage TBD
|
|
|
|
<p>
|
|
The koji tool authenticates to the central server using Kerberos, so you will need
|
|
to have a valid Kerberos ticket to use many features. However, many of the read-only
|
|
commands will work without authentication.
|
|
|
|
<h2>Building a package</h2>
|
|
<p>Builds are initiated with the command line tool.
|
|
To build a package, the syntax is:</p>
|
|
<pre>$ koji build <build target> <cvs URL></pre>
|
|
|
|
<p>For example:</p>
|
|
<pre>$ koji build dist-fc7-scratch 'cvs://cvs.example.com/cvs/dist?rpms/kernel/FC-7#kernel-2_6_20-1_2925_fc7'</pre>
|
|
<p>
|
|
The <code>koji build</code> command creates a build task in Koji. By default
|
|
the tool will wait
|
|
and print status updates until the build completes. You can override this with
|
|
the <code>--nowait</code> option. To view other options to the build command use the
|
|
<code>--help</code> option.
|
|
</p>
|
|
|
|
<pre>$ koji build --help
|
|
</pre>
|
|
|
|
<h2>Build Options</h2>
|
|
<p>
|
|
There are a few options to the build command. Here are some more detailed explanations
|
|
of them:
|
|
</p>
|
|
|
|
<dl>
|
|
<dt>--skip-tag</dt>
|
|
<dd>Normally the package is tagged after the build completes. This option causes
|
|
the tagging step to be skipped. The package will be in the system, but untagged
|
|
(you can later tag it with the tag-pkg command)</dd>
|
|
<dt>--scratch</dt>
|
|
<dd>This makes the build into a scratch build. The build will not be
|
|
imported into the db, it will just be built. The rpms will land under
|
|
<topdir>/scratch. Scratch builds are not tracked and can never
|
|
be tagged, but can be convenient for testing. Scratch builds are
|
|
typically removed from the filesystem after one week.
|
|
</dd>
|
|
<dt>--nowait</dt>
|
|
<dd>As stated above, this prevents the cli from waiting on the build task.</dd>
|
|
<dt>--arch-override</dt>
|
|
<dd>This option allows you to override the base set of arches to build for.
|
|
This option is really only for testing during the beta period, but it may
|
|
be retained for scratch builds in the future.</dd>
|
|
</dl>
|
|
|
|
<h2>Build Failures</h2>
|
|
<p>If your package fails to build, you will see something like this.</p>
|
|
<pre>
|
|
420066 buildArch (kernel-2.6.18-1.2739.10.9.el5.jjf.215394.2.src.rpm,
|
|
ia64): open (build-1.example.com) -> FAILED: BuildrootError:
|
|
error building package (arch ia64), mock exited with status 10
|
|
</pre>
|
|
|
|
<p>You can figure out why the build failed by looking at the log files. If
|
|
there is a build.log, start there. Otherwise, look at init.log</p>
|
|
|
|
<pre>
|
|
$ ls -1 <topdir>/work/tasks/420066/*
|
|
<topdir>/work/tasks/420066/build.log
|
|
<topdir>/work/tasks/420066/init.log
|
|
<topdir>/work/tasks/420066/mockconfig.log
|
|
<topdir>/work/tasks/420066/root.log
|
|
</pre>
|
|
|
|
<h2>Filing Bugs</h2>
|
|
|
|
<p>bug tracking TBD
|
|
|
|
<h1>Koji Architecture</h1>
|
|
|
|
<h2>Terminology</h2>
|
|
|
|
In Koji, it is sometimes necessary to distinguish between the a package in general,
|
|
a specific build of a package, and the various rpm files created by a build. When
|
|
precision is needed, these terms should be interpreted as follows:
|
|
|
|
<dl>
|
|
<dt>Package</dt>
|
|
<dd>The name of a source rpm. This refers to the package in general and not
|
|
any particular build or subpackage. For example: kernel, glibc, etc.</dd>
|
|
<dt>Build</dt>
|
|
<dd>A particular build of a package. This refers to the entire build: all arches
|
|
and subpackages. For example: kernel-2.6.9-34.EL, glibc-2.3.4-2.19.</dd>
|
|
<dt>RPM</dt>
|
|
<dd>A particular rpm. A specific arch and subpackage of a build.
|
|
For example: kernel-2.6.9-34.EL.x86_64, kernel-devel-2.6.9-34.EL.s390,
|
|
glibc-2.3.4-2.19.i686, glibc-common-2.3.4-2.19.ia64</dd>
|
|
</dl>
|
|
|
|
|
|
<h2>Koji Components</h2>
|
|
|
|
Koji is comprised of several components:
|
|
|
|
<ul>
|
|
<li><em>koji-hub</em> is the center of all Koji operations. It is an XML-RPC server
|
|
running under mod_python in Apache. koji-hub is passive in that it only
|
|
receives XML-RPC calls and relies upon the build daemons and other
|
|
components to initiate communication. koji-hub is the only component that
|
|
has direct access to the database and is one of the two components that have
|
|
write access to the file system.</li>
|
|
|
|
<li><em>kojid</em> is the build daemon that runs on each of the build machines. Its
|
|
primary responsibility is polling for incoming build requests and handling
|
|
them accordingly. Koji also has support for tasks other than building.
|
|
Creating install images is one example. kojid is responsible for handling
|
|
these tasks as well.
|
|
|
|
<p>kojid uses mock for building. It also creates a fresh buildroot for
|
|
every build. kojid is written in Python and communicates with koji-hub via
|
|
XML-RPC.</p></li>
|
|
|
|
<li><em>koji-web</em> is a set of scripts that run in mod_python and use the Cheetah
|
|
templating engine to provide an web interface to Koji. koji-web exposes a
|
|
lot of information and also provides a means for certain operations, such as
|
|
cancelling builds.</li>
|
|
|
|
<li><em>koji</em> is a CLI written in Python that provides many hooks into
|
|
Koji. It allows the user to query much of the data as well as perform
|
|
actions such as build initiation.</li>
|
|
|
|
<li><em>kojirepod</em> is a daemon that keeps the build root repodata
|
|
updated.</li>
|
|
|
|
</ul>
|
|
|
|
<h2>Package Organization</h2>
|
|
<p><i>Tags and Targets</i></p>
|
|
<p>Koji organizes packages using tags. In Koji a tag is roughly analogous to
|
|
a beehive collection instance, but differ in a number of ways:</p>
|
|
<ul>
|
|
<li>Tags are tracked in the database but not on disk</li>
|
|
<li>Tags support multiple inheritance</li>
|
|
<li>Each tag has its own list of valid packages (inheritable)</li>
|
|
<li>Package ownership can be set per-tag (inheritable)</li>
|
|
<li>Tag inheritance is more configurable</li>
|
|
<li>When you build you specify a <i>target</i> rather than a tag</li>
|
|
</ul>
|
|
<p>
|
|
A build target specifies where a package should be built and how it
|
|
should be tagged afterwards. This allows target names to remain fixed
|
|
as tags change through releases. You can get a full list of build targets
|
|
with the following command:</p>
|
|
<pre>$ koji list-targets
|
|
</pre>
|
|
You can see just a single target with the <code>--name</code> option:
|
|
<pre>$ koji list-targets --name dist-fc7
|
|
Name Buildroot Destination
|
|
---------------------------------------------------------------------------------------------
|
|
dist-fc7 dist-fc7-build dist-fc7
|
|
</pre>
|
|
This tells you a build for target dist-fc7 will use a buildroot with packages
|
|
from the tag dist-fc7-build and tag the resulting packages as dist-fc7.
|
|
<p>
|
|
You can get a list of tags with the following command:</p>
|
|
<pre>$ koji list-tags
|
|
</pre>
|
|
<p><i>Package lists</i></p>
|
|
<p>
|
|
As mentioned above, each tag has its own list of packages that may be placed
|
|
in the tag. To see that list for a tag, use the <code>list-pkgs</code> command:</p>
|
|
<pre>$ koji list-pkgs --tag dist-fc7
|
|
Package Tag Extra Arches Owner
|
|
----------------------- ----------------------- ---------------- ----------------
|
|
ElectricFence dist-fc6 pmachata
|
|
GConf2 dist-fc6 rstrode
|
|
lucene dist-fc6 dbhole
|
|
lvm2 dist-fc6 lvm-team
|
|
ImageMagick dist-fc6 nmurray
|
|
m17n-db dist-fc6 majain
|
|
m17n-lib dist-fc6 majain
|
|
MAKEDEV dist-fc6 clumens
|
|
...
|
|
</pre>
|
|
The first column is the name of the package, the second tells you which tag
|
|
the package entry has been inherited from, and the third tells you the owner
|
|
of the package.
|
|
<p><i>Latest Builds</i></p>
|
|
<p>
|
|
To see the latest builds for a tag, use the <code>latest-pkg</code> command:</p>
|
|
<pre>$ koji latest-pkg --all dist-fc7
|
|
Build Tag Built by
|
|
---------------------------------------- -------------------- ----------------
|
|
ConsoleKit-0.1.0-5.fc7 dist-fc7 davidz
|
|
ElectricFence-2.2.2-20.2.2 dist-fc6 jkeating
|
|
GConf2-2.16.0-6.fc7 dist-fc7 mclasen
|
|
ImageMagick-6.2.8.0-3.fc6.1 dist-fc6-updates nmurray
|
|
MAKEDEV-3.23-1.2 dist-fc6 nalin
|
|
MySQL-python-1.2.1_p2-2 dist-fc7 katzj
|
|
NetworkManager-0.6.5-0.3.cvs20061025.fc7 dist-fc7 caillon
|
|
ORBit2-2.14.6-1.fc7 dist-fc7 mclasen
|
|
</pre>
|
|
The output gives you not only the latest builds, but which tag they have
|
|
been inherited from and who built them (note: for builds imported from beehive
|
|
the "built by" field may be misleading)
|
|
|
|
|
|
<h2>Exploring Koji</h2>
|
|
|
|
<p>We've tried to make Koji self-documenting wherever possible. The command
|
|
line tool will print a list of valid commands and each command supports
|
|
<code>--help</code>. For example:</p>
|
|
|
|
<pre>
|
|
$ koji help
|
|
Koji commands are:
|
|
build Build a package from source
|
|
cancel-task Cancel a task
|
|
help List available commands
|
|
latest-build Print the latest rpms for a tag
|
|
latest-pkg Print the latest builds for a tag
|
|
...
|
|
$ koji build --help
|
|
usage: koji build [options] tag URL
|
|
(Specify the --help global option for a list of other help options)
|
|
|
|
options:
|
|
-h, --help show this help message and exit
|
|
--skip-tag Do not attempt to tag package
|
|
--scratch Perform a scratch build
|
|
--nowait Don't wait on build
|
|
...
|
|
</pre>
|
|
|
|
<h1>Getting Involved</h1>
|
|
|
|
If you would like to be more involved with the Koji project...
|
|
|
|
<p>Project data TBD
|
|
|
|
</body>
|
|
</html>
|