particle-os/debian-koji
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Initial code drop

Browse source
This commit is contained in:
Jesse Keating 2007-02-14 11:25:01 -05:00
parent 4967a2a434
commit 5d7e66a17e
126 changed files with 27342 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

321
docs/HOWTO.html Normal file
View file

@ -0,0 +1,321 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Koji HOWTO</title>
</head>
<body>
<h1>Introduction</h1>
Koji is a system for building and tracking RPMs. It was designed with the following
features in mind:
<p>
<b>Security</b>
<ul>
<li>New buildroot for each build</li>
<li>nfs is used (mostly) read-only</li>
</ul>
<b>Leverage other software</b>
<ul>
<li>Uses Yum and Mock open-source components</li>
<li>XML-RPC APIs for easy integration with other tools</li>
</ul>
<b>Flexibility</b>
<ul>
<li>rich data model</li>
<li>active code base</li>
</ul>
<b>Usability</b>
<ul>
<li>Web interface with Kerberos authentication</li>
<li>Thin, portable client</li>
<li>Users can create local buildroots</li>
</ul>
<b>Reproducibility</b>
<ul>
<li>Buildroot contents are tracked in the database</li>
<li>Versioned data</li>
</ul>
<p> This HOWTO document covers the basic tasks that a developer needs to be
able to accomplish with Koji.
</p>
<h1>Getting started</h1>
<h2>The web interface</h2>
<p>The primary interface for viewing Koji data is a web application. Most of the interface
is read-only, but if you are logged in (see below) and have sufficient privileges there
are some actions that can be performed though the web. For example:
<ul>
<li>Cancel a build</li>
<li>Resubmit a failed task</li>
</ul>
Those with admin privileges will find additional actions, such as:
<ul>
<li>Create/Edit/Delete a tag</li>
<li>Create/Edit/Delete a target</li>
<li>Enable/Disable a build host</li>
</ul>
<p>The web site utilizes Kerberos authentication. In order to log in you will
need a valid Kerberos ticket and your web browser will need to be configured to send the
Kerberos information to the server.
<p>In Firefox or Mozilla, you will need to use the about:config page to set a few parameters.
Use the search term 'negotiate' to filter the list. Change
network.negotiate-auth.trusted-uris to the domain you want to authenticate against,
e.g .example.com. You can leave network.negotiate-auth.delegation-uris blank, as it
enables Kerberos ticket passing, which is not required. If you do not see those two
config options listed, your version of Firefox or Mozilla may be too old to support
Negotiate authentication, and you should consider upgrading.
<p>In order to obtain a Kerberos ticket, use the kinit command.
<h2>Installing the Koji cli</h2>
<p>There is a single point of entry for most operations. The command is
called 'koji' and is included in the main koji package.
<p>Repos/webpage TBD
<p>
The koji tool authenticates to the central server using Kerberos, so you will need
to have a valid Kerberos ticket to use many features. However, many of the read-only
commands will work without authentication.
<h2>Building a package</h2>
<p>Builds are initiated with the command line tool.
To build a package, the syntax is:</p>
<pre>$ koji build &lt;build target&gt; &lt;cvs URL&gt;</pre>
<p>For example:</p>
<pre>$ koji build dist-fc7-scratch 'cvs://cvs.example.com/cvs/dist?rpms/kernel/FC-7#kernel-2_6_20-1_2925_fc7'</pre>
<p>
The <code>koji build</code> command creates a build task in Koji. By default
the tool will wait
and print status updates until the build completes. You can override this with
the <code>--nowait</code> option. To view other options to the build command use the
<code>--help</code> option.
</p>
<pre>$ koji build --help
</pre>
<h2>Build Options</h2>
<p>
There are a few options to the build command. Here are some more detailed explanations
of them:
</p>
<dl>
<dt>--skip-tag</dt>
<dd>Normally the package is tagged after the build completes. This option causes
the tagging step to be skipped. The package will be in the system, but untagged
(you can later tag it with the tag-pkg command)</dd>
<dt>--scratch</dt>
<dd>This makes the build into a scratch build. The build will not be
imported into the db, it will just be built. The rpms will land under
&lt;topdir&gt;/scratch. Scratch builds are not tracked and can never
be tagged, but can be convenient for testing. Scratch builds are
typically removed from the filesystem after one week.
</dd>
<dt>--nowait</dt>
<dd>As stated above, this prevents the cli from waiting on the build task.</dd>
<dt>--arch-override</dt>
<dd>This option allows you to override the base set of arches to build for.
This option is really only for testing during the beta period, but it may
be retained for scratch builds in the future.</dd>
</dl>
<h2>Build Failures</h2>
<p>If your package fails to build, you will see something like this.</p>
<pre>
420066 buildArch (kernel-2.6.18-1.2739.10.9.el5.jjf.215394.2.src.rpm,
ia64): open (build-1.example.com) -> FAILED: BuildrootError:
error building package (arch ia64), mock exited with status 10
</pre>
<p>You can figure out why the build failed by looking at the log files. If
there is a build.log, start there. Otherwise, look at init.log</p>
<pre>
$ ls -1 &lt;topdir&gt;/work/tasks/420066/*
&lt;topdir&gt;/work/tasks/420066/build.log
&lt;topdir&gt;/work/tasks/420066/init.log
&lt;topdir&gt;/work/tasks/420066/mockconfig.log
&lt;topdir&gt;/work/tasks/420066/root.log
</pre>
<h2>Filing Bugs</h2>
<p>bug tracking TBD
<h1>Koji Architecture</h1>
<h2>Terminology</h2>
In Koji, it is sometimes necessary to distinguish between the a package in general,
a specific build of a package, and the various rpm files created by a build. When
precision is needed, these terms should be interpreted as follows:
<dl>
<dt>Package</dt>
<dd>The name of a source rpm. This refers to the package in general and not
any particular build or subpackage. For example: kernel, glibc, etc.</dd>
<dt>Build</dt>
<dd>A particular build of a package. This refers to the entire build: all arches
and subpackages. For example: kernel-2.6.9-34.EL, glibc-2.3.4-2.19.</dd>
<dt>RPM</dt>
<dd>A particular rpm. A specific arch and subpackage of a build.
For example: kernel-2.6.9-34.EL.x86_64, kernel-devel-2.6.9-34.EL.s390,
glibc-2.3.4-2.19.i686, glibc-common-2.3.4-2.19.ia64</dd>
</dl>
<h2>Koji Components</h2>
Koji is comprised of several components:
<ul>
<li><em>koji-hub</em> is the center of all Koji operations. It is an XML-RPC server
running under mod_python in Apache. koji-hub is passive in that it only
receives XML-RPC calls and relies upon the build daemons and other
components to initiate communication. koji-hub is the only component that
has direct access to the database and is one of the two components that have
write access to the file system.</li>
<li><em>kojid</em> is the build daemon that runs on each of the build machines. Its
primary responsibility is polling for incoming build requests and handling
them accordingly. Koji also has support for tasks other than building.
Creating install images is one example. kojid is responsible for handling
these tasks as well.
<p>kojid uses mock for building. It also creates a fresh buildroot for
every build. kojid is written in Python and communicates with koji-hub via
XML-RPC.</p></li>
<li><em>koji-web</em> is a set of scripts that run in mod_python and use the Cheetah
templating engine to provide an web interface to Koji. koji-web exposes a
lot of information and also provides a means for certain operations, such as
cancelling builds.</li>
<li><em>koji</em> is a CLI written in Python that provides many hooks into
Koji. It allows the user to query much of the data as well as perform
actions such as build initiation.</li>
<li><em>kojirepod</em> is a daemon that keeps the build root repodata
updated.</li>
</ul>
<h2>Package Organization</h2>
<p><i>Tags and Targets</i></p>
<p>Koji organizes packages using tags. In Koji a tag is roughly analogous to
a beehive collection instance, but differ in a number of ways:</p>
<ul>
<li>Tags are tracked in the database but not on disk</li>
<li>Tags support multiple inheritance</li>
<li>Each tag has its own list of valid packages (inheritable)</li>
<li>Package ownership can be set per-tag (inheritable)</li>
<li>Tag inheritance is more configurable</li>
<li>When you build you specify a <i>target</i> rather than a tag</li>
</ul>
<p>
A build target specifies where a package should be built and how it
should be tagged afterwards. This allows target names to remain fixed
as tags change through releases. You can get a full list of build targets
with the following command:</p>
<pre>$ koji list-targets
</pre>
You can see just a single target with the <code>--name</code> option:
<pre>$ koji list-targets --name dist-fc7
Name Buildroot Destination
---------------------------------------------------------------------------------------------
dist-fc7 dist-fc7-build dist-fc7
</pre>
This tells you a build for target dist-fc7 will use a buildroot with packages
from the tag dist-fc7-build and tag the resulting packages as dist-fc7.
<p>
You can get a list of tags with the following command:</p>
<pre>$ koji list-tags
</pre>
<p><i>Package lists</i></p>
<p>
As mentioned above, each tag has its own list of packages that may be placed
in the tag. To see that list for a tag, use the <code>list-pkgs</code> command:</p>
<pre>$ koji list-pkgs --tag dist-fc7
Package Tag Extra Arches Owner
----------------------- ----------------------- ---------------- ----------------
ElectricFence dist-fc6 pmachata
GConf2 dist-fc6 rstrode
lucene dist-fc6 dbhole
lvm2 dist-fc6 lvm-team
ImageMagick dist-fc6 nmurray
m17n-db dist-fc6 majain
m17n-lib dist-fc6 majain
MAKEDEV dist-fc6 clumens
...
</pre>
The first column is the name of the package, the second tells you which tag
the package entry has been inherited from, and the third tells you the owner
of the package.
<p><i>Latest Builds</i></p>
<p>
To see the latest builds for a tag, use the <code>latest-pkg</code> command:</p>
<pre>$ koji latest-pkg --all dist-fc7
Build Tag Built by
---------------------------------------- -------------------- ----------------
ConsoleKit-0.1.0-5.fc7 dist-fc7 davidz
ElectricFence-2.2.2-20.2.2 dist-fc6 jkeating
GConf2-2.16.0-6.fc7 dist-fc7 mclasen
ImageMagick-6.2.8.0-3.fc6.1 dist-fc6-updates nmurray
MAKEDEV-3.23-1.2 dist-fc6 nalin
MySQL-python-1.2.1_p2-2 dist-fc7 katzj
NetworkManager-0.6.5-0.3.cvs20061025.fc7 dist-fc7 caillon
ORBit2-2.14.6-1.fc7 dist-fc7 mclasen
</pre>
The output gives you not only the latest builds, but which tag they have
been inherited from and who built them (note: for builds imported from beehive
the "built by" field may be misleading)
<h2>Exploring Koji</h2>
<p>We've tried to make Koji self-documenting wherever possible. The command
line tool will print a list of valid commands and each command supports
<code>--help</code>. For example:</p>
<pre>
$ koji help
Koji commands are:
build Build a package from source
cancel-task Cancel a task
help List available commands
latest-build Print the latest rpms for a tag
latest-pkg Print the latest builds for a tag
...
$ koji build --help
usage: koji build [options] tag URL
(Specify the --help global option for a list of other help options)
options:
-h, --help show this help message and exit
--skip-tag Do not attempt to tag package
--scratch Perform a scratch build
--nowait Don't wait on build
...
</pre>
<h1>Getting Involved</h1>
If you would like to be more involved with the Koji project...
<p>Project data TBD
</body>
</html>

4
docs/Makefile Normal file
View file

@ -0,0 +1,4 @@
install:
clean:
rm -f *.o *.so *.pyc *~

607
docs/schema.sql Normal file
View file

@ -0,0 +1,607 @@
-- vim:noet:sw=8
-- still needs work
DROP TABLE build_notifications;
DROP TABLE log_messages;
DROP TABLE buildroot_listing;
DROP TABLE rpmfiles;
DROP TABLE rpmdeps;
DROP TABLE rpminfo;
DROP TABLE group_package_listing;
DROP TABLE group_req_listing;
DROP TABLE group_config;
DROP TABLE groups;
DROP TABLE tag_listing;
DROP TABLE tag_packages;
DROP TABLE buildroot;
DROP TABLE repo;
DROP TABLE build_target_config;
DROP TABLE build_target;
DROP TABLE tag_config;
DROP TABLE tag_inheritance;
DROP TABLE tag;
DROP TABLE build;
DROP TABLE task;
DROP TABLE host_channels;
DROP TABLE host;
DROP TABLE channels;
DROP TABLE package;
DROP TABLE user_groups;
DROP TABLE user_perms;
DROP TABLE permissions;
DROP TABLE sessions;
DROP TABLE users;
DROP TABLE event_labels;
DROP TABLE events;
DROP FUNCTION get_event();
DROP FUNCTION get_event_time(INTEGER);
BEGIN WORK;
-- We use the events table to sequence time
-- in the event that the system clock rolls back, event_ids will retain proper sequencing
CREATE TABLE events (
id SERIAL NOT NULL PRIMARY KEY,
time TIMESTAMP NOT NULL DEFAULT NOW()
) WITHOUT OIDS;
-- A function that creates an event and returns the id, used as DEFAULT value for versioned tables
CREATE FUNCTION get_event() RETURNS INTEGER AS '
INSERT INTO events (time) VALUES (''now'');
SELECT currval(''events_id_seq'')::INTEGER;
' LANGUAGE SQL;
-- A convenience function for converting events to timestamps, useful for
-- quick queries where you want to avoid JOINs.
CREATE FUNCTION get_event_time(INTEGER) RETURNS TIMESTAMP AS '
SELECT time FROM events WHERE id=$1;
' LANGUAGE SQL;
-- this table is used to label events
-- most events will be unlabeled, so keeping this separate saves space
CREATE TABLE event_labels (
event_id INTEGER NOT NULL REFERENCES events(id),
label VARCHAR(255) UNIQUE NOT NULL
) WITHOUT OIDS;
-- User and session data
CREATE TABLE users (
id SERIAL NOT NULL PRIMARY KEY,
name VARCHAR(255) UNIQUE NOT NULL,
password VARCHAR(255),
status INTEGER,
usertype INTEGER,
krb_principal VARCHAR(255) UNIQUE
) WITHOUT OIDS;
CREATE TABLE permissions (
id SERIAL NOT NULL PRIMARY KEY,
name VARCHAR(50) UNIQUE NOT NULL
) WITHOUT OIDS;
-- Some basic perms
INSERT INTO permissions (name) VALUES ('admin');
INSERT INTO permissions (name) VALUES ('build');
INSERT INTO permissions (name) VALUES ('repo');
INSERT INTO permissions (name) VALUES ('runroot');
CREATE TABLE user_perms (
user_id INTEGER NOT NULL REFERENCES users(id),
perm_id INTEGER NOT NULL REFERENCES permissions(id),
-- versioned - see VERSIONING
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
revoke_event INTEGER REFERENCES events(id),
active BOOLEAN DEFAULT 'true' CHECK (active),
CONSTRAINT active_revoke_sane CHECK (
(active IS NULL AND revoke_event IS NOT NULL )
OR (active IS NOT NULL AND revoke_event IS NULL )),
PRIMARY KEY (create_event, user_id, perm_id),
UNIQUE (user_id,perm_id,active)
) WITHOUT OIDS;
-- groups are represented as users w/ usertype=2
CREATE TABLE user_groups (
user_id INTEGER NOT NULL REFERENCES users(id),
group_id INTEGER NOT NULL REFERENCES users(id),
-- versioned - see VERSIONING
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
revoke_event INTEGER REFERENCES events(id),
active BOOLEAN DEFAULT 'true' CHECK (active),
CONSTRAINT active_revoke_sane CHECK (
(active IS NULL AND revoke_event IS NOT NULL )
OR (active IS NOT NULL AND revoke_event IS NULL )),
PRIMARY KEY (create_event, user_id, group_id),
UNIQUE (user_id,group_id,active)
) WITHOUT OIDS;
-- a session can create subsessions, which are just new sessions whose
-- 'master' field points back to the session. This field should
-- always point to the top session. If the master session is expired,
-- the all its subsessions should be expired as well.
-- If a session is exclusive, it is the only session allowed for its
-- user. The 'exclusive' field is either NULL or TRUE, never FALSE. This
-- is so exclusivity can be enforced with a unique condition.
CREATE TABLE sessions (
id SERIAL NOT NULL PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users(id),
expired BOOLEAN NOT NULL DEFAULT FALSE,
master INTEGER REFERENCES sessions(id),
key VARCHAR(255),
authtype INTEGER,
hostip VARCHAR(255),
callnum INTEGER,
start_time TIMESTAMP NOT NULL DEFAULT NOW(),
update_time TIMESTAMP NOT NULL DEFAULT NOW(),
exclusive BOOLEAN CHECK (exclusive),
CONSTRAINT no_exclusive_subsessions CHECK (
master IS NULL OR "exclusive" IS NULL),
CONSTRAINT exclusive_expired_sane CHECK (
expired IS FALSE OR "exclusive" IS NULL),
UNIQUE (user_id,exclusive)
) WITHOUT OIDS;
CREATE INDEX sessions_master ON sessions(master);
CREATE INDEX sessions_active_and_recent ON sessions(expired, master, update_time) WHERE (expired IS NOT TRUE AND master IS NULL);
-- Channels are used to limit which tasks are run on which machines.
-- Each task is assigned to a channel and each host 'listens' on one
-- or more channels. A host will only accept tasks for channels it is
-- listening to.
CREATE TABLE channels (
id SERIAL NOT NULL PRIMARY KEY,
name VARCHAR(128) UNIQUE NOT NULL
) WITHOUT OIDS;
-- create default channel
INSERT INTO channels (name) VALUES ('default');
INSERT INTO channels (name) VALUES ('runroot');
-- Here we track the build machines
-- each host has an entry in the users table also
-- capacity: the hosts weighted task capacity
CREATE TABLE host (
id SERIAL NOT NULL PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users (id),
name VARCHAR(128) UNIQUE NOT NULL,
arches TEXT,
task_load FLOAT CHECK (NOT task_load < 0) NOT NULL DEFAULT 0.0,
capacity FLOAT CHECK (capacity > 1) NOT NULL DEFAULT 2.0,
ready BOOLEAN NOT NULL DEFAULT 'false',
enabled BOOLEAN NOT NULL DEFAULT 'true'
) WITHOUT OIDS;
CREATE INDEX HOST_IS_READY_AND_ENABLED ON host(enabled, ready) WHERE (enabled IS TRUE AND ready IS TRUE);
CREATE TABLE host_channels (
host_id INTEGER NOT NULL REFERENCES host(id),
channel_id INTEGER NOT NULL REFERENCES channels(id),
UNIQUE (host_id,channel_id)
) WITHOUT OIDS;
-- tasks are pretty general and may refer to all sorts of jobs, not
-- just package builds.
-- tasks may spawn subtasks (hence the parent field)
-- top-level tasks have NULL parent
-- the request and result fields are base64-encoded xmlrpc data.
-- this means each task is effectively an xmlrpc call, using this table as
-- the medium.
-- the host_id field indicates which host is running the task. This field
-- is used to lock the task.
-- weight: the weight of the task (vs. host capacity)
-- label: this field is used to label subtasks. top-level tasks will not
-- have a label. some subtasks may be unlabeled. labels are used in task
-- failover to prevent duplication of work.
CREATE TABLE task (
id SERIAL NOT NULL PRIMARY KEY,
state INTEGER,
create_time TIMESTAMP NOT NULL DEFAULT NOW(),
completion_time TIMESTAMP,
channel_id INTEGER NOT NULL REFERENCES channels(id),
host_id INTEGER REFERENCES host (id),
parent INTEGER REFERENCES task (id),
label VARCHAR(255),
waiting BOOLEAN,
awaited BOOLEAN,
owner INTEGER REFERENCES users(id) NOT NULL,
method TEXT,
request TEXT,
result TEXT,
eta INTEGER,
arch VARCHAR(16) NOT NULL,
priority INTEGER,
weight FLOAT CHECK (NOT weight < 0) NOT NULL DEFAULT 1.0,
CONSTRAINT parent_label_sane CHECK (
parent IS NOT NULL OR label IS NULL),
UNIQUE (parent,label)
) WITHOUT OIDS;
CREATE INDEX task_by_state ON task (state);
-- CREATE INDEX task_by_parent ON task (parent); (unique condition creates similar index)
CREATE INDEX task_by_host ON task (host_id);
-- by package, we mean srpm
-- we mean the package in general, not an individual build
CREATE TABLE package (
id SERIAL NOT NULL PRIMARY KEY,
name TEXT UNIQUE NOT NULL
) WITHOUT OIDS;
-- CREATE INDEX package_by_name ON package (name);
-- (implicitly created by unique constraint)
-- here we track the built packages
-- this is at the srpm level, since builds are by srpm
-- see rpminfo for isolated packages
-- even though we track epoch, we demand that N-V-R be unique
-- task_id: a reference to the task creating the build, may be
-- null, or may point to a deleted task.
CREATE TABLE build (
id SERIAL NOT NULL PRIMARY KEY,
pkg_id INTEGER NOT NULL REFERENCES package (id) DEFERRABLE,
version TEXT NOT NULL,
release TEXT NOT NULL,
epoch INTEGER,
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
completion_time TIMESTAMP,
state INTEGER NOT NULL,
task_id INTEGER REFERENCES task (id),
owner INTEGER NOT NULL REFERENCES users (id),
CONSTRAINT build_pkg_ver_rel UNIQUE (pkg_id, version, release),
CONSTRAINT completion_sane CHECK ((state = 0 AND completion_time IS NULL) OR
(state != 0 AND completion_time IS NOT NULL))
) WITHOUT OIDS;
CREATE INDEX build_by_pkg_id ON build (pkg_id);
CREATE INDEX build_completion ON build(completion_time);
CREATE TABLE changelogs (
id SERIAL NOT NULL PRIMARY KEY,
build_id INTEGER NOT NULL REFERENCES build (id),
date TIMESTAMP NOT NULL,
author TEXT NOT NULL,
text TEXT
) WITHOUT OIDS;
CREATE INDEX changelogs_by_date on changelogs (date);
CREATE INDEX changelogs_by_build on changelogs (build_id);
-- Note: some of these CREATEs may seem a little out of order. This is done to keep
-- the references sane.
CREATE TABLE tag (
id SERIAL NOT NULL PRIMARY KEY,
name VARCHAR(50) UNIQUE NOT NULL
) WITHOUT OIDS;
-- CREATE INDEX tag_by_name ON tag (name);
-- (implicitly created by unique constraint)
-- VERSIONING
-- Several tables are versioned with the following scheme. Since this
-- is the first, here is the explanation of how it works.
-- The versioning fields are: create_event, revoke_event, and active
-- The active field is either True or NULL, it is never False!
-- The create_event and revoke_event fields refer to the event table
-- A version is active if active is not NULL
-- (an active version also has NULL revoke_event.)
-- A UNIQUE condition can incorporate the 'active' field, making it
-- apply only to the active versions.
-- When a version is made inactive (revoked):
-- revoke_event is set
-- active is set to NULL
-- Query for current data with WHERE active is not NULL
-- (should be same as WHERE revoke_event is NULL)
-- Query for data at event e with WHERE create_event <= e AND e < revoke_event
CREATE TABLE tag_inheritance (
tag_id INTEGER NOT NULL REFERENCES tag(id),
parent_id INTEGER NOT NULL REFERENCES tag(id),
priority INTEGER NOT NULL,
maxdepth INTEGER,
intransitive BOOLEAN NOT NULL DEFAULT 'false',
noconfig BOOLEAN NOT NULL DEFAULT 'false',
pkg_filter TEXT,
-- versioned - see desc above
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
revoke_event INTEGER REFERENCES events(id),
active BOOLEAN DEFAULT 'true' CHECK (active),
CONSTRAINT active_revoke_sane CHECK (
(active IS NULL AND revoke_event IS NOT NULL )
OR (active IS NOT NULL AND revoke_event IS NULL )),
PRIMARY KEY (create_event, tag_id, priority),
UNIQUE (tag_id,priority,active),
UNIQUE (tag_id,parent_id,active)
) WITHOUT OIDS;
CREATE INDEX tag_inheritance_by_parent ON tag_inheritance (parent_id);
-- XXX - need more config options listed here
-- perm_id: the permission that is required to apply the tag. can be NULL
--
CREATE TABLE tag_config (
tag_id INTEGER NOT NULL REFERENCES tag(id),
arches TEXT,
perm_id INTEGER REFERENCES permissions(id),
locked BOOLEAN NOT NULL DEFAULT 'false',
-- versioned - see desc above
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
revoke_event INTEGER REFERENCES events(id),
active BOOLEAN DEFAULT 'true' CHECK (active),
CONSTRAINT active_revoke_sane CHECK (
(active IS NULL AND revoke_event IS NOT NULL )
OR (active IS NOT NULL AND revoke_event IS NULL )),
PRIMARY KEY (create_event, tag_id),
UNIQUE (tag_id,active)
) WITHOUT OIDS;
-- a build target tells the system where to build the package
-- and how to tag it afterwards.
CREATE TABLE build_target (
id SERIAL NOT NULL PRIMARY KEY,
name VARCHAR(50) UNIQUE NOT NULL
) WITHOUT OIDS;
CREATE TABLE build_target_config (
build_target_id INTEGER NOT NULL REFERENCES build_target(id),
build_tag INTEGER NOT NULL REFERENCES tag(id),
dest_tag INTEGER NOT NULL REFERENCES tag(id),
-- versioned - see desc above
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
revoke_event INTEGER REFERENCES events(id),
active BOOLEAN DEFAULT 'true' CHECK (active),
CONSTRAINT active_revoke_sane CHECK (
(active IS NULL AND revoke_event IS NOT NULL )
OR (active IS NOT NULL AND revoke_event IS NULL )),
PRIMARY KEY (create_event, build_target_id),
UNIQUE (build_target_id,active)
) WITHOUT OIDS;
-- track repos
CREATE TABLE repo (
id SERIAL NOT NULL PRIMARY KEY,
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
tag_id INTEGER NOT NULL REFERENCES tag(id),
state INTEGER
) WITHOUT OIDS;
-- here we track the buildroots on the machines
CREATE TABLE buildroot (
id SERIAL NOT NULL PRIMARY KEY,
host_id INTEGER NOT NULL REFERENCES host(id),
repo_id INTEGER NOT NULL REFERENCES repo (id),
arch VARCHAR(16) NOT NULL,
task_id INTEGER REFERENCES task (id),
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
retire_event INTEGER,
state INTEGER,
dirtyness INTEGER
) WITHOUT OIDS;
-- this table associates tags with builds. an entry here tags a package
CREATE TABLE tag_listing (
build_id INTEGER NOT NULL REFERENCES build (id),
tag_id INTEGER NOT NULL REFERENCES tag (id),
-- versioned - see earlier description of versioning
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
revoke_event INTEGER REFERENCES events(id),
active BOOLEAN DEFAULT 'true' CHECK (active),
CONSTRAINT active_revoke_sane CHECK (
(active IS NULL AND revoke_event IS NOT NULL )
OR (active IS NOT NULL AND revoke_event IS NULL )),
PRIMARY KEY (create_event, build_id, tag_id),
UNIQUE (build_id,tag_id,active)
) WITHOUT OIDS;
CREATE INDEX tag_listing_tag_id_key ON tag_listing(tag_id);
-- this is a per-tag list of packages, with some extra info
-- so this allows you to explicitly state which packages belong where
-- (as opposed to beehive where this can only be done at the collection level)
-- these are packages in general, not specific builds.
-- this list limits which builds can be tagged with which tags
-- if blocked is true, then the package is specifically not included. this
-- prevents the package from being included via inheritance
CREATE TABLE tag_packages (
package_id INTEGER NOT NULL REFERENCES package (id),
tag_id INTEGER NOT NULL REFERENCES tag (id),
owner INTEGER NOT NULL REFERENCES users(id),
blocked BOOLEAN NOT NULL DEFAULT FALSE,
extra_arches TEXT,
-- versioned - see earlier description of versioning
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
revoke_event INTEGER REFERENCES events(id),
active BOOLEAN DEFAULT 'true' CHECK (active),
CONSTRAINT active_revoke_sane CHECK (
(active IS NULL AND revoke_event IS NOT NULL )
OR (active IS NOT NULL AND revoke_event IS NULL )),
PRIMARY KEY (create_event, package_id, tag_id),
UNIQUE (package_id,tag_id,active)
) WITHOUT OIDS;
-- package groups (per tag). used for generating comps for the tag repos
CREATE TABLE groups (
id SERIAL NOT NULL PRIMARY KEY,
name VARCHAR(50) UNIQUE NOT NULL
-- corresponds to the id field in a comps group
) WITHOUT OIDS;
-- if blocked is true, then the group is specifically not included. this
-- prevents the group from being included via inheritance
CREATE TABLE group_config (
group_id INTEGER NOT NULL REFERENCES groups (id),
tag_id INTEGER NOT NULL REFERENCES tag (id),
blocked BOOLEAN NOT NULL DEFAULT FALSE,
exported BOOLEAN DEFAULT TRUE,
display_name TEXT NOT NULL,
is_default BOOLEAN,
uservisible BOOLEAN,
description TEXT,
langonly TEXT,
biarchonly BOOLEAN,
-- versioned - see earlier description of versioning
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
revoke_event INTEGER REFERENCES events(id),
active BOOLEAN DEFAULT 'true' CHECK (active),
CONSTRAINT active_revoke_sane CHECK (
(active IS NULL AND revoke_event IS NOT NULL )
OR (active IS NOT NULL AND revoke_event IS NULL )),
PRIMARY KEY (create_event, group_id, tag_id),
UNIQUE (group_id,tag_id,active)
) WITHOUT OIDS;
CREATE TABLE group_req_listing (
group_id INTEGER NOT NULL REFERENCES groups (id),
tag_id INTEGER NOT NULL REFERENCES tag (id),
req_id INTEGER NOT NULL REFERENCES groups (id),
blocked BOOLEAN NOT NULL DEFAULT FALSE,
type VARCHAR(25),
is_metapkg BOOLEAN NOT NULL DEFAULT FALSE,
-- versioned - see earlier description of versioning
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
revoke_event INTEGER REFERENCES events(id),
active BOOLEAN DEFAULT 'true' CHECK (active),
CONSTRAINT active_revoke_sane CHECK (
(active IS NULL AND revoke_event IS NOT NULL )
OR (active IS NOT NULL AND revoke_event IS NULL )),
PRIMARY KEY (create_event, group_id, tag_id, req_id),
UNIQUE (group_id,tag_id,req_id,active)
) WITHOUT OIDS;
-- if blocked is true, then the package is specifically not included. this
-- prevents the package from being included in the group via inheritance
-- package refers to an rpm name, not necessarily an srpm name (so it does
-- not reference the package table).
CREATE TABLE group_package_listing (
group_id INTEGER NOT NULL REFERENCES groups (id),
tag_id INTEGER NOT NULL REFERENCES tag (id),
package TEXT,
blocked BOOLEAN NOT NULL DEFAULT FALSE,
type VARCHAR(25) NOT NULL,
basearchonly BOOLEAN,
requires TEXT,
-- versioned - see earlier description of versioning
create_event INTEGER NOT NULL REFERENCES events(id) DEFAULT get_event(),
revoke_event INTEGER REFERENCES events(id),
active BOOLEAN DEFAULT 'true' CHECK (active),
CONSTRAINT active_revoke_sane CHECK (
(active IS NULL AND revoke_event IS NOT NULL )
OR (active IS NOT NULL AND revoke_event IS NULL )),
PRIMARY KEY (create_event, group_id, tag_id, package),
UNIQUE (group_id,tag_id,package,active)
) WITHOUT OIDS;
-- rpminfo tracks individual rpms (incl srpms)
-- buildroot_id can be NULL (for externally built packages)
-- even though we track epoch, we demand that N-V-R.A be unique
-- we don't store filename b/c filename should be N-V-R.A.rpm
CREATE TABLE rpminfo (
id SERIAL NOT NULL PRIMARY KEY,
build_id INTEGER REFERENCES build (id),
buildroot_id INTEGER REFERENCES buildroot (id),
name TEXT NOT NULL,
version TEXT NOT NULL,
release TEXT NOT NULL,
epoch INTEGER,
arch VARCHAR(16) NOT NULL,
payloadhash TEXT NOT NULL,
size INTEGER NOT NULL,
buildtime BIGINT NOT NULL,
CONSTRAINT rpminfo_unique_nvra UNIQUE (name,version,release,arch)
) WITHOUT OIDS;
CREATE INDEX rpminfo_build ON rpminfo(build_id);
-- sighash is the checksum of the signature header
CREATE TABLE rpmsigs (
rpm_id INTEGER NOT NULL REFERENCES rpminfo (id),
sigkey TEXT NOT NULL,
sighash TEXT NOT NULL,
CONSTRAINT rpmsigs_no_resign UNIQUE (rpm_id, sigkey)
) WITHOUT OIDS;
-- buildroot_listing needs to be created after rpminfo so it can reference it
CREATE TABLE buildroot_listing (
buildroot_id INTEGER NOT NULL REFERENCES buildroot(id),
rpm_id INTEGER NOT NULL REFERENCES rpminfo(id),
is_update BOOLEAN NOT NULL DEFAULT FALSE,
UNIQUE (buildroot_id,rpm_id)
) WITHOUT OIDS;
CREATE INDEX buildroot_listing_rpms ON buildroot_listing(rpm_id);
-- this table holds the requires, provides, obsoletes, and conflicts
-- for an rpminfo entry
CREATE TABLE rpmdeps (
pkey SERIAL NOT NULL PRIMARY KEY,
rpm_id INTEGER NOT NULL REFERENCES rpminfo (id),
dep_name TEXT NOT NULL,
dep_version TEXT,
dep_flags INTEGER,
dep_type INTEGER NOT NULL
) WITHOUT OIDS;
CREATE INDEX rpmdeps_by_rpm_id ON rpmdeps (rpm_id);
CREATE INDEX rpmdeps_by_depssolve ON rpmdeps (dep_type, dep_name, dep_flags, dep_version);
CREATE TABLE rpmfiles (
rpm_id INTEGER NOT NULL REFERENCES rpminfo (id),
filename TEXT NOT NULL,
filemd5 VARCHAR(32) NOT NULL,
filesize INTEGER NOT NULL,
fileflags INTEGER NOT NULL,
PRIMARY KEY (filename, rpm_id)
) WITHOUT OIDS;
CREATE INDEX rpmfiles_by_rpm_id ON rpmfiles (rpm_id);
CREATE INDEX rpmfiles_by_filename ON rpmfiles (filename);
CREATE TABLE log_messages (
id SERIAL NOT NULL PRIMARY KEY,
message TEXT NOT NULL,
message_time TIMESTAMP NOT NULL DEFAULT NOW(),
logger_name VARCHAR(200) NOT NULL,
level VARCHAR(10) NOT NULL,
location VARCHAR(200),
host VARCHAR(200)
) WITHOUT OIDS;
CREATE TABLE build_notifications (
id SERIAL NOT NULL PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users (id),
package_id INTEGER REFERENCES package (id),
tag_id INTEGER REFERENCES tag (id),
success_only BOOLEAN NOT NULL DEFAULT FALSE,
email TEXT NOT NULL
) WITHOUT OIDS;
GRANT SELECT ON build, package, task, tag,
tag_listing, tag_config, tag_inheritance, tag_packages,
rpminfo, rpmdeps,
rpmfiles TO PUBLIC;
-- example code to add initial admins
-- insert into users (name, usertype, krb_principal) values ('admin', 0, 'admin@EXAMPLE.COM');
-- insert into user_perms (user_id, perm_id)
-- select users.id, permissions.id from users, permissions
-- where users.name in ('admin')
-- and permissions.name = 'admin';
COMMIT WORK;