particle-os/debian-koji
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

doc: livecd/livemedia updates

Browse source 
Fixes: https://pagure.io/koji/issue/2094
This commit is contained in:
Tomas Kopecek 2020-09-22 14:36:12 +02:00
parent dcf235e845
commit 87146dc4b9
1 changed files with 91 additions and 81 deletions
Download patch file Download diff file Expand all files Collapse all files

172
docs/source/image_build.rst
View file

@ -6,9 +6,9 @@ Image Building
==============
Koji is capable of building many different types of images or appliances. They
broadly fall into a few categories: LiveCDs, Disk Images, and Containers. All
types of images end up with a Name-Version-Release just like an RPM build.
What you need to provide Koji to perform a build varies by category, and the
broadly fall into a few categories: LiveMedias, Disk Images, and Containers. All
types of images end up with a Name-Version-Release just like an RPM build. What
you need to provide Koji to perform a build varies by category, and the
specifics will be explained below.
For additional questions and information, ask around in ``#koji`` on FreeNode
@ -67,100 +67,111 @@ If we passed ``--ksurl`` the string above, and gave
``server-ec2.ks``, and pass that to Anaconda
* Perform an automated install
.. _building-livecds:
.. _building-livemedia:
Building LiveCDs
================
Building LiveMedias
===================
Let's describe building LiveCDs and the mechanics behind it.
Let's describe building LiveMedias and the mechanics behind it. We also still
suport older method LiveCD which is deprecated now.
Getting Started
---------------
What you need before proceeding:
* a name for your LiveCD, and a version number
* a name for your LiveMedia, and a version number
* a Koji build target
* what architectures you want
* a flattened kickstart file
* you may also need yum repositories generated by release engineering if you
require signed packages be installed into the LiveCD
* the livecd permission in Koji
require signed packages be installed into the LiveMedia
* the permission for building these in Koji - in default installation permission
is not needed, but in production environments there will be probably some
permission or at least policy in place
The Koji command that creates a LiveCD is called ``spin-livecd``. It calls out
to `livecd-tools`_ in a mock chroot to construct the LiveCD. To run a LiveCD
build, use the spin-livecd command like so:
The Koji command that creates a LiveMedia is called ``spin-livemedia``. It calls
out to `livemedia-creator`_ from `lorax`_ project in a mock chroot to construct
the LiveMedia. To run a LiveMedia build, use the ``spin-livemedia`` command like
so:
::
$ koji spin-livecd --release 4 fedora-workstation 23 f23-image-build fedora-workstation.ks
$ koji spin-livemedia --release 20200922.n.0 \
--ksurl 'git+https://pagure.io/fedora-kickstarts.git?#b4956c05028a088c641d0ce6e1a31b6d8b20176f' \
--install-tree-url 'https://kojipkgs.fedoraproject.org/compose/branched/Fedora-33-20200922.n.0/compose/Everything/$basearch/os' \
--repo 'https://kojipkgs.fedoraproject.org/compose/branched/Fedora-33-20200922.n.0/compose/Everything/$basearch/os' \
--can-fail ppc64le,aarch64 \
Fedora-Workstation-Live 33 f33 x86_64,ppc64le,aarch64 fedora-live-workstation.ks
In this example a LiveCD will be created with the N-V-R of
fedora-workstation-23-4. If ``--release`` was not included an incrementing
value would be computed by Koji instead. spin-livecd takes a minimum of 5
arguments:
In this example (real task is `here
<https://koji.fedoraproject.org/koji/taskinfo?taskID=52013359>`_) a LiveMedia will
be created with the N-V-R of ``Fedora-Workstation-Live-33-20200922.n.0``. If
``--release`` was not included an incrementing value would be computed by Koji
instead. spin-livemedia takes a minimum of 4 arguments:
Name
the name of the image, without versioning information. Examples could be
``fedora`` or ``fedora-workstation``.
``Fedora-Live`` or ``Fedora-Workstation-Live``.
Version
an arbitrary version string. For Fedora images, this usually matches the
release version, such as 21, 22, or 23.
release version, such as 31, 32, or 33.
Build Target
just like RPM builds Koji must be told which target to use. This controls
what tag the image will be tagged into, and what packages are available to
install into the image.
Architecture
only x86_64 or i386 is supported
Kickstart File
this is a recipe file that performs drives the construction of the image.
Note, that you're can either upload your kickstart or just point to some SCM
(``--ksurl``). In our example fedora repo contains
``fedora-live-workstation.ks`` kickstart file.
Use ``--help`` to discover more options to ``spin-livecd``. You can override
the Release field with ``--release``, or ``--repo`` to override what yum repo
is used to provide packages to install to the image. Note that regardless of
what repo you use, it must contain RPMs built by Koji, otherwise you will get
an error. (unless you are building a scratch image)
Use ``--help`` to discover more options to ``spin-livemedia``. You can override
the Release field with ``--release``, or ``--repo`` to override what yum repo is
used to provide packages to install to the image. Note that regardless of what
repo you use, it must contain RPMs built by Koji, otherwise you will get an
error. (unless you are building a scratch image). ``--can-fail`` options says
which architectures are optional and can fail without failing the whole build.
Mechanics
---------
The way LiveCDs are created in Koji is fairly straightforward. A chroot is
The way LiveMedias are created in Koji is fairly straightforward. A chroot is
initialized and populated with the packages and their dependencies from the
``livecd-build`` package group. Next, the kickstart file is copied into it if
``livemedia-build`` package group. Next, the kickstart file is copied into it if
it was provided from local storage. If not, it is checked out into it from an
SCM. It is then modified to use the repo associated with the build tag for the
target specified in the command unless the ``--repo`` option was given. Both
the original and the modified kickstart files are saved as part of the output
for the task for later review. A `livecd-creator`_ command is executed using the
mock('--chroot', ...) method.
target specified in the command unless the ``--repo`` option was given. Both the
original and the modified kickstart files are saved as part of the output for
the task for later review. A `livemedia-creator`_ command is executed using the
``mock('--chroot', ...)`` method.
.. note::
This process runs as root. This produces the desired image which is
uploaded to /mnt/koji/images/<image>/$imageID if it is not a scratch image.
This process runs as root. This produces the desired image which is uploaded
to ``/mnt/koji/images/<image>/$imageID`` if it is not a scratch image.
.. _caveats-for-livecds:
.. _caveats-for-livemedias:
Caveats for LiveCDs
-------------------
Caveats for LiveMedias
----------------------
There are some known caveats with using the spin-livecd command that users
There are some known caveats with using the spin-livemedia command that users
should be aware of.
%include macros in the kickstart
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A word of caution about kickstart files and the ``%include`` macro.
`livecd-creator`_ is smart enough to search the current directory of the
submitted kickstart file if it has %include macros. If the kickstart specified
to koji is from local storage, only that kickstart file will be copied into
the chroot, and this creates a problem if it has ``%include`` macros, because
the other kickstart files it needs will be inaccessible. This issue is not
present when the kickstart file is retrieved from a remote SCM (such as the
spin-kickstarts git repo), because the entire repository is checked out.
`livemedia-creator`_ is smart enough to search the current directory of the
submitted kickstart file if it has ``%include`` macros. If the kickstart
specified to koji is from local storage, only that kickstart file will be copied
into the chroot, and this creates a problem if it has ``%include`` macros,
because the other kickstart files it needs will be inaccessible. This issue is
not present when the kickstart file is retrieved from a remote SCM (such as the
fedora-kickstarts git repo), because the entire repository is checked out.
Presumably it will include any other kickstart files the specified one is
including in the same directory. A workaround for the issue would be to use
``ksflatten`` (from pykickstart) on kickstart files with ``%include`` macros
@ -170,11 +181,11 @@ Package Groups in the Kickstart File
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Package Groups in the kickstart file cause a problem if the Koji repos do not
define them, which they most likely don't since Koji's comps.xml is based on
the "groups" set up from the CLI. `livecd-creator`_'s behavior is to ignore
define them, which they most likely don't since Koji's comps.xml is based on the
"groups" set up from the CLI. `livemedia-creator`_'s behavior is to ignore
package groups that are not defined in the repo it is using, so this can be
troublesome when creating the image since packages could be left out. There
are a couple possible workarounds:
troublesome when creating the image since packages could be left out. There are
a couple possible workarounds:
* do not use package groups in the kickstart file and just specify a huge list
of packages
@ -205,21 +216,22 @@ system.
%post Section in Kickstart
^^^^^^^^^^^^^^^^^^^^^^^^^^
While livecd-tools does support building on SELinux disabled hosts, you can run
into denials when booting if you create and use new files in the ``%post``
section of your kickstart file. If you do, you should either set the labels
appropriately at the end of the ``%post`` section, or instigate an autorelabel
at boot time.
While `livemedia-creator`_ does support building on SELinux disabled hosts, you
can run into denials when booting if you create and use new files in the
``%post`` section of your kickstart file. If you do, you should either set the
labels appropriately at the end of the ``%post`` section, or instigate an
autorelabel at boot time.
Troubleshooting
---------------
If your build fails, you will be notified on the command line. In the output
will be a URL to the Koji UI, visit that and click on the red subtask link.
From that page review ``root.log`` and ``livecd.log`` for errors. Often errors
are caused by packages being missing, or malformed kickstart files. The log
files are at the bottom of the page. If you're stuck, contact Release
Engineering.
From that page review ``root.log`` and ``livemedia-out.log`` for errors. Often
errors are caused by packages being missing, or malformed kickstart files. The
log files are at the bottom of the page. If problem occurs later during
installation `livemedia-creator`_ will also upload a lot of other logs like
anaconda's, etc. If you're stuck, contact Release Engineering.
Build System Preparation
------------------------
@ -228,32 +240,30 @@ This section assumes you have know-how required to install and configure a new
instance of Koji, and that you have already done so. You can learn how to do so
:doc:`here <server_howto>` if you need to. Please ensure you are using the
latest version of the software and that your database schema is updated as
well. You should also have some familiarity with how `livecd-creator`_ works.
This section only covers preparation for LiveCD builds.
well. You should also have some familiarity with how `livemedia-creator`_ works.
This section only covers preparation for LiveMedia builds.
Follow this procedure step by step to get things prepared they way they need to
be.
#. ``koji add-host-to-channel <your-host> livecd``
add a builder to the livecd channel
#. ``koji grant-permission livecd <user>``
#. ``koji add-host-to-channel <your-host> livemedia``
add a builder to the livemedia channel
#. ``koji grant-permission livemedia <user>``
grant the permission to build an image type to a user. This step is
optional since admins have all permissions.
#. You will need a tag and target to build the images from. The yum repo
generated for the build tag of the target is what Koji will use to populate
the LiveCDs with by default. (the alternative is to use the ``--repo``
the LiveMedias with by default. (the alternative is to use the ``--repo``
option, more on that later)
#. ``koji add-group <build-tag> livecd-build``
add the livecd-build group
#. ``koji add-group-pkg <build-tag> livecd-build <pkg> ...``
add packages to the livecd-build group. These package lists vary has
packages and dependencies change. As of October, 2015 for Fedora 23 the
#. ``koji add-group <build-tag> livemedia-build``
add the livemedia-build group
#. ``koji add-group-pkg <build-tag> livemedia-build <pkg> ...``
add packages to the livemedia-build group. These package lists vary has
packages and dependencies change. As of September, 2020 for Fedora 33 the
needed packages for each image type are:
* bash, coreutils, fedora-logos, fedora-release, livecd-tools,
policycoreutils, python-dbus, sed, selinux-policy-targeted,
shadow-utils, squashfs-tools, sssd-client, tar, unzip, util-linux,
which, yum
* bash, coreutils, glibc-all-langpacks, lorax-lmc-novirt,
selinux-policy-targeted, shadow-utils, util-linux
Building Disk Images
@ -268,7 +278,7 @@ Getting Started
What you need:
* a name for your LiveCD, and a version number
* a name for your image, and a version number
* what architectures you want
* a Koji build target
* kickstart file
@ -869,13 +879,13 @@ error. (unless you are building a scratch image)
Mechanics
---------
The way Appliances are created in Koji is the same as :ref:`building-livecds`.
The way Appliances are created in Koji is the same as :ref:`building-livemedia`.
Caveats for Appliances
----------------------
Caveats for using ``spin-appliance`` are the same as using ``spin-livecd`` to
:ref:`caveats-for-livecds`.
Caveats for using ``spin-appliance`` are the same as using ``spin-livemedia`` to
:ref:`caveats-for-livemedias`.
Troubleshooting
---------------
@ -939,8 +949,8 @@ to be.
https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst
.. _Anaconda Kickstart Guide:
https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst
.. _livecd-tools: https://github.com/rhinstaller/livecd-tools/
.. _livecd-creator: https://fedoraproject.org/wiki/FedoraLiveCD
.. _lorax: https://github.com/weldr/lorax
.. _livemedia-creator: https://weldr.io/lorax/livemedia-creator.html
.. _ImageFactory: http://imgfac.org/
.. _Oz: https://github.com/clalancette/oz
.. _how to use Oz: https://github.com/clalancette/oz/wiki