The `osbuild` executable requires that the exact same version of the
corresponding python library is installed, but this was not enforced
in the RPM package. Thus a old version of osbuild could be installed
alongside an older version of python3-osbuild, which results in an
osbuild crash (see below).
Therefore, enforce that both installed packages have matching
versions by specifying the exact version for the `python3-osbuild`
dependency of the `osbuild` package.
Example osbuild crash if versions are not matching:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
ws.require(__requires__)
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (osbuild 10 (/usr/lib/python3.8/site-packages), Requirement.parse('osbuild==11'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/osbuild", line 6, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3252, in <module>
def _initialize_master_working_set():
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 585, in _build_master
return cls._build_from_requirements(__requires__)
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'osbuild==11' distribution was not found and is required by the application
RPM is smart about dependencies - it goes over all shebangs of packaged files
and adds Requires to them. Prior this commit osbuild package depended on
python 3.6, because rhel runner has /usr/bin/python3.6 as shebang.
This commit fixes it by turning off dependency generators for our internal
stuff: assemblers, runners and stages. They run in a container, so I think
there's no reason to generate dependencies from them.
Use `make man` rather than hard-coding man-page creation in the
spec-file. Furthermore, install all man-pages, not just the base one.
The commands are adjusted to account for possibly more man-pages being
added. Tree-wide the only place to adjust for new man-pages being added
is the `%files` list in the spec-file.
We already indent the values in the spec-file to all start at the same
column. However, we use different indentation sizes for different
blocks of values. This makes it really confusing to read. Lets use a
consistent indentation and align everything with the main assignments in
the spec-file.
Detect the host dynamically from os-release(5) instead of relying on the
`org.osbuild.host` symlink.
It is awkward to install a symlink that tells osbuild which distro is is
running on, when there is a standard way to detect this.
This makes it easier to run osbuild from sources and removes the need to
include every host in the spec file. The latter became hard to do,
because there's no obvious way to distinguish RHEL minor releases.
This changes the name of the source tarball, but apart from that has
no effect as-is.
However, if the `commit` global is set, then the source tarball is
expected to be the standard git-archive at that commit, and the
release field is set to contain the date and the git sha. This is
useful for when we want to build rpms between releases.
This works analogously to how osbulid-composer currently works.
See <https://fedoraproject.org/wiki/Forge-hosted_projects_packaging_automation>
for details.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This source adds support for downloaded files. The files are
indexed by their content hash, and the only option is their URL.
The main usecase for this will be downloading rpms. Allowing depsolving
to be done outside of osbuild, network access to be restricted and
downloaded rpms to be reused between runs.
Each source is now passed two additional arguments, a cache directory
and an output directory. Both are in the source's namespace, and
the source is responsible for managing them. Each directory may
contain contents from previous runs, but neither is ever guaranteed
to do so.
Downloaded contents may be saved to the cache and resued between
runs, and the requested content should be written to the output dir.
If secrets are used, the source must only ever write contents to
the output that corresponds to the available secrets (rather than
contents from the cache from previous runs).
Each stage is passed an additional argument, a sources directory.
The directory is read-only, and contains a subdirectory named after
each used source, which will contain the requseted contents when
the `Get()` call returns (if the source uses this functionality).
Based on a patch by Lars Karlitski.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The Fedora 31 and Fedora 32 runners are symlinks but the spec file
only looked for files and not symlinks. Fix that.
Follow up by 6a14ba40f7. Found by
Lars.
osbuild currently throws an error when not passing a build environment
on the command line, because the runner is unset. This is annoying on
hosts which only need a runner set, but no build pipeline.
To simplify running osbuild in this common case, introduce
`org.osbuild.host`, which is a runner that is defined to work on the
host that osbuild is installed on. Use this runner by default and
include a symlink to the right runner in the Fedora and RHEL packages.
Also add `runners/org.osbuild.host` to `.gitignore`, so that developers
can set the symlink when running osbuild from the source directory.
Fixes#171
We've been using a generic `osbuild-run`, which sets up the build
environment (and works around bugs) for all build roots. It is already
getting unwieldy, because it tries to detect the OS for some things it
configures. It's also about to cause problems for RHEL, which doesn't
currently support a python3 shebang without having /etc around.
This patch changes the `build` key in a pipeline to not be a pipeline
itself, but an object with `runner` and `pipeline` keys. `pipeline` is
the build pipeline, as before. `runner` is the name of the runner to
use. Runners are programs in the `runners` subdirectory.
Three runners are included in this patch. They're copies of osbuild-run
for now (except some additions for rhel82). The idea is that each of
them only contains the minimal setup code necessary for an OS, and that
we can review what's needed when updating a build root.
Also modify the `--build-pipeline` command line switch to accept such a
build object (instead of a pipeline) and rename it accordingly, to
`--build-env`.
Correspondingly, `OSBUILD_TEST_BUILD_PIPELINE` → `OSBUILD_TEST_BUILD_ENV`.
_libdir is platform dependant, but that is not what we want because we
would need additional runtime logic to handle platforms. this patch
overrides the defautl location
The repository now contains a Vagrantfile for running the testing script
against an RPM package created locally using `make rpm`. To run this
test use `make vagrant-test`. setup.py was also modified to adhere to
packaging guidelines and not to install system-level executables.
The lincense is now included in the Python package using the MANIFEST.in
file.
