Commit graph

4399 commits

Author SHA1 Message Date
Christian Kellner
59edcc4a46 distro/fedora: only pass kernelName to NewOSPipeline for bootable images
If the image is not bootable it means that we do not need the kernel
version and thus we should not pass it along to `NewOSPipeline`,
because we might not even have a kernel in the package set.
2022-06-29 10:02:46 +02:00
Christian Kellner
49559d53d7 pipeline/os: only infer kernel version if name is not empty
Do not try to infer the kernel verison if `kernelName` is the
empty string.
2022-06-29 10:02:46 +02:00
Christian Kellner
776dfd2da6 distro: option to exclude docs during installation
We don't ship documentation in container images, add an option to
ImageConfig to disable the installation of docs during installation.
2022-06-29 10:02:46 +02:00
Christian Kellner
f37a791bd2 distro: new image config to disable selinux
We don't need to label container artifacts, so include an option
in the image config to disable it.
NB: This is a boolean that with the default false in order to
not have to change every existing image configuration.
2022-06-29 10:02:46 +02:00
Christian Kellner
4b7dc3bbec pipelines/os: new option to exclude documentation
Add a new `ExcludeDocs` to control the if documentation is included
during the package installation.
2022-06-29 10:02:46 +02:00
Christian Kellner
d88bde8ab7 pipelines/os: new option to control selinux labeling
Make the SELinux labeling of the tree optional via a `SELinux`
variable.
2022-06-29 10:02:46 +02:00
Christian Kellner
516f30de17 distro: new blueprint helper where kernel package is optional
Add a new extended helper `GetPackagesEx` that includes a flag
to specify if the target is bootlable or not and only include
the kernel package in case it is set to true. Delegate to this
new helper from the existing `GetPackages`.
2022-06-29 10:02:46 +02:00
Christian Kellner
986f076276 container: add support for uploading to registries
Add a new generic container registry client via a new `container`
package. Use this to create a command line utility as well as a
new upload target for container registries.

The code uses the github.com/containers/* project and packages to
interact with container registires that is also used by skopeo,
podman et al. One if the dependencies is `proglottis/gpgme` that
is using cgo to bind libgpgme, so we have to add the corresponding
devel package to the BuildRequires as well as installing it on CI.

Checks will follow later via an integration test.
2022-06-29 10:02:46 +02:00
Jakub Rusz
d136a075bc manifests: regenerate RHEL-8.6 qcow2 test cases 2022-06-29 10:15:50 +03:00
Ondřej Budai
c6c401a322 worker/config: add tests for parseConfig
Tests are always nice, aren't they?

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-06-28 15:55:31 +01:00
Ondřej Budai
9af6d1d43b worker: make default config values more idiomatic
The only functional change is that
base_path = ""

will be now parsed as:

config.BasePath == ""

which wasn't possible before.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-06-28 15:55:31 +01:00
Ondřej Budai
dc78b05a19 worker: flip error handling when parsing the config
It's more idiomatic this way.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-06-28 15:55:31 +01:00
Ondřej Budai
c7716c2072 worker/config: extract all subtypes
...to make this testable.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-06-28 15:55:31 +01:00
Ondřej Budai
04f053689d worker: factor out config parsing
The struct is factored out 1:1. The only functional change in this commit is
worker now logging in case of a missing config (which means just loading the
defaults).

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-06-28 15:55:31 +01:00
Ondřej Budai
e4a66b0373 worker: clean up the RelaxTimeoutFactor
This commit moves the field to the koji struct where it actually belongs.
Also, it renames it to relax_timeout_factor for the sake of consistency.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-06-28 15:55:31 +01:00
Ondřej Budai
3b32480d45 worker: factor out kojiServer struct
In order to match the config structure.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-06-28 15:55:31 +01:00
Ondřej Budai
8666abc4ba worker: add a forgotten toml tag
The TOML library translates the field names 1:1, so now you have to use:

[Composer]
proxy: "abcd"

This is not idiomatic though so let's add the toml tag to make it [composer].

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-06-28 15:55:31 +01:00
Juan Abia
fa38246575 remove test skip
COMPOSER-1397 temporarily skipped azure-rhui image. Now that we test on
rhel 8.7, this skip can be removed.
2022-06-28 15:23:52 +02:00
dependabot[bot]
2c6e28b01d build(deps): bump github.com/aws/aws-sdk-go from 1.44.4 to 1.44.43
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.4 to 1.44.43.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.4...v1.44.43)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-28 14:25:12 +02:00
Chloe Kaubisch
765e93bfc8 cloudapi: get specific error from openapi
Error handling is structured in such a way that typically, a ServiceCodeError is passed
through the echo HTTP error, in reference to internally defined errors. We want to be able
to obtain and return specific external errors, for example during validation from openapi3.
Add a 'details' field to the serviceError struct, to contain extra / externally defined
information. Modify HTTPErrorHandler to anticipate either a string or a ServiceErrorCode
from echo, and respond accordingly. Edit the affected tests to expect the appropriate response.
2022-06-28 10:02:55 +02:00
Achilleas Koutsou
d59d870574 dnf-json: fix depsolve error handling
When a DepsolveError exception occurs, the error message would print the
packages in the request.  When the request arguments changed, the error
message handling wasn't updated and would fail to produce the correct
error message.

Compile a list of packages from all transactions and print them in the
error message as a comma-separated list.
2022-06-27 20:41:34 +02:00
Tom Gundersen
be2195b235 pipeline: add a bit more documentation
Make sure that each of the types have at least a bit of documentation.
2022-06-27 19:11:26 +01:00
Tom Gundersen
4556312d22 pipeline: split qcow2 into format specific pipelines
Stages are procedural and named after the tool they wrap, but pipelines are declarative and should
be named after the kind of artefact they produce.

This splits the qemu (the tool) pipeline into qcow2, vmdk, and vpc (the formats) pipelines. In theory
we may have wanted to implemented through some shared helpers, but for now it seems trivial
enough that it is not worth it.
2022-06-27 19:11:26 +01:00
Tom Gundersen
1fa92f9091 pipeline: distinguish between optional and mandatory properties
The ideal is that the constructor takes mandatory properties as arguments, and fields in the struct
are all optional.

This clarifies that across the pipelines (or leaves TODOs where work remains), and where possible
makes fields optional by providing a valid default value.
2022-06-27 19:11:26 +01:00
Tom Gundersen
c001af63ec pipeline: further cleanups
This adds more documentation and makes more properties implicitly inherited rather than
repeated. This makes for less boilerplate, and gives us fewer things to keep in sync.
2022-06-27 19:11:26 +01:00
Tom Gundersen
e5ef7933ac pipeline: drop stage_options.go
This was barely used. Making it explicit seemed more readable.
2022-06-27 19:11:26 +01:00
Tom Gundersen
f60092033b pipelines: make OSPipeline own the PartitionTable
The OSPipeline might need to know what disk layout it will be put onto, enforce this by making
the PartitionTable a property of the OSPipeline, and require child pipelines to query it when needed.
2022-06-27 19:11:26 +01:00
Tom Gundersen
ae34513d18 pipeline: move package to top level
The pipeline package is exists conceptually between the distro and the osbuild packages, so move
it to the top level rather than as a child of distro.

No functional change.
2022-06-27 19:11:26 +01:00
Tom Gundersen
e024ceb138 pipelines: cleanups
This is a collection of minor cleanups:
 - Start documenting the API
 - Enforce dependent pipelines have the correct type where necessary
 - Use data from dependent pipelines where possible
 - Start enforcing required fields
 - Move logic into the pipeline implementation where we can
2022-06-27 19:11:26 +01:00
Tom Gundersen
040a4ef6a1 pipelines: add Fedora IoT installer pipelines 2022-06-27 19:11:26 +01:00
Tom Gundersen
284eb9af59 distro/fedora: drop dead code 2022-06-27 19:11:26 +01:00
Tom Gundersen
0f015801d7 pipelines: add Fedora IoT commit and container pipelines 2022-06-27 19:11:26 +01:00
Tom Gundersen
1243f84cb0 pipelines: add fedora liveimg and qemu pipelines 2022-06-27 19:11:26 +01:00
Tom Gundersen
562a5b1127 pipelines: add fedora OS pipeline
This pulls out the OS pipeline, without changing the parameters. The dependency
between the OS pipeline and build pipeline, is now explicit, rather than by name.
2022-06-27 19:11:26 +01:00
Tom Gundersen
b6d6626a5d pipelines: introduce pipeline abstractions
Create a new package to encapsulate pipelines. This introduces only the build
pipeline and uses it in fedora.

No functional change.
2022-06-27 19:11:26 +01:00
Tom Gundersen
b8815aab4c distro/fedora: avoid double import
Fix typo. No functional chnage.
2022-06-27 19:11:26 +01:00
Ygal Blum
534625fa38 Fedora - Use vendor instead of rpm dependencies
RPM Spec
--------
Remove all Go dependecies
Add Start and End marker comments for bundling information
Add '-k' to goprep to preserve the vendor directory

tools
-----
Add script to update the RPM spec file to generate the indication lines
based on vendor/modules.txt

Packit
------
Run the new script as a post-upstream-clone hook

Makefile
--------
Run the new script on the generated spec file before generating the RPM

mockbuild.sh
------------
Run the new script before creating the RPM
2022-06-23 18:02:09 +02:00
Jakub Rusz
8fef39c4cf rpmbuild: add fedora-36 2022-06-23 12:30:20 +02:00
Achilleas Koutsou
365e1d109c distro/rhel7: remove vhd image type
1. Broken: specifies packages that aren't available in the repositories
   (langpacks-en and dhcp-client).
2. Not needed.
2022-06-23 10:12:35 +02:00
Christian Kellner
292ffd59a0 cmd/osbuild-upload-gcp: fix typo in skip-import's help string 2022-06-22 12:37:34 +02:00
Jakub Rusz
0201b0bff9 tests: Update compose version in Upgrade test 2022-06-21 10:40:58 +02:00
Jakub Rusz
2eee0632d6 manifests: regenerate rhel-91-gce-boot
We don't want to do gpg check for the google repositories.
2022-06-21 10:40:58 +02:00
Jakub Rusz
70688173bb tests: update RHEL version in unsupported tests 2022-06-21 10:40:58 +02:00
Jakub Rusz
81c66b5128 CI: move centos out of RHEL runners
We have this lits of separate RHEL runners only to build test rpms
during nightly pipelines.
2022-06-21 10:40:58 +02:00
Jakub Rusz
eb78fcd85e tests/koji: build latest RHEL when running on RHEL
Based on
aff680542d
this test is supposed to always build the latest available compose.
2022-06-21 10:40:58 +02:00
Jakub Rusz
5fcf0b1d53 tests/ostree: Update to support new RHEL nightlies 2022-06-21 10:40:58 +02:00
Jakub Rusz
1829ff24ea Update terraform SHA 2022-06-21 10:40:58 +02:00
Jakub Rusz
d1c6b46ac3 Schutzfile: update snapshots and osbuild commits
New osbuild commmit SHA and newer rpmrepo snapshots
osbuild commit: stages/mkfs.fat: pass -I command line option
2022-06-21 10:40:58 +02:00
Jakub Rusz
d87c60857c mockbuild: update mockbuild script after RHEL-9 GA
Removing rhel-9 templates as they are shipped upstream already and
making necessary modifications to mockbuild.sh
2022-06-21 10:40:58 +02:00
Alexander Todorov
0fd77f4594 Update osbuild commit 2022-06-21 10:40:58 +02:00