Commit graph

2244 commits

Author SHA1 Message Date
sanne
c53283f5e9 distribution: Change clouddot template to allow for unavailable pods
Because the state directory, which is still currently used, only allows
for a single pod to mount it, allow for 0 available pods when updating
the deployment. Otherwise it will block updating until the state
directory is available for mounting, which will be never.
2021-07-14 22:19:20 +02:00
sanne
d0b5aaae95 distribution: Switch to ConfigMap for configuration
This will allow us to drop vault secrets, and make it easier to
configure composer.
2021-07-14 18:43:34 +02:00
Ondřej Budai
984717628e tests: use rpmrepo for f33 aarch64
This commit also makes aarch64 and x84_64 consistent with each other
and updates the updates repos for x86_64

Fixes #1551

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-14 17:54:24 +02:00
Alexander Todorov
648bf26fca tests: Don't overwrite CI statuses on GitHub branched for nightly pipelines
Closes #1527.
2021-07-14 12:12:48 +02:00
Ondřej Budai
5e8c022e6d docs: document the release process
This commit adds RELEASING.md guide describing the process of making a new
upstream release and pushing it into Fedora and CentOS Stream 9.

Additionally, a new helper is added to the repository - update-distgit.py.
The purpose of this helper is to simplify work with the dist-git.

Random thoughts:
I decided to left off RHEL 8 because the guide would be full of internal
URLs and tools. I will probably write a similar guide for it and put it
into internal guides.

I decided to just reference the RHEL Developer Guide for CentOS Stream 9.
It's pretty well written and I didn't feel like duplicating the effort.

We should definitely switch to PR-based approach for Fedora and implement
at least some smoke tests. I believe that the current guide is good enough
and we can iterate later.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-14 12:11:35 +02:00
Ondřej Budai
f8fd3d04b8 test/fedora33: use rpmrepo instead of kernel mirrors
mirrors.kernel.org are sometimes desynchronized, and thus causing issues with
the image tests. This commit switches the image tests for F33 to use RPMRepo.

All test cases were regenerated.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-14 12:05:45 +02:00
Jakub Rusz
0b962c9185 ci: add slack notfications
This sends a message to a slack channel if a scheduled pipeline is
ran and reports either failure or success.
2021-07-12 11:21:01 +03:00
Brian C. Lane
2b99e9e894 tests: Copy over test repos for RHEL point releases 2021-07-12 08:58:42 +02:00
Brian C. Lane
b5f0400bd4 weldr: Filter the list of supported distributions
Some distributions do not have repositories and therefore cannot be
built. This filters the list of supported distributions by checking for
repos when starting up. All other requests use the api.distros list or
api.getDistro() function.
2021-07-12 08:58:42 +02:00
Brian C. Lane
fabe0d28d2 reporegistry: Add a DistroHasRepos helper function
This returns the repos and the found flag, useful for testing if a
distro has any valid repos.
2021-07-12 08:58:42 +02:00
Brian C. Lane
963ce1d050 weldr: Use host distro name from registry list
The name of the distro you get from distros.FromHost() may not match any of
the names in the registry's list. Use the actual name of the distro
instead of the mangled name.

Also removes api.distro which is unused.
2021-07-12 08:58:42 +02:00
Brian C. Lane
1aa5d49696 docs: Add release note about building for other distros 2021-07-12 08:58:42 +02:00
Brian C. Lane
b0f9d38a95 api_test: Test new source with 2nd distro 2021-07-12 08:58:42 +02:00
Brian C. Lane
f731d7a606 api_test: Test compose with 2nd distribution 2021-07-12 08:58:42 +02:00
Brian C. Lane
788daf91a4 api_test: Test projects/depsolve with 2nd distribution 2021-07-12 08:58:42 +02:00
Brian C. Lane
e010e2c421 api_test: Test modules/info with 2nd distribution 2021-07-12 08:58:42 +02:00
Brian C. Lane
2ad4b84af6 api_test: Test projects/info with 2nd distribution 2021-07-12 08:58:42 +02:00
Brian C. Lane
c9f4b1ac8b api_test: Test modules/list with 2nd distribution 2021-07-12 08:58:42 +02:00
Brian C. Lane
05f327c796 api_test: Test projects/list with 2nd distribution 2021-07-12 08:58:42 +02:00
Brian C. Lane
43171b4169 api_test: Test compose types with 2nd distribution 2021-07-12 08:58:42 +02:00
Brian C. Lane
5021ef06f7 Add 2nd distribution to the NewTestAPI function
This will help test support for multiple distributions.
2021-07-12 08:58:42 +02:00
Brian C. Lane
c1da403dc0 weldr: Use GetAllDistroSources in allRepository* functions 2021-07-12 08:58:42 +02:00
Brian C. Lane
76e65e88c8 store: Add GetAllDistroSources function 2021-07-12 08:58:42 +02:00
Brian C. Lane
e2b170e754 weldr: Check source POST for valid distros 2021-07-12 08:58:42 +02:00
Brian C. Lane
2b56e4d8e9 store: Add Distros to Source structs 2021-07-12 08:58:42 +02:00
Brian C. Lane
f79f399808 weldr: Add optional distro selection to compose/types 2021-07-12 08:58:42 +02:00
Brian C. Lane
f233f540e5 weldr: Fix PROJECTS_ERROR response 2021-07-12 08:58:42 +02:00
Brian C. Lane
881e1c5652 weldr: Add optional distro selection to projects/depsolve route 2021-07-12 08:58:42 +02:00
Brian C. Lane
2b63b3cbed weldr: Add optional distro selection to modules and projects/info route 2021-07-12 08:58:42 +02:00
Brian C. Lane
b98cf3c4f7 weldr: Add optional distro selection to modules/list route 2021-07-12 08:58:42 +02:00
Brian C. Lane
d1d2f6d491 weldr: Add optional distro selection to projects/list route 2021-07-12 08:58:42 +02:00
Brian C. Lane
453d2dfd5f weldr: Select the distribution specific ImageType
This uses the image type based on the distribution selected by the
blueprint, or the host distro if none is present. This enables compose
to build images for the selected distribution.

It adds a helper, getImageType(), to return the ImageType based on the
distro name and compose type.
2021-07-12 08:58:42 +02:00
Brian C. Lane
dd6a7550f1 weldr: Add distribution selection allRepositories and its callers
And use it for blueprints/depsolve results.
2021-07-12 08:58:42 +02:00
Brian C. Lane
d63dd09686 weldr: Setup NewTestAPI correctly
The host distro needs to be passed to New in the first position, AND
second so that it ends up in the distro map. Without this
distros.GetDistro() will fail because it cannot lookup the host distro
name.
2021-07-12 08:58:42 +02:00
Brian C. Lane
3caa6ba24d weldr: Check blueprint POST distro or set it to host 2021-07-12 08:58:42 +02:00
Brian C. Lane
b476078570 Move isStringInSlice to common.IsStringInSlice 2021-07-12 08:58:42 +02:00
Brian C. Lane
1abdd9a1f7 weldr: Add distro field to blueprints
An optional distribution name can be included with the blueprint. If is
is not then the blueprint will be depsolved/built using the current host
distribution.

depsolveBlueprint and depsolveBlueprintForImageType check for the empty
Distro name and set it to the host distro before using it. The function
signatures have also been changed to use the value instead of a pointer
so that changes don't effect anything outside the depsolve function.
2021-07-12 08:58:42 +02:00
Brian C. Lane
aa54fe842f weldr: Add distros/list route
This returns the list of supported distributions as a list of strings.
eg.

{
    distros: ["fedora-33", "fedora-34", "fedora-35"]
}
2021-07-12 08:58:42 +02:00
Brian C. Lane
9818b4b6b1 osbuild-composer: Move InitWeldr code into weldr.New
This will make it easier to support new features related to building for
other distribution releases.
2021-07-12 08:58:42 +02:00
Brian C. Lane
a330d84739 reporegistry: Make ReposByArchName function public
This will be used to select distributions that do not match the host
distro.
2021-07-12 08:58:42 +02:00
Jakub Rusz
1b5ca0f163 tests: make azure.sh easier to run locally
We've dropped Jenkins already and this change helps by setting
CI variables to a default value in case they're not present.
2021-07-09 19:52:52 +01:00
Jakub Rusz
4994a858c3 tests: Define speccific runners for libvirt.sh
Libvirt.sh tests are failing without kvm support on centos-stream-8
and on RHEL-8.5. More in rhbz#1975242 and issue #1454
2021-07-09 10:10:33 +02:00
Achilleas Koutsou
5fe1ae41b1 schutzbot: akoutsou ssh keys 2021-07-09 09:55:40 +02:00
sanne
4385c39d66 worker: Introduce heartbeats
An occupied worker checks about every 15 seconds if it's current job was
cancelled. Use this to introduce a heartbeat mechanism, where if
composer hasn't heard from the worker in 2 minutes, the job times out
and is set to fail.
2021-07-08 21:14:38 +01:00
sanne
0fcb44e617 worker: Move job tokens to the queue itself
This removes state from the worker server, as it no longer contains the
list of running jobs. Instead only the queue knows if jobs are running
or not.
2021-07-08 21:14:38 +01:00
Tomas Hozza
34d52aa8e1 osbuild2: add support for org.osbuild.authselect stage
Add support for the `org.osbuild.authselect` osbuild stage [1],
which allows one to set system identity profile and authentication
sources using `authselect`.

Add unit test cases for the newly added stage.

[1] https://github.com/osbuild/osbuild/pull/696

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-08 19:37:09 +02:00
Tomas Hozza
006ff98025 osbuild2: support for x11-keymap option in org.osbuild.keymap
Add support for the newly added `x11-keymap` option in the
`org.osbuild.keymap` osbuild stage [1]. The option allows one to
set layouts for the X11 keyboard.

Add unit test cases for the added functionality.

[1] https://github.com/osbuild/osbuild/pull/693

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-08 19:37:09 +02:00
Tomas Hozza
f06e66b94b osbuild2: support for servers option in org.osbuild.chrony
Add support for the newly added `servers` option in the
`org.osbuild.chrony` osbuild stage [1]. The option allows one to
specify timeservers to be used by chrony, including a subset of
lower-level configuration options per each server.

Implement a custom JSON marshaller method for `ChronyStageOptions` to
ensure that exactly one of 'timeservers' or 'servers' is specified, as
mandated by the stage schema.

Optional values in `ChronyConfigServer` are declared as pointers to
distinguish the case when the value was explicitly set by the user from
the default value when the structure instance is created. All of these
options should be omitted from the JSON, but only when not explicitly
set, not when their value us "0" for int or "false" for bool. Downside
of this approach is that one can not easily use pointer to a basic
type literal in the struct literal. Passing the basic type literal has
to be workarounded using an intermediate variable, which address is
used in the struct literal.

Add unit test cases for the added functionality.

[1] https://github.com/osbuild/osbuild/pull/692

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-08 19:37:09 +02:00
Tomas Hozza
8367542f25 osbuild2: support for subscription-manager option in org.osbuild.rhsm
Add support for the newly added `subscription-manager` option in the
`org.osbuild.rhsm` osbuild stage [1]. The option allows one to
create configure subset of subscription-manager options in its
configuration file `/etc/rhsm/rhsm.conf`.

Add unit test cases for the added functionality.

[1] https://github.com/osbuild/osbuild/pull/659

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-08 19:37:09 +02:00
Tomas Hozza
29ade764ce osbuild2: support for unit-dropins option in org.osbuild.systemd
Add support for the newly added `unit-dropins` option in the
`org.osbuild.systemd` osbuild stage [1]. The stage allows one to
create `.service` unit files drop-in configuration files under
`/usr/lib/systemd/system/`.

Add unit test cases for the added functionality.

[1] https://github.com/osbuild/osbuild/pull/664

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-08 19:37:09 +02:00