Commit graph

3736 commits

Author SHA1 Message Date
Xiaofeng Wang
75a3426515 test: Add a new user in installer bp and test new user created 2022-03-28 14:08:57 +02:00
Achilleas Koutsou
14aa3b9f54 test: save installer test manifest and build log
Same as with ostree tests.
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
5825294dad distro/rhel86+90: enable the anaconda user modeule for image-installer
Enable the user module unconditionally for the image-installer:
- If users are specified for the kickstart file, the module is required
  to set up the users.
- If no users are specified, the module can be used at install time to
  create users.

Updated relevant test cases (manifests).
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
fe514b9c2e test: create new test cases (manifests)
New test cases for edge- and image-installer with users.
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
3ac1f940a4 format-request-map: add installers with users
Add two new test cases:
- image-installer-with-users
- edge-installer-with-users
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
ebc3330cbd distro/rhel86+90: don't create users in the payload for image-installer
Users are created at install time now.
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
974b0a434b osbuild2: deduplicate anacondaStageOptions()
Use single NewAnacondaStageOptions() from osbuild2 instead of
implementing in each distro.

The new function conditionally adds the user module when there are users
that need to be created at install time (image- and edge-installers).
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
cb186df208 obsuild2: deduplicate *kickstartStageOptions()
Use single NewKickstartStageOptions() and replace image-type-specific
implementations from each distro.
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
2eef6e6e2d osbuild2: deduplicate groupStageOptions()
Use single NewGroupsStageOptions() from osbuild2 instead of implementing
in each distro.

The new function does not set the Group.Name field anymore.  The field
does not exist in the osbuild schema and was silently ignored.
The field in the stage has been marked 'omitempty' and the relevant
manifests have been updated.
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
ca8b371142 osbuild2: deduplicate userStageOptions()
Use single NewUsersStageOptions() from osbuild2 instead of implementing
in each distro.
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
d022a23b8c osbuild2: add Users and Groups to kickstart stage
Supported by the osbuild stage for a while now.
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
db2910fed9 distro/rhel86+90: common error message for unsupported customizations
Use the same error message for both edge-installer and
edge-simplified-installer when an unsupported customization is added.
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
3a1765a5a8 distro/rhel90: allow user and group customizations for edge-installer 2022-03-28 14:08:57 +02:00
Achilleas Koutsou
f34380d5b5 distro/rhel86: allow user and group customizations for edge-installer 2022-03-28 14:08:57 +02:00
Sanne Raymaekers
11890682b7 templates/composer: Drop unused variables 2022-03-28 12:02:37 +02:00
Sanne Raymaekers
eba355bb60 templates/composer: Remove unused acl claims
This leaves fedora and consoledot tenants.
2022-03-28 11:38:48 +02:00
Alexander Todorov
7ea3920cdf Add Xiaofeng to notifications for nightly pipeline 2022-03-28 11:30:44 +02:00
Diaa Sami
68639b4bf9 koji: increment retry counter only when retrying 2022-03-26 09:33:36 +01:00
Diaa Sami
6b08b8ed63 koji: don't decrement retry counter on the first call
After examining the logic of retryablehttp library, the callback does not happen for the first HTTP call, so no need to decrement when counting.
2022-03-26 09:33:36 +01:00
Diaa Sami
3496efe70d koji: initialize retryable client properly
Previously used client has MaxRetries of zero, so was not effectively
retrying
Fixes COMPOSER-1420
2022-03-26 09:33:36 +01:00
Jakub Rusz
f1006c5adb ci/tests: re-enable Installers.sh test
Related bug has been fixed. Updating snapshots and enabling the test.
2022-03-25 16:00:39 +01:00
Jakub Rusz
b639a44899 tests/ostree-simplified-installer: destroy VM when installation is stuck
This test get stuck randomly on centos-stream-8 and is making the CI
unreliable. Adding hard wait limit and destroying the VM afterwards
helps the test get unstuck and continue as expected. See
https://github.com/osbuild/osbuild-composer/issues/2413 for details.
2022-03-25 16:00:39 +01:00
Jakub Rusz
fbefd432c1 Schutzfile: Update centos-stream-9 repos 2022-03-25 16:00:39 +01:00
Jakub Rusz
417c381b44 Schutzfile: fix centos-stream-9 repo pin
During runner repo override the runner name is compared with the key in
Schutzfile. There was the word 'stream' missing for centos runners so
the repos were never acutally overwritten before. See
https://github.com/osbuild/gitlab-ci-terraform-executor/blob/main/prepare#L69
for more context
2022-03-25 16:00:39 +01:00
Jakub Rusz
8390709614 ci/details: list system repositories and run in after_script too
It is good to know what system repositories are used exactly on the
runner. Also running the ci_details.sh in after_script can help during
any debugging as packages that got updated or installed during the run
can be easily identified.
2022-03-25 16:00:39 +01:00
Ondřej Budai
fc86ffd968 container: fix liveness probe
We don't have permissions to write to /run when running on OpenShift so let's
just use /tmp and change the filename to prevent any conflicts.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-25 14:02:12 +01:00
Ondřej Budai
7a194bfcb5 cloudapi: require gpgkey if check_gpg is set
If a user requires that packages from a certain repository are checked using
a GPG key, they should specify it. Now, this is enforced to catch this issue
earlier than in osbuild.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-25 13:23:32 +01:00
Ondřej Budai
af44202b1c cloudapi: rename gpg_key field to gpgkey
Oh no, we made a mistake here: Both our json repositories and repo files in
/etc/yum.repos.d have the GPG key in a field named `gpgkey`. Unfortunately,
cloudapi uses a field named `gpg_key`. One consequence of this issue is that
our api.sh test is meant to pass GPG keys in the compose request but since
it's using a bad field name (`gpgkey`), the key is actually not used.

I've decided to fix this in cloudapi: The `gpg_key` field is now renamed to
`gpgkey`. This is a breaking change but no one is using this API anyway so
we think it's better to do this now than introducing weird backward
compatible hacks.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-25 13:23:32 +01:00
Ondřej Budai
80af9cac98 cloudapi: enable edge-commit on Fedora
We have to do a small hack to enable edge-commit on Fedora because its name
is different. We can also change this in the image definition but I want to
iterate quickly on the Fedora Integration MVP and don't want to run in
any conflicts with
https://github.com/osbuild/osbuild-composer/pull/2461

This commit also enables a test for Fedora IoT built through the API.

While enabling the test, I also simplified our decision logic for SSH_USER
and DISTRO.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-25 13:23:32 +01:00
Antonio Murdaca
bf46113251 test/data/repositories: bump cs8
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-03-24 18:23:39 +01:00
Jakub Rusz
7bf019efcb test/repositories: update rhel-90 repository gpg keys
SHA-1 keys are deprecated, updating to the newly used ones.
2022-03-24 18:23:39 +01:00
yih
cd0f762e58 add fdo sshkey and rootcert test 2022-03-24 18:23:39 +01:00
Antonio Murdaca
66bc1e8b85 rhel8/9: fix path to fdo diun root certificates
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-03-24 18:23:39 +01:00
Tomas Hozza
2ec55b4c53 RHEL-90: use XBOOTLDR partition GUID for /boot
Use the Extended Bootloader Partition GUID for `/boot`, instead of the
Linux filesystem data GUID. This is useful for autodetection of a
partition purpose based on its GUID without reading the `/etc/fstab`
first.

Ensure that when creating mountpoints, e.g. when converting the
partitions layout to LVM, the `/boot` partition get the proper GUID
assigned.

Regenerate RHEL-90 and centos-9 image test cases.

Related to https://bugzilla.redhat.com/show_bug.cgi?id=2057231
2022-03-24 15:40:10 +01:00
Sanne Raymaekers
9368b60401 templates/composer: Add prod service accounts owner 2022-03-23 16:43:10 +01:00
schutzbot
3571447fc4 Post release version bump
[skip ci]
2022-03-23 14:30:52 +00:00
Simon Steinbeiss
11593b8318 create-tag: Set the release time to 13UTC 2022-03-23 13:57:56 +01:00
Simon Steinbeiss
53cb872aca create-tag: Don't check week for manual triggers
When a release is manually triggered we want to ignore whether it is an
odd or even week but allow the release in any case.
2022-03-23 13:57:56 +01:00
Simon Steinbeiss
64be96017f create-tag: Fix bash typo
This seems to be a bit special in GitHub Actions with bash:
https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-environment-variable
2022-03-23 13:57:56 +01:00
Tomas Hozza
60db6ad06f image tests: update GPG keys used for RHEL-9.0 repos
SHA-1 key is no longer supported, update it with a new one from
redhat-release.

Related rhbz#2058497

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-03-23 13:22:13 +01:00
Simon Steinbeiss
86bf942780 create-tag: Fix upstream release schedule
We're counting the even and odd weeks since the Unix epoch, so this
doesn't match the odd/even calendar weeks. Consequently let's switch to
odd weeks for composer.
2022-03-23 11:30:32 +01:00
Tomas Hozza
d11a2f3bc7 api.sh: encrypt the DB dump artifact 2022-03-23 09:45:09 +01:00
Simon Steinbeiss
f521e9a800 Enable scheduled upstream releases
Instead of a human pushing a tag with the release notes let a bot do the
work.
The bot is part of our composite action in osbuild/release-action on the
create-tag branch. It calculates the next subsequent release version and
creates a tag based on pull request titles associated with the changes
since the last release.
Finally the tag is pushed to the repository.

Unfortunately GH Actions don't allow for reliably fortnightly schedules,
so we do an additional check that determines if this is an even or an
odd week. This will help with correctly scheduling alternating osbuild
and osbuild-composer releases.
2022-03-22 17:50:57 +01:00
Ondřej Budai
e9ce9370c6 dbjoqbqueue: actually use the transaction object when a tx is created
Transactions are tied to a connection so this is actually not a functional
change. Nevertheless, I think it's nice to explicitly state that we are
using a transaction.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-22 17:49:22 +01:00
Ondřej Budai
187eb188da dbjoqbqueue: wait for listener to become ready before returning from New
Otherwise, there might be an already waiting dequeuer and if something is
enqueued before `sqlListen` is called, we will lost this notification.

Also, a small log message was added when shutting down the listener.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-22 17:49:22 +01:00
Tom Gundersen
c21596cd99 containers/composer: forward SIGTERM to children
In docker compose and openshift SIGTERM is used to tell containers
to shut down gracefully. Forward the signal to the processes we spawned.
2022-03-22 14:17:37 +01:00
Tom Gundersen
367444635a containers/composer: terminate composer first
Composer may depend on dnf-json and the worker to shut down cleanly.
2022-03-22 14:17:37 +01:00
Tom Gundersen
c3d66b5a33 cmd/composer: gracefully shut down on SIG{INT,TERM}
Call `Shutdown()` on all http servers. This means we will finish processing
any pending requests (including depsolving), but we will not listen to new
ones.

In particular, we will not answer to the readiness probe, so no new traffic
will be routed to this container.

Once all pending requests have been handled composer will shut down
gracefully and the liveness probe will return failure.

Note that in order for this to work correctly no requests should ever take longer
than the shutdown timeout (by default 30s).
2022-03-22 14:17:37 +01:00
Tom Gundersen
d3cd3197c0 container: make liveness probe independent of webserver
Currently liveness and readiness was treated the same. However, their
behaviour at shutdown is meant to be different. When a service is not read
no new connections are made to it, and when a service is not live it can be
cleaned up.

By considering our service live if and only if it listens to HTTP requests we
don't have the opportunity to clean up after we stop listening to new requests.

Leave readiness probes as they are, and instead use a file in the filesystem to
indicate when the service is live. It is created before composer is spawned and
deleted once composer exits.
2022-03-22 14:17:37 +01:00
Jakub Rusz
15c2044b3c tests/upgrade: update gpg key
We need to use a new gpg key after the SHA-1 deprecation. Also don't
fail immediately on compose failure to be able to retrieve logs from the
test VM.
2022-03-22 10:54:30 +01:00