We can now send webhook data to an SQS queue at AWS without signing the
request with credentials. This allows us to trigger Schutzbot from
forks and from branches on the main repository.
Signed-off-by: Major Hayden <major@redhat.com>
The org.osbuild.sysconfig stage is now supported. Config updates can be
made to the kernel and network files. Currently, the same values are
used for all image types in rhel84. The image-info script is updated to
allow testing the sysconfig info.
In order to add the newly supported sysconfig stage, the osbuild
dependency needs to be updated to version 24. The version is updated to
24 in the specfile and README. Also, for testing purposes, the Schutzfile
now points to the commit hash for osbuild version 24 for rhel-8.3 tests.
The test case generation script contained a typo in the blueprint
customizations dictionary: "hosname" instead of "hostname".
Fixed typo and regenerated relevant test cases.
Expose a more detailed job status result - specifically, include upload status
alongside image status. Expand openapi.yml accordingly and add an UploadStatus
field to the OSBuildJobResult struct. At the moment, only represent the
"success" and "failure" states of UploadStatus - to differentiate between
"pending" and "running" would involve significant design decisions and should be
addressed in a separate commit.
There is some confusion surrounding the format of the source TOML that
can be sent to the server. The format it accepts doesn't match the
output from composer-cli which includes the source id in [] eg.
[k8s]
name = "kubernetes packages"
...
This patch changes the parsing to allow the id to be set as 'id = "k8s"'
or passed as a map in [k8s]. If the id is passed in the body it takes
precedence over the map name.
The algorithm from calculating changed in osbuild 24, thus breaking the
testsuite. As the the inputhash is merely a implementation detail, there's
no need to test it in the image tests. Nevertheless, the inputhash is also
tested in the osbuild's testsuite.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Due to https://github.com/Azure/azure-storage-blob-go/issues/236 , we had to
use a weird version of this library (see 1b051922).
A new release came out yesterday that's tagged correctly so let's use it
so we can remove the hack from go.mod.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Tom was too quick to merge PR#1178 and as a result, one image test case
was not properly regenerated in that PR. The CI passed for PR#1178,
because RHEL-8.3 images in CI are not tested on aarch64.
Add properly regenerated image test case for rhel-8 aarch64 edge commit.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Remove 'subscription' customization from 'format-request-map.json',
because it currently has no effect on image test cases. The reason is
that this customization of image is possible only via Cloud API in
composer and setting imageOptions. However image test cases currently
support image customization only via Blueprint customizations.
Update repo links in 'repos.json' to the latest rpmrepo snapshots.
Regenerate all image test cases affected by updates to 'repos.json'
and 'format-request-map.json'. This includes all `rhel-8` and `rhel-84`
image test cases.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Blueprint overrides from `format-request-map.json` were previously not applied
when generating image test cases due to a bug. As a result, there were
multiple "blueprint" sections in the generated image test case, if the
image type had an override configured.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The oldest distros we support are Fedora 32 and RHEL 8.3. As both have
Go 1.14, we're safe to upgrade.
Also, I had to change prepare-source.sh because go fmt now refuses to run on
a project which has issues in go.mod, go.sum or modules.text. I think this
should be a harmless change.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
See the comment in tools.go, I cannot fully explain what's happening here.
Somehow, Go 1.14 wants to use the vendored version of oapi-codegen but
without this file, oapi-codegen isn't vendored so the generation fails.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
`image-info` tools parses output of `systemctl list-unit-files` run on a
different tree (with `--root` option), to determine the list of enabled
and disabled services on the inspected image. However since Fedora 33
(and presumably since systemd v246), the output of `systemctl
list-unit-files` changed. Some units previously reported as "enabled" or
"disabled" are now reported as "alias", which means, that they are just
a symlink to a different unit.
There is no systemd command, that would take an "alias" unit and would
report its state as "enabled" or "disabled" and could run on a different
tree (with "--root" option).
To make the list of reported services in the given state consistent on
systems with older and new (v246+) systemd version, check all "alias"
units and append them to the list of services with a specific status,
if their target is also listed in in the list.
Example of the `systemctl list-unit-files` output change:
~]# rpm -q systemd
systemd-246.6-3.fc33.x86_64
~]# systemctl list-unit-files ctrl-alt-del.target
UNIT FILE STATE VENDOR PRESET
ctrl-alt-del.target alias -
~]# rpm -q systemd
systemd-245.8-2.fc32.x86_64
~]# systemctl list-unit-files ctrl-alt-del.target
UNIT FILE STATE VENDOR PRESET
ctrl-alt-del.target enabled disabled
This change makes it possible to produce consistent output for an
inspected image, regardless if the `image-info` tool is run on Fedora
32, Fedora 33 or RHEL-8.
Also regenerate all Fedora 33 test cases, since this commit changes the
content of produced list of enabled / disabled services since Fedora 33.
The list is now consistent with what would be produced by `image-info`
for an image on older Fedora (e.g. 32) or RHEL-8.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `generate-all-test-cases` tool, which generates all defined image
test cases based on provided "distro x arch x image-type" combination
matrix and CLI arguments. Test cases are generated by running the
`generate-test-cases` tool inside an architecture-specific VM.
The tool requires a qcow2 image to be provided for each requested
architecture. These images are used to create ephemeral VMs used
for test case generation.
There appears to be an issue with entropy on all VMs for other
architecture than the host's one. It makes the generation of the first
test case usually fail due to the `go get` command failing with a "TLS
handshake timeout" error. There is a workaround to retry the generation
of a test case a maximum of 3 times in this case. In the usual case,
the test case generation passes on the second try. All subsequent test
case generations should complete successfully on the first try.
Add the `distro-arch-imagetype-map.json` file as the default source of
the "distro x arch x image-type" combination matrix for generating test
cases. Its content is based on the current content of the
`test/data/manifests` directory.
Update `test/README.md` to mention the `generate-all-test-cases` tool.
Add a note, that `generate-test-cases` must be run on a host with the
same architecture, as the intended test case's.
Known issues:
- The tool does not work with RHEL images because of missing "9p"
filesystem support.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Container configuration directory can be set in the .env file which is
automatically used by docker-compose. The default value is the one used
previously (containers/config).
The HACKING.md guide is updated to describe how to set this and how it
affects other commands.
We add both localhost and the internal container names of each service
to the certificates so both VM and containerised setups should work with
the same setup.
System repos cannot be overridden by users, return an error if they try
to push a source with the same name/id as a system source.
Resolves: rhbz#1915359
The last imagefactory nightly did not contain dnf-plugin-spacewalk,
fwupd, nss, or udisks2. These packages are now excluded. The
udisks2.service and mdmonitor.service are no longer enabled. Also, the
fwupd-refresh, mdcheck_continue, mdcheck_start, and mdmonitor-oneshot
timers are no longer listed as disabled services.
The packages in the last imagefactory nightly differ from ours. The
following packages are now added:
oddjob
oddjob-mkhomedir
psmisc
authselect-compat
rng-tools
dbxtool
Also, the rngd and nfs-convert services are enabled.
Adding the tag called `Name` to the AMI ensures that the name appears in
the *Name* column inside AWS' web console.
Fixes#1171.
Signed-off-by: Major Hayden <major@redhat.com>
An image only had a systemd stage added if its blueprint contained
services or if its image type contained enabled services. The systemd
stage is now also added if the image type contains disabled services or
a default target.
The RHEL 8.4 qcow2 image type now specifies the multi-user default target.
In order to test this the image-info tool now includes the default
target in its output. Image test manifests are updated to include this
change.
Send requests to the compose/{id}, compose/{id}/logs, and
compose/{id}/manifests using job IDs for non-finalize type jobs to test
the type verification.
The type verification introduced in the previous commit is now also used
when retrieving the job status (GET /compose/{id}) and the logs (GET
/compose/{id}/logs).
In these cases, job retrieval needs to be performed twice:
1. First the job parameters are retrieved (Job()) to check the type.
2. Then the job result is retrieved (JobStatus()) for the status or
logs.
This makes it unlikely (essentially impossible) that the retrieval will
fail with "not found" on the second retrieval (JobStatus()), but it's
still a good sanity check for the system.
Verifies the Koji job types when retrieving Init and Build jobs as well.
When a job's arguments are retrieved (for the /manifests API endpoint),
the incoming ID should correspond to a Finalize Job. The new
worker.Job() method helps us verify the type and produce an error if the
wrong type is found.
Similarly, the dependencies of a Finalize Job should be in order (Init
Job first followed by Build Jobs). The types are validated while
iterating the dependency list.
Added convenience functions that check the retrieved job type and return
the initialised struct or an error if the ID is not found or does not
match the type.
Currently the getInitJob() function isn't used but it will be useful
later.
