The oldest distros we support are Fedora 32 and RHEL 8.3. As both have
Go 1.14, we're safe to upgrade.
Also, I had to change prepare-source.sh because go fmt now refuses to run on
a project which has issues in go.mod, go.sum or modules.text. I think this
should be a harmless change.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
See the comment in tools.go, I cannot fully explain what's happening here.
Somehow, Go 1.14 wants to use the vendored version of oapi-codegen but
without this file, oapi-codegen isn't vendored so the generation fails.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
`image-info` tools parses output of `systemctl list-unit-files` run on a
different tree (with `--root` option), to determine the list of enabled
and disabled services on the inspected image. However since Fedora 33
(and presumably since systemd v246), the output of `systemctl
list-unit-files` changed. Some units previously reported as "enabled" or
"disabled" are now reported as "alias", which means, that they are just
a symlink to a different unit.
There is no systemd command, that would take an "alias" unit and would
report its state as "enabled" or "disabled" and could run on a different
tree (with "--root" option).
To make the list of reported services in the given state consistent on
systems with older and new (v246+) systemd version, check all "alias"
units and append them to the list of services with a specific status,
if their target is also listed in in the list.
Example of the `systemctl list-unit-files` output change:
~]# rpm -q systemd
systemd-246.6-3.fc33.x86_64
~]# systemctl list-unit-files ctrl-alt-del.target
UNIT FILE STATE VENDOR PRESET
ctrl-alt-del.target alias -
~]# rpm -q systemd
systemd-245.8-2.fc32.x86_64
~]# systemctl list-unit-files ctrl-alt-del.target
UNIT FILE STATE VENDOR PRESET
ctrl-alt-del.target enabled disabled
This change makes it possible to produce consistent output for an
inspected image, regardless if the `image-info` tool is run on Fedora
32, Fedora 33 or RHEL-8.
Also regenerate all Fedora 33 test cases, since this commit changes the
content of produced list of enabled / disabled services since Fedora 33.
The list is now consistent with what would be produced by `image-info`
for an image on older Fedora (e.g. 32) or RHEL-8.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `generate-all-test-cases` tool, which generates all defined image
test cases based on provided "distro x arch x image-type" combination
matrix and CLI arguments. Test cases are generated by running the
`generate-test-cases` tool inside an architecture-specific VM.
The tool requires a qcow2 image to be provided for each requested
architecture. These images are used to create ephemeral VMs used
for test case generation.
There appears to be an issue with entropy on all VMs for other
architecture than the host's one. It makes the generation of the first
test case usually fail due to the `go get` command failing with a "TLS
handshake timeout" error. There is a workaround to retry the generation
of a test case a maximum of 3 times in this case. In the usual case,
the test case generation passes on the second try. All subsequent test
case generations should complete successfully on the first try.
Add the `distro-arch-imagetype-map.json` file as the default source of
the "distro x arch x image-type" combination matrix for generating test
cases. Its content is based on the current content of the
`test/data/manifests` directory.
Update `test/README.md` to mention the `generate-all-test-cases` tool.
Add a note, that `generate-test-cases` must be run on a host with the
same architecture, as the intended test case's.
Known issues:
- The tool does not work with RHEL images because of missing "9p"
filesystem support.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Container configuration directory can be set in the .env file which is
automatically used by docker-compose. The default value is the one used
previously (containers/config).
The HACKING.md guide is updated to describe how to set this and how it
affects other commands.
We add both localhost and the internal container names of each service
to the certificates so both VM and containerised setups should work with
the same setup.
System repos cannot be overridden by users, return an error if they try
to push a source with the same name/id as a system source.
Resolves: rhbz#1915359
The last imagefactory nightly did not contain dnf-plugin-spacewalk,
fwupd, nss, or udisks2. These packages are now excluded. The
udisks2.service and mdmonitor.service are no longer enabled. Also, the
fwupd-refresh, mdcheck_continue, mdcheck_start, and mdmonitor-oneshot
timers are no longer listed as disabled services.
The packages in the last imagefactory nightly differ from ours. The
following packages are now added:
oddjob
oddjob-mkhomedir
psmisc
authselect-compat
rng-tools
dbxtool
Also, the rngd and nfs-convert services are enabled.
Adding the tag called `Name` to the AMI ensures that the name appears in
the *Name* column inside AWS' web console.
Fixes#1171.
Signed-off-by: Major Hayden <major@redhat.com>
An image only had a systemd stage added if its blueprint contained
services or if its image type contained enabled services. The systemd
stage is now also added if the image type contains disabled services or
a default target.
The RHEL 8.4 qcow2 image type now specifies the multi-user default target.
In order to test this the image-info tool now includes the default
target in its output. Image test manifests are updated to include this
change.
Send requests to the compose/{id}, compose/{id}/logs, and
compose/{id}/manifests using job IDs for non-finalize type jobs to test
the type verification.
The type verification introduced in the previous commit is now also used
when retrieving the job status (GET /compose/{id}) and the logs (GET
/compose/{id}/logs).
In these cases, job retrieval needs to be performed twice:
1. First the job parameters are retrieved (Job()) to check the type.
2. Then the job result is retrieved (JobStatus()) for the status or
logs.
This makes it unlikely (essentially impossible) that the retrieval will
fail with "not found" on the second retrieval (JobStatus()), but it's
still a good sanity check for the system.
Verifies the Koji job types when retrieving Init and Build jobs as well.
When a job's arguments are retrieved (for the /manifests API endpoint),
the incoming ID should correspond to a Finalize Job. The new
worker.Job() method helps us verify the type and produce an error if the
wrong type is found.
Similarly, the dependencies of a Finalize Job should be in order (Init
Job first followed by Build Jobs). The types are validated while
iterating the dependency list.
Added convenience functions that check the retrieved job type and return
the initialised struct or an error if the ID is not found or does not
match the type.
Currently the getInitJob() function isn't used but it will be useful
later.
JobArgs() function replaced with more general Job() function that
returns all the parameters used to originally define a job during
Enqueue(). This new function enables access to the type of a job in the
queue, which wasn't available until now (except when Dequeueing).
1. Test retrieving the job arguments from the worker using an
OSBuildJob.
Very basic test with an empty manifest.
2. Test retrieving job manifests through the Koji API. Uses the existing
TestCompose test. The returned manifests are empty for all cases.
Add and implement an API endpoint that returns the manifests for a given
osbuild-koji job. The route returns an array of manifests, one for each
image in the request.
Retrieves the arguments of each Build Job to extract the Manifest.
Wraps jobqueue.JobArgs() method and unmarshals data into the provided
concrete struct value. The struct should match the requested job type of
the given ID.
JobArgs() returns the arguments submitted with a job in raw form.
Since the structure of the args are opaque to job queue, it's the
responsibility of the caller to deserialize the arguments.
Args retrieval is added to the existing TestArgs() function.
With our limited number of machines in OpenStack, we cannot run unlimited jobs
in parallel. When there's a lot of activity in the repository, there's a high
probability that a job must wait for a free VM for quite a long. In these cases,
tests commonly fail due to the timeout.
I decided to prolong the timeout to 12 hours. This should lower the amount of
these kinds of jobs.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Add the latest information about the intended use of this type of
image. Also add note about the most common way to consume Red Hat
content using this type of image.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The majority of pull requests do not fix a downstream issue. In
practice, people usually delete the whole suggested content.
Move it into a comment, so that this step is unnecessary.
RHEl 8.4 guest images need to have the default timezone of EST/EDT
unless the user specifies one in their blueprint. New York is a major
location for this timezone.
The directory with image-tests test cases has been renamed from `cases`
to `manifests`. This has not been previously reflected in the test/README.md
and osbuild-image-tests code. osbuild-image-tests hardcodes the test
cases directory path and uses it in case no test case are passed
to it on the command line. Since the image_tests.sh CI test case looks
for image-tests test cases in the correct directory and passes the
relevant ones to osbuild-image-tests, the CI didn't detect this issue.
Running osbuild-images-tests without any argument and let it run all
test cases from the default test cases directory as part of CI probably
does not make sense. Due to this reason, I'm not adding any new test.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
testjobqueue did not implement the JobQueue interface correctly (noted
in its package comment), making it impossible to write tests for
JobQueue itself.
Replace its use everywhere with fsjobqueue operating on a temporary
directory.
