When generating an osbuild manifest for an image type, we take a
customizations struct, which specifies the image-type-independent
customizations to apply. We also take the size argument, which is
specific to the image build and not part of the blueprint.
Introduce a new argument ImageOptions, which for now just wraps the size
argument. These options are specific to the image build/type, and
therefore does not belong with the other customizations.
For now this is a non-functional change, but follow-up commits will
introduce more types of image options.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This adds the wrapper object and a simple test for the rpm-ostree stage.
See the osbuild documentation for details.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This adds the wrapper object and a simple test for the ostree.commit
assembler.
See the osbuild documentation for details.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Exactly one of the UUID or the label must be set. The helpers are kept
the same and only supports the UUID for now, but these should likely be
dropped in favor of open-coding the structs anyway.
We do not enforce that the UUID or label is set, but osbuild will.
Signed-off-by: Tom Gundersen <teg@jklm.no>
see this comment from @teg:
https://github.com/osbuild/osbuild-composer/issues/356#issuecomment-630766947
Most of the calls within these tests should be converted to
use the plain text version. However some functions need the
JSON b/c they parse it and return a response, e.g. startCompose(),
getComposeStatus().
We are now building RPMs via mock in Jenkins, so we don't need these RPM
build jobs in GitHub Workflows.
Signed-off-by: Major Hayden <major@redhat.com>
As it turns out, the default expectation is not to distinguish between
these. We will now produce whatever is the most recent minor release by
default, and image tests will still be pinned at a given snapshot to be
reproducible.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Now that mock building seems to be stable, let's start building RPMs for
RHEL 8.3 so we can onboard it for base/image testing soon.
Signed-off-by: Major Hayden <major@redhat.com>
Build the RPMs in a mock using a simple script so that ansible-osbuild
can focus fully on deployment rather than compiling RPMs.
Signed-off-by: Major Hayden <major@redhat.com>
The go test coverage report does not cover other packages unless you
list them with the -coverpkg= argument. This results in an incomplete
coverage report with oddly missing lines.
This commit lists all of the packages so that they will all be included
when running the tests and gathering the results.
The osbuild CI uses the osbuild-composer CI testing playbooks but
there's no need to run the EC2 import for osbuild's CI. Add a check in
the Ansible playbook to look for the AWS_BUCKET environment variable and
skip the EC2 import if the bucker variable is missing.
Signed-off-by: Major Hayden <major@redhat.com>
Now that EC2 image testing is working well for Fedora 32, enable it for
all of the distros. Also, make it less special and allow it to run with
the other image tests.
Remove the `ami-boot` test from the list of regular image test cases
since it will be thoroughly tested via an import into AWS.
Signed-off-by: Major Hayden <major@redhat.com>
When we ran into these timeouts, they were always false negatives. It is
hard to find right values for them.
If someone does introduce a bug that hangs one of the tests, we still
have the global timeout.
Fixes#648
This fixes permission for the weldr sockets: allow users in group
`weldr` to access them, but nobody else. Also add this group to the
sysusers file.
Fixes#646
Importing an image into an EBS snapshot can sometimes take a long time.
The customer experience for a timeout is not great.
Allow the import to continue running for as long as needed without
timing out. This allows a customer to monitor the process of their
import until it has a success or failure status.
Fixesosbuild/osbuild-composer#632.
Signed-off-by: Major Hayden <major@redhat.com>
GetImageBuildResult() method is not called anywhere, so we can drop it. As
it was the only reader of result.json file, we can also drop all the code
which wrote to that file.
Our current tests cannot currently check if the logs can reach all the way
from the worker to the weldr API. This commit adds a simple check that
the /compose/log route returns at least something.
Also the logs are printed when the compose fails.
The store-jobqueue split broke the retrieval of logs in weldr API. Weldr API
still retrieved the logs from store but they are now stored in the jobqueue.
This commit fixes that by taking advantage of the fact that getComposeStatus()
now returns the compose result from which the logs can be generated.
Let's speak why I merged JobResult and JobStatus together:
Both methods actually called jobqueue.JobStatus(), so performance-wise
there's no real difference and it feels to me that it makes the code simpler:
You don't have to decide which method to call, you just get all the data
about a job in one call. We could split those again when we see some perf
issues with retrieving logs on each status check but I don't think we should
optimize prematurely. Let's leave some work for the future us.
When a job doesn't exist for a compose, just return an empty result.
In the future this will cause logs disappear from jobs created before the
store-jobqueue split but we decided we can live with that, they are still
in /var for those who really want them.
The reasoning is the same as in one of the previous commits. I think that
returning multiple values from one method can be confusing and potentially
error-prone. Also, this method will return ComposeResult soon too.
Also, I chose to create a new composeStatus struct instead of reusing
worker.JobStatus. I think that's a good thing in terms of consistency and
reducing the dependencies between packages.
Not a functional change.
Personally, I don't like methods returning too much values. I think it's
easy to assign the values in an incorrect order and thus interpreting them
in a wrong way. Also, the following commits will make JobStatus() return
also the JobResult, which would make the number of returned values even
higher.
Not a functional change.
The image tests are now able to upload images to Azure and boot them there.
However, the documentation on how to set up the required resources in Azure
was missing. This commit adds it.
This does not yet actually upload the image, and it only supports empty
images. You need to place a an empty file named <filename>, with a valid
extension (e.g., .qcow2) in /mnt/koji/work/<directory>/.
Signed-off-by: Tom Gundersen <teg@jklm.no>
If a pipeline runs for three hours, stop the pipeline. This can happen
if a network connection breaks down or if jenkins loses control of one
of its agents due to infrastructure issues.
Signed-off-by: Major Hayden <major@redhat.com>
By the same logic as the previous patch.
This exposed an apparent bug, where we are sorting by the version field
of the blueprint in the change objects, but these were never
(un)marshalled, so would always be set to zero in the past.
Remove the no-op sorting for now, as it never had an effect, and
put a note whether we should instantiate the blueprint in the change
object somehow.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Apply the backwards compatibility where the ImageBuild struct is
assembled. This should make the code simpler to reason about, by
reducing the structs we produce and later fix up.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The store only serves the weldr API, and that hard-codes the assumption
of only one image build per compose all over the place. Move this
assumption into the json serialization handler.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Serialize to a plain string, and do the translation for compatibility
from string to string.
This removes a lot of code, and if we change the store format, we should
just skip the conversion altogether.
Signed-off-by: Tom Gundersen <teg@jklm.no>
