Ensure that logs are artifacted after a test run. Also, ensure that
these logs are in unique directories so that they are not overwritten by
Jenkins as it builds the final artifact archive.
For example, if two jobs have a `test.log`, Jenkis will only keep one of
those logs. Each log must be named differently or placed in a unique
directory.
Signed-off-by: Major Hayden <major@redhat.com>
When Jenkins clones a repository for testing, it does the base clone
first and then merges the code from the PR afterwards. This ensures that
the code merges properly and is tested properly, but it also makes a SHA
that only exists inside Jenkins. 😢
Tell ansible-osbuild to use the repository that Jenkins made so that the
SHA is valid.
Signed-off-by: Major Hayden <major@redhat.com>
t.Parallel() makes defer weird. This commit moves the temporary dir setup
and teardown to the parallel subtest, so the teardown gets called after
the tests are finished.
More detailed explanation:
Consider this code:
```go
t.Run("group", func(t *testing.T) {
setUp()
t.Run("parallel test1", parallelTest1)
t.Run("parallel test2", parallelTest1)
t.Run("sequential test", sequentialTest)
cleanUp()
})
```
When the group subtest is started, firstly parallelTest1 is run. When
t.Parallel() in the parallelTest1 is hit, its execution is suspended and
parallelTest2 is started. Once again, t.Parallel() is hit and the
parallelTest2 is suspended. Now, sequentialTest is started. It doesn't
contain t.Parallel() so it's run without any interruptions. The group subtest
continues its execution and calls the cleanUp method. After the cleanup is
done, the group subtest returns and it's time for the important part:
The parallelTest1 and 2 are resumed and run in parallel. In other words,
the code after t.Parallel is run when the PARENT subtest returns.
The important (and weird) part is that the parent subtest of the parallel
subtests returns before its parallel children finish. However, just the inner
anonymous function returns. The containing t.Run("group", func(){...})
function does not return until all the parallel subtests are finished.
How to solve this? One solution is to move the cleanup out of the group
subtest:
```go
t.Run("group", func(t *testing.T) {
setUp()
t.Run("parallel test1", parallelTest1)
t.Run("parallel test2", parallelTest1)
t.Run("sequential test", sequentialTest)
})
cleanUp()
```
How does this work? The group subtest is not marked as parallel, so it just
runs from its beginning to its end (including the parallel subtests). After
it finishes, then the cleanUp is called.
In this commit I chose a different method: I just moved the clean up into each
parallel subtest. I think this is clearer than all that parallel magic I
tried explaining in previous paragraphs.
See the section "Run a group of tests in parallel":
https://blog.golang.org/subtests
and this paragraph:
https://golang.org/pkg/testing/#hdr-Subtests_and_Sub_benchmarks
Make it easier to see progress in Jenkins for image tests and keep their
logs separated for easier diagnosis of problems.
Signed-off-by: Major Hayden <major@redhat.com>
Empty names are not allowed, and blueprint names should only contain
characters matching: ^[a-zA-Z0-9._-]+$
This also adds tests for the various places where the blueprint name
could potentially be wrong.
This makes the queue more type safe and allows to get rid of the
`pendingChannel` and `pendingChannels` helpers, which only existed to
create not-yet-existing pending channels.
The enum is redundant information that can be deduced from the job's
times: queuedAt, startedAt, and finishedAt. Not having it reduces the
potential for inconsistent state.
After the queue rework, TestCompose() was the only user of
UpdateImageBuildInCompose() and Compose.UpdateState(). Set all required
fields in TestCompose() directly instead of going through those.
Since 2 releases `osbuild` accepts an `--output-directory=DIR` argument
which lets us decide where to place generated artifacts. Switch over to
it, rather than digging into the store, to make sure we will not access
the osbuild store when parallel cleanups are ongoing (which are not yet
a thing, though).
This is the same as 6c611b487, but for the jenkins directory
(configuration for Jenkins inside Red Hat's VPN).
Use golang's `test.timeout` to set the timeout instead of using
ansible's `async` property. The latter doesn't respect `register` when
running into a timeout, which means there are no logs in that case.
Use golang's `test.timeout` to set the timeout instead of using
ansible's `async` property. The latter doesn't respect `register` when
running into a timeout, which means there are no logs in that case.
A timeout doesn't make sense on this level, because it is very difficult
to estimate how long downloading rpm metadata takes. Drop it completely
in favor of higher-level timeouts in the test runner.
Fixes#601
We're only using code coverage as additional information. Put in a 5%
threshold so that small fixes that decrease coverage slightly don't
block pull requests.
Attempt osbuild-composer testing on the internal Jenkins deployment with
nodes that are destroyed after each use. The internal Jenkins looks for
a Jenkinsfile inside the `schutzbot` directory.
Let's not remove the `jenkins` directory (used by jenkins.osbuild.org)
yet until we know the internal Jenkins is stable and performs well.
Signed-off-by: Major Hayden <major@redhat.com>
The store is responsible for two things: user state and the compose queue. This
is problematic, because the rcm API has slightly different semantics from weldr
and only used the queue part of the store. Also, the store is simply too
complex.
This commit splits the queue part out, using the new jobqueue package in both
the weldr and the rcm package. The queue is saved to a new directory `queue/`.
The weldr package now also has access to a worker server to enqueue and list
jobs. Its store continues to track composes, but the `QueueStatus` for each
compose (and image build) is deprecated. The field in `ImageBuild` is kept for
backwards compatibility for composes which finished before this change, but a
lot of code dealing with it in package compose is dropped.
store.PushCompose() is degraded to storing a new compose. It should probably be
renamed in the future. store.PopJob() is removed.
Job ids are now independent of compose ids. Because of that, the local
target gains ComposeId and ImageBuildId fields, because a worker cannot
infer those from a job anymore. This also necessitates a change in the
worker API: the job routes are changed to expect that instead of a
(compose id, image build id) pair. The route that accepts built images
keeps that pair, because it reports the image back to weldr.
worker.Server() now interacts with a job queue instead of the store. It gains
public functions that allow enqueuing an osbuild job and getting its status,
because only it knows about the specific argument and result types in the job
queue (OSBuildJob and OSBuildJobResult). One oddity remains: it needs to report
an uploaded image to weldr. Do this with a function that's passed in for now,
so that the dependency to the store can be dropped completely.
The rcm API drops its dependencies to package blueprint and store, because it
too interacts only with the worker server now.
Fixes#342
Now that the "old" `jobqueue` package was renamed to `worker`, add a new
package that contains an interface to an actual job queue. Also add two
implementations: fsjobqueue, a job queue backed by the file system, and
testjobqueue, which can be used as a mock implementation for testing.
These packages are not yet used.
Code that's calling PushCompose() had to depsolve packages and fetch the
right ImageType from a distro, but not create the osbuild manifest. That
was left for PushCompose to do. Move it out of there to the callers, so
that the store is mainly concerned with storing things.
This also simplifies the argument list of PushCompose().
This function had two modes (`uuids == nil` and `uuids != nil`). Instead
of splitting it up into one function for each mode, inline it at call
sites. It was only used three times.
Instead of iterating over all composes, iterate over the passed UUIDs
(which amount to all composes when `*` is passed). This is more
straight-forward to read and more efficient in the case of requesting
only one (or a few) statuses.
`uuids` was initialized with a length, but only appended to. This lead
to a slice with lots of empty uuids, followed by the actual ones.
Fix this by initializing to an empty slice.
This happened to work because there's a non-fatal check for non-existing
composes further down, and the empty string fails that test.
By using a small Jenkins pipeline in the repository, we can define
almost all of our testing parameters in the repo itself and not inside
Jenkins.
This also allows us to use the GitHub Branch Source plugin and
auto-discover new repositories without `ok to test` bombs in
pull requests.
Signed-off-by: Major Hayden <major@redhat.com>
