Commit graph

836 commits

Author SHA1 Message Date
Major Hayden
41e8090e95 🔮 Add dnf integration tests
Run the dnf integration tests with every pull request.

Signed-off-by: Major Hayden <major@redhat.com>
2020-04-06 19:06:06 +02:00
Ondřej Budai
7dd09443cf tests/image/ami: test ami images on AWS
Prior this commit the ami images were tested locally using qemu. This does
not reflect at all how they're used in practice. This commit introduces
the support for running them in the actual AWS. Yay!

The structure of code reflects that we want to switch to osbuild-composer
to build the images soon.
2020-04-06 16:38:28 +02:00
Ondřej Budai
92e69dcb85 tests/image: extract one test method for each boot type
Shorter functions are always nicer, right? Also, the AWS and Azure tests will
be longer, this change prepares for that.
2020-04-06 16:38:28 +02:00
Ondřej Budai
00f903b879 tests/image: refactor constants
It's not very clear that the constants are indeed constants. This commit moves
them to a new struct. This way it should be more clear that those values are
constants.
2020-04-06 16:38:28 +02:00
Ondřej Budai
640a9fcd92 tests/image: pass the ssh key from boot types
Soon, images will be run non-locally (AWS, Azure). For the remote ones it's
potentially dangerous to use the publicly available key-pair. This change
prepares the codebase for specifying different keys than the pre-generated
one.
2020-04-06 16:38:28 +02:00
Ondřej Budai
1a8379f2cd tests/image: extract withTempDir helper
It will be needed on another place at some point, I promise.
2020-04-06 16:38:28 +02:00
Ondřej Budai
ca3c469bdc tests/image: move the namespace creation closer to the boot types
Soon, images will be run non-locally (AWS, Azure). For those boot types
there's no need to have an unshared network namespace. This commit prepares
the code for that.
2020-04-06 16:38:28 +02:00
Ondřej Budai
cd51cfdcdf tests/image: allow to specify host address
Soon, images will be run non-locally (AWS, Azure). This commit adds support
to test ssh for different machines than those on localhost.
2020-04-06 16:38:28 +02:00
Ondřej Budai
7baacda6ef tests/image: allow to run ssh without network namespace
Soon, images will be run non-locally (AWS, Azure). For this, we cannot
use ssh in a namespaced environment. This commit makes the namespace for ssh
optional.
2020-04-06 16:38:28 +02:00
Ondřej Budai
e593bceede tests/image: use NoError when asserting errors
This is not a functional change (except for the slightly different output).
NoError() is more understandable (both in the code and in the output) than
Nil() when asserting errors. Also the rest of the tests uses mostly NoError()
when asserting errors, this change should lead to a better consistency.
2020-04-06 16:38:28 +02:00
Ondřej Budai
e6f323e6b6 tests/image: format osbuild output in case of a failure
testify library cannot deal with error messages with length > 64k. Sadly,
osbuild output is very long one line. This commits formats the output before
making the error from it.
2020-04-06 16:38:28 +02:00
Lars Karlitski
2435163fc9 worker: move running osbuild into separate function
Setting up a command to run is quite involved. Separate that from the
logic of running it.
2020-04-06 12:11:54 +02:00
Lars Karlitski
ada058f2c3 jobqueue: move Job struct to client.go
`Job` is now a simple data struct, which is only used to return
multiple arguments from `AddJob()`. Move it to that file.
2020-04-06 12:11:54 +02:00
Lars Karlitski
1ece08414c jobqueue: move Job.Run() to the worker
This makes the jobqueue package independent of forking osbuild, the
choices for which (exact invocation, location of the cache directory)
should be made in the worker.
2020-04-06 12:11:54 +02:00
Lars Karlitski
d3b9a3515d worker: inline handleJob()
It's a small function that's only called once.
2020-04-06 12:11:54 +02:00
Lars Karlitski
db5dd1ee2c worker: remove redundant UpdateJob() call
A job is already set to be running when it is returned from the API (see
Store.PopJob()).
2020-04-06 12:11:54 +02:00
Lars Karlitski
54b63a645f jobqueue: remove JSON tags on Job
Job is not used for serialization anymore.
2020-04-06 12:11:54 +02:00
Lars Karlitski
ab2c8aff08 jobqueue: make import declarations consistent
The usual style is three blocks: standard library, external
dependencies, internal dependencies.

Also run through `go fmt`, which sorts each of those blocks
alphabetically.
2020-04-06 12:11:54 +02:00
Lars Karlitski
1f06d78362 jobqueue: rename ID to ComposeID in job structs
It's not an id of the job, but the compose id.
2020-04-06 12:11:54 +02:00
Lars Karlitski
269988a737 jobqueue: start moving API types to json.go
Similar to `weldr/json.go`, this file will contain structs that are
shared between client and server and only meant to be used to
serialize from and to JSON.

This immediate change allows us to make fields in `Job` private in a
future commit.
2020-04-06 12:11:54 +02:00
Ondřej Budai
9a96908c1e upload/aws: delete the s3 object after the snapshot is imported
After the snapshot is imported to EC2, there's no point in having the s3
object anymore. If the s3 object is not deleted, the user will be charged
for it periodically, while the object doesn't provide any value.
2020-04-04 02:49:00 +02:00
Major Hayden
2f94a87dab 🎲 Enable rcm integration tests
Now that Jenkins is hooked up to the repository, let's run the rcm
tests after we deploy all of the services into a VM.

Signed-off-by: Major Hayden <major@redhat.com>
2020-04-03 19:38:10 +00:00
Tom Gundersen
8797cd4704 common: drop unused image types
The Alibaba and GCE image types were not used, and they were
causing problems when translated from and to strings as they were
mapping to the same internal image type. The end result was that
AWS jobs were shown as Alibaba or GCE in the UI.

This fixes #369.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-04-03 08:46:50 +02:00
Major Hayden
b57f70e3e5 🔌 Plug osbuild-composer into Jenkins
Let's throw schutz to the wind and add Jenkins to the osbuild-composer
repository. Jenkins will execute the `jenkins/run_tests.sh` script and that
will deploy osbuild, osbuild-composer, and cockpit-composer on Fedora 31
and 32.

We can add extra integration tests gradually after this is merged.

Signed-off-by: Major Hayden <major@redhat.com>
2020-04-02 21:11:37 +00:00
Jakub Rusz
8861c6dbb9 tests: increase coverage of internal/crypt
fixes #444
2020-04-02 20:12:06 +02:00
Jakub Rusz
4f160447ee tests: use asserts in states_test.go 2020-04-02 09:17:09 +02:00
Jakub Rusz
1cf2af10d0 tests: add coverage for common/helpers.go
This also changes the helpers.go to enable mocking all the different architectures.
2020-04-02 09:14:40 +02:00
Ondřej Budai
09109b4f8e 9
Release osbuild-composer version 9
2020-04-01 15:56:57 +02:00
Ondřej Budai
c081844954 ci/github: bump the checkout action to version 2
actions/checkout@v1 sometimes fails, bumping the version should fix it. See:
https://github.com/actions/checkout/issues/23
2020-04-01 15:03:48 +02:00
Ondřej Budai
bbd71efeff distro/rhel: remove dracut-config-generic from build packages
The build pipeline doesn't run dracut nor install kernel, the package is
therefore useless in it.
2020-04-01 12:29:59 +02:00
Ondřej Budai
c48728a049 distro/rhel: move dracut-config-generic to bootloader packages
This change makes the dracut-config-generic installation in par with
Fedora. Also, by having it in bootloader packages, it is not installed in
non-bootable images, where it provides no value.
2020-04-01 12:29:59 +02:00
Ondřej Budai
f14224ede1 distro/fedora: add dracut-config-generic to bootloader packages
Dracut is unfortunately very host-dependant by default. The package
dracut-config-generic forces it use a generic configuration instead of a
configuration generated from the host environment.

This change should make the image generation more reproducible. For example
it was not possible to boot ami images built on Travis on AWS prior this
commit.

Also, the tests were re-generated in this commit.
2020-04-01 12:29:59 +02:00
Jiri Kortus
b0e28689fb Add unit test for systemd stage
Refs: #406
2020-04-01 09:52:10 +02:00
Lars Karlitski
3a957f4b16 store: don't add custom repositories in PushCompose()
The weldr API already passes custom repositories, so they end up in the
list twice. This is a bit awkward, because weldr passes something to the
store that the store already knows about. It only works this way right
now though, because the rcm API doesn't need these repositories at all.
It expects all repositories to be passed.
2020-04-01 00:29:13 +02:00
Lars Karlitski
cef7267a2d store: create compose id from PushCompose
Every caller had to make their own uuid and pass that into
PushCompose(). Make PushCompose() create the id and return it.
2020-04-01 00:29:13 +02:00
Lars Karlitski
a6338410bc rcm: remove outdated comment
PushCompose() does not inject a local target anymore.
2020-04-01 00:29:13 +02:00
Lars Karlitski
eb10cf3124 store: remove unused arguments from PushCompose 2020-04-01 00:29:13 +02:00
Jiri Kortus
e516180640 Add unit tests for SELinux stage
Refs: #406
2020-04-01 00:26:22 +02:00
Jiri Kortus
44b501d401 Add unit test for timezone stage
Refs: #406
2020-04-01 00:25:59 +02:00
Jiri Kortus
8dafb64f1b Add unit tests for script stage
Refs: #406
2020-04-01 00:25:34 +02:00
Jiri Kortus
8c808831d3 Add unit test for users stage
Refs: #406
2020-04-01 00:24:39 +02:00
David Rheinsberg
6319d52fe4 docs: import changelog as NEWS.md
Import a new `NEWS.md` file with the changelog for each release. This is
targetted at distributors, so it should contain information on new
features, modified features, and anything important to packagers. This
is not a replacement for the git-changelog, but rather a human-readable
instruction to packagers what needs to be changed when using the new
release.

Additionally, copy `make release` from *osbuild* to help in creating new
release entries in NEWS.md.
2020-04-01 00:22:59 +02:00
David Rheinsberg
1c32a30092 docs: rework README and align with osbuild
This aligns the layout of `README.md` with the osbuild project. It now
has a clear structure with individual sections that we can continue to
improve on.

This tries to salvage most of the information that was previously in
README.md, but removes the bits and pieces that were either moved into
the osbuild-composer(7) man-page, or feels misplaced in a project
readme.
2020-04-01 00:22:59 +02:00
David Rheinsberg
d344e967e5 build: rework Makefile for man-page target
This adds `make man` with SRCDIR and BUILDDIR support. This is copied
from *osbuild*. Since `make man` will be the official way to build the
man-pages, lets also import `make help` and some documentation for the
Makefile.
2020-04-01 00:22:59 +02:00
David Rheinsberg
b36227e4da docs: add osbuild-composer(7)
Create a new man-page source for `osbuild-composer(7)`. This will be
the overview man-page for composer, as well as a good entrypoint to
learn about the entire osbuild setup.

For now, just include some basic information, as well as copy the bits
from README.md that make sense.

Contributions welcome! This could be a lot more verbose and helpful.
2020-04-01 00:22:59 +02:00
David Rheinsberg
c5293edbab docs: break lines in markdown
Avoid overlong lines in markdown files. This does not affect the
rendered output, but makes it a lot easier to navigate through the file
from a text-editor when editing. Furthermore, it improves *Diffs* of
our commits, and allows reading markdown without rendering it first.
2020-04-01 00:22:59 +02:00
Jiri Kortus
ea02b33d2b Add unit test for RPM stage
Refs: #406
2020-04-01 00:19:43 +02:00
Jakub Rusz
703039495b tests: add coverage for internal/blueprint/customizations.go 2020-03-31 16:47:00 +02:00
Jakub Rusz
4807b64dea tests: add coverage for compose_result.go 2020-03-31 16:32:34 +02:00
Jakub Rusz
2b87ca41b5 Separate nested structure in compose_result.go
This separated the nested structure into their own struct types.
2020-03-31 16:32:34 +02:00