Commit graph

716 commits

Author SHA1 Message Date
Martin Sehnoutka
db4b6e71ed distro_test: add test that covers BuildPackages
The main reason here is not to make sure we have the right set of build
packages, but rather to make sure the pointer hierarchy (distro->arch->
image type) works as expected.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
91f572e961 fedora30,32: merge fedora3XImageType into imageType struct
This patch uses the same approach as fedora31 few commits ago. It moves
the arch pointer to the imageType struct and sets it while creating the
distro->arch->image type hierarchy in the setImageType function. The
rest of the patch is just renaming.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
ff065cc7d0 fedora30,32: merge fedora3XArch structure into arch
It uses the same approach as fedora31 few commits ago. The pointer to
distro is moved to "arch" and "arch" now contains a map of allowed
image types for each arch.

Also include setArches and setImageTypes helper functions to ease the
creation of pointers from the structures to a parent structure while
creating the distro->arches->image types hierarchy.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
507194772b fedora31: create setImageTypes and setArches helpers
Previously we had to type the name of the architecture and image type
multiple times (in the definition, when inserting it into each parent
structure etc.) and it would get even worse because we would have to do
the same for all distros. These helper functions make it less error
prone by copying the name in the function automatically.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
adae7110af fedora31: merge fedora31ImageType with imageType
The former one was just a wrapper to the second one. In this patch I
removed the fedora31ImageType struct and instead moved the pointer to
the imageType struct. The previous solution always created a new wrapper
and returned it to the called. This new solution creates a shallow copy
of imageType, assignes arch field, and then returns the result to the
caller. It should achieve the same without the additional wrapper.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
473fa3b341 fedora31: merge fedora31Arch struct into arch
It was unnecessary to have these separate any more.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
6d2d259e57 distro: remove unused FilenameFromType() function
This information is now provided only when an architecture is specified,
so it is necessary to first obtain object implementing the Arch interface
then object implementing the ImageType interface and then you can get
the filename and mime type.

Tests are changed accordingly to the new API.
2020-03-28 00:21:31 +01:00
Martin Sehnoutka
2f576e0964 weldr: stop using Distro.FilenameFromType
The method is available in the Distro interface, but the distro does not
have the information needed to provide this kind of information any
more. The logic is now split into Arch and ImageType interfaces. This
patch will allow us to get rid of some old code and move forward.
2020-03-28 00:21:31 +01:00
Jiri Kortus
5fc1f10ecb Add unit test for GRUB2 stage
Refs: #405
2020-03-27 19:18:14 +01:00
Jiri Kortus
3306a3b20e Add unit test for groups stage
Refs: #405
2020-03-27 19:11:13 +01:00
Jiri Kortus
bc52b93ece Add unit tests for fstab stage
Refs: #405
2020-03-27 19:10:42 +01:00
Jiri Kortus
ea3fccd643 Add unit tests for chrony stage
Refs: #405
2020-03-27 19:09:30 +01:00
Brian C. Lane
fc205786fc weldr: Return a JSON API error response for all unknown requests
This matches the lorax-composer behavior.
2020-03-27 19:07:33 +01:00
Brian C. Lane
e3934108f7 client: Handle the differences between unit tests and integration tests
The mock server used by unit tests is slightly different than the
running server, mostly related to package names that are hard-coded.

This adds a bool to testState that can be used in the tests to alter the
expected behavior. It should be used as little as possible.
2020-03-27 19:07:33 +01:00
Brian C. Lane
856eb59edf client: Move the weldrcheck integration tests to client
With this change the integration tests can now also be run as unit tests
against the mocked server. The way it works is this:

internal/client/unit_test.go sets up the mock server and is built
when the `integration` build tag is *not* included.

internal/client/integration_test.go sets up the connection to an
existing server and is built when the `integration` build tag *is*
included.

The test code is built and run for both cases.

Currently they all pass for the integration test run. The unit test
cases need some work because the mocked server isn't a real server with
real depsolving and package lists. A future commit will fix this.
2020-03-27 19:07:33 +01:00
Brian C. Lane
816a09fd9d client: Add integration test setup code 2020-03-27 19:07:33 +01:00
Brian C. Lane
b5b5bd96df client: Move test setup into unit_test.go
Copy weldrcheck's utils.go into client, switch to using TestState struct
to hold global test data. Only build unit_test.go if integration has not
been selected.

This is in preparation to moving weldrcheck code into client *_test.go
files so that the test code can be shared and run against a mock server
during unit testing, or against a running WELDR API server during
integration testing.
2020-03-27 19:07:33 +01:00
Jiri Kortus
dd3fc6bd7c Add coverage for *assembler.go
Resolves: #363
2020-03-27 16:37:12 +02:00
Jiri Kortus
2d8b9c3178 Add unit test for firewall stage
Refs: #405
2020-03-27 15:22:08 +01:00
Jiri Kortus
dae77de5ac Add unit test for Fix BLS stage
Refs: #405
2020-03-27 15:21:47 +01:00
Lars Karlitski
32d7c68cf1 image-types: start describing image types
This is the start to document the image types that osbuild-composer
produces on a higher level than the code in the `distro` package.

The idea is to document the purpose for each of the image types as well
as the reasons why deviating from a standard install is necessary.
2020-03-26 19:16:30 +01:00
Tom Gundersen
895a4769e3 osbuild: remove dead code
The helper functions were never used, we should aim to use the
osbuild types just as regular structs for serialization purposes.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-26 00:31:37 +01:00
Alexander Todorov
7967ecbbf8 tests: Make TestSources distro independent. Refs #315
by calling `composer-cli sources info` on a user defined repository
2020-03-25 16:30:52 +01:00
Ondřej Budai
f6b3c4a994 test/images: regenerate the test cases
Previous commit changed the output of image-info, therefore the cases must be
regenerated.
2020-03-25 16:14:43 +01:00
Ondřej Budai
a040369513 tools/image-info: deduplicate and sort the services
systemctl list-unit-files doesn't produce machine readable output.
parse_unit_files isn't very good at reading it and can produce duplicate
records.

This commit fixes it by deduplicating and sorting the units. This is a bit
hacky solution, but should work just alright. In the future we might
dump list-unit-files and do the job ourself, but let's not recreate the
systemd logic for now.
2020-03-25 16:14:43 +01:00
Ondřej Budai
9b3375ccca tools/image-info: fix crash when parsing output from systemd >245
systemd 245 added a third column to systemctl list-unit-files, causing
the unpacking to fail.
2020-03-25 16:14:43 +01:00
Ondřej Budai
4c93d5a045 tools/image-info: fix warning when running with python 3.8
Python 3.8 introduced this warning:

tools/image-info:53: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if line[0] is '#':

This commit fixes this warning. More info in Python 3.8 release notes:
https://docs.python.org/3.8/whatsnew/3.8.html#changes-in-python-behavior
2020-03-25 16:14:43 +01:00
Lars Karlitski
3b5d5a73d3 worker: drop default port
We require passing the address from the unit file. Do the same for the
socket, using host:port syntax.

Overriding the port was broken before, because we unconditionally
appended ":8700" to every address.
2020-03-25 14:05:44 +01:00
Lars Karlitski
f8982f4a1a worker: don't hard code path to unix domain socket
Introduce a mandatory argument `address`, which is interpreted as a path
to a unix socket when `-unix` is given or a network address otherwise.

Move the default path to the service file.

Add a more useful usage message when passing `-help` or no arguments.
2020-03-25 14:05:44 +01:00
Lars Karlitski
b5432e78b9 worker: move ComposerClient to jobqueue package
This moves the client code into the same package as the server code,
which makes it easier to change (and version) the two in sync. Also, it
will allow to make some structs private to the jobqueue package and to
test `Client`.

Also rename it to jobqueue.Client.
2020-03-25 14:05:44 +01:00
Lars Karlitski
cb4421b69f worker: remoteAddress → address 2020-03-25 14:05:44 +01:00
Lars Karlitski
94183d14a8 worker: split NewClient()
Use the default dialing functions for tcp connections and set the tls
config on the transport directly. This makes the code easier to follow,
because the only special case is overriding the DialContext() for unix
connections.
2020-03-25 14:05:44 +01:00
Lars Karlitski
845ba6e8e5 worker: don't hard code upload URL
This doesn't work with remote workers.
2020-03-25 14:05:44 +01:00
Lars Karlitski
9e71df234a worker: load tls certificates once on startup 2020-03-25 14:05:44 +01:00
Lars Karlitski
16cd243300 worker: set remoteAddress once on startup 2020-03-25 14:05:44 +01:00
Ondřej Budai
ef7e8e0772 drop packit configuration file
Packit is not used currently, therefore let's drop its configuration from the
repository. The change can be always reverted if we change our minds.
2020-03-25 11:43:39 +01:00
Ondřej Budai
0d5c2d141c spec: add notes to remove the lines related to renaming in F34
The packaging guidelines requires that these provides and obsoletes stays
in the spec file for 2 fedora versions. The name change was introduced in
Fedora 32, there the lines should be removed in F34.
2020-03-25 11:43:39 +01:00
Ondřej Budai
247e8cb205 update readmes to use the new osbuild-composer package 2020-03-25 11:43:39 +01:00
Ondřej Budai
b45119c471 spec: sync the line order and whitespace changes
In the new spec file, the tests-related code is always at the bottom of the
containing block to make the conditional code easier to read. Do the same
in the old spec file.

Also, in the new spec file, some whitespaces are different (rpmlint reported
mix used of tabs and spaces), let's do the same changes in the old spec file.
2020-03-25 11:43:39 +01:00
Ondřej Budai
cdc4248909 spec: use the standard way to represent a go dependency
This isn't a functional change.
2020-03-25 11:43:39 +01:00
Ondřej Budai
64181e7ba0 ci/rpm: use the old spec file for f31
Prior this commit the rpm for fedora 31 was built using the new spec file.
The old spec file should be used for that Fedora version.
2020-03-25 11:43:39 +01:00
Ondřej Budai
48a51611ec makefile: use the new spec file for rpm targets
This commit switches the rpm and srpm targets to use the new spec file.
Also, it introduces old-rpm and old-srpm targets to build an rpm from
the old spec file.

Note that the rpm target builds also the tests subpackages (they are not
built by default).
2020-03-25 11:43:39 +01:00
Ondřej Budai
0b08eb1904 spec: add spec file for osbuild-composer package
We successfully renamed the package in Fedora to osbuild-composer
(RHBZ #1815577). This commit adds the new spec file.

Changes: the tests subpackage is not built by default, to build it
--with tests parameter must be passed to rpmbuild.

Let's hope both spec files stay in sync.
2020-03-25 11:43:39 +01:00
Lars Karlitski
ee752b0ab8 tree-wide: panic when json marshalling fails
According to the new guidelines in docs/errors.md.

Note that this does not include code that marshals to a writer that
might fail (when a connection drops, for example).
2020-03-25 10:22:16 +01:00
Lars Karlitski
ad1a12f7aa tree-wide: use uuid.MustParse() when possible
According to the new guidelines in docs/errors.md.
2020-03-25 10:22:16 +01:00
Lars Karlitski
9625c589b0 docs: add errors.md
docs/errors.md are guidelines for how we handle errors. It's meant to be
expanded as we figure out more rules.
2020-03-25 10:22:16 +01:00
Brian C. Lane
899d38422c weldrcheck: Adding integration tests for modules 2020-03-25 10:01:14 +01:00
Brian C. Lane
b5da15e2d8 client: Add /modules/ support 2020-03-25 10:01:14 +01:00
Brian C. Lane
d530d9a87e weldr: Fix projects/depsolve route handling
Return a 400 with json error message when no packages are included, with
or without trailing /
2020-03-25 10:01:14 +01:00
Brian C. Lane
3bddfd3449 weldrcheck: Adding integration tests for projects 2020-03-25 10:01:14 +01:00