Commit graph

54 commits

Author SHA1 Message Date
Florian Schüller
a4068b328d splunk_logger: move environment hook to splunk_logger pt2
for reusability also in image-builder
2024-08-28 16:41:07 +02:00
Florian Schüller
9006836afc logrus: add deployment channel as field to the logs 2024-08-07 12:32:57 +02:00
Lukas Zapletal
5ce8f65a58 cloudapi: propagate operation/external id
Signed-off-by: Lukas Zapletal <lzap+git@redhat.com>
2024-06-25 13:58:53 +02:00
Lukas Zapletal
f3c0daebbf cmd/osbuild-composer: journald support 2024-06-25 13:58:53 +02:00
Tomáš Hozza
43e87632fb Drop common.CurrentArch() in favor of osbuild/images
Drop `common.CurrentArch()` implementation and use
`arch.Current().String()` from the osbuild/images instead.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
c4e5ab5aca Drop internal/common/distro.go in favor of osbuild/images
Drop `common.GetHostDistroName()` implementation and use
`distro.GetHostDistroName()` from the osbuild/images instead.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
d7e960f0c1 Internal: delete unused common.VersionLessThan()
The function is a leftover from the image definitions split and it is
not used. Moreover, the `images` copy of it is being reimplemented by
[1]. It is better to remove this copy to prevent any unintended use of
it or confusion.

[1] https://github.com/osbuild/images/pull/195

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-10 14:30:58 +02:00
Tomáš Hozza
3c95ba8476 Include osbuild/images version in Manifest job result
Include the osbuild/images module version in the Manifest job result.
The module has direct impact on image definitions and the content of
produced manifest, therefore including this information in the Manifest
job result is very helpful for various purposes (debugging,
traceability).

This will enable to embed this information in the Koji build metadata.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-09-27 12:39:28 +02:00
Ondřej Budai
abe6ccfb50 upload/azure: migrate from azure-storage-blob-go to azure-sdk-for-go
https://github.com/Azure/azure-storage-blob-go/ is deprecated, the main SDK
should be now used instead. Let's migrate the code. There should be no
functional changes.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-04-04 09:09:43 +02:00
Ondřej Budai
b997142db0 common: merge all *ToPtr methods to one generic ToPtr
After introducing Go 1.18 to a project, it's required by law to convert at
least one method to a generic one.

Everyone hates IntToPtr, StringToPtr, BoolToPtr and Uint64ToPtr, so let's
convert them to the ultimate generic ToPtr one.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-01-09 14:03:18 +01:00
Tomas Hozza
ff28b0f5d3 common: define storage units as constants
Define all used storage units as constants. Use them in
`DataSizeToUint64()`, instead of literal multiples.
2022-09-15 08:50:33 +02:00
Achilleas Koutsou
91e581462b common: move VersionLessThan() util function
Move VersionLessThan() utility function to the common package to be
reused.
2022-09-13 16:06:19 +01:00
Lukas Zapletal
a8afca4634 Introduce logging adapter for jobqueue 2022-09-09 16:27:38 +02:00
Tomas Hozza
af65baa6fe worker/osbuild: use os-release to determine host OS
When running an osbuild job, we read `/etc/redhat-release` to get the
host OS name to attach as metadata to the job result.

Only Fedora and RHEL ship this file, which makes the osbuild job always
fail on other distributions.

The main reason to report host OS back to the worker server is due to
Koji composes and the koji-finalize job, which pushes it to Koji. The
motivation is to have enough information to potentially re-instantiate
/ identify the original builder host OS. There are no specific
requirements on the string.

Modify the code to use `/etc/os-release` to determine the host OS. Fall
back to using `linux` as the host OS, in case reading `os-release`
fails, log the error and continue with the job. The `linux` fallback is
suggested by the `os-release` spec [1]

[1] https://www.freedesktop.org/software/systemd/man/os-release.html#ID=

Co-authored-by: Achilleas Koutsou <achilleas@koutsou.net>
2022-08-25 08:25:35 +01:00
Tomas Hozza
f378209bf3 Simplify trimming of quotes in readOSRelease()
Use standard library function to trim surrounding quotes from read
values. This makes the code shorter and easier to read.
2022-08-25 08:25:35 +01:00
Tomas Hozza
2fba4c2c1f Drop special handling for RHEL < 8.4 in GetHostDistroName()
RHEL-8.3 is EOL and we do not explicitly support it. There is no need to
mangle the host distro name for RHEL-8 releases older than 8.4.
2022-08-25 08:25:35 +01:00
Tomas Hozza
c7e5e3c9c2 Move GetRedHatRelease() and GetHostDistroName() to common package
The `distro` package is now used for distro definitions supported by
osbuild-composer, not for introspecting the Host system. Move
`GetRedHatRelease()` and `GetHostDistroName()` functions to the `common`
package.
2022-06-10 14:48:18 +01:00
Martin Sehnoutka
7d6dadb598 internal/common: introduce function to convert data sizes
This function will be used to parse filesystem sizes specified as
string.
2021-11-02 18:32:47 +01:00
Diaa Sami
179009fec4 cloudapi: use Logrus as default logger in Echo
And remove log.Logger references
2021-09-30 18:34:05 +02:00
Martin Sehnoutka
d7d4e02c8c internal/common: introduce git revision and rpm version
The variables are set to the git revision from which the build is
triggered and rpm version from the spec file, if it is build using RPM.

This can be later used to query exact source version while
running osbuild-composer.

It is necessary to use both, because none of them is available in all
possible scenarios.

Use either git-rev (preferably) or RPM version (NEVRA) instead of the
"devel" build type. It was just a placeholder.
2021-09-24 14:13:23 +02:00
sanne
5a9d8c792b cloudapi: V2
V2 is compliant with api.openshift.com design guidelines.

Errors are predefined, have codes, and are queryable.

All requests have an operationId set: a unique identifier which is
sortable by time. This is added to the response in case of an error.

All returned objects have the href, id, and kind field set.
2021-09-14 15:32:21 +02:00
Tomas Hozza
6c32ff048a internal/common: add function to get a pointer to string literal
Add a new helper function to get a pointer to a string literal.

Add unit test for the new function.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-02 19:05:09 +02:00
Achilleas Koutsou
dc62275d4d common: helper function for uint64 ptr 2021-07-24 15:54:00 +02:00
Brian C. Lane
b476078570 Move isStringInSlice to common.IsStringInSlice 2021-07-12 08:58:42 +02:00
Tomas Hozza
9386a02984 internal/common: add helper functions to get pointer to a basic type value
Add two new helper functions `IntToPtr()` and `BoolToPtr()` to the
`internal/common` package, which can be used to conveniently set
basic type literals in a struct literal, in which pointer to the basic
type is expected.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-08 19:37:09 +02:00
Gianluca Zuccarelli
9aa8ebdfff fix: unused function arg 2021-07-02 10:38:04 +02:00
Lars Karlitski
bcd57a77e2 weldr: move common.ComposeState to weldr
ComposeState is only used by the weldr API.

Drop the JSON marshaller and unmarshaller, because ComposeState is not
used in an JSON-exported field anymore.
2020-11-09 14:17:19 +01:00
Tom Gundersen
ac5f69e757 osbuild: move result serialization from common
In the same way `osbuild.Manifest` is the input to the osbuild API,
`osbuild.Result` is the output. Move it to the `osbuild` package where
it belongs.

This is not a functional change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-08-26 12:12:37 +02:00
Tom Gundersen
2c3790f20c common/ImageType: move to the store package
This is now only used for marshalling in the store, so keep the logic
where it is used.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
3f2e7be9cf common/types: drop dead code
Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-18 11:50:15 +02:00
Tom Gundersen
8797cd4704 common: drop unused image types
The Alibaba and GCE image types were not used, and they were
causing problems when translated from and to strings as they were
mapping to the same internal image type. The end result was that
AWS jobs were shown as Alibaba or GCE in the UI.

This fixes #369.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-04-03 08:46:50 +02:00
Jakub Rusz
4f160447ee tests: use asserts in states_test.go 2020-04-02 09:17:09 +02:00
Jakub Rusz
1cf2af10d0 tests: add coverage for common/helpers.go
This also changes the helpers.go to enable mocking all the different architectures.
2020-04-02 09:14:40 +02:00
Jakub Rusz
4807b64dea tests: add coverage for compose_result.go 2020-03-31 16:32:34 +02:00
Jakub Rusz
2b87ca41b5 Separate nested structure in compose_result.go
This separated the nested structure into their own struct types.
2020-03-31 16:32:34 +02:00
Tom Gundersen
4a7519807e common/Architecture: drop unused code
The intended use for this will be repalced by Arch objects, currently
it was unused, so drop it.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 15:46:57 +01:00
Tom Gundersen
87b0bb6e5d distro: mass cleanup
Delete unused methods and make types and fields private where
possible. Some code is moved around, but apart from that there
is no change in behavior.

The naming of the distros were moved back into the distro
packages as the common types now only had one user, and this
allowed us to drop some redundant error checking.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-24 14:13:03 +01:00
Lars Karlitski
72f8b07e8b store: move ComposeRequest to the store package
A ComposeRequest is data used to submit a compose to the store, so it
should live in that package.

Remove the json marshalling test, because ComposeRequest is never
marshalled to JSON.

This will allow to use types from `distro` in the ComposeRequest struct.
2020-03-12 20:01:24 +01:00
Lars Karlitski
17ca73ab14 osbuild-pipeline: validate distro when fetching it
We were verifying two things: if the passed distroArg exists in the
distribution mapping in common/types.go and if the it is an actually
registered distro. Since you cannot have distros registered that don't
correspond to a type, the first test is unnecessary.

Merge the two tests by moving the (much better) error message down into
the second test. This makes DistributionExists redundant, because
Registry.GetDistro() checks this implicitly.

Also, move ListDistributions() to the Registry object, because we want
to show distributions that are actually registered.

Add a test which checks that Registry.List() works and that all included
distributions register correctly.
2020-03-03 16:21:13 +01:00
Ondřej Budai
d7cbc22da4 lint: fix unhandled errors 2020-03-02 14:28:55 +01:00
Ondřej Budai
efb100fbca lint: fix gosimple/S1005 errors 2020-03-02 14:28:55 +01:00
Jacob Kozol
2a3a5b318b distro: add rhel-8.1 support
Images can be built for rhel 8.1. The pipeline generation and distro
tests are based off of the rhel 8.2 ones. Repository information as been
added for rhel 8.1. The repo urls are internal ones and will only work
if the user is on the Red Hat vpn.
2020-02-29 01:17:50 +01:00
Ondřej Budai
80f0888896 cmd/composer: move currentArch helper to common package
The helper function might be useful also in different parts of the project.
2020-02-26 16:58:39 +01:00
Martin Sehnoutka
923a0b0b97 rcm: introduce rpmmd member of the api structure
This is needed for unit tests, because it wasn't possible to mock the
rpmmd module before. This also requires that the checksum is moved to
the compose request and evaluated in the endpoint handler instead of
push compose. I think it makes sense to have the checksum in the compose
request directly.

Also a "module platform ID" is required now, but we don't have the
"global" distribution any more, so this patch introduces mapping from a
distribution to the module platform ID.
2020-02-20 13:04:28 +01:00
Martin Sehnoutka
5b0611983a common: fix bug in architecture ToString implementation
it used a wrong mapping, replace it with the right one
2020-02-20 13:04:28 +01:00
Martin Sehnoutka
6b957f01a0 common: fix method signature for UnmarshalJSON
There was a bug in the previous implementation which used to pass the
argument as a value but that does not work because we need to change the
value of it. The new implementation uses pass by reference.

Create a test to cover this scenario.
2020-02-20 13:04:28 +01:00
Jacob Kozol
fae51095d2 distro: add fedora 32 support
Images can be built for fedora 32. The pipeline generation and distro
 tests are based off of the fedora 30 ones. Repository information has
 also been added for the fedora 32 repos.
2020-02-19 14:18:58 +01:00
Lars Karlitski
4f2ac6a687 tree-wide: format code with gofmt 2020-02-17 16:09:17 +01:00
Martin Sehnoutka
62d186cd1b common: refactor build states into a separate file
These states will be used for tracking the image builds and compose
states in the rest of our codebase. There should be no change in the
behavior. It is a 1 to 1 replacement with the only difference of using
type alias instead of plain string.
2020-02-12 11:17:26 +01:00
Martin Sehnoutka
9c91b12076 common: compose request should include full bp and repositories
we want to move to a state where composer is a stateless service,
therefore we cannot take blueprints by name (and fetch them from a
storage) or assume repositories by the distribution. This commit
introduces both of these parameters as part of the structure.
2020-02-12 11:17:26 +01:00