Commit graph

686 commits

Author SHA1 Message Date
Tomas Kopecek
638f6cc6f2 replace xmlrpc_client exception with requests
It is broken in startup function for kojid and kojivmd from the
introduction of requests.

Fixes: https://pagure.io/koji/issue/1914
2020-02-21 12:40:42 +01:00
Atanas Zhelev
b4b88206fe Add smtp authentication support 2020-02-12 14:38:18 +01:00
Tomas Kopecek
e2edc37153 ensure that all keys in distrepo are lowered
Fixes: https://pagure.io/koji/issue/1978
2020-02-10 15:31:30 +01:00
Tomas Kopecek
a392467bce fix nvr/dict params
Related: https://pagure.io/koji/issue/1619
2020-01-02 12:19:57 +01:00
Tomas Kopecek
ac42f9b8ba basic zchunk support for dist-repo
Fixes: https://pagure.io/koji/issue/1198
2020-01-02 11:30:55 +01:00
Mike McLean
2ff52c0dc3 refactor 2020-01-02 10:45:48 +01:00
Yu Ming Zhu
8d4a5c7129 kojid: use binary msg for python3 in *Notification tasks
fixes: #1882
2020-01-02 10:44:20 +01:00
Tomas Kopecek
0e5042832d split admin_emails option in koji.add_mail_logger
If there are multiple e-mailes separated by comma of space, they should
be split for correct usage of smtplib.sendmail.

Fixes: https://pagure.io/koji/issue/1240
2019-12-19 10:31:58 +01:00
Yu Ming Zhu
79a387e90a sort imports in builder 2019-12-12 11:00:58 +00:00
Tomas Kopecek
418376dc81 build can wait for actual repo
New options --wait-for-build and --wait-for-repo for build command are
simplification of using wait-repo + build command succession. They have
same semantices as the original one.

Fixes: https://pagure.io/koji/issue/1619
2019-12-12 15:29:27 +01:00
Tim Smith
80021ab904 Provide for passing credentials to SRPMfromSCM
Some places force every access to their systems to be authenticated for
audit purposes, which means even an https fetch of a source tarball
needs access to some sort of credential.

Provide for a directory intended to contain credentials to be
bind-mounted into the BuildRoot for the BuildSRPMFromSCM task as
"/credentials". Whatever runs a a result of the source_cmd ("make
sources") is responsible for doing something useful with the contents.
2019-12-12 15:22:01 +01:00
Yuming Zhu
93ab8c9df4 fix conflict -r option for kernel version
fixes: #793
related: #821
2019-12-03 09:45:57 +01:00
Yuming Zhu
7a6ebc851f improve test and clean targets in Makefiles 2019-11-27 15:42:30 +01:00
Tomas Kopecek
16279ceefa use --update for dist-repos if possible
Fixes: https://pagure.io/koji/issue/1031
2019-11-27 15:29:56 +01:00
Tomas Kopecek
d662f7e501 Log kernel version used for buildroot
Fixes: https://pagure.io/koji/issue/793
2019-11-25 14:54:34 +01:00
Mohan Boddu
5b3dbd4fd8 Remove title option for livemedia-creator
Signed-off-by: Mohan Boddu <mboddu@bhujji.com>
2019-11-21 15:19:58 +01:00
Tomas Kopecek
956ad744af add _remote.repositories to ignored maven files
Fixes: https://pagure.io/koji/issue/1731
2019-11-07 11:43:09 -05:00
Tomas Kopecek
8d4d74fca0 check existence of maven symlink
Fixes: https://pagure.io/koji/issue/1593
2019-11-06 13:49:04 -05:00
Yu Ming Zhu
135ac0d6ae clean python compiled binaries for non *.py code
fixes: #1698
2019-10-31 13:20:53 +01:00
Tomas Kopecek
3f64a486a9 mostly only mock exit code 10 ends in build.log
Fixes: https://pagure.io/koji/issue/1679
2019-10-16 09:08:44 +02:00
Yu Ming Zhu
081aacbdeb use RawConfigParser for kojid 2019-10-15 09:14:16 +02:00
Yu Ming Zhu
4c1928f377 extract read_config_files util for config parsing 2019-10-15 09:14:16 +02:00
Tomas Kopecek
dd8fd935f4 use preferred arch if there is more options
https://pagure.io/koji/pull-request/323 introduced option to force an
arch for noarch packages. Nevertheless, it is only task architecture and
not buildroot one. So, if builder can provide more possible archs (32/64
bit), it will pick one of these randomly and can go against preferred arch.

Patch introduces "preferred_arch" option for find_arch and use the
task's one when it works with noarch rpm.

Fixes: https://pagure.io/koji/issue/789
2019-10-14 10:12:40 +02:00
Yu Ming Zhu
2034695e88 add an option to set server realm for all clients 2019-10-10 16:37:56 +02:00
Brendan Reilly
8eea824490 Handle sys.exc_clear in Python 3
Fixes: #1641
2019-10-08 14:38:09 +02:00
Tomas Kopecek
0c9ecf3f7c write binary data to ks file
Fixes: https://pagure.io/koji/issue/1617
2019-08-28 15:16:20 +02:00
Tomas Kopecek
74ec938f69 fix krbV error handling
Fixes: https://pagure.io/koji/issue/1575
2019-08-14 11:47:38 -04:00
Patrick Uiterwijk
184fbf02f4 Allow builder to attempt krb if gssapi is available
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2019-08-14 11:47:38 -04:00
Tomas Kopecek
6afee115d0 remove deprecated BuildRoot.uploadDir()
Fixes: https://pagure.io/koji/issue/1455
2019-08-12 17:58:35 -04:00
Mike McLean
c9ade098ed adjust tag macro namespace 2019-07-15 10:40:54 -04:00
David Abdurachmanov
7e1809bafd Add support for tag/target macros for Mageia
Discussions with Son_Goku (from Mageia) on IRC revealed that they need
per tag/target macro support in Koji. We store tag macros in 'Extra Options'
and macro has to start with '%'.

Example:

    % koji taginfo f29
    Tag: f29 [11]
    Arches: riscv64
    Groups:
    Required permission: 'admin'
    Tag options:
      %mymacro : 'value123'
      %packager : 'Fedora Project RISC-V'
      mock.package_manager : 'dnf'
    [..]

Tag macros can be added using koji edit-tag.

The tag macros are allowed to overwrite macros set in koji.genMockConfig.
For example, %packager is set in koji.genMockConfig by default. The mock
configuration generated by kojid was:

    [..]
    config_opts['macros']['%_host'] = 'riscv64-koji-linux-gnu'
    config_opts['macros']['%mymacro'] = 'value123'
    config_opts['macros']['%_host_cpu'] = 'riscv64'
    config_opts['macros']['%vendor'] = 'Fedora Project'
    config_opts['macros']['%distribution'] = 'Fedora Project'
    config_opts['macros']['%_topdir'] = '/builddir/build'
    config_opts['macros']['%_rpmfilename'] = '%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm'
    config_opts['macros']['%packager'] = 'Fedora Project RISC-V'
    [..]

This two-line change was tested on Fedora RISC-V koji instance.

Signed-off-by: David Abdurachmanov <david.abdurachmanov@gmail.com>
2019-07-15 10:10:29 -04:00
Tomas Kopecek
4d248f460b use RawConfigParser for kojid
We are using percent signs in host_principal_format. As python3 replaced
ConfigParser with SafeConfigParser, parsing now fails.

Fixes: https://pagure.io/koji/issue/1543
2019-07-12 09:50:23 -04:00
Kevin Fenzi
05cd2fe120 kojibuilder: Pass mergerepo_c --all for bare mode as well.
The '--koji' mode of mergerepo_c implies --all, and thats what we want
for bare mode as well. The default merge mode is 'repo' which uses the
first found package by name if multiple packages with the same name are
in the multiple repos. This causes chaos with rhel8 repos as GA and
updates content are in the same repo and if it hits an old version of
one package and an update of another that depends on the new version
it blows up.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2019-07-11 15:14:28 -04:00
Tomas Kopecek
a5014ac29b set module_hotfixes=1 in yum.conf via tag config
New field in tag extra 'mock.yum.module_hotfixes' can be used to set
module_hotfixes=0/1 in yum.conf's inside mock's buildroots.

Fixes: https://pagure.io/koji/issue/1521
2019-07-11 15:12:47 -04:00
Tomas Kopecek
5882bc249b remove unneeded chown 2019-06-26 09:06:48 +02:00
Tomas Kopecek
2bef7c16dc rename source_srpm to srpm 2019-06-26 09:06:48 +02:00
Mike McLean
17128a67fd use getBuildConfig with event_id to check for rebuild_srpm setting 2019-06-26 09:06:48 +02:00
Tomas Kopecek
279e6533f6 rebuildSRPM task
New task rebuildSRPM for srpms uploaded to koji. If policy
'rebuild_srpm' returns True, build task will spawn rebuildSRPM task
first to recreate SRPM with updated buildroot macros, typically 'dist'.
Policy has access to same data as build_from_srpm and build_from_repo_id
policies. (user_id, source, task_id, build_tag, skip_tag, target, tag)

Fixes: https://pagure.io/koji/issue/1396
2019-06-26 09:06:48 +02:00
Mike McLean
0899f74de3 Don't pass block list in bare merge mode
Related: https://pagure.io/koji/issue/1404

mergerepos_c doesn't allow -b without --koji
2019-06-19 11:16:35 -04:00
Tomas Kopecek
be1a56647a createrepo_c is used by default now
Fixes: https://pagure.io/koji/issue/716
2019-06-06 16:24:17 -04:00
Tomas Kopecek
e3c5a03156 deprecate BuildRoot.uploadDir method
Fixes: https://pagure.io/koji/issue/839
2019-05-31 19:58:22 -04:00
Mikolaj Izdebski
3d058c50db Allow generating separate src repo for build repos
Fixes #1266
2019-05-23 10:01:12 -04:00
Mike McLean
fc250d09c2 typo in comment 2019-05-20 17:07:50 -04:00
Tomas Kopecek
3cb64be496 py2.6 compatibility fix
Python 2.6 doesn't support context manager for GzipFile. Revert to
original behaviour.

Fixes: https://pagure.io/koji/issue/1431
2019-05-20 17:07:50 -04:00
Mike McLean
be6ae53fb6 handle bare merge mode
Fixes https://pagure.io/koji/issue/1404
2019-05-06 11:23:19 -04:00
Brian Stinson
af1efed6fa build_srpm: Wait until after running the sources command to check for
alt_sources_dir

In the RPM layout, it's possible that the SOURCES directory might be
completely empty save for the lookaside payload. In this case a SOURCES
directory wouldn't exist in SCM, and wouldn't be created until the
sources command is run.
2019-05-06 10:47:36 -04:00
Igor Gnatenko
a66b4ddfb4
kojid: Download only 'repomd.xml'
We have pretty slow connection from s390x koji which helped to uncover
this part. Kojid downloads all files from repomd.xml (incl. filelists)
which is really big. What we really want is just find location of
'origin' (used later in the code).

Signed-off-by: Igor Gnatenko <ignatenkobrain@fedoraproject.org>
2019-04-17 07:38:03 +02:00
Tomas Kopecek
0f8e61554c Use createrepo_update even for first repo run
createrepo_update is currently reusing only old repos from same tag.
Nevertheless, for first newRepo there is no old data, but there is a
high chance, that we inherit something. This inherited repo can be used
also for significant speedup.

Fixes: https://pagure.io/koji/issue/1354
2019-04-09 15:36:32 -04:00
Tomas Kopecek
47ed809a8a Rely on ozif_enabled switch in BaseImageTask
Fixes: https://pagure.io/koji/issue/1345
2019-03-15 10:27:21 -04:00
Tomas Kopecek
12cabb0f2c Remove 'keepalive' option
keepalive is not used anymore anywhere in koji

Fixes: https://pagure.io/koji/issue/1239
2019-03-12 10:34:36 -04:00