Commit graph

117 commits

Author SHA1 Message Date
Yu Ming Zhu
f41b8c70a7 Bandit [B411]: use defusedxml to prevent remote XML attacks
- putting xmlrpc stuff into koji.xmlrpcplus
- adding koji.xmlrpcplus.xmlrpc_server to refer
- replacing refs of original xmlrpc.client.dumps to enhanced
  koji.xmlrpcplus.dumps

fixes: #3964
2024-02-02 09:08:18 +01:00
Mike McLean
1c1807ef6d drop stray typing hint
typo from https://pagure.io/koji/pull-request/3679
2023-07-03 17:13:43 -04:00
Tomas Kopecek
39ecf0538a vm: ignore B113: request_without_timeout
Bandit warning for missing timeout. It would be caught on VM level
https://bandit.readthedocs.io/en/1.7.5/plugins/b113_request_without_timeout.html
2023-04-20 10:00:34 +02:00
Tomas Kopecek
3e8d856bb1 use wrapper only in main thread 2023-03-09 15:17:32 +01:00
Tomas Kopecek
7cc3fe6a52 try to close failing connection 2023-03-09 15:17:32 +01:00
Tomas Kopecek
22c88fcce3 vm: Retry libvirt connection
Related: https://pagure.io/koji/issue/985
2023-03-09 15:17:32 +01:00
Jana Cupova
e2e0914eb9 Set daemon = true instead of call deaprecated setDaemon
Fixes: https://pagure.io/koji/issue/3698
2023-03-03 09:36:25 +01:00
Ken Dreyer
bf6a52c936 kojivmd: check for HTTP errors in getFile()
kojivmd proxies files from Koji's topurl through the getFile() RPC.
In some cases (misconfigurations), kojivmd can fail to download files
from topurl.

Prior to this change, if kojivmd failed to download a file (for example,
a 404 error), it would silently cache the 404 error HTTP body contents
and pass those on to verifyChecksum(). As a result, kojivmd would verify
the checksum of an HTML error page, rather than checksuming the intended
Koji archive file. In this scenario, it's difficult for administrators
to diagnose why checksums are not matching.

Check the HTTP response for errors before doing anything with the
response contents. With this change, winbuild tasks will fail with an
easier-to-understand "HTTP 404 not found" HTTPError, rather than a Koji
BuildError about checksums.
2022-11-08 17:12:03 -05:00
Tomas Kopecek
bd95153cb3 vm: handle waitrepo tasks in kojivmd
When moved there, kojivmd can import it and wait for
regen repos it started.

Related: https://pagure.io/koji/issue/3536
2022-10-21 14:23:47 +02:00
Ken Dreyer
b194a9ad3c kojivmd: cleanup VMs with UNDEFINE_NVRAM
Newer virtual machines may use nvram, and libvirt needs a special flag
to clean these up. Prior to this change (with libvirt-8.0.0-8.1.el9_0):

  File "/usr/sbin/kojivmd", line 1072, in cleanupVM
    vm.undefine()
  File "/usr/lib64/python3.9/site-packages/libvirt.py", line 3178, in undefine
    raise libvirtError('virDomainUndefine() failed')
  libvirt.libvirtError: Requested operation is not valid: cannot undefine
    domain with nvram

RHEL 7 first introduced the libvirt.VIR_DOMAIN_UNDEFINE_NVRAM flag in
libvirt-python-1.2.8-7.el7 (rhbz#1144284)
2022-09-16 09:10:19 +02:00
Ken Dreyer
953bbba183 kojivmd: narrow error handling for missing VMs
lookupByName() could raise libvirt.libvirtError for many different
reasons (libvirt connection problems, etc).

If this kojivmd host does not have this VM available
(VIR_ERR_NO_DOMAIN), and we should log the "VM not available" message
and skip taking the task.

If the error is something else, this is unexpected, and we should raise
it in the logs so the administrator can see it at non-debug log levels.
2022-09-16 09:04:27 +02:00
Ken Dreyer
4db109cbaf kojivmd: import xmlrpc.server
The xmlrpc.server module is separate from the main xmlrpc module, and we
must explicitly import it in order to use it in DaemonXMLRPCServer.
2022-09-16 09:00:21 +02:00
Ken Dreyer
fa5cba7117 kojivmd: pass "-F qcow2" to qemu-img create
In RHEL 9 (qemu-img-6.2.0-11.el9_0.3), the "qemu-img create" command
requires the "-F" flag that specifies the second image's format.

qemu-img-1.5.3-175.el7_9.6 recognizes this -F flag also, so it should be
safe to use on all supported platforms.
2022-09-15 11:58:04 -04:00
Tomas Kopecek
ba2e1e520b basic security checks with bandit
Fixes: https://pagure.io/koji/issue/3042
2021-10-20 15:20:14 +02:00
Tomas Kopecek
73399e517c kojivmd: py3 compatibility issues
Fixes: https://pagure.io/koji/issue/3059
2021-09-30 12:39:23 +02:00
Yu Ming Zhu
0489ad7187 kojivmd: change opts allowed_scms_by_* to allowed_scms_use_*
fixes: #3049
2021-09-29 09:19:36 +02:00
Yu Ming Zhu
a3f19e0f12 more reasonable parameter name, and more doc strs 2021-08-16 17:55:26 +00:00
Yu Ming Zhu
47c4b5d70b kojid: extend SCM.assert_allowed with hub policy
This is a simple extention of `SCM.assert_allowed`

- `assert_allowed_by_policy` will set the default "use_common" to False which is different to the old behavior
- `channel`, `user_id`, `scratch` are passed in the `policy_data` with scminfo right now.

This is a prototype for this change, and there are some other solutions could be implemented too

- Use a scmpolicy plugin as `postSCMCheckout` callback, the pro is that we can do more checks after the source is initialized on builder, meanwhile, the con is that the source will be downloaded even it is denied by policy. It might be a potential risk?
- Do the scm check in hub's `make_task`, this looks straightforward, but may lack some builder's information

fixes: #2757
2021-08-16 17:55:26 +00:00
Tomas Kopecek
96ae0ecef5 explicit encoding for text file operations
Fixes: https://pagure.io/koji/issue/2641
2021-01-26 13:36:25 +01:00
Tomas Kopecek
883f34cde4 kojivmd: fix typo
Fixes: https://pagure.io/koji/issue/2625
2021-01-06 17:02:35 +01:00
Tomas Kopecek
0f56d39487 remove deprecated --ca option
Fixes: https://pagure.io/koji/issue/2192
2020-11-23 11:03:38 +01:00
Tomas Kopecek
5a4efd6d07 fix merge problems 2020-10-06 15:10:22 +02:00
Tomas Kopecek
48434f4393 vm in py3 2020-10-06 15:00:23 +02:00
Tomas Kopecek
f06a8c0bca fix flake8 errors 2020-08-25 13:05:02 +02:00
Tomas Kopecek
f8419e3626 remove deprecated krbV support
Fixes: https://pagure.io/koji/issue/1991
2020-07-24 15:08:47 -04:00
Tomas Kopecek
bb0ea0d6d5 vm: clone mac address via xml
Fixes: https://pagure.io/koji/issue/2289
2020-06-24 14:43:17 +02:00
Yuming Zhu
65f5b6489b fix param name for uploadFile 2020-06-24 14:42:01 +02:00
Tomas Kopecek
50bcb6f3bc replace md5 with sha256 2020-06-24 14:42:01 +02:00
Yu Ming Zhu
a893e8bf2b a wrapper ignoring FIPS for hashlib.md5 2020-06-24 14:42:00 +02:00
Tomas Kopecek
0132313f92 replace logging.warn with warning
call is deprected in python 3.7

Fixes: https://pagure.io/koji/issue/2135
2020-04-22 13:47:15 +02:00
Yuming Zhu
cf34706f04 use Exception instead of BaseException for bare expection 2020-03-03 21:38:22 +08:00
Yuming Zhu
2a2c5cb729 flake8: apply W rules (prefering W503) 2020-03-03 21:38:22 +08:00
Yuming Zhu
c5db34a8e1 flake8: apply E501 with max-line-length=99 2020-03-03 21:38:22 +08:00
Yuming Zhu
f1ba6cefd7 flake8: apply all rest E7 rules 2020-03-03 21:38:22 +08:00
Yuming Zhu
6dd77e0fa4 flake8: apply E502 rule 2020-03-03 21:38:22 +08:00
Yuming Zhu
0f727a2ab4 flake8: apply E3 rules 2020-03-03 21:38:22 +08:00
Yuming Zhu
05340b146b flake8: apply E2 rules except E266 2020-03-03 21:38:21 +08:00
Yuming Zhu
ce1f9928af flake8: apply E1 rules 2020-03-03 21:38:21 +08:00
Yu Ming Zhu
49504073b1 refine import style 2020-03-03 21:35:08 +08:00
Yuming Zhu
97cfaa4fcf flake8: follow E265 rule 2020-03-03 21:35:08 +08:00
Yuming Zhu
642508ccf6 flake8: follow all F rules 2020-03-03 21:35:08 +08:00
Tomas Kopecek
638f6cc6f2 replace xmlrpc_client exception with requests
It is broken in startup function for kojid and kojivmd from the
introduction of requests.

Fixes: https://pagure.io/koji/issue/1914
2020-02-21 12:40:42 +01:00
Tomas Kopecek
0e5042832d split admin_emails option in koji.add_mail_logger
If there are multiple e-mailes separated by comma of space, they should
be split for correct usage of smtplib.sendmail.

Fixes: https://pagure.io/koji/issue/1240
2019-12-19 10:31:58 +01:00
Yu Ming Zhu
5b83c682e4 remove unused imports 2019-12-12 11:00:58 +00:00
Yu Ming Zhu
77bc43c07a sort imports in vm 2019-12-12 11:00:56 +00:00
Tomas Kopecek
98b83eff69 fix downloads w/o content-length
Fixes: https://pagure.io/koji/issue/982
2019-11-25 14:51:08 +01:00
Yu Ming Zhu
4c1928f377 extract read_config_files util for config parsing 2019-10-15 09:14:16 +02:00
Tomas Kopecek
46f85c9b86 verifyChecksum fails for non-output files
PR #967 introduced another error. Before verifyChecksum was used only
from kojikamid and it made sense, that it failed on other files. Now it
is used also for build requires, etc. As it makes no harm (read-only),
we can extend this check for other paths.

Fixes: https://pagure.io/koji/issue/1669
2019-10-15 09:10:10 +02:00
Yu Ming Zhu
2034695e88 add an option to set server realm for all clients 2019-10-10 16:37:56 +02:00
Tomas Kopecek
63648170df Fix typo preventing vm builds
Bug introduced in PR #967 prevents vm builds to work. There is an
additional option to method verifyChecksum.

Fixes: https://pagure.io/koji/issue/1665
2019-10-09 09:15:14 +02:00