Commit graph

181 commits

Author SHA1 Message Date
Yu Ming Zhu
f41b8c70a7 Bandit [B411]: use defusedxml to prevent remote XML attacks
- putting xmlrpc stuff into koji.xmlrpcplus
- adding koji.xmlrpcplus.xmlrpc_server to refer
- replacing refs of original xmlrpc.client.dumps to enhanced
  koji.xmlrpcplus.dumps

fixes: #3964
2024-02-02 09:08:18 +01:00
Yu Ming Zhu
444f4394c1 remove unnecessary reject_draft refs in HostExports
As we should be able to guarantee that builder won't do that.
2024-01-05 10:00:49 +00:00
Yu Ming Zhu
72fa1b8c58 reject draft build in buildroot of winbuild 2024-01-05 10:00:40 +00:00
Mike McLean
1c1807ef6d drop stray typing hint
typo from https://pagure.io/koji/pull-request/3679
2023-07-03 17:13:43 -04:00
Tomas Kopecek
39ecf0538a vm: ignore B113: request_without_timeout
Bandit warning for missing timeout. It would be caught on VM level
https://bandit.readthedocs.io/en/1.7.5/plugins/b113_request_without_timeout.html
2023-04-20 10:00:34 +02:00
Tomas Kopecek
3e8d856bb1 use wrapper only in main thread 2023-03-09 15:17:32 +01:00
Tomas Kopecek
7cc3fe6a52 try to close failing connection 2023-03-09 15:17:32 +01:00
Tomas Kopecek
22c88fcce3 vm: Retry libvirt connection
Related: https://pagure.io/koji/issue/985
2023-03-09 15:17:32 +01:00
Jana Cupova
e2e0914eb9 Set daemon = true instead of call deaprecated setDaemon
Fixes: https://pagure.io/koji/issue/3698
2023-03-03 09:36:25 +01:00
Ken Dreyer
35512061fa koijkamid: remove clamav scanner
The ClamAV virus scanner in Cygwin is buggy and unmaintained. It takes
significant time to run the scanner during each build, and Windows has
better malware scanners now.

Remove the ClamAV virus scanning from kojikamid.
2023-01-10 12:02:16 +01:00
Ken Dreyer
bb1d61569e kojikamid: docstring for getFile method
Document the getFile method.
2022-11-10 09:50:42 +01:00
Ken Dreyer
64a856f62b kojikamid: write buildrequires files as bytes
Fix a TypeError when downloading buildrequires files on Python 3.
base64.b64decode() returns bytes, so we must open the file for writing
in bytes mode, not text mode.

Add a unit test that verifies this behavior.
2022-11-09 16:50:17 -05:00
Ken Dreyer
e0682ac810 kojivmd: update example topurl
The usual name for this location is "kojifiles", not "kojiroot". Update
the example configuration to match what users will normally expect.
2022-11-08 17:23:02 -05:00
Ken Dreyer
bf6a52c936 kojivmd: check for HTTP errors in getFile()
kojivmd proxies files from Koji's topurl through the getFile() RPC.
In some cases (misconfigurations), kojivmd can fail to download files
from topurl.

Prior to this change, if kojivmd failed to download a file (for example,
a 404 error), it would silently cache the 404 error HTTP body contents
and pass those on to verifyChecksum(). As a result, kojivmd would verify
the checksum of an HTML error page, rather than checksuming the intended
Koji archive file. In this scenario, it's difficult for administrators
to diagnose why checksums are not matching.

Check the HTTP response for errors before doing anything with the
response contents. With this change, winbuild tasks will fail with an
easier-to-understand "HTTP 404 not found" HTTPError, rather than a Koji
BuildError about checksums.
2022-11-08 17:12:03 -05:00
Tomas Kopecek
bd95153cb3 vm: handle waitrepo tasks in kojivmd
When moved there, kojivmd can import it and wait for
regen repos it started.

Related: https://pagure.io/koji/issue/3536
2022-10-21 14:23:47 +02:00
Ken Dreyer
b194a9ad3c kojivmd: cleanup VMs with UNDEFINE_NVRAM
Newer virtual machines may use nvram, and libvirt needs a special flag
to clean these up. Prior to this change (with libvirt-8.0.0-8.1.el9_0):

  File "/usr/sbin/kojivmd", line 1072, in cleanupVM
    vm.undefine()
  File "/usr/lib64/python3.9/site-packages/libvirt.py", line 3178, in undefine
    raise libvirtError('virDomainUndefine() failed')
  libvirt.libvirtError: Requested operation is not valid: cannot undefine
    domain with nvram

RHEL 7 first introduced the libvirt.VIR_DOMAIN_UNDEFINE_NVRAM flag in
libvirt-python-1.2.8-7.el7 (rhbz#1144284)
2022-09-16 09:10:19 +02:00
Ken Dreyer
953bbba183 kojivmd: narrow error handling for missing VMs
lookupByName() could raise libvirt.libvirtError for many different
reasons (libvirt connection problems, etc).

If this kojivmd host does not have this VM available
(VIR_ERR_NO_DOMAIN), and we should log the "VM not available" message
and skip taking the task.

If the error is something else, this is unexpected, and we should raise
it in the logs so the administrator can see it at non-debug log levels.
2022-09-16 09:04:27 +02:00
Ken Dreyer
4db109cbaf kojivmd: import xmlrpc.server
The xmlrpc.server module is separate from the main xmlrpc module, and we
must explicitly import it in order to use it in DaemonXMLRPCServer.
2022-09-16 09:00:21 +02:00
Ken Dreyer
fa5cba7117 kojivmd: pass "-F qcow2" to qemu-img create
In RHEL 9 (qemu-img-6.2.0-11.el9_0.3), the "qemu-img create" command
requires the "-F" flag that specifies the second image's format.

qemu-img-1.5.3-175.el7_9.6 recognizes this -F flag also, so it should be
safe to use on all supported platforms.
2022-09-15 11:58:04 -04:00
Tomas Kopecek
ba2e1e520b basic security checks with bandit
Fixes: https://pagure.io/koji/issue/3042
2021-10-20 15:20:14 +02:00
Tomas Kopecek
73399e517c kojivmd: py3 compatibility issues
Fixes: https://pagure.io/koji/issue/3059
2021-09-30 12:39:23 +02:00
Yu Ming Zhu
0489ad7187 kojivmd: change opts allowed_scms_by_* to allowed_scms_use_*
fixes: #3049
2021-09-29 09:19:36 +02:00
Yu Ming Zhu
a3f19e0f12 more reasonable parameter name, and more doc strs 2021-08-16 17:55:26 +00:00
Yu Ming Zhu
47c4b5d70b kojid: extend SCM.assert_allowed with hub policy
This is a simple extention of `SCM.assert_allowed`

- `assert_allowed_by_policy` will set the default "use_common" to False which is different to the old behavior
- `channel`, `user_id`, `scratch` are passed in the `policy_data` with scminfo right now.

This is a prototype for this change, and there are some other solutions could be implemented too

- Use a scmpolicy plugin as `postSCMCheckout` callback, the pro is that we can do more checks after the source is initialized on builder, meanwhile, the con is that the source will be downloaded even it is denied by policy. It might be a potential risk?
- Do the scm check in hub's `make_task`, this looks straightforward, but may lack some builder's information

fixes: #2757
2021-08-16 17:55:26 +00:00
Yu Ming Zhu
edd2dbadc5 fix flake8 2021-08-13 18:46:03 +00:00
Tomas Kopecek
06528a5650 use decode for py3 bytes/strings
Fixes: https://pagure.io/koji/issue/2976
2021-08-11 12:41:58 +02:00
Tomas Kopecek
0cfb2c0280 py3 Popen text mode fix 2021-08-11 12:41:58 +02:00
Tomas Kopecek
13f4023bf5 flake8 fixes 2021-02-16 14:43:49 +01:00
Tomas Kopecek
c6e69b4f8b backport py27 compatible file open with encoding
client and builder needs to run on py27 which doesn't support
open(encoding='utf-8')

Related: https://pagure.io/koji/issue/2641
2021-02-16 14:13:03 +01:00
Tomas Kopecek
96ae0ecef5 explicit encoding for text file operations
Fixes: https://pagure.io/koji/issue/2641
2021-01-26 13:36:25 +01:00
Tomas Kopecek
883f34cde4 kojivmd: fix typo
Fixes: https://pagure.io/koji/issue/2625
2021-01-06 17:02:35 +01:00
lrossett
1c787a323f flake8 fixes 2020-12-14 13:26:00 +01:00
Tomas Kopecek
0f56d39487 remove deprecated --ca option
Fixes: https://pagure.io/koji/issue/2192
2020-11-23 11:03:38 +01:00
Tomas Kopecek
5a4efd6d07 fix merge problems 2020-10-06 15:10:22 +02:00
Tomas Kopecek
0c833c1236 remove sysv from Makefiles 2020-10-06 15:00:23 +02:00
Tomas Kopecek
48434f4393 vm in py3 2020-10-06 15:00:23 +02:00
Tomas Kopecek
732be23bb5 vm: revert to RawConfigParser
Fixes: https://pagure.io/koji/issue/2277
2020-08-26 09:27:29 +02:00
Tomas Kopecek
f06a8c0bca fix flake8 errors 2020-08-25 13:05:02 +02:00
Tomas Kopecek
f8419e3626 remove deprecated krbV support
Fixes: https://pagure.io/koji/issue/1991
2020-07-24 15:08:47 -04:00
Tomas Kopecek
bb0ea0d6d5 vm: clone mac address via xml
Fixes: https://pagure.io/koji/issue/2289
2020-06-24 14:43:17 +02:00
Yuming Zhu
65f5b6489b fix param name for uploadFile 2020-06-24 14:42:01 +02:00
Tomas Kopecek
50bcb6f3bc replace md5 with sha256 2020-06-24 14:42:01 +02:00
Yu Ming Zhu
a893e8bf2b a wrapper ignoring FIPS for hashlib.md5 2020-06-24 14:42:00 +02:00
Tomas Kopecek
64a0033150 fix additional flake8 problems 2020-05-13 09:56:36 +02:00
Tomas Kopecek
0132313f92 replace logging.warn with warning
call is deprected in python 3.7

Fixes: https://pagure.io/koji/issue/2135
2020-04-22 13:47:15 +02:00
Yuming Zhu
cf34706f04 use Exception instead of BaseException for bare expection 2020-03-03 21:38:22 +08:00
Yuming Zhu
2a2c5cb729 flake8: apply W rules (prefering W503) 2020-03-03 21:38:22 +08:00
Yuming Zhu
c5db34a8e1 flake8: apply E501 with max-line-length=99 2020-03-03 21:38:22 +08:00
Yuming Zhu
f1ba6cefd7 flake8: apply all rest E7 rules 2020-03-03 21:38:22 +08:00
Yuming Zhu
6dd77e0fa4 flake8: apply E502 rule 2020-03-03 21:38:22 +08:00