debian-forge-composer/internal
Achilleas Koutsou a7a1f1ac07 dnfjson: size-based cache management
Functions for managing repository cache management based on a max
desirable size for the entire dnf-json cache directory.
While none of the functions are currently used, the workflow should
be as follows:
- Update the timestamp of a repository whenever it's used in a
  transaction by calling `touchRepo()` with the repository ID and the
  current time.
- Update the internal cache information when desired by calling
  `updateInfo()`.  This should be called for example after multiple
  depsolve transactions are run for a single build request.
- Shrink the cache to below the configured maxSize by calling
  `shrink()`.

The most important work happens in `updateInfo()`.  It collects all the
information it needs from the on-disk cache directories and organises it
in a way that makes it convenient for the `shrink()` function to run
efficiently.  It stores three important pieces of information:
1. repoElements: a map that links a repository ID with all the
   information about a repository's cache:
    - the top-level elements (files and directories) for the cache
    - size of the repository cache (total of all elements)
    - most recent mtime from all the elements which, if the
      `touchRepo()` call is consistently used, should reflect the most
      recent time the repository was used
2. repoRecency: a list of repository IDs sorted by mtime (oldest first)
3. size: the total size of the cache (total of all repository caches)

This way, when `shrink()` is called, the paths associated with the
least-recently-used repositories can be easily deleted by iterating on
repoRecency, obtaining the repository info from the map, deleting every
path in the repoElements array, and subtracting the repository's size
from the total.  The `shrink()` function stops when the new size is
below the maxSize (or when all repositories have been deleted).
2022-06-10 12:45:41 +01:00
..
auth go.mod: Update openshift-online/ocm-sdk-go 2022-05-19 22:18:42 +02:00
blueprint blueprint: sanitize user home directories 2022-03-28 14:08:57 +02:00
boot distro: housekeeping with cpu arch and arch.Name() 2022-05-10 19:53:41 +02:00
client tests: use TestMain() instead of Init() to compile mock-dnf-json 2022-06-01 11:36:52 +01:00
cloud Upload to HTTPS S3 - Support self signed certificate 2022-05-26 13:46:00 +03:00
cloudapi simplify package set chain handling 2022-06-01 11:36:52 +01:00
common internal/common: introduce function to convert data sizes 2021-11-02 18:32:47 +01:00
crypt crypt: add automatic darwin support 2020-07-17 07:48:22 +02:00
disk disk: test minimum directory size calculations 2022-04-29 08:55:36 +02:00
distro Blacklist amdgpu module on Azure images 2022-06-09 14:18:45 +01:00
distroregistry distro/fedora: refactor based on RHEL 9.0 code 2022-05-09 12:25:21 +02:00
dnfjson dnfjson: size-based cache management 2022-06-10 12:45:41 +01:00
jobqueue osbuild-service-maintenance: Vacuum tables 2022-06-08 21:12:46 +02:00
jsondb test: use T.TempDir to create temporary test directory 2022-04-05 09:27:43 +02:00
kojiapi dnfjson: cache cleanup 2022-06-01 11:36:52 +01:00
mocks dnfjson: drop repo checksums 2022-06-01 11:36:52 +01:00
osbuild1 osbuild1: support VMDK subformat in QEMU assembler 2022-05-04 16:22:29 +02:00
osbuild2 distro/rhel86: ignore SRIOV interface via new udev rule on azure-rhui 2022-05-16 15:46:46 +02:00
ostree ostree: rename errors and introduce ParameterComboError 2022-02-23 11:08:24 +01:00
prometheus prometheus: add tenant label 2022-06-07 16:35:03 +02:00
reporegistry reporegistry: remove unused ReposByArch() function 2022-03-09 23:22:06 +00:00
rhsm internal/rhsm: implement fallback to the previous mechanism 2021-08-09 12:40:23 +02:00
rpmmd simplify package set chain handling 2022-06-01 11:36:52 +01:00
store distro/fedora: refactor based on RHEL 9.0 code 2022-05-09 12:25:21 +02:00
target Support Generic S3 upload in Composer API 2022-06-02 16:12:53 +03:00
test internal/test: add small library for tests 2022-03-08 12:07:00 +01:00
upload koji: fix excessive logging & monitoring 2022-04-05 23:48:30 +02:00
weldr Support Generic S3 upload in Composer API 2022-06-02 16:12:53 +03:00
worker prometheus: add tenant label 2022-06-07 16:35:03 +02:00
tools.go go: vendor the oapi-codegen cmd 2021-02-01 15:32:58 +01:00