diff --git a/README.md b/README.md index bbdb3af..f7e7ad1 100644 --- a/README.md +++ b/README.md @@ -24,10 +24,12 @@ These are general guidelines for writing official bash modules and their documen - If you want to insert another regular string as a suffix or prefix to the `"${variable_name}"`, you should do that in this format: `"prefix-${variable_name}-suffix"` - Use `set -euo pipefail` at the start of the script, to ensure that module will fail the image build if error is caught. - You can also use `set -euxo pipefail` during debugging, where each executed command is printed. This should not be used in a published module. + +Using [Shellcheck](https://www.shellcheck.net/) in your editor is recommended. ### Documentation -Every public module should have a `module.yml` ([reference](https://blue-build.org/reference/module/#moduleyml)) file for metadata and a `README.md` file for an in-depth description. +Every public module should have a `module.yml` (see below) file for metadata and a `README.md` file for an in-depth description. For the documentation of the module in `README.md`, the following guidelines apply: - At the start of each _paragraph_, refer to the module using its name or with "the module", not "it" or "the script". @@ -37,6 +39,56 @@ For the documentation of the module in `README.md`, the following guidelines app For the short module description (`shortdesc:`), the following guidelines apply: - The description should start with a phrase like "The glorb module reticulates splines" or "The tree module can be used to plant trees". +### `module.yml` + +A `module.yml` is the metadata file for a public module, used on the website to generate module reference pages. May be used in future projects to showcase modules and supply some defaults for them. + +#### `name:` + +The name of the module, same as the name of the directory and script. + +#### `shortdesc:` + +A short description of the module, ideally not more than one sentence long. This is used in website metadata or anywhere a shorter module description is needed. + +#### `readme:` + +The URL to the raw contents of the module’s `README.md` in plain text, not HTML. The README may include a top-level heading for readability, but it will be stripped out in favor of `name:` when the README is ingested for the website. + +#### `typespec:` + +The URL to the raw contents of the module’s `.tsp` [TypeSpec](https://typespec.io/) definition in plain text. This will be used for configuration validation in the editor and CLI, and for generating documentation for the module. Please document each configuration option carefully. + +#### `example:` + +A YAML string of example configuration showcasing the configuration options available with inline documentation to describe them. Some of the configuration options may be commented out, with comments describing why one might enable them. The intention here is that the example would be a good place to copy-paste from to get started. + +### [TypeSpec](https://typespec.io/) schema + +Every module folder should include a `.tsp` file containing a model of the module's valid configuration options. This schema syntax should be familiar to programmers used to typed languages, especially TypeScript. The schemas will be compiled to the [JSON Schema](https://json-schema.org/) format and used for validation in editors and CLI. + +- When creating a new module, you can get started easily by copying relevant parts of the `.tsp` file of a module with similar configuration. + - Make sure to change all references to the module's name. + - Here's an example of an empty `.tsp` file for a module. Replace `` with the module's name in kebab-case, and `` with the module's name in PascalCase. + ```tsp + import "@typespec/json-schema"; + using TypeSpec.JsonSchema; + + @jsonSchema("/modules/.json") + model Module { + /** + * https://blue-build.org/reference/modules// + */ + type: "", + } + ``` +- Use docstrings with the `/** */` syntax liberally to describe every option in the configuration. + - Even the `type:` key should be documented as in the example above. + - See [the TypeSpec documentation](https://typespec.io/docs/language-basics/documentation). +- Remember to use the `?` syntax to declare all properties which are not required to use the module successfully as optional. Also declare default values when applicable. + - See [the TypeSpec documentation](https://typespec.io/docs/language-basics/models#optional-properties). +- Make sure to add a semicolon `;` to the end of all property definitions. Without this, the schema compilation will fail. + ### Boot-time Modules > [!IMPORTANT] > Build-time modules are preferred over boot-time modules for better system reliability. diff --git a/modules.json b/modules.json index 2b625dc..ab77836 100644 --- a/modules.json +++ b/modules.json @@ -1,13 +1,14 @@ [ + "https://raw.githubusercontent.com/blue-build/modules/main/modules/files/module.yml", "https://raw.githubusercontent.com/blue-build/modules/main/modules/akmods/module.yml", + "https://raw.githubusercontent.com/blue-build/modules/main/modules/chezmoi/module.yml", "https://raw.githubusercontent.com/blue-build/modules/main/modules/bling/module.yml", "https://raw.githubusercontent.com/blue-build/modules/main/modules/brew/module.yml", - "https://raw.githubusercontent.com/blue-build/modules/main/modules/chezmoi/module.yml", "https://raw.githubusercontent.com/blue-build/modules/main/modules/default-flatpaks/module.yml", - "https://raw.githubusercontent.com/blue-build/modules/main/modules/files/module.yml", "https://raw.githubusercontent.com/blue-build/modules/main/modules/fonts/module.yml", "https://raw.githubusercontent.com/blue-build/modules/main/modules/gnome-extensions/module.yml", "https://raw.githubusercontent.com/blue-build/modules/main/modules/gschema-overrides/module.yml", + "https://raw.githubusercontent.com/blue-build/modules/main/modules/justfiles/module.yml", "https://raw.githubusercontent.com/blue-build/modules/main/modules/rpm-ostree/module.yml", "https://raw.githubusercontent.com/blue-build/modules/main/modules/script/module.yml", "https://raw.githubusercontent.com/blue-build/modules/main/modules/signing/module.yml", diff --git a/modules/akmods/README.md b/modules/akmods/README.md index 1473414..1b82511 100644 --- a/modules/akmods/README.md +++ b/modules/akmods/README.md @@ -22,6 +22,8 @@ See available tags here: https://github.com/ublue-os/akmods/#how-its-organized ## Known issues +### Outdated akmods compared to the current kernel version fail the build + When the upstream base image is failing to build for some time, you will probably notice that this module fails too with this error: ``` Resolving dependencies...done @@ -32,3 +34,20 @@ Problem: package "version_of_akmod" from @commandline requires "version_of_kerne Just wait for the base image build to resolve & akmods module will start working again. If this issue happens for a prolonged period of time, report it to the upstream repo if not already reported or worked on. + +### Some akmods are not installing due to lack of some additional akmod package + +Example of the error: +``` +Resolving dependencies...done +error: Could not depsolve transaction; 1 problem detected: +Problem: conflicting requests +- nothing provides kvmfr-kmod-common >= 0.0.git.21.ba370a9b needed by kmod-kvmfr-6.9.4-200.fc40.x86_64-0.0.git.21.ba370a9b-1.fc40.x86_64 from @commandline +``` + +This happens when the mentioned akmod is not pulled from ublue-os/akmods COPR repo, but from some other one. +Those akmods are rare & they are residing in `extra` akmods stream. +There is also the information of repo source of the akmod, where you can see which akmod is the "exotic" one. +All this information can be seen in [`akmods` repo](https://github.com/ublue-os/akmods#kmod-packages). + +The solution to this problem is to add the affected akmod repo to [`rpm-ostree`](https://blue-build.org/reference/modules/rpm-ostree/) module in `repos` section. diff --git a/modules/akmods/akmods.sh b/modules/akmods/akmods.sh index 7c863e9..fc3193b 100644 --- a/modules/akmods/akmods.sh +++ b/modules/akmods/akmods.sh @@ -15,7 +15,6 @@ function SET_HIGHER_PRIORITY_AKMODS_REPO { } get_yaml_array INSTALL '.install[]' "$1" -SURFACE=$(rpm -qa --queryformat '%{NAME}\n' | awk '$0 == "kernel-surface"') INSTALL_PATH=("${INSTALL[@]/#/\/tmp/rpms/kmods/*}") INSTALL_PATH=("${INSTALL_PATH[@]/%/*.rpm}") @@ -24,15 +23,8 @@ INSTALL_STR=$(echo "${INSTALL_PATH[*]}" | tr -d '\n') if [[ ${#INSTALL[@]} -gt 0 ]]; then echo "Installing akmods" echo "Installing: $(echo "${INSTALL[*]}" | tr -d '\n')" - if [[ -n "$SURFACE" ]]; then - SET_HIGHER_PRIORITY_AKMODS_REPO - ENABLE_MULTIMEDIA_REPO - rpm-ostree install kernel-surface-devel-matched $INSTALL_STR - DISABLE_MULTIMEDIA_REPO - else - SET_HIGHER_PRIORITY_AKMODS_REPO - ENABLE_MULTIMEDIA_REPO - rpm-ostree install kernel-devel-matched $INSTALL_STR - DISABLE_MULTIMEDIA_REPO - fi + SET_HIGHER_PRIORITY_AKMODS_REPO + ENABLE_MULTIMEDIA_REPO + rpm-ostree install $INSTALL_STR + DISABLE_MULTIMEDIA_REPO fi diff --git a/modules/akmods/akmods.tsp b/modules/akmods/akmods.tsp new file mode 100644 index 0000000..73639ad --- /dev/null +++ b/modules/akmods/akmods.tsp @@ -0,0 +1,23 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/akmods.json") +model AkmodsModule { + /** The akmods module is a tool used for managing and installing kernel modules built by Universal Blue. + * https://blue-build.org/reference/modules/akmods/ + */ + type: "akmods"; + + /** The kernel your images uses. + * - main: stock kernel / main and nvidia images + * - asus: asus kernel / asus images + * - fsync: fsync kernel / bazzite images + * - surface: surface kernel / surface images + */ + base?: "main" | "asus" | "fsync" | "surface" = "main"; + + /** List of akmods to install. + * See all available akmods here: https://github.com/ublue-os/akmods#kmod-packages + */ + install: Array; +} \ No newline at end of file diff --git a/modules/akmods/module.yml b/modules/akmods/module.yml index 2c355c7..0cc4381 100644 --- a/modules/akmods/module.yml +++ b/modules/akmods/module.yml @@ -1,6 +1,7 @@ name: akmods shortdesc: The akmods module is a tool used for managing and installing kernel modules built by Universal Blue. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/akmods/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/akmods/akmods.tsp example: | type: akmods base: asus # if not specified, classic "main" base is used by default diff --git a/modules/bling/bling.tsp b/modules/bling/bling.tsp new file mode 100644 index 0000000..7a94d77 --- /dev/null +++ b/modules/bling/bling.tsp @@ -0,0 +1,13 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/bling.json") +model BlingModule { + /** The bling module can be used to pull in small "bling" into your image. + * https://blue-build.org/reference/modules/bling/ + */ + type: "bling"; + + /** List of bling submodules to run / things to install onto your system. */ + install: Array<"ublue-update" | "1password" | "dconf-update-service" | "gnome-vrr" | "laptop" | "flatpaksync">; +} \ No newline at end of file diff --git a/modules/bling/module.yml b/modules/bling/module.yml index 73c0eaf..d1abba3 100644 --- a/modules/bling/module.yml +++ b/modules/bling/module.yml @@ -1,12 +1,13 @@ name: bling shortdesc: The bling module can be used to pull in small "bling" into your image. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/bling/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/bling/bling.tsp example: | type: bling install: - # - ublue-update # https://github.com/ublue-os/ublue-update - # - 1password # install 1Password (stable) and `op` CLI tool - # - dconf-update-service # a service unit that updates the dconf db on boot - # - gnome-vrr # enables gnome-vrr for your image - # - laptop # installs TLP and configures your system for laptop usage - # - flatpaksync # allows synchronization of user-installed flatpaks, see separate documentation section + - ublue-update # https://github.com/ublue-os/ublue-update + - 1password # install 1Password (stable) and `op` CLI tool + - dconf-update-service # a service unit that updates the dconf db on boot + - gnome-vrr # enables gnome-vrr for your image + - laptop # installs TLP and configures your system for laptop usage + - flatpaksync # allows synchronization of user-installed flatpaks, see separate documentation section diff --git a/modules/brew/README.md b/modules/brew/README.md index 061d73e..22bfd91 100644 --- a/modules/brew/README.md +++ b/modules/brew/README.md @@ -51,54 +51,6 @@ The brew module installs [Homebrew / Linuxbrew](https://brew.sh/) on your system - `brew-update` runs at the specified time to update Brew to the latest version - `brew-upgrade` runs at the specified time to upgrade Brew packages -## Configuration Options - -### Update - -The Brew update operation updates the Brew binary to latest version. - -#### `auto-update` (optional: boolean, default: true) -If false, disables automatic activation of `brew-update.timer`. - -#### `update-interval` (optional: string, default: '6h') -Defines how often the Brew update service should run. The string is passed directly to `OnUnitInactiveSec` in systemd timer. (Syntax: ['1d', '6h', '10m']). - -#### `update-wait-after-boot` (optional: string, default: '10min') -Time delay after system boot before the first Brew update runs. The string is passed directly to `OnBootSec` in systemd timer. (Syntax: ['1d', '6h', '10m']). - -### Upgrade - -The Brew upgrade operation upgrades all installed Brew packages to latest version. - -#### `auto-upgrade` (optional: boolean, default: true) -If false, disables automatic activation of `brew-upgrade.timer`. - -#### `upgrade-interval` (optional: string, default: '8h') -Defines how often the Brew upgrade service should run. The string is passed directly to `OnUnitInactiveSec` in systemd timer. (Syntax: ['1d', '6h', '10m']). - -#### `upgrade-wait-after-boot` (optional: string, default: '30min') -Time delay after system boot before the first Brew package upgrade runs. The string is passed directly to `OnBootSec` in systemd timer. (Syntax: ['1d', '6h', '10m']). - -### Analytics - -The Homebrew project uses analytics to anonymously collect the information about Brew usage & your system in order to improve the experience of Brew users. - -#### `brew-analytics` (optional: boolean, default: true) -Determines whether to opt-out of Brew analytics. When set to true, analytics are enabled. - -:::caution -Please review the Brew documentation carefully before modifying the settings above. -::: - -### Nofile limits - -Nofile limit refers to the maximum number of open files for a single process. For more information about this, you can read this thread: -https://serverfault.com/questions/577437/what-is-the-impact-of-increasing-nofile-limits-in-etc-security-limits-conf - -#### `nofile-limits` (optional: boolean, default: false) -Determines whether to increase nofile limits for Brew installations. -When set to true, it increases the nofile limits to prevent certain "I/O heavy" Brew packages from failing due to "too many open files" error. However, it's important to note that increasing nofile limits can have potential security implications for malicious applications which would try to abuse storage I/O. Defaults to false for security purposes. - ## Development Setting `DEBUG=true` inside `brew.sh` will enable additional output for debugging purposes during development. diff --git a/modules/brew/brew.tsp b/modules/brew/brew.tsp new file mode 100644 index 0000000..6de2e0c --- /dev/null +++ b/modules/brew/brew.tsp @@ -0,0 +1,42 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/brew.json") +model BrewModule { + /** The brew module installs Homebrew / Linuxbrew at build time and ensures the package manager remains up-to-date. + * https://blue-build.org/reference/modules/brew/ + */ + type: "brew"; + + /** Whether to auto-update the Brew binary using a systemd service. */ + "auto-update"?: boolean = true; + + /** Defines how often the Brew update service should run. The string is passed directly to `OnUnitInactiveSec` in systemd timer. (Syntax: ['1d', '6h', '10m']). */ + "update-interval"?: string = "6h"; + + /** Time delay after system boot before the first Brew update runs. The string is passed directly to `OnBootSec` in systemd timer. (Syntax: ['1d', '6h', '10m']). */ + "update-wait-after-boot"?: string = "10min"; + + /** Whether to auto-upgrade all installed Brew packages using a systemd service. */ + "auto-upgrade"?: boolean = true; + + /** Defines how often the Brew upgrade service should run. The string is passed directly to `OnUnitInactiveSec` in systemd timer. (Syntax: ['1d', '6h', '10m']). */ + "upgrade-interval"?: string = "8h"; + + /** Time delay after system boot before the first Brew package upgrade runs. The string is passed directly to `OnBootSec` in systemd timer. (Syntax: ['1d', '6h', '10m']). */ + "upgrade-wait-after-boot"?: string = "30min"; + + /** Whether to increase nofile limits (limits for number of open files) for Brew installations. + * When set to true, it increases the nofile limits to prevent certain "I/O heavy" Brew packages from failing due to "too many open files" error. + * However, it's important to note that increasing nofile limits can have potential security implications for malicious applications which would try to abuse storage I/O. + * Defaults to false for security purposes. + * + * https://serverfault.com/questions/577437/what-is-the-impact-of-increasing-nofile-limits-in-etc-security-limits-conf + */ + "nofile-limits"?: boolean = false; + + /** Whether to enable Brew analytics. + * The Homebrew project uses analytics to anonymously collect the information about Brew usage & your system in order to improve the experience of Brew users. + */ + "brew-analytics"?: boolean = true; +} \ No newline at end of file diff --git a/modules/brew/module.yml b/modules/brew/module.yml index 5ab4ba2..5c6002f 100644 --- a/modules/brew/module.yml +++ b/modules/brew/module.yml @@ -1,21 +1,8 @@ name: brew shortdesc: The brew module installs Homebrew / Linuxbrew at build time and ensures the package manager remains up-to-date. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/brew/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/brew/brew.tsp example: | type: brew - # Auto-update Brew binary - auto-update: true # Optional - Default: true - Expects type: boolean - # Interval between Brew updates - update-interval: '6h' # Optional - Default: '6h' - Expects type: string - # Time delay after boot before first Brew update - update-wait-after-boot: '10min' # Optional - Default: '10min' - Expects type: string - # Auto-upgrade Brew packages - auto-upgrade: true # Optional - Default: true - Expects type: boolean - # Interval between Brew package upgrades - upgrade-interval: '8h' # Optional - Default: '8h' - Expects type: string - # Time delay after boot before first Brew upgrade - upgrade-wait-after-boot: '30min' # Optional - Default: '30min' - Expects type: string - # Apply nofile limits for Brew installations - nofile-limits: false # Optional - Default: false - Expects type: boolean - # Control Brew analytics - brew-analytics: true # Optional - Default: true - Expects type: boolean + nofile-limits: true # increase nofile limits + brew-analytics: false # disable telemetry diff --git a/modules/chezmoi/README.md b/modules/chezmoi/README.md index 82da397..1a2e816 100644 --- a/modules/chezmoi/README.md +++ b/modules/chezmoi/README.md @@ -34,7 +34,7 @@ You can enable them manually instead when the system has been installed: To enable the services for a single user, run the following command as that user: ```bash -systemctl enable --user chezmoi-init.service chezmoi-update.timer` +systemctl enable --user chezmoi-init.service chezmoi-update.timer ``` To manually enable the services for all users, run the following command with sudo: diff --git a/modules/chezmoi/chezmoi.tsp b/modules/chezmoi/chezmoi.tsp new file mode 100644 index 0000000..a6b1d71 --- /dev/null +++ b/modules/chezmoi/chezmoi.tsp @@ -0,0 +1,31 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/chezmoi.json") +model ChezmoiModule { + /** The chezmoi module installs the latest chezmoi release at build time, along with services to clone a dotfile repository and keep it up-to-date. + * https://blue-build.org/reference/modules/chezmoi/ + */ + type: "chezmoi"; + + /** Git repository to initialize. */ + repository: string; + + /** Whether to enable the modules services globally for all users, if false users need to enable services manually. */ + "all-users"?: boolean = true; + + /** Dotfiles will be updated with this interval. */ + "run-every"?: string = "1d"; + + /** Dotfile updates will wait this long after a boot before running. */ + "wait-after-boot"?: string = "5m"; + + /** Disable the service that initializes `repository` on users that are logged in or have linger enabled UI. */ + "disable-init"?: boolean = false; + + /** Disable the timer that updates chezmoi with the set interval. */ + "disable-update"?: boolean = false; + + /** What to do when file different that exists on your repo is has been changed or exists locally. Accepts "skip" or "replace". */ + "file-conflict-policy"?: "skip" | "replace" = "skip"; +} \ No newline at end of file diff --git a/modules/chezmoi/module.yml b/modules/chezmoi/module.yml index a715dc8..299af96 100644 --- a/modules/chezmoi/module.yml +++ b/modules/chezmoi/module.yml @@ -1,20 +1,10 @@ name: chezmoi shortdesc: The chezmoi module installs the latest chezmoi release at build time, along with services to clone a dotfile repository and keep it up-to-date. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/chezmoi/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/chezmoi/chezmoi.tsp example: | type: chezmoi - # Git repository to initialize - repository: "https://example.org/user/dotfiles" # Required - Default: n/a - Expects type: string - # Whether to enable the modules services globally for all users, if false users need to enable services manually - all-users: true # Optional - Default: true - Expects type: boolean - # Dotfiles will be updated with this interval - run-every: '1d' # Optional - Default: '1d' - Expects type: string - # Dotfile updates will wait this long after a boot before running - wait-after-boot: '5m' # Optional - Default: '5m' - Expects type: string - # Disable the service that initializes `repository` on users that are logged in or have linger enabled - disable-init: false # Optional - Default: false - Expects type: boolean - # Disable the timer that updates chezmoi with the interval set above - disable-update: false # Optional - Default: false - Expects type: boolean - # What to do when file different that exists on your repo is has been changed or exists locally. Accepts "skip" or "replace" - file-conflict-policy: "skip" # Optional - Default: "skip" - Expects type: string + repository: "https://github.com/octocat/dotfiles" # my dotfiles repo + all-users: false # make users have to enable chezmoi manually + file-conflict-policy: replace # override changed files with those from the repo diff --git a/modules/default-flatpaks/default-flatpaks.sh b/modules/default-flatpaks/default-flatpaks.sh index 2d69ada..18af9b1 100644 --- a/modules/default-flatpaks/default-flatpaks.sh +++ b/modules/default-flatpaks/default-flatpaks.sh @@ -100,6 +100,49 @@ configure_lists () { fi } +check_flatpak_id_validity_from_flathub () { + if [[ -f "/usr/share/bluebuild/default-flatpaks/system/repo-info.yml" ]]; then + SYSTEM_FLATHUB_REPO=$(yq .repo-url "/usr/share/bluebuild/default-flatpaks/system/repo-info.yml") + else + SYSTEM_FLATHUB_REPO="" + fi + if [[ -f "/usr/share/bluebuild/default-flatpaks/user/repo-info.yml" ]]; then + USER_FLATHUB_REPO=$(yq .repo-url "/usr/share/bluebuild/default-flatpaks/user/repo-info.yml") + else + USER_FLATHUB_REPO="" + fi + FLATHUB_REPO_LINK="https://dl.flathub.org/repo/flathub.flatpakrepo" + URL="https://flathub.org/apps" + CONFIG_FILE="${1}" + INSTALL_LEVEL="${2}" + get_yaml_array INSTALL ".$INSTALL_LEVEL.install[]" "${CONFIG_FILE}" + get_yaml_array REMOVE ".$INSTALL_LEVEL.remove[]" "${CONFIG_FILE}" + if [[ "${SYSTEM_FLATHUB_REPO}" == "${FLATHUB_REPO_LINK}" ]] || [[ "${USER_FLATHUB_REPO}" == "${FLATHUB_REPO_LINK}" ]]; then + echo "Safe-checking if ${INSTALL_LEVEL} flatpak IDs are typed correctly. If test fails, build also fails" + if [[ ${#INSTALL[@]} -gt 0 ]]; then + for id in "${INSTALL[@]}"; do + if ! curl --output /dev/null --silent --head --fail "${URL}/${id}"; then + echo "ERROR: This ${INSTALL_LEVEL} install flatpak ID '${id}' doesn't exist in FlatHub repo, please check if you typed it correctly in the recipe." + exit 1 + fi + done + fi + if [[ ${#REMOVE[@]} -gt 0 ]]; then + for id in "${REMOVE[@]}"; do + if ! curl --output /dev/null --silent --head --fail "${URL}/${id}"; then + echo "ERROR: This ${INSTALL_LEVEL} removal flatpak ID '${id}' doesn't exist in FlatHub repo, please check if you typed it correctly in the recipe." + exit 1 + fi + done + fi + else + if ! ${MESSAGE_DISPLAYED}; then + echo "NOTE: Flatpak ID safe-check is only available for FlatHub repo" + MESSAGE_DISPLAYED=true + fi + fi +} + echo "Enabling flatpaks module" mkdir -p /usr/share/bluebuild/default-flatpaks/{system,user} mkdir -p /usr/etc/bluebuild/default-flatpaks/{system,user} @@ -130,6 +173,10 @@ if [[ ! $(echo "$1" | yq -I=0 ".user") == "null" ]]; then configure_lists "$1" "user" fi +MESSAGE_DISPLAYED=false +check_flatpak_id_validity_from_flathub "${1}" "system" +check_flatpak_id_validity_from_flathub "${1}" "user" + echo "Configuring default-flatpaks notifications" NOTIFICATIONS=$(echo "$1" | yq -I=0 ".notify") CONFIG_NOTIFICATIONS="/usr/share/bluebuild/default-flatpaks/notifications" diff --git a/modules/default-flatpaks/default-flatpaks.tsp b/modules/default-flatpaks/default-flatpaks.tsp new file mode 100644 index 0000000..4254cbe --- /dev/null +++ b/modules/default-flatpaks/default-flatpaks.tsp @@ -0,0 +1,49 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/default-flatpaks.json") +model DefaultFlatpaksModule { + /** The default-flatpaks module can be used to install or uninstall flatpaks from a configurable remote on every boot. + * https://blue-build.org/reference/modules/default-flatpaks/ + */ + type: "default-flatpaks"; + + /** Whether to send a notification after the install/uninstall is finished. */ + notify?: boolean = false; + + /** Configuration for system flatpaks. */ + system?: { + /** URL of the repo to add. Defaults to Flathub's URL. */ + "repo-url"?: string = "https://dl.flathub.org/repo/flathub.flatpakrepo"; + + /** Name for the repo to add. */ + "repo-name"?: string = "flathub"; + + /** Pretty title for the repo to add. Not set by default. */ + "repo-title"?: string; + + /** List of Flatpak IDs to install from the repo. */ + install?: Array; + + /** List of Flatpak IDs to remove. */ + remove?: Array; + }; + + /** Configuration for user flatpaks. */ + user?: { + /** URL of the repo to add. Defaults to Flathub's URL. */ + "repo-url"?: string = "https://dl.flathub.org/repo/flathub.flatpakrepo"; + + /** Name for the repo to add. */ + "repo-name"?: string = "flathub"; + + /** Pretty title for the repo to add. Not set by default. */ + "repo-title"?: string; + + /** List of Flatpak IDs to install from the repo. */ + install?: Array; + + /** List of Flatpak IDs to remove. */ + remove?: Array; + }; +} \ No newline at end of file diff --git a/modules/default-flatpaks/module.yml b/modules/default-flatpaks/module.yml index 5e2d9dc..0bdcf89 100644 --- a/modules/default-flatpaks/module.yml +++ b/modules/default-flatpaks/module.yml @@ -1,6 +1,7 @@ name: default-flatpaks shortdesc: The default-flatpaks module can be used to install or uninstall flatpaks from a configurable remote on every boot. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/default-flatpaks/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/default-flatpaks/default-flatpaks.tsp example: | modules: # configured multiple times to highlight how options are overridden - type: default-flatpaks diff --git a/modules/files/README.md b/modules/files/README.md index fae4ce2..84b6ef0 100644 --- a/modules/files/README.md +++ b/modules/files/README.md @@ -1,32 +1,32 @@ # `files` -The `files` module can be used to copy directories from `config/files` to +The `files` module can be used to copy directories from `files/` to any location in your image at build time, as long as the location exists at -build time (e.g. you can't put files in `/home/`, because users +build time (e.g. you can't put files in `/home//`, because users haven't been created yet prior to first boot). :::note -If you want to place files into `/etc`, there are two ways to do it: +If you want to place files into `/etc/`, there are two ways to do it: -1. copying a directory in `config/files` directly to `/etc` to add all of its +1. copying a directory in `files/` directly to `/etc` to add all of its files at build time, or -2. putting the files you want there in `/usr/etc` as part of copying things - over to `/usr`, which `rpm-ostree` will then copy to `/etc` at runtime/boot. +2. putting the files you want there in `/usr/etc/` as part of copying things + over to `/usr/`, which `rpm-ostree` will then copy to `/etc/` at runtime/boot. -Typically, you will want to use the latter option (putting files in `/usr/etc`) +Typically, you will want to use the latter option (putting files in `/usr/etc/`) in almost all cases, since that is the proper directory for "system" -configuration templates on atomic Fedora distros, whereas `/etc` is meant for +configuration templates on atomic Fedora distros, whereas `/etc/` is meant for manual overrides and editing by the machine's admin *after* installation (see issue https://github.com/blue-build/legacy-template/issues/28). However, if you -really need something to be in `/etc` *at build time* --- for instance, if you -for some reason need to place a repo file in `/etc/yum.repos.d` in such a way +really need something to be in `/etc/` *at build time* --- for instance, if you +for some reason need to place a repo file in `/etc/yum.repos.d/` in such a way that it is used by a `rpm-ostree` module later on --- then the former option will be necessary. ::: :::caution The `files` module **cannot write to directories that will later be symlinked -to point to other places (typically `/var`) by `rpm-ostree`**. +to point to other places (typically `/var/`) by `rpm-ostree`**. This is because it doesn't make sense for a directory to be both a symlink and a real directory that has had actual files directly copied to it, so the @@ -39,13 +39,13 @@ documentation](https://docs.fedoraproject.org/en-US/fedora-silverblue/technical- here is a list of the directories that `rpm-ostree` symlinks to other locations: -- `/home` → `/var/home` -- `/opt` → `/var/opt` -- `/srv` → `/var/srv` -- `/root` → `/var/roothome` -- `/usr/local` → `/var/usrlocal` -- `/mnt` → `/var/mnt` -- `/tmp` → `/sysroot/tmp` +- `/home/` → `/var/home/` +- `/opt/` → `/var/opt/` +- `/srv/` → `/var/srv/` +- `/root/` → `/var/roothome/` +- `/usr/local/` → `/var/usrlocal/` +- `/mnt/` → `/var/mnt/` +- `/tmp/` → `/sysroot/tmp/` So don't use `files` to copy any files to any of the directories on the left, because at runtime `rpm-ostree` will want to link them to the ones on the diff --git a/modules/files/files.sh b/modules/files/files.sh index 6bfbd91..cbb7107 100644 --- a/modules/files/files.sh +++ b/modules/files/files.sh @@ -5,21 +5,43 @@ set -euo pipefail get_yaml_array FILES '.files[]' "$1" -cd "$CONFIG_DIRECTORY/files" -shopt -s dotglob +# Support for legacy "/tmp/config/" to satisfy transition period to "/tmp/files/" +if [[ "${CONFIG_DIRECTORY}" == "/tmp/config" ]]; then + FILES_DIR="${CONFIG_DIRECTORY}/files" +elif [[ "${CONFIG_DIRECTORY}" == "/tmp/files" ]]; then + FILES_DIR="${CONFIG_DIRECTORY}" +fi +cd "${FILES_DIR}" +shopt -s dotglob + if [[ ${#FILES[@]} -gt 0 ]]; then echo "Adding files to image" for pair in "${FILES[@]}"; do + # Support for legacy recipe format to satisfy transition period to new source/destination recipe format + if [[ $(echo $pair | yq '.source') == "null" || -z $(echo $pair | yq '.source') ]] && [[ $(echo $pair | yq '.destination') == "null" || -z $(echo $pair | yq '.destination') ]]; then + echo "ATTENTION: You are using the legacy module recipe format" + echo " It is advised to switch to new module recipe format," + echo " which contains 'source' & 'destination' YAML keys" + echo " For more details, please visit 'files' module documentation:" + echo " https://blue-build.org/reference/modules/files/" FILE="$PWD/$(echo $pair | yq 'to_entries | .[0].key')" DEST=$(echo $pair | yq 'to_entries | .[0].value') + else + FILE="$PWD/$(echo $pair | yq '.source')" + DEST=$(echo $pair | yq '.destination') + fi if [ -d "$FILE" ]; then if [ ! -d "$DEST" ]; then mkdir -p "$DEST" fi - echo "Copying $FILE to $DEST" + echo "Copying $FILE/* to $DEST" cp -rf "$FILE"/* $DEST - rm -f "$DEST"/.gitkeep + if [[ "${DEST}" =~ *"/" ]] || [[ "${DEST}" == "/" ]]; then + rm -f "${DEST}.gitkeep" + else + rm -f "${DEST}/.gitkeep" + fi elif [ -f "$FILE" ]; then DEST_DIR=$(dirname "$DEST") if [ ! -d "$DEST_DIR" ]; then @@ -27,9 +49,13 @@ if [[ ${#FILES[@]} -gt 0 ]]; then fi echo "Copying $FILE to $DEST" cp -f $FILE $DEST - rm -f "$DEST"/.gitkeep + if [[ "${DEST}" =~ *"/" ]] || [[ "${DEST}" == "/" ]]; then + rm -f "${DEST}.gitkeep" + else + rm -f "${DEST}/.gitkeep" + fi else - echo "File or Directory $FILE Does Not Exist in $CONFIG_DIRECTORY/files" + echo "File or Directory $FILE Does Not Exist in ${FILES_DIR}" exit 1 fi done diff --git a/modules/files/files.tsp b/modules/files/files.tsp new file mode 100644 index 0000000..f6a826a --- /dev/null +++ b/modules/files/files.tsp @@ -0,0 +1,16 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/files.json") +model FilesModule { + /** Copy files to your image at build time + * https://blue-build.org/reference/modules/files/ + */ + type: "files"; + + /** List of files / folders to copy. */ + files: Array> | Array<{ + source: string; + destination: string; + }>; +} diff --git a/modules/files/module.yml b/modules/files/module.yml index 38eab11..bc89139 100644 --- a/modules/files/module.yml +++ b/modules/files/module.yml @@ -1,12 +1,12 @@ name: files shortdesc: Copy files to your image at build time readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/files/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/files/files.tsp example: | modules: - type: files files: - - usr: /usr # copies config/files/usr into your images /usr - # put configuration files you want in /etc/ on a *booted* - # system in /usr/etc/ in the image. - - etc: /etc # copies config/files/etc into your image's /etc immediately, - # for use with further modules. + - source: system # copies `files/system/*` (* means everything inside it) into your image's root folder `/` + destination: / + - source: my-image/usr # copies `files/my-image/usr/*` to `/usr/` inside the image + destination: /usr diff --git a/modules/fonts/fonts.tsp b/modules/fonts/fonts.tsp new file mode 100644 index 0000000..a7c10bb --- /dev/null +++ b/modules/fonts/fonts.tsp @@ -0,0 +1,18 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/fonts.json") +model FontsModule { + /** The fonts module can be used to install fonts from Nerd Fonts or Google Fonts. + * https://blue-build.org/reference/modules/fonts/ + */ + type: "fonts"; + + fonts: { + /** List of Nerd Fonts to install (without the "Nerd Font" suffix). */ + "nerd-fonts"?: Array; + + /** List of Google Fonts to install. */ + "google-fonts"?: Array; + }; +} \ No newline at end of file diff --git a/modules/fonts/module.yml b/modules/fonts/module.yml index a1b5a8a..125b2f1 100644 --- a/modules/fonts/module.yml +++ b/modules/fonts/module.yml @@ -1,6 +1,7 @@ name: fonts shortdesc: The `fonts` module can be used to install fonts from Nerd Fonts or Google Fonts. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/fonts/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/fonts/fonts.tsp example: | type: fonts fonts: diff --git a/modules/gnome-extensions/README.md b/modules/gnome-extensions/README.md index 2c6aff1..a951ee8 100644 --- a/modules/gnome-extensions/README.md +++ b/modules/gnome-extensions/README.md @@ -58,13 +58,45 @@ How to uninstall extensions using the module: ## Known issues -### Some extensions complain about missing gschema.compiled file +### Some extensions use extension-only gschemas.compiled file location -This is a rarity, but some extensions might complain about this one, due to the way they are programmed with hard-coded gschema locations. +This is a rarity, but some extensions might have this issue, due to the way they are programmed with hard-coded gschema locations. Most extensions which follow Gnome extension standards don't have this issue. +Standard location for global `gschema.compiled` file is: +`/usr/share/glib-2.0/schemas/gschema.compiled` + +Those problematic extensions explicitly ask for this extension-only location instead: +`/usr/share/gnome-shell/extensions/$EXT_UUID/schemas/gschemas.compiled` + If you get the error similar to this one (Fly-Pie extension example): -`GLib.FileError: Failed to open file “/usr/share/gnome-shell/extensions/flypie@schneegans.github.com/schemas/gschemas.compiled”: open() failed: No such file or directory` +``` +GLib.FileError: Failed to open file “/usr/share/gnome-shell/extensions/flypie@schneegans.github.com/schemas/gschemas.compiled”: open() failed: No such file or directory +``` Then please open the issue in BlueBuild Modules GitHub repo with the affecting extension, as it's trivial to fix. https://github.com/blue-build/modules/issues/new + +### Some extensions published in https://extensions.gnome.org are hard-coded to user locations + +Those type of extensions are fixed to these locations (... indicates further folders): +- `/usr/local/share/...` (local system location) +- `$HOME/.local/share/...` (user location) + +Those locations are not writable in build-time. + +`/usr/share/...` is the standard location for system Gnome extensions, as outlined in "What does this module do?" section. + +That means that the extension has build instructions for packagers to build the extension either system-wide or user-wide. + +While some extensions might not have this limit even with the instructions above, some extensions might have. + +GSConnect from https://extensions.gnome.org has this limitation & requires the system version of the extension to make it work successfully. +Those system versions are usually provided by the system packagers. + +So the solution is to install the extension from system repository instead if available. + +In this scenario, you will notice the extension error similar to this when trying to run it (notice the explicit request to `/usr/local/share/...` location): +``` +GLib.FileError: Failed to open file “/usr/local/share/glib-2.0/schemas/gschemas.compiled”: open() failed: No such file or directory +``` diff --git a/modules/gnome-extensions/gnome-extensions.tsp b/modules/gnome-extensions/gnome-extensions.tsp new file mode 100644 index 0000000..00cf946 --- /dev/null +++ b/modules/gnome-extensions/gnome-extensions.tsp @@ -0,0 +1,20 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/gnome-extensions.json") +model GnomeExtensionsModule { + /** The gnome-extensions module can be used to install GNOME extensions inside system directory. + * https://blue-build.org/reference/modules/gnome-extensions/ + */ + type: "gnome-extensions"; + + /** List of GNOME extensions to install. + * (case sensitive extension names or extension IDs from https://extensions.gnome.org/) + */ + install?: Array; + + /** List of system GNOME extensions to uninstall. + * Only use this to remove extensions not installed by your package manager. Those extensions should be uninstalled using the package manager instead. + */ + uninstall?: Array; +} diff --git a/modules/gnome-extensions/module.yml b/modules/gnome-extensions/module.yml index f4be325..c8bb78a 100644 --- a/modules/gnome-extensions/module.yml +++ b/modules/gnome-extensions/module.yml @@ -1,6 +1,7 @@ name: gnome-extensions -shortdesc: The gnome-extensions module can be used to install Gnome extensions inside system directory. +shortdesc: The gnome-extensions module can be used to install GNOME extensions inside system directory. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/gnome-extensions/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/gnome-extensions/gnome-extensions.tsp example: | type: gnome-extensions install: diff --git a/modules/gschema-overrides/README.md b/modules/gschema-overrides/README.md index 43ce535..50cf20e 100644 --- a/modules/gschema-overrides/README.md +++ b/modules/gschema-overrides/README.md @@ -6,27 +6,27 @@ This module is similar to using `dconf` configuration, but is better because it What does this module do? -- It copies all content from `/usr/share/glib-2.0/schemas`, except existing gschema.overrides to avoid conflicts, into temporary test location. -- It copies your gschema.overrides you provided in this module from `config/gschema-overrides` into temporary test location. +- It copies all content from `/usr/share/glib-2.0/schemas/`, except existing gschema.overrides to avoid conflicts, into temporary test location. +- It copies your gschema.overrides you provided in this module from `files/gschema-overrides/` into temporary test location. - It tests them for errors in temporary test location by using `glib-compile-schemas` with `--strict` flag. If errors are found, build will fail. -- If test is passed successfully, it copies your gschema.overrides to `/usr/share/glib-2.0/schemas`. -- It compiles gschema using `glib-compile-schemas` in `/usr/share/glib-2.0/schemas` location to include your changes. +- If test is passed successfully, it copies your gschema.overrides to `/usr/share/glib-2.0/schemas/`. +- It compiles gschema using `glib-compile-schemas` in `/usr/share/glib-2.0/schemas/` location to include your changes. Temporary test location is: -`/tmp/bluebuild-schema-test` +`/tmp/bluebuild-schema-test/` ## Usage To use this module, you need to include your gschema.override file(s) in this location (make folder if it doesn't exist): -`config/gschema-overrides` +`files/gschema-overrides/` Then you need to include those file(s) in recipe file, like in example configuration. It is highly recommended to use `zz1-` prefix before your gschema.override name, to ensure that your changes are going to be applied. -Also don't forget to rename your file(s) too with this prefix in `config/gschema-overrides`. +Also don't forget to rename your file(s) too with this prefix in `files/gschema-overrides/`. ## Creating gschema.override files diff --git a/modules/gschema-overrides/gschema-overrides.sh b/modules/gschema-overrides/gschema-overrides.sh index 707f8aa..bf9b01e 100644 --- a/modules/gschema-overrides/gschema-overrides.sh +++ b/modules/gschema-overrides/gschema-overrides.sh @@ -4,7 +4,7 @@ set -euo pipefail get_yaml_array INCLUDE '.include[]' "$1" -schema_include_location="/tmp/config/gschema-overrides" +schema_include_location="${CONFIG_DIRECTORY}/gschema-overrides" schema_test_location="/tmp/bluebuild-schema-test" schema_location="/usr/share/glib-2.0/schemas" gschema_extension=false diff --git a/modules/gschema-overrides/gschema-overrides.tsp b/modules/gschema-overrides/gschema-overrides.tsp new file mode 100644 index 0000000..3f3280a --- /dev/null +++ b/modules/gschema-overrides/gschema-overrides.tsp @@ -0,0 +1,13 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/gschema-overrides.json") +model GschemaOverridesModule { + /** The gschema-overrides module can be used for including system-setting overrides for GTK-based desktop environments. + * https://blue-build.org/reference/modules/gschema-overrides/ + */ + type: "gschema-overrides"; + + /** Gschema override files to test and copy to the correct place. */ + include?: Array; +} \ No newline at end of file diff --git a/modules/gschema-overrides/module.yml b/modules/gschema-overrides/module.yml index c6a27b3..be24147 100644 --- a/modules/gschema-overrides/module.yml +++ b/modules/gschema-overrides/module.yml @@ -1,6 +1,7 @@ name: gschema-overrides shortdesc: The `gschema-overrides` module can be used for including system-setting overrides for GTK-based desktop environments. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/gschema-overrides/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/gschema-overrides/gschema-overrides.tsp example: | type: gschema-overrides include: diff --git a/modules/justfiles/README.md b/modules/justfiles/README.md new file mode 100644 index 0000000..6b0f063 --- /dev/null +++ b/modules/justfiles/README.md @@ -0,0 +1,51 @@ +# `justfiles` + +:::note +The module is only compatible with Universal Blue images. +::: + +The `justfiles` module makes it easy to include [just](https://just.systems/) recipes from multiple files in Universal Blue -based images. It can be useful for example when utilizing DE-specific justfiles when building multiple images. On the other hand, you likely wont need the module if you're building just one image or need just one justfile for all your images. + +## What is just ? + +Just is a command runner (kind of like make) that can be used to supply arbitrary scripts under a single shell command. Images based on Universal Blue bundle a set of these scripts, called recipes, which can be accessed with the `ujust` command. + +For more information, refer to these links: + +* [Official just documentation](https://just.systems/man/en) +* [Universal Blue documentation](https://universal-blue.discourse.group/docs?topic=42) +* [BlueBuild documentation](https://blue-build.org/learn/universal-blue/#custom-just-recipes) + +## What the module does + +1. The module checks if the `files/justfiles/` folder is present. + + * If it's not there, it fails. + +2. The module finds all `.just` files inside of the `files/justfiles/` folder or starting from the relative path specified under `include`. + + * If no `.just` files are found, it fails. + + * The structure of the `files/justfiles/` folder does not matter, folders/files can be placed in there however desired, the module will find all `.just` files. + + * Optionally, the `.just` files can be validated. + +3. The module copies over the files/folders containing `.just` files to `/usr/share/bluebuild/justfiles/`. + + * The folder structure of the copy destination remains the same as in the config folder. + +4. The module generates import lines and appends them to the `/usr/share/ublue-os/just/60-custom.just` file. + + * The module does not overwrite the destination file. New lines are added to an existing file. + + * If the generated import lines are already present, the module skips them to avoid duplications. + +## How to use the module + +Place all your `.just` files or folders with `.just` files inside the `files/justfiles/` folder. If that folder doesn't exist, create it. + +By default, the module will import all files with names ending in `.just` from `files/justfiles/`. You can also specify files or subfolders under `include`, and they will be the only ones imported. + +If you also want to validate your justfiles, set `validate: true`. The validation can be very unforgiving and is turned off by default. + +* The validation command usually prints huge number of lines. To avoid cluttering up the logs, the module will only tell you which files did not pass the validation. You can then use the command `just --fmt --check --unstable --justfile ` to troubleshoot them. diff --git a/modules/justfiles/justfiles.sh b/modules/justfiles/justfiles.sh new file mode 100644 index 0000000..8592b34 --- /dev/null +++ b/modules/justfiles/justfiles.sh @@ -0,0 +1,83 @@ +#!/usr/bin/env bash + +set -euo pipefail + +get_yaml_array CONFIG_SELECTION '.include[]' "$1" +VALIDATE="$(echo "$1" | yq -I=0 ".validate")" + +IMPORT_FILE="/usr/share/ublue-os/just/60-custom.just" +CONFIG_FOLDER="${CONFIG_DIRECTORY}/justfiles" +DEST_FOLDER="/usr/share/bluebuild/justfiles" + +# Abort if justfiles folder is not present +if [ ! -d "${CONFIG_FOLDER}" ]; then + echo "Error: The config folder '${CONFIG_FOLDER}' was not found." + exit 1 +fi + +# Include all files in the folder if none specified +if [[ ${#CONFIG_SELECTION[@]} == 0 ]]; then + CONFIG_SELECTION=($(find "${CONFIG_FOLDER}" -mindepth 1 -maxdepth 1 -exec basename {} \;)) +fi + +for SELECTED in "${CONFIG_SELECTION[@]}"; do + + echo "------------------------------------------------------------------------" + echo "--- Adding folder/file '${SELECTED}'" + echo "------------------------------------------------------------------------" + + # Find all justfiles, starting from 'SELECTED' and get their paths + JUSTFILES=($(find "${CONFIG_FOLDER}/${SELECTED}" -type f -name "*.just" | sed "s|${CONFIG_FOLDER}/||g")) + + # Abort if no justfiles found at 'SELECTED' + if [[ ${#JUSTFILES[@]} == 0 ]]; then + echo "Error: No justfiles were found in '${CONFIG_FOLDER}/${SELECTED}'." + exit 1 + fi + + # Validate all found justfiles if set to do so + if [ "${VALIDATE}" == "true" ]; then + + echo "Validating justfiles" + VALIDATION_FAILED=0 + for JUSTFILE in "${JUSTFILES[@]}"; do + if ! /usr/bin/just --fmt --check --unstable --justfile "${CONFIG_FOLDER}/${JUSTFILE}" &> /dev/null; then + echo "- The justfile '${JUSTFILE}' FAILED validation." + VALIDATION_FAILED=1 + fi + done + + # Exit if any justfiles are not valid + if [ ${VALIDATION_FAILED} -eq 1 ]; then + echo "Error: Some justfiles didn't pass validation." + exit 1 + else + echo "- All justfiles passed validation." + fi + + fi + + # Copy 'SELECTED' to destination folder + echo "Copying folders/files" + mkdir -p "${DEST_FOLDER}/$(dirname ${SELECTED})" + cp -rfT "${CONFIG_FOLDER}/${SELECTED}" "${DEST_FOLDER}/${SELECTED}" + echo "- Copied '${CONFIG_FOLDER}/${SELECTED}' to '${DEST_FOLDER}/${SELECTED}'." + + # Generate import lines for all found justfiles + echo "Adding import lines" + for JUSTFILE in "${JUSTFILES[@]}"; do + + # Create an import line + IMPORT_LINE="import \"${DEST_FOLDER}/${JUSTFILE}\"" + + # Skip the import line if it already exists, else append it to import file + if grep -wq "${IMPORT_LINE}" "${IMPORT_FILE}"; then + echo "- Skipped: '${IMPORT_LINE}' (already present)" + else + echo "${IMPORT_LINE}" >> "${IMPORT_FILE}" + echo "- Added: '${IMPORT_LINE}'" + fi + + done + +done diff --git a/modules/justfiles/justfiles.tsp b/modules/justfiles/justfiles.tsp new file mode 100644 index 0000000..caf2d3b --- /dev/null +++ b/modules/justfiles/justfiles.tsp @@ -0,0 +1,16 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/justfiles.json") +model JustfilesModule { + /** The justfiles module makes it easy to include just recipes from multiple files in Universal Blue -based images. + * https://blue-build.org/reference/modules/justfiles/ + */ + type: "justfiles"; + + /** Whether to validate the syntax of the justfiles against `just --fmt`. (warning: can be very unforgiving) */ + validate?: boolean = false; + + /** List of files or subfolders to include into this image. If omitted, all justfiles will be included. */ + include?: Array; +} \ No newline at end of file diff --git a/modules/justfiles/module.yml b/modules/justfiles/module.yml new file mode 100644 index 0000000..c38d29b --- /dev/null +++ b/modules/justfiles/module.yml @@ -0,0 +1,12 @@ +name: justfiles +shortdesc: The justfiles module makes it easy to include just recipes from multiple files in Universal Blue -based images. +readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/justfiles/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/justfiles/justfiles.tsp +example: | + type: justfiles + validate: true + include: + - common + - gnome/monitors.just + - shared/flatpak/fix-theming.just + - justfile1.just diff --git a/modules/rpm-ostree/README.md b/modules/rpm-ostree/README.md index 537fa8d..03a8383 100644 --- a/modules/rpm-ostree/README.md +++ b/modules/rpm-ostree/README.md @@ -10,6 +10,8 @@ Then the module installs the packages declared under `install:` using `rpm-ostre Installing RPM packages directly from a `http(s)` url that points to the RPM file is also supported, you can just put the URLs under `install:` and they'll be installed along with the other packages. The magic string `%OS_VERSION%` is substituted with the current VERSION_ID (major Fedora version) like with the `repos:` property. +The module can also replace base RPM packages with packages from COPR repo. Under `replace:`, the module finds every pair of keys `- from-repo:` and `packages:`. (Multiple pairs are supported.) The module downloads the COPR repository file declared by `- from-repo:` into `/etc/yum.repos.d/`, and from that repository replaces packages declared under `packages:` using the command `rpm-ostree override replace`. The COPR repository file is then deleted. The magic string `%OS_VERSION%` is substituted with the current VERSION_ID (major Fedora version) as already said above. At the moment, only COPR repo is supported. + :::note [Removed packages are still present in the underlying ostree repository](https://coreos.github.io/rpm-ostree/administrator-handbook/#removing-a-base-package), what `remove` does is kind of like hiding them from the system, it doesn't free up storage space. ::: diff --git a/modules/rpm-ostree/module.yml b/modules/rpm-ostree/module.yml index 72224a2..2771ab0 100644 --- a/modules/rpm-ostree/module.yml +++ b/modules/rpm-ostree/module.yml @@ -1,17 +1,26 @@ name: rpm-ostree shortdesc: The rpm-ostree module offers pseudo-declarative package and repository management using rpm-ostree. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/rpm-ostree/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/rpm-ostree/rpm-ostree.tsp example: | type: rpm-ostree repos: - https://copr.fedorainfracloud.org/coprs/atim/starship/repo/fedora-%OS_VERSION%/atim-starship-fedora-%OS_VERSION%.repo # when including COPR repos, use the %OS_VERSION% magic string - - https://pkgs.tailscale.com/stable/fedora/tailscale.repo - https://brave-browser-rpm-release.s3.brave.com/brave-browser.repo keys: - https://brave-browser-rpm-release.s3.brave.com/brave-core.asc + optfix: + - Tabby # needed because tabby installs into /opt/Tabby install: - - python3-pip - - libadwaita + - starship + - brave-browser + - https://github.com/Eugeny/tabby/releases/download/v1.0.209/tabby-1.0.209-linux-x64.rpm remove: - firefox - firefox-langpacks + replace: + - from-repo: https://copr.fedorainfracloud.org/coprs/trixieua/mutter-patched/repo/fedora-%OS_VERSION%/trixieua-mutter-patched-fedora-%OS_VERSION%.repo + packages: + - mutter + - mutter-common + - gdm \ No newline at end of file diff --git a/modules/rpm-ostree/rpm-ostree.sh b/modules/rpm-ostree/rpm-ostree.sh index d068c4a..bf1afbf 100644 --- a/modules/rpm-ostree/rpm-ostree.sh +++ b/modules/rpm-ostree/rpm-ostree.sh @@ -76,3 +76,44 @@ elif [[ ${#REMOVE[@]} -gt 0 ]]; then echo "Removing: ${REMOVE_STR[*]}" rpm-ostree override remove $REMOVE_STR fi + +get_yaml_array REPLACE '.replace[]' "$1" + +# Override-replace RPM packages +if [[ ${#REPLACE[@]} -gt 0 ]]; then + for REPLACEMENT in "${REPLACE[@]}"; do + + # Get repository + REPO=$(echo "${REPLACEMENT}" | yq -I=0 ".from-repo") + REPO="${REPO//%OS_VERSION%/${OS_VERSION}}" + + # Ensure repository is provided + if [[ "${REPO}" == "null" ]]; then + echo "Error: Key 'from-repo' was declared, but repository URL was not provided." + exit 1 + fi + + # Get info from repository URL + MAINTAINER=$(awk -F'/' '{print $5}' <<< "${REPO}") + REPO_NAME=$(awk -F'/' '{print $6}' <<< "${REPO}") + FILE_NAME=$(awk -F'/' '{print $9}' <<< "${REPO}") + + # Get packages to replace + get_yaml_array PACKAGES '.packages[]' "${REPLACEMENT}" + REPLACE_STR="$(echo "${PACKAGES[*]}" | tr -d '\n')" + + # Ensure packages are provided + if [[ ${#PACKAGES[@]} == 0 ]]; then + echo "Error: No packages were provided for repository '${REPO_NAME}'." + exit 1 + fi + + echo "Replacing packages from repository: '${REPO_NAME}' owned by '${MAINTAINER}'" + echo "Replacing: ${REPLACE_STR}" + + curl --output-dir "/etc/yum.repos.d/" -O "${REPO//[$'\t\r\n ']}" + rpm-ostree override replace --experimental --from repo=copr:copr.fedorainfracloud.org:${MAINTAINER}:${REPO_NAME} ${REPLACE_STR} + rm "/etc/yum.repos.d/${FILE_NAME}" + + done +fi diff --git a/modules/rpm-ostree/rpm-ostree.tsp b/modules/rpm-ostree/rpm-ostree.tsp new file mode 100644 index 0000000..cb5ee2c --- /dev/null +++ b/modules/rpm-ostree/rpm-ostree.tsp @@ -0,0 +1,33 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/rpm-ostree.json") +model RpmOstreeModule { + /** The rpm-ostree module offers pseudo-declarative package and repository management using rpm-ostree. + * https://blue-build.org/reference/modules/rpm-ostree/ + */ + type: "rpm-ostree"; + + /** List of links to .repo files to download into /etc/yum.repos.d/. */ + repos?: Array; + + /** List of links to key files to import for installing from custom repositories. */ + keys?: Array; + + /** List of folder names under /opt/ to enable for installing into. */ + optfix?: Array; + + /** List of RPM packages to install. */ + install?: Array; + + /** List of RPM packages to remove. */ + remove?: Array; + + /** List of configurations for `rpm-ostree override replace`ing packages. */ + replace?: Array<{ + /** URL to the source COPR repo for the new packages. */ + "from-repo": string, + /** List of packages to replace using packages from the defined repo. */ + packages: Array, + }>; +} \ No newline at end of file diff --git a/modules/script/README.md b/modules/script/README.md index ecdde85..05d5945 100644 --- a/modules/script/README.md +++ b/modules/script/README.md @@ -3,7 +3,7 @@ The `script` module can be used to run arbitrary bash snippets and scripts at image build time. This is intended for running commands that need no YAML configuration. The snippets, which are run in a bash subshell, are declared under `snippets:`. -The scripts, which are run from the `config/scripts` directory, are declared under `scripts:`. +The scripts, which are run from the `files/scripts/` directory, are declared under `scripts:`. ## Creating a Script @@ -16,4 +16,4 @@ When creating a script, please make sure - ...it starts with a [shebang]() like `#!/usr/bin/env bash`. - This ensures the script is ran with the correct interpreter / shell. - ...it contains the command `set -euo pipefail` right after the shebang. - - This will make the image build fail if your script fails. If you do not care if your script works or not, you can omit this line. \ No newline at end of file + - This will make the image build fail if your script fails. If you do not care if your script works or not, you can omit this line. diff --git a/modules/script/module.yml b/modules/script/module.yml index a3b02da..f52ba24 100644 --- a/modules/script/module.yml +++ b/modules/script/module.yml @@ -1,6 +1,7 @@ name: script shortdesc: The script module can be used to run arbitrary bash snippets and scripts at image build time. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/script/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/script/script.tsp example: | type: script snippets: diff --git a/modules/script/script.tsp b/modules/script/script.tsp new file mode 100644 index 0000000..222ab35 --- /dev/null +++ b/modules/script/script.tsp @@ -0,0 +1,16 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/script.json") +model ScriptModule { + /** The script module can be used to run arbitrary bash snippets and scripts at image build time. + * https://blue-build.org/reference/modules/script/ + */ + type: "script"; + + /** List of bash one-liners to run. */ + snippets?: Array; + + /** List of script files to run. */ + scripts?: Array; +} \ No newline at end of file diff --git a/modules/signing/module.yml b/modules/signing/module.yml index 8626bdf..08c348d 100644 --- a/modules/signing/module.yml +++ b/modules/signing/module.yml @@ -1,5 +1,6 @@ name: signing shortdesc: The signing module is used to install the required signing policies for cosign image verification with rpm-ostree and bootc. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/signing/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/signing/signing.tsp example: | type: signing # this sets up the proper policy & signing files for signed images to work fully diff --git a/modules/signing/signing.tsp b/modules/signing/signing.tsp new file mode 100644 index 0000000..d60c621 --- /dev/null +++ b/modules/signing/signing.tsp @@ -0,0 +1,10 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/signing.json") +model SigningModule { + /** The signing module is used to install the required signing policies for cosign image verification with rpm-ostree and bootc. + * https://blue-build.org/reference/modules/signing/ + */ + type: "signing"; +} \ No newline at end of file diff --git a/modules/systemd/README.md b/modules/systemd/README.md index 41adbfd..5ffda91 100644 --- a/modules/systemd/README.md +++ b/modules/systemd/README.md @@ -4,8 +4,8 @@ The `systemd` module streamlines the management of systemd units during image bu You can also include your systemd units to be copied into system directories into these locations, depending if your unit is `system` or `user` based: -`config/systemd/system` -`config/systemd/user` +`files/systemd/system/` +`files/systemd/user/` Those units are then copied into these folders (depending on unit base): `/usr/lib/systemd/system` diff --git a/modules/systemd/module.yml b/modules/systemd/module.yml index 03b79e0..8cd4cb8 100644 --- a/modules/systemd/module.yml +++ b/modules/systemd/module.yml @@ -1,23 +1,15 @@ name: systemd shortdesc: The systemd module streamlines the management of systemd units during image building. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/systemd/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/systemd/systemd.tsp example: | type: systemd + # this example disables automatic flatpak updates and enables a custom service unit for all users system: - enabled: - - example.service # Enabled (runs on system boot) disabled: - - example.target # Disabled (does not run on system boot, unless other unit strictly requires it) - masked: - - example.service # Masked (does not run on system boot, under any circumstances) - unmasked: - - example.service # Unmasked (runs on system boot, even if previously masked) + - flatpak-system-update.timer user: enabled: - - example.timer # Enabled (runs for the user) + - my-custom.service disabled: - - example.service # Disabled (does not run for the user, unless other unit strictly requires it) - masked: - - example.service # Masked (does not run for the user, under any circumstances) - unmasked: - - example.service # Unmasked (runs for the user, even if previously masked) + - flatpak-user-update.timer \ No newline at end of file diff --git a/modules/systemd/systemd.tsp b/modules/systemd/systemd.tsp new file mode 100644 index 0000000..d8cdecd --- /dev/null +++ b/modules/systemd/systemd.tsp @@ -0,0 +1,40 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/systemd.json") +model SystemdModule { + /** The systemd module streamlines the management of systemd units during image building. + * https://blue-build.org/reference/modules/systemd/ + */ + type: "systemd"; + + /** System unit configuration. */ + system?: { + /** List of systemd units to enable. (runs on system boot) */ + enabled?: Array; + + /** List of systemd units to disable. (does not run on system boot, unless another unit strictly requires it) */ + disabled?: Array; + + /** List of systemd units to mask. (does not run on system boot, under any circumstances) */ + masked?: Array; + + /** List of systemd units to unmask. (runs on system boot, even if previously masked) */ + unmasked?: Array; + }; + + /** User unit configuration (with --global to make changes for all users). */ + user?: { + /** List of systemd units to enable. (runs for the users) */ + enabled?: Array; + + /** List of systemd units to disable. (does not run for the users, unless another unit strictly requires it) */ + disabled?: Array; + + /** List of systemd units to mask. (does not run for the users, under any circumstances) */ + masked?: Array; + + /** List of systemd units to unmask. (runs for the users, even if previously masked) */ + unmasked?: Array; + }; +} \ No newline at end of file diff --git a/modules/yafti/module.yml b/modules/yafti/module.yml index 74aa253..cf65c7d 100644 --- a/modules/yafti/module.yml +++ b/modules/yafti/module.yml @@ -1,6 +1,7 @@ name: yafti shortdesc: The yafti module can be used to install yafti and set it up to run on first boot. readme: https://raw.githubusercontent.com/blue-build/modules/main/modules/yafti/README.md +typespec: https://raw.githubusercontent.com/blue-build/modules/main/modules/yafti/yafti.tsp example: | type: yafti custom-flatpaks: diff --git a/modules/yafti/yafti.tsp b/modules/yafti/yafti.tsp new file mode 100644 index 0000000..6b47136 --- /dev/null +++ b/modules/yafti/yafti.tsp @@ -0,0 +1,13 @@ +import "@typespec/json-schema"; +using TypeSpec.JsonSchema; + +@jsonSchema("/modules/yafti.json") +model YaftiModule { + /** The yafti module can be used to install yafti and set it up to run on first boot. + * https://blue-build.org/reference/modules/yafti/ + */ + type: "yafti"; + + /** List of custom Flatpaks to inject to the default yafti.yml. Format is: `PrettyName: org.example.flatpak_id` */ + "custom-flatpaks"?: Array>; +} \ No newline at end of file