This will help make the Containerfile just a little bit easier to read
(ignoring all the mounts lol). This would also allow us to add logic
later to support modules that run executables other than `*.sh`.
Recipe files can now be put into their own directory `./recipes/`. This
directory is NEVER copied into the build so changes to a recipe will no
longer cause cache misses for builds. Here is an example of my build
changing the second to last module and only requiring the last 2 `RUN`
layers to be run again.
```
=> CACHED [stage-config 1/1] COPY ./config /config 0.0s
=> CACHED [stage-modules 1/2] COPY --from=ghcr.io/blue-build/modules:latest /modules /modules 0.0s
=> CACHED [stage-modules 2/2] COPY ./modules /modules 0.0s
=> CACHED [stage-keys 1/1] COPY cosign.pub /keys/jp-desktop-gaming.pub 0.0s
=> CACHED [stage-4 2/16] RUN --mount=type=bind,from=stage-keys,src=/keys,dst=/tmp/keys mkdir -p /usr/etc/pki/containers/ && cp /tmp/keys/* /usr/et 0.0s
=> CACHED [stage-bins 1/3] COPY --from=gcr.io/projectsigstore/cosign /ko-app/cosign /bins/cosign 0.0s
=> CACHED [stage-bins 2/3] COPY --from=docker.io/mikefarah/yq /usr/bin/yq /bins/yq 0.0s
=> CACHED [stage-bins 3/3] COPY --from=ghcr.io/blue-build/cli:main-installer /out/bluebuild /bins/bluebuild 0.0s
=> CACHED [stage-4 3/16] RUN --mount=type=bind,from=stage-bins,src=/bins,dst=/tmp/bins mkdir -p /usr/bin/ && cp /tmp/bins/* /usr/bin/ && ostree 0.0s
=> CACHED [stage-4 4/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> CACHED [stage-4 5/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> CACHED [stage-4 6/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> CACHED [stage-4 7/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> CACHED [stage-4 8/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> CACHED [stage-4 9/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> CACHED [stage-4 10/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> CACHED [stage-4 11/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> CACHED [stage-4 12/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> CACHED [stage-4 13/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> CACHED [stage-4 14/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind 0.0s
=> [stage-4 15/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind,from= 33.4s
=> [stage-4 16/16] RUN --mount=type=tmpfs,target=/var --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw --mount=type=bind,from=s 0.7s
```
Support was also added to put all build files into `./files/` instead of
`./config/`. This is an all or nothing operation, meaning if there
exists a directory of `files` then the `config` directory will be
completely ignored. Work will have to be done in
https://github.com/blue-build/modules to allow users to put their files
directly in `./files/` and not `./files/files` for the `files` module or
`./files/scripts` for the scripts module.
Support was also added to move the `./config/containerfiles/` directory
to the root of the project. Now the directories you can find in the root
of projects are:
```
files/
containerfiles/
recipes/
```
I noticed that making changes to the exports script before a release
could cause modules to not build properly if breaking changes got pushed
out. To prevent this, I'm making it so that the hash of the commit is
put in the tag for the exports script image and that the CLI tool will
use that hash when building the `Containerfile`.
This exists in the legacy template:
489ebea77b/build.sh (L13C1-L13C39)
And is also used in many modules, though with
`MODULE_DIRECTORY="${MODULE_DIRECTORY:-"/tmp/modules"}"` to not break
things.
And is documented on the website, because I assumed it existed:
https://blue-build.org/reference/module/#module_directory
I know this probably won't change much, but it's still useful for the
scripts.
I've been investigating more into how ostree works and how it relates to
running `ostree container commit` for each layer. I've decided to move
our pre-installed bins and public keys into their own stages and then
bind mount them into a `RUN` instruction so that we can just use `cp` to
get the files into the image and then call `ostree container commit`.
Now all of our layers in the image (after the base image) will be in the
ostree commit tree.
https://github.com/blue-build/cli/issues/118#issuecomment-1992536977
as mentioned here reverting to the bash files module simplifies the
build process (and allows for better usage of `ostree container
commit`). It also means there are less overall layers and (ideally) a
smaller image as a result. Tested locally and template generation seems
to work just fine
the `os_version` is defaulting to the `image_tag` inside containers and
causing our template to use latest tag
---------
Co-authored-by: Gerald Pinder <gmpinder@gmail.com>
Since the command `ostree container commit` checks for the presence of
mounted directories, we will be running it at the end of each module
run. We have also updated the final commit to remove from /tmp/ and
/var/ again in case a user creates extra files through custom
instructions.
Closes#101#95
