particle-os/debian-forge-composer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: document the release process

Browse source 
This commit adds RELEASING.md guide describing the process of making a new
upstream release and pushing it into Fedora and CentOS Stream 9.

Additionally, a new helper is added to the repository - update-distgit.py.
The purpose of this helper is to simplify work with the dist-git.

Random thoughts:
I decided to left off RHEL 8 because the guide would be full of internal
URLs and tools. I will probably write a similar guide for it and put it
into internal guides.

I decided to just reference the RHEL Developer Guide for CentOS Stream 9.
It's pretty well written and I didn't feel like duplicating the effort.

We should definitely switch to PR-based approach for Fedora and implement
at least some smoke tests. I believe that the current guide is good enough
and we can iterate later.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This commit is contained in:
Ondřej Budai 2021-06-14 15:59:36 +02:00 • committed by Ondřej Budai
parent f8fd3d04b8
commit 5e8c022e6d
2 changed files with 290 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

166
RELEASING.md Normal file
View file

@ -0,0 +1,166 @@
# Making a new release
This guide describes the process of releasing osbuild-composer both [upstream][upstream-git] and into [Fedora][fedora-distgit] and [CentOS Stream][centos-distgit].
> This guide assumes that the number of the new release is stored in an environment variable called `VERSION`.
## Upstream release
Osbuild-composer release is a tagged commit that's merged into the `main` branch as any other contribution.
### Making a release PR
Firstly, a new branch from (up-to-date) main needs to be created:
```
git checkout main
git pull
git checkout -b release-version-$VERSION
```
All unreleased news have to copied over to a new directory:
```
mkdir docs/news/$VERSION
mv docs/news/unreleased/* docs/news/$VERSION
```
Now, the `NEWS.md` file must be updated. A template for a new release can be generated using `make release`:
```
make release
your-favourite-editor NEWS.md
```
The next step is to update the version number in the spec file. You can use `your-favourite-editor`, or this lovely `sed` script:
```
sed -i -E "s/(Version:\\s+)[0-9]+/\1$VERSION/" osbuild-composer.spec
```
At this moment, everything in the checkout is ready for making the release commit:
```
git add osbuild-composer.spec NEWS.md docs/news/unreleased docs/news/$VERSION
git commit -s -m $VERSION -m "Release osbuild-composer $VERSION"
```
As the final step, push the branch into your fork and open a PR against the `main` branch. The PR now needs to be approved and merged.
### After the release PR is merged
Firstly, make sure to update the local checkout of the `main` branch:
```
git checkout main
git pull
```
Tag the release and push the tag:
```
git tag -s -m 'osbuild-composer $VERSION' v$VERSION HEAD
git push upstream v$VERSION
```
The last thing to do is to create a new release on GitHub. The form can be found at [this webpage][upstream-draft-a-new-release]. The tag version is simply `v$VERSION`, and the release title is just `$VERSION`. The release description should contain the newly added entry from `NEWS.md`.
The upstream release process is now done, now it's time to push the newly born release into downstreams.
## Fedora release
In order to push the new release into Fedora, you need to be [a Fedora packager][new-fedora-packager] and have commit rights for [the repository][fedora-distgit]. If you don't have them, create [an issue upstream][upstream-new-issue].
Start by cloning the dist-git repository and selecting the `rawhide` branch:
```
fedpkg clone osbuild-composer
fedpkg switch-branch rawhide
```
> You should always start with updating the latest Fedora release.
>
> Also, note that `fedpkg` is in many cases just a wrapper around `git`. You can use `git checkout` instead of `fedpkg switch-branch`, `git push` instead of `fedpkg push` etc.
Now, you need to update the sources and specfile. Luckily, we have a handy script in the repository that does the following:
- It merges the upstream spec file with changelog from the downstream spec file and adds there an entry for the new release.
- It uploads the tarball into the distgit's look-side cache.
- It commits the updated spec file and sources file.
Note that the script needs to have a valid Kerberos ticket in order to upload the tarball into the lookaside cache. To use the helper, just run:
```
$OSBUILD_COMPOSER/tools/update-distgit.py \
--version $VERSION \
--author "Name Surname <email@example.com>" \
--pkgtool fedpkg
```
`$OSBUILD_COMPOSER` contains the path to your local upstream osbuild-composer checkout.
After the script is finished, it doesn't hurt to perform two checks:
- Review the commit that the helper created.
- Make a scratch build so you know that the new version is indeed buildable in Fedora.
You can do these steps by running:
```
git show HEAD
fedpkg scratch-build --srpm
```
After the scratch build successfully finishes, push the changes and make a real build:
```
fedpkg push
fedpkg build
```
After you are done with rawhide, switch to the newest stable release for Fedora and do the same change there. It's preferred to reuse the commit from `rawhide`:
```
fedpkg switch-branch f34
git merge --ff-only rawhide
```
If a fast-forward merge it not possible, you can for example cherry-pick the latest commit from `rawhide` using `git cherry-pick rawhide`. If it doesn't apply cleanly, you need to figure out what happened in the git history. Note that you cannot force-push into dist-git. Once something is there, it cannot be removed.
Now, it's time for the scratch-build check. If it passes, we can safely push the changes into the dist-git and make a regular build.
```
fedpkg scratch-build --srpm
fedpkg push
fedpkg build
```
For stable Fedora releases, it's also needed to create an update in Bodhi:
```
fedpkg update --type enhancement --notes "Update osbuild-composer to the latest version"
```
> Feeling lazy? Just run the following line, grab a cup of coffee and watch it do all the hard work for you:
> ```
> fedpkg scratch-build --srpm && fedpkg push && fedpkg build && fedpkg update --type enhancement --notes "Update osbuild-composer to the latest version"
> ```
After this is done, continue with the older stable release(s). After all of them are done, the work on Fedora is over. The updates will not appear immediately, it takes them a week to get through the `updates-testing` repository.
## CentOS Stream 9 release
There's a wonderful guide on this topic in the section 5 of RHEL Developer Guide. `update-distgit.py` can also save you some time here. The only differences from Fedora are that `--pkgtool centos` and `--release c9s` flags need to be used.
At the time of writing this document, gating tests are not run in CentOS dist-git. You also have to simultaneously open a PR in RHEL dist-git to verify that the test suite passes. Close this PR once the PR in CentOS dist-git is merged.
## Spreading the word on osbuild.org
The last of releasing a new version is to create a new post on osbuild.org. Just open a PR in [osbuild/osbuild.github.io]. You can find a lot of inspiration in existing release posts.
[upstream-git]: https://github.com/osbuild/osbuild-composer
[fedora-distgit]: https://src.fedoraproject.org/rpms/osbuild-composer
[centos-distgit]: https://gitlab.com/redhat/centos-stream/rpms/osbuild-composer
[upstream-draft-a-new-release]: https://github.com/osbuild/osbuild-composer/releases/new
[new-fedora-packager]: https://fedoraproject.org/wiki/Join_the_package_collection_maintainers
[upstream-new-issue]: https://github.com/osbuild/osbuild-composer/issues/new/choose
[osbuild/osbuild.github.io]: https://github.com/osbuild/osbuild.github.io

124
tools/update-distgit.py Executable file
View file

@ -0,0 +1,124 @@
#!/usr/bin/python3
import argparse
import contextlib
import os
import subprocess
import tempfile
import urllib.request
from datetime import datetime
def read_file(path) -> str:
"""
Return the content of the file on the given path.
"""
with open(path) as f:
return f.read()
def download_github_release_tarball(user: str, project: str, version: str) -> str:
"""
Download a github release tarball of the given project.
"""
project_url = f"https://github.com/{user}/{project}"
tarball_url = f"{project_url}/archive/v{version}.tar.gz"
local_path = f"{project}-{version}.tar.gz"
urllib.request.urlretrieve(
tarball_url,
local_path
)
return local_path
@contextlib.contextmanager
def extracted_tarball(path: str):
"""
Extract a tarball into a temporary directory.
Yields the temporary directory.
"""
with tempfile.TemporaryDirectory() as tempdir:
subprocess.run(["tar", "-xf", path, "--directory", tempdir], check=True)
yield tempdir
def merge_specfiles(upstream: str, downstream: str, version: str, author: str):
"""
Merge the upstream specfile with the changelog from downstream and add a new
changelog entry.
"""
upstream_spec_lines = upstream.splitlines()
downstream_spec_lines = downstream.splitlines()
# Find where changelog starts in both specfiles
changelog_start_in_up_spec = upstream_spec_lines.index("%changelog")
changelog_start_in_down_spec = downstream_spec_lines.index("%changelog")
# Create a new changelog entry
date = datetime.now().strftime("%a %b %d %Y")
changelog = f"""\
* {date} {author} - {version}-1
- New upstream release
"""
# Join it all together:
# Firstly, let's take upstream spec file including the %changelog directory
# Then, put the newly created changelog entry
# Finally, put there the changelog from the downstream spec file
merged_lines = upstream_spec_lines[:changelog_start_in_up_spec + 1] + \
changelog.splitlines() + \
downstream_spec_lines[changelog_start_in_down_spec + 1:]
return "\n".join(merged_lines) + "\n"
def update_distgit(user: str, project: str, version: str, author: str, pkgtool: str, release: str):
"""
Update the dist-git for a new release.
"""
specfile = f"{project}.spec"
tarball = download_github_release_tarball(user, project, version)
old_downstream_specfile = read_file(specfile)
with extracted_tarball(tarball) as path:
upstream_specfile = read_file(f"{path}/{project}-{version}/{specfile}")
new_downstream_specfile = merge_specfiles(upstream_specfile, old_downstream_specfile, version, author)
with open(specfile, "w") as f:
f.write(new_downstream_specfile)
release_arg = ["--release", release] if release else []
subprocess.check_call([pkgtool, *release_arg, "new-sources", tarball])
subprocess.check_call(["git", "add", ".gitignore", specfile, "sources"])
commit_message = f"Update to {version}"
subprocess.check_call(["git", "commit", "-m", commit_message])
if __name__ == "__main__":
parser = argparse.ArgumentParser(allow_abbrev=False)
parser.add_argument("--version", metavar="VERSION", type=str, help="version to be released to downstream",
required=True)
parser.add_argument("--author", metavar="AUTHOR", type=str,
help="author of the downstream change (format: Name Surname <email@example.com>", required=True)
parser.add_argument("--pkgtool", metavar="PKGTOOL", type=str, help="fedpkg, centpkg, or rhpkg", required=True)
parser.add_argument("--release", metavar="RELEASE", type=str, help="distribution release (required only for centpkg)")
args = parser.parse_args()
if args.pkgtool not in ["fedpkg", "centpkg", "rhpkg"]:
raise RuntimeError("--pkgtool must be fedpkg, centpkg, or rhpkg!")
update_distgit(
"osbuild",
"osbuild-composer",
args.version,
args.author,
args.pkgtool,
args.release,
)