From d5a22097b81e8ba1887b8cf7a8c94b78bb7b00d7 Mon Sep 17 00:00:00 2001 From: Alexander Todorov Date: Mon, 12 Oct 2020 13:33:21 +0300 Subject: [PATCH] tests: Add notes for downstream testing also includes PR template to make this more visible to everyone! --- .github/PULL_REQUEST_TEMPLATE.md | 13 +++++++++++++ test/README.md | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) create mode 100644 .github/PULL_REQUEST_TEMPLATE.md diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 000000000..2d8cc400e --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,13 @@ +# Description + + + +**Downstream testing notes:** + +- [ ] 1st commit of any `rhbz#` related PR contains bug reproducer; CI reports FAIL or +- [ ] PR contains automated tests for new functionality and +- [ ] QE has approved reproducer/new tests and +- [ ] Subsequent commits provide bug fixes without modifying the reproducer; CI reports PASS and +- [ ] QE approves this PR; RHBZ status is set to `MODIFIED + Verified=Tested` + +For more info see [test/README.md](../test/README.md#downstream-testing-notes)! diff --git a/test/README.md b/test/README.md index 852500da1..8bd8b4ed4 100644 --- a/test/README.md +++ b/test/README.md @@ -191,3 +191,35 @@ from the local directory without installing it: * `make build` - will build everything under `cmd/` * `./osbuild-composer-cli-tests` - will execute the freshly built integration test suite + + +## Downstream testing notes + +To make it easier for us to test & verify downstream builds we are going to +move most of the work upstream and apply the following rules: + +1. Preferably the 1st commit of any PR will contain a bug reproducer. + First push a draft PR only containing that commit which will cause CI to FAIL. +2. QE will review and approve the reproducer (can happen in parallel with next item) +3. Subsequent commits provide bug fixes without modifying the reproducer and + CI reports PASS. Push these on top of the approved reproducer. +4. QE has done final review and approved the PR; RHBZ status is set to + `MODIFIED + Verified=Tested` +5. Devel has done final review and approved the PR + +**NOTES for devel:** + +Pull requests related to new functionality may add their +automated tests together or after commit(s) adding said functionality! + +All PRs containing commits referencing `rhbz#` number and/or +all PRs against a dedicated `rhel-` branch should follow the above rules! + +**NOTE for QE:** + +CI results are also reported against each commit and these +can be used to review the test automation state during a PR lifecycle. + +`qa_ack+` on RHBZ will be granted **after** a reproducer has been +identified and with the mutual understanding that PRs related to +that RHBZ must include an automated test reproducer.