Share Test Code
In order to make the CI workflow reliable and efficient it is crucial to keep the test coverage in a good shape at all times. Sharing test code between several packages (even within multiple branches of the same package) may significantly help to:
Prevent test code duplication
Minimize test maintenance
Catch incompatibilities early
In general, tests define how the software works and the basic functionality of many packages doesn’t change that often. We try hard to keep the backward compatibility where possible. Thus it seems natural that, for such components, tests guarding the spec could change at a slower pace than the distribution branches.
See the whole https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/thread/55U6V6UHA54MJLD2F6JF46EOLMVRUAE7[ci-list discussion] for some more context.
Store test code in your preferred repository and reference the tests from the dist-git yaml file. There is also a special `tests` namespace dedicated for storing Fedora CI integration tests:
Use `fedpkg` to quickly clone repositories from the tests namespace:
fedpkg clone tests/shell
Some of the xref:standard-test-roles.adoc[Standard Test Roles] (currently basic and beakerlib) support fetching test code from remote repositories directly in their config in this way:
- role: standard-test-beakerlib
- repo: "https://src.fedoraproject.org/tests/shell.git"
dest: "shell"
It is also possible to specify version (branch, commit hash) which should be fetched from the remote repository:
- role: standard-test-beakerlib
- repo: "https://src.fedoraproject.org/tests/shell.git"
dest: "shell"
version: "devel"
Testing Tests
It is a good idea to ensure that updating tests in the shared repository does not negatively impact packages which they are testing. To enable pull request pipeline for tests stored in the Fedora dist git tests namespace simply include `tests.yml` file in the root of the test repository.
- hosts: localhost
- role: standard-test-basic
- classic
- smoke27:
dir: smoke
run: VERSION=2.7 METHOD=virtualenv ./venv.sh
- smoke36:
dir: smoke
run: VERSION=3.6 METHOD=virtualenv ./venv.sh
- smoke37:
dir: smoke
run: VERSION=3.7 METHOD=virtualenv ./venv.sh
- python27
- python36
- python37