English French
ansible-playbook tests.yml
You can find output artifacts of the tests in an `artifacts/` or specify a specific directory like this:
TEST_ARTIFACTS=/tmp/output ansible-playbook tests.yml
You can filter which kinds of tests are run by providing a `--tags` argument. To only run tests that are suited for classic systems installed by `yum` or `dnf` you can use a command like:
ansible-playbook --tags=classic tests.yml
When run by a CI System the tests are invoked according to the xref:standard-test-interface.adoc[Standard Test Interface]. Look here for more details and standard invocation variables.
Package
When you run the tests as above, the tests assume that the system to be tested is the same as the system invoking the tests. In particular, the test assumes that the thing to be tested is already installed.
A _test subject_ is what we call the thing to be tested. RPMs are a particular kind of _test subject_. To turn a test subject into a launched, installed system to be tested, we use http://docs.ansible.com/ansible/intro_dynamic_inventory.html[Ansible dynamic inventory]. Let's invoke the tests with an inventory and a specific version of gzip:
curl -o gzip.rpm https://kojipkgs.fedoraproject.org//packages/gzip/1.8/2.fc26/x86_64/gzip-1.8-2.fc26.x86_64.rpm
export TEST_SUBJECTS=$PWD/gzip.rpm
ansible-playbook tests.yml
You'll notice that the RPM is installed into the testable system before invoking the tests. Some tests contain their own inventory, that is their own instructions for turning a _test subject_ into one or more testable systems. But in this case we use the default `standard-test-roles` inventory in `/usr/share/ansible/inventory` to do this.
Container
Another example is to use a _test subject_ of a container image. This is also a fully formed and integrated deliverable. The _test subject_ again represents the thing to be tested. For testing containers there is an additional dependency needed:
sudo dnf install standard-test-roles-inventory-docker
The container image is pulled from a registry and launched using docker by an http://docs.ansible.com/ansible/intro_dynamic_inventory.html[Ansible dynamic inventory].
export TEST_SUBJECTS=docker:docker.io/library/fedora:27
ansible-playbook --tags=container tests.yml
If you watch closely you'll notice the image is pulled if not already local, launched as a container, and then prepared for the tests to run on. The first time this may take a little longer. Not all tests are able to function in the somewhat different environment of a container. In fact, for certain tests, the software to be tested may not be included in the container. But many of the tests for core packages should work here.
The `--tags` argument filters out tests that are not suitable for running in a container, either because the system functions differently, or the correct packages are not installable.
See the <<_debug>> section for instructions how to log into a running container and diagnose why the tests failed.
Additional arguments for Docker