[Pki-devel] Python integration tests with betamax, Vagrant and Ansible

Hi, I have made some progress with integration tests of our Python API. You can find my experiments at https://github.com/tiran/pki/commits/betamax . The integration tests are using pytest fixtures and betamax. What is betamax? ---------------- https://betamax.readthedocs.org/en/latest/ Betamax is recorder and player. More precisely it is a testing library for python-requests to record requests and replay responses. In recording mode it records all requests and stores the requests and responses in cassettes (JSON files). In replay mode (aka off-line) it intercepts requests and serves responses from pre-recorded cassettes. With betamax we can run integration tests against a life Dogtag installation. Once our tests have been recorded, we don't need a Dogtag server any more. The recorded test cases become self-sustained and fully reproducible. Betamax is in Fedora 23! How to record? -------------- Of course we also need a simple to record the initial set, new tests or to update recordings. Please welcome Ansible and Vagrant! I wrote a Vagrant file and Ansible playbook that do all the heavily lifting in the background. The code is currently located at https://github.com/tiran/pki-vagans . Once you installed and set up Vagrant and Ansible on your machine, it takes just one command and about 7 to 10 minutes to install a VM with Dogtag. The command 'vagrant up' will download Fedora 23, upgrade the box, set up 389 DS and a Dogtag instance with CA and KRA. With 'vagrant provision' you can update the machine and even install custom builds from a directory with RPMs. What's left to do? ------------------ * write more tests (obviously) * pki-vagans and integration tests are tuned to my machine. The setup might not work on your box. * Tests are currently read-only. I need to figure out a good way for tests that create data, probably a fixture on module or class level. Please test :) Christian