Posted in Testing

By Felix Carmona

Backend Developer

@felixcarmona46

Running PHPUnit tests in parallel

Usually the automated test suites run in series. The main reason for this behaviour is about the consistency of the data which is accessed by the tests. This means, for example, that if a test A drops the users' database, and the test B does the same thing, then they are not able to run at the same time.

To work around this issue we need to think about the scope of the test and be sure that, if a test should be an integration one, it must preserve its infrastructure dependencies (database/file system writing, etc), or otherwise if it should be a unit test, then the infrastructure dependencies should be mocked.

For a solid test automation strategy, consider the Mike Cohn's Testing Pyramid.

Test Pyramid

For example, if there are 20 use cases (with their respective test cases) that needs an User Repository, we have two ways to test it:

  • Bring the MySQL or the Redis implementation strategy of the UserRepository interface.

    The test running cost would be 20 integration tests that cannot run in parallel (or 40 if we want to test them twice: storing the users in MySQL and storing them in Redis)

  • Mock the UserRepository or implement a fixture like InMemoryUserRepository, use it in these 20 use cases, and then test the integration with the infrastructure in the MySql/RedisUserRepositoryTest.

    The cost would be 20 unit tests that can run in parallel (+2 integration test, one for each strategy of the UserRepository, as they can run in parallel too because they use different databases)

If we still want to opt for the first option, we must find a way to not interfere and corrupt the data between the tests.

This means that if the tests use Redis, we must delimit the keys, generating a random key prefix, and applying it to all the keys of the test case and then delete without using flushall, or use the different redis databases etc. Or in the case of MySQL, generate a random database name in runtime and then delete it, etc

If some tests still can not be isolated from each other, define test suites where each must run in a new parallel thread, and run the containing test cases in serial.

For example, if we have the two following test suites:

  • InDisk suite, which executes its tests in series
  • Redis suite, which executes its tests in series too

Both test suites can be run in two parallel threads, so the Redis tests won’t wait for the filesystem resource and the InDisk ones won’t wait for the RAM resource.

In Social Point we have thousands of tests, and run them takes 20 seconds to finish when running in parallel versus 2-3 minutes to finish when running in series, which is a huge improvement.

Here is our open source tool to run a phpunit test cases in parallel: parallel-phpunit


Comments

comments powered by Disqus