Faster Integration Tests Execution with Terraform and Selenoid

Photo by Matthew Henry on Unsplash

Integration tests are typically much slower to execute in contrast to your unit tests, but they are as important as your unit tests since they test your system as a whole rather than its components in isolation. However, the execution time can become an obstacle in this DevOps era in which everybody wants fast and continuous feedback. The good news is that will all the tools that we have at our disposal today you can significantly lower the execution time of your Selenium tests by making the most of Infrastructure as code and the scalability of the cloud.

Selenoid is a custom selenium grid that runs docker containers as the nodes in the grid. You can tell Selenoid how many parallel tests you want to execute and it will automatically fire up containers on demand when you start executing your test suite, rather than having a static grid with static nodes. This enables you to scale your grid as you need, so that it adapts to the amount of tests that you have, allowing you to lower the execution time of your test suite.

We are going to create a terraform script that you can easily execute as part of your CI pipeline. The script will create a spot instance with Selenoid running on it. After the script finishes executing then you can execute your tests pointing to the IP of the spot instance created by the Terraform scripts. When your test suite finishes then you can destroy your grid by destroying the terraform resources and save on costs.

The Terraform Script

We are going to use this Selenoid terraform module to make things significantly easier for us. (Disclaimer: I created the module). The following script makes use of the module, be sure to replace the variables with yours.

module "selenoid" {
  source  = "ryudice/selenoid/aws"
  version = "0.0.1"
  spot_price      = "0.69"
  instance_type   = "c5.4xlarge"
  security_groups = ["your security groups"]
  subnet_id       = "subnet-id"
  cpu_count       = 8
  key_pair        = "your-keypair"
  instance_role   = "your-role"
}
output "ip" {
  # value = aws_spot_instance_request.selenium.private_ip
  value = module.selenoid.private_ip
}

Replace the values of the module variables with yours. To find out more about the supported variables for the module you can go to the documentation.

The above Terraform script will output the private IP of the Selenium grid instance running Selenoid. We will pass on this IP to our test scripts through an environment variable.

This is a sample Python script with a simple selenium test so you get the idea (DISCLAIMER: I didn’t test it):

from selenium import webdriver
from selenium import Keys
import os
SELENIUM_GRID = f"http://{os.getenv('SELENIUM_GRID')}:4444/wg/hub" # This is where we get the IP of the selenium grid we created with terraform
driver = driver.Remote(
    command_executor = SELENIUM_GRID,
    desired_capabilities = {
        'browserName': 'chrome'
    }
)
# User the driver to execute your tests

And this is the Bash script that you could use to run everything together:

terraform apply -auto-approve # Create the Selenium grid using terraform
SELENIUM_GRID="$(terraform output ip)" # Store the IP of the created grid in an env variable
python run.py # Run the tests. The tests will read the above created env variable to point to the grid
terraform destroy -auto-approve # Destroy everything so you dont get billed for the instance after

There you go, now you can scale your grid as your test suite grows by just increasing the capacity of the instance. And not only that, you only pay for the time that your tests are running.