Learn Selenium - Selenium Grid & Parallel Execution

#Introduction

A test suite that runs one by one on a single machine takes hours. Episode 10 introduces Selenium Grid — the solution for running tests in parallel across many browsers, even many machines. After this episode, you'll be able to drastically cut suite duration and test several browsers at once.

Grid introduces two concepts: a hub/router that accepts session requests and nodes that run browsers. Selenium 4 overhauled Grid with a more modular new architecture. You'll see how to run it locally, with Docker, and how to manage session load.

#Selenium Grid Architecture

#Grid Components in Selenium 4

Selenium 4 Grid consists of several separate components: a router as the entry point, a distributor that assigns sessions to nodes, a session map that tracks active sessions, and nodes that actually run browsers. The good news: for general use you don't need to understand every component's detail — just know that all of it can run with a single command.

#Standalone vs Hub-Node Mode

Grid has two popular modes: standalone for a single machine (simplest) and hub-node for larger scale. Standalone is enough for parallel tests on a developer machine; hub-node is for teams using many browser nodes.

#Running a Local Grid

#Standalone Mode

Start from the Selenium Server jar file:

curl -sSL https://github.com/SeleniumHQ/selenium/releases/download/selenium-4.29.0/selenium-server-4.29.0.jar -o selenium-server.jar
java -jar selenium-server.jar standalone --port 4444

The java -jar selenium-server.jar standalone command starts the router, distributor, and node in one process on port 4444. To use the endpoint, make sure curl http://localhost:4444/status responds with grid status JSON.

#Hub-Node Mode

For larger scale, run the router separately then register nodes:

java -jar selenium-server.jar hub --port 4444
java -jar selenium-server.jar node --port 5555

java -jar selenium-server.jar hub starts the router on 4444, and node --port 5555 registers one node that runs browsers. You can add as many nodes as needed, on the same or different machines.

#Docker-based Grid

#Docker Compose for the Grid

Docker makes the grid consistent and easy to reproduce. Save this configuration as docker-compose.yml:

services:
  chrome:
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - hub
    environment:
      - SE_EVENT_BUS_HOST=hub
  hub:
    image: selenium/hub:latest
    ports:
      - "4444:4444"

The docker-compose.yml above defines the selenium/hub:latest image as the center and selenium/node-chrome:latest as a Chrome node with 2GB of shared memory. Run it with:

docker compose up -d

The docker compose up -d command starts the hub and node together. Combining Docker and Grid is the industry standard for parallel testing in CI.

#Remote WebDriver Inside Tests

To connect tests to the grid, replace the local driver constructor with Remote:

Remote WebDriver to the grid

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
 
opsi = Options()
driver = webdriver.Remote(
    command_executor="http://localhost:4444/wd/hub",
    options=opsi,
)
driver.get("https://example.com")
print(driver.title)
driver.quit()

webdriver.Remote(command_executor="http://localhost:4444/wd/hub", options=opsi) requests a session from the grid. The grid picks a node with capacity, then the test runs as usual — your code doesn't change for parallelism.

#Running Parallel with pytest

Combine the grid with pytest-xdist:

pip install pytest-xdist
pytest tests -n 4

pytest tests -n 4 runs four tests in parallel. Each worker opens a session to the grid, and the load spreads across available nodes.

#Managing Node and Session Load

#Monitoring Grid Status

Node load can be monitored through the status endpoint:

curl http://localhost:4444/status | python3 -m json.tool

curl http://localhost:4444/status shows the number of nodes, supported browsers, and active sessions. Get into the habit of checking this endpoint when the grid starts feeling slow — the cause is usually nodes running out of capacity.

#Node Capacity Rules

Node capacity must be set realistically: one Chrome node with 4GB RAM comfortably handles two to three concurrent sessions. Too many sessions per node actually makes tests slow and flaky. Start small, observe status, then increase capacity gradually.

#Conclusion

Episode 10 opens a new dimension of test execution: local grids in standalone and hub-node modes, Docker-based grids that are consistent and easy to replicate, test connections via webdriver.Remote, and node capacity management to keep the grid stable.

Key takeaways:

• Grid separates router and nodes; standalone for one machine, hub-node for large scale.
• java -jar selenium-server.jar standalone is enough for local parallel tests.
• Docker Compose makes the grid reproducible for CI.
• Remote WebDriver connects tests to the grid without changing code.
• Monitor node capacity via the status endpoint and set it realistically.

In episode 11 next, we'll cover CI/CD integration — running Selenium tests in GitHub Actions, GitLab CI, Jenkins, and Azure Pipelines, headless execution in pipelines, test reporting, artifacts, and failure feedback, plus strategies for handling flaky tests in CI.