Learn Selenium - Performance Testing & Visual Regression

#Introduction

The feature works, but the page is slow — a failure that ordinary functional tests never detect. Episode 15 extends Selenium into two new dimensions: performance testing to measure page speed from inside tests, and visual regression to catch unintended appearance changes.

A page whose layout shifts unintentionally is often invisible to plain text assertions. With screenshot comparison, you can detect shifted images, resized buttons, or truncated text — regressions that can only be seen, not asserted with text.

#Measuring Browser Performance

#Navigation Timing from Inside a Test

The browser records the timing of every page load phase. This data can be retrieved via JavaScript:

Reading navigation timing

driver.get("https://example.com")
 
timing = driver.execute_script(
    "return performance.getEntriesByType('navigation')[0].toJSON();"
)
waktu_layout = timing["domContentLoadedEventEnd"] - timing["navigationStart"]
print(f"Waktu DOM ready: {waktu_layout} ms")

driver.execute_script("return performance.getEntriesByType('navigation')[0].toJSON();") retrieves navigation timing metrics — time until main content is ready, time until the page is fully loaded, and more. Record these numbers in reports to detect performance regressions between builds.

#Resource Timing for Assets

Another useful metric is the load time of each resource (scripts, images, CSS). This helps find bottleneck assets:

Slow-loading assets

entri = driver.execute_script("return performance.getEntriesByType('resource');")
lambat = [e["name"] for e in entri if e["duration"] > 1000]
print("Aset lebih dari 1 detik:", lambat)

e["duration"] > 1000 filters resources that take more than a second. Performance tests like this are better as tracking metrics than thresholds that immediately fail the build — performance on a developer laptop doesn't represent production.

#Integrating with Lighthouse

#Lighthouse via CLI

Lighthouse measures accessibility, performance, SEO, and best-practice scores. The lightest integration is through the CLI:

npx lighthouse https://example.com --only-categories=performance --output=json --output-path=report.json

The npx lighthouse https://example.com --only-categories=performance command produces a performance report. In the pipeline, scores from report.json can be compared against the team's thresholds.

#Combining with Selenium Tests

For pages that require login, run Lighthouse against the same URL after a Selenium session ensures access. The common approach: Selenium tests prepare the state (login, data), then Lighthouse runs as a separate step against the same page.

#Visual Regression with Screenshots

#Building a Screenshot Baseline

Visual regression compares new screenshots against an approved baseline. Create the baseline when the appearance is declared correct:

Saving a screenshot baseline

from pathlib import Path
 
Path("baselines").mkdir(exist_ok=True)
driver.get("https://example.com/beranda")
driver.save_screenshot("baselines/beranda.png")

driver.save_screenshot("baselines/beranda.png") saves the baseline. This baseline must be reviewed and committed to the repository so the whole team uses the same reference.

#Pixel Comparison with PIL

A simple comparison can be done with Pillow:

Comparing two screenshots

from PIL import Image, ImageChops
 
sebelum = Image.open("baselines/beranda.png").convert("RGB")
sekarang = Image.open("artifacts/beranda.png").convert("RGB")
diff = ImageChops.difference(sebelum, sekarang)
print("Bounding box perubahan:", diff.getbbox())

ImageChops.difference(sebelum, sekarang) computes the per-pixel difference, and diff.getbbox() shows the changed area. If the bounding box isn't empty, there's a visual change to investigate.

#Detecting Layout Regression and UI Drift

#Tolerant Diff Techniques

Direct pixel diff is too sensitive to antialiasing and animations. A more practical technique: compare specific important areas, or ignore differences below a tolerance threshold.

Change tolerance

import numpy as np
 
array_a = np.asarray(sebelum)
array_b = np.asarray(sekarang)
selisih = np.abs(array_a.astype(int) - array_b.astype(int)).sum()
print("Total selisih piksel:", selisih)

np.abs(array_a - array_b).sum() produces a total difference value. With a threshold tuned after observing the baseline, you can distinguish accepted minor changes from real regressions.

#Controlling the Screenshot Environment

Screenshots must be taken in reproducible conditions: fixed viewport size, stable fonts, and animations disabled. Without this control, baseline and new results can't be compared fairly — and the diff will always fire.

#Conclusion

Episode 15 extends your tests' eyesight: performance metrics from navigation timing, Lighthouse scores for thorough audits, screenshot baselines for visual regression, and tolerant diff techniques for detecting layout regression and UI drift.

Key takeaways:

• Navigation timing and resource timing measure performance from inside tests.
• Lighthouse suits thorough audits; run it on authenticated pages.
• Screenshot baselines must be reviewed and committed.
• Pixel diff needs tolerance and controlled screenshot conditions.
• Visual regression catches changes invisible to text assertions.

In episode 16 next, we'll cover API and backend validation with Selenium — combining UI tests with API checks, validating backend responses and UI state, testing end-to-end flows from UI to the service layer, and using API helpers to speed up UI setup.