Visual Regression Testing with a Screenshot API: Simpler Than You Think

Visual regression testing answers a simple question: "Does this page still look right?" It catches CSS breaks, layout shifts, missing assets, and rendering bugs that unit tests and integration tests can't see.

The traditional approach involves running Puppeteer or Playwright locally, managing headless browsers, and maintaining baseline images. It works, but it's heavy. Here's a lighter approach using a screenshot API.

The Basic Pattern

import requests
import hashlib
import json
import sys

SCREENSHOT_API = "https://hermesforge.dev/api/screenshot"
BASELINE_FILE = "visual_baselines.json"

def capture(url, width=1280):
    resp = requests.get(SCREENSHOT_API, params={
        "url": url,
        "width": width,
        "format": "png",
        "block_ads": "true",
    }, timeout=30)
    resp.raise_for_status()
    return hashlib.sha256(resp.content).hexdigest()

def load_baselines():
    try:
        with open(BASELINE_FILE) as f:
            return json.load(f)
    except FileNotFoundError:
        return {}

def save_baselines(baselines):
    with open(BASELINE_FILE, "w") as f:
        json.dump(baselines, f, indent=2)

# Pages to test
PAGES = {
    "home": "https://yoursite.com",
    "pricing": "https://yoursite.com/pricing",
    "docs": "https://yoursite.com/docs",
    "login": "https://yoursite.com/login",
}

def update_baselines():
    """Capture new baselines for all pages."""
    baselines = {}
    for name, url in PAGES.items():
        baselines[name] = capture(url)
        print(f"  Captured baseline: {name}")
    save_baselines(baselines)
    print(f"Saved {len(baselines)} baselines to {BASELINE_FILE}")

def run_tests():
    """Compare current screenshots against baselines."""
    baselines = load_baselines()
    if not baselines:
        print("No baselines found. Run with --update first.")
        sys.exit(1)

    failures = []
    for name, url in PAGES.items():
        current = capture(url)
        expected = baselines.get(name)
        if expected is None:
            print(f"  SKIP {name}: no baseline")
        elif current != expected:
            failures.append(name)
            print(f"  FAIL {name}: visual change detected")
        else:
            print(f"  PASS {name}")

    if failures:
        print(f"\n{len(failures)} visual regression(s) detected: {', '.join(failures)}")
        sys.exit(1)
    else:
        print(f"\nAll {len(PAGES)} pages match baselines.")

if __name__ == "__main__":
    if "--update" in sys.argv:
        update_baselines()
    else:
        run_tests()

How It Works

1. Capture baselines: Run python visual_test.py --update after a known-good deployment
2. Run tests: Run python visual_test.py in your CI pipeline or locally
3. Detect changes: Any visual difference fails the test

The hash comparison is binary — any pixel change triggers a failure. This is intentional for CI/CD: you want to know about every visual change, even small ones.

Adding to CI/CD

GitHub Actions

name: Visual Regression Tests
on: [push, pull_request]

jobs:
  visual-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install requests
      - run: python visual_test.py

Store visual_baselines.json in your repository. When tests fail, review the change, and if intentional, run --update and commit the new baselines.

GitLab CI

visual-regression:
  image: python:3.12-slim
  script:
    - pip install requests
    - python visual_test.py
  only:
    - merge_requests
    - main

Testing Multiple Viewport Sizes

Mobile and desktop layouts can break independently. Test both:

VIEWPORTS = {
    "desktop": 1280,
    "tablet": 768,
    "mobile": 375,
}

PAGES = {
    "home": "https://yoursite.com",
    "pricing": "https://yoursite.com/pricing",
}

def run_tests():
    baselines = load_baselines()
    failures = []

    for name, url in PAGES.items():
        for viewport_name, width in VIEWPORTS.items():
            key = f"{name}_{viewport_name}"
            current = capture(url, width=width)
            expected = baselines.get(key)
            if expected and current != expected:
                failures.append(key)
                print(f"  FAIL {key}")
            elif expected:
                print(f"  PASS {key}")

    return len(failures) == 0

When This Approach Works Best

Good for: - Marketing sites, landing pages, documentation - Sites that change infrequently - Quick smoke tests after deployments - Teams without existing visual testing infrastructure

Less ideal for: - Pages with dynamic content (timestamps, user-specific data, random elements) - Single-page apps with animations - Pages requiring authentication (though the API supports custom JS injection for cookie setting)

For dynamic content, you'll need to either: - Inject JS to freeze dynamic elements: js=document.querySelector('.timestamp').remove() - Use pixel-level comparison with a tolerance threshold instead of exact hash matching - Mock the dynamic data at the application level

Comparison with Dedicated Visual Testing Tools

The screenshot API approach won't replace Percy or Chromatic for teams that need detailed visual diffs, approval workflows, and browser matrix testing. But for a quick "did anything break?" check in CI, it's dramatically simpler to set up.

Get Started

No signup needed. Test the API directly:

curl "https://hermesforge.dev/api/screenshot?url=https://example.com&width=1280&format=png" -o screenshot.png

For CI/CD usage with higher rate limits, get a free API key.