# Visual regression testing with screenshots: Puppeteer, pixelmatch and what makes diffs flaky

> Measured with Puppeteer 25.7: why the same page differs between shots, how to stabilise it, and which pixelmatch thresholds hide real changes.

Published 2026-11-07 · https://shareglint.com/en/guides/visual-regression-screenshots

Visual regression testing means taking a screenshot of a page, keeping it as the **baseline**, and comparing every later screenshot of the same page with it: if pixels change that should not, the test fails and shows you where. The idea is simple; making it reliable is not. The same page, shot twice a second apart, rarely gives identical pixels — and an image diff with default settings can miss a real change. Every number on this page was measured on 2026-11-07 with Puppeteer 25.7.0 (Chrome 152) in the official `ghcr.io/puppeteer/puppeteer` image and [pixelmatch](https://github.com/mapbox/pixelmatch) 6, on small local test pages.

For taking the screenshots themselves — viewport, element, retina, waiting — see the [Puppeteer screenshot guide](https://shareglint.com/en/guides/puppeteer-screenshot).

## What we measured

| Test | Differing pixels |
|---|---|
| Same page twice, as is (spinner, timestamp, focused input) | 264 of 320,000 |
| Same page twice, animations, caret and timestamp frozen | **0** of 320,000 |
| Same static page, two separate browser launches, same image | 0 |
| Same static page, Puppeteer image vs Playwright image (Chrome 152 vs Chromium 156) | 2,195 at threshold 0; 1,277 at 0.1; 401 at 0.2 |
| Button colour #3366cc → #2f5fc4, threshold 0 | 2,674 |
| Button colour #3366cc → #2f5fc4, threshold 0.1 (pixelmatch default) | **0** |
| Button colour #3366cc → #2f5fc4, threshold 0.2 (Playwright default) | **0** |

Three lessons come out of that table. A page has to be **stabilised** before it can be compared. The baseline and the new shot have to come from the **same environment** — the same static page differed in 2,195 pixels between two official browser images, which ship different browser versions and fonts — on a page of plain text and one button. And the **threshold** is a colour tolerance per pixel, not a share of the image: at the defaults a whole button changing colour went unnoticed.

## A baseline and a compare script

```javascript
import fs from 'node:fs';
import puppeteer from 'puppeteer';
import { PNG } from 'pngjs';
import pixelmatch from 'pixelmatch';

const [url, name] = process.argv.slice(2);
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800, deviceScaleFactor: 1 });
await page.goto(url, { waitUntil: 'networkidle2' });

// Stabilise: no animations, transitions or caret; fonts loaded.
await page.addStyleTag({ content: `*, *::before, *::after {
  animation: none !important; transition: none !important; caret-color: transparent !important; }` });
await page.evaluate(() => document.fonts.ready);

const shot = PNG.sync.read(await page.screenshot());
await browser.close();

const baselinePath = `baselines/${name}.png`;
if (!fs.existsSync(baselinePath)) {
  fs.writeFileSync(baselinePath, PNG.sync.write(shot));
  console.log('baseline written');
  process.exit(0);
}

const base = PNG.sync.read(fs.readFileSync(baselinePath));
if (base.width !== shot.width || base.height !== shot.height) {
  console.error(`size changed: ${base.width}×${base.height} → ${shot.width}×${shot.height}`);
  process.exit(1);
}

const diff = new PNG({ width: base.width, height: base.height });
const changed = pixelmatch(base.data, shot.data, diff.data, base.width, base.height, { threshold: 0 });
fs.writeFileSync(`diffs/${name}.png`, PNG.sync.write(diff));
console.log(`${changed} pixels differ`);
process.exit(changed > 0 ? 1 : 0);
```

Commit the `baselines/` folder. When a change is intended, delete the baseline (or add an `--update` flag) and run again. The diff image paints changed pixels red over a faded copy of the page, which is what makes a failure readable in a pull request.

**Check the size first.** pixelmatch compares two buffers of the same size; it cannot compare a 1280-wide shot with a 1281-wide one, and a full-page screenshot changes height whenever content above the fold grows. A size change is a result in itself — report it, do not crop it away.

## Stabilising the page

Each of these made a difference in our test or is a common cause of flaky diffs:

- **Animations and transitions.** A spinner rotates between shots. The CSS above stops them; for JavaScript animations, wait for the element to settle or hide it. Emulating `prefers-reduced-motion` only helps on sites that respect it.
- **The text caret.** A focused input blinks; `caret-color: transparent` removes it.
- **Dates, times and random data.** "Updated 2 minutes ago", today's date, a random avatar. Use fixed test data, or replace the text before the shot (`page.evaluate`), or cover the element.
- **Fonts.** Wait for `document.fonts.ready`; a web font arriving after the shot changes every line of text. Missing system fonts change glyphs too ([fonts and emoji in Puppeteer](https://shareglint.com/en/guides/puppeteer-emoji-and-fonts)).
- **Lazy images and third parties.** Images below the fold, ads, chat widgets and cookie banners. Load or block them deliberately ([full-page screenshots](https://shareglint.com/en/guides/puppeteer-full-page-screenshot) shows lazy loading).
- **Viewport and pixel ratio.** Fix both in code; a different `deviceScaleFactor` changes every pixel.
- **The environment.** Generate baselines and run comparisons in the same Docker image, in CI. Our two-image test shows what happens otherwise: hundreds of pixels differ in plain text with nothing changed. A baseline made on a laptop is likely to fail in CI.

Hiding an element with `display: none` changes the layout below it; covering it with a solid box of the same size keeps the layout. For a timestamp in a fixed-size box, either works; for a banner of varying height, cover it.

## Choosing a threshold

The pixelmatch README describes `threshold` as "a per-pixel perceptual color difference" from 0 to 1, with 0.1 as the default — "not a share of the image". In our test the button colour changing from #3366cc to #2f5fc4 (a small change, the kind a design token edit makes) counted as **0 differing pixels** at 0.1 and at 0.2. A plain 200×100 block of those two colours differed fully at threshold 0.01 and not at all at 0.05.

So two separate decisions:

1. **How different must a pixel be?** In a fixed environment, 0 or close to it — the environment is the same, so any colour difference is real.
2. **How many pixels may differ?** Allow a small number for anti-aliasing (pixelmatch already ignores pixels it detects as anti-aliased unless `includeAA` is set), and fail above it.

If your environments differ (developers' machines, several CI runners), raising the threshold hides the noise and real changes with it. Unify the environment instead.

## Playwright's toHaveScreenshot

If you test with Playwright, its [`toHaveScreenshot`](https://playwright.dev/docs/test-snapshots) does most of the above for you: it writes the baseline on the first run, stores one per browser and platform, retries until two consecutive screenshots match, and by default disables CSS animations and transitions (`animations: "disabled"`) and hides the caret (`caret: "hide"`). The `mask` option covers elements with a solid box, and `stylePath` injects your own CSS before the shot. Its `threshold` defaults to **0.2** — in our test that also missed the button colour change — so set `threshold`, `maxDiffPixels` or `maxDiffPixelRatio` deliberately. The Playwright docs warn, as our measurement shows, that rendering varies with the OS, settings and hardware, and recommend running in the environment where the baselines were made.

With Puppeteer you assemble the same pieces yourself, as in the script above. That is a few dozen lines, and it suits teams who already use Puppeteer for other work. Python teams can do the same with Playwright for Python ([Python screenshots](https://shareglint.com/en/guides/screenshots-in-python-playwright-pyppeteer)).

## Where a screenshot API fits

Visual regression tests mostly run against pages that are not public yet — a branch preview, localhost, a component in Storybook — and those belong to your own Puppeteer or Playwright in CI. Our [screenshot API](https://shareglint.com/en/screenshot-api) takes screenshots of public pages only, so it fits a different job: watching production. A daily check of your public pages, from one consistent browser in one place, catches the change that slipped through — a broken stylesheet after a CDN change, a cookie banner covering the page, a third-party widget pushing content down.

Things to know if you use it that way:

- The same URL with the same options within 24 hours returns the existing file, which suits a daily check. To force a new shot, change the URL, for example with a `?check=2026-11-07` query parameter your site ignores.
- `hide_selectors` hides changing parts (`display: none`, so the layout below moves up), `wait_for` waits for an element, `block_cookie_banners` and `block_ads` remove noise, and `reduced_motion` emulates `prefers-reduced-motion`.
- A viewport shot costs 1 render, a full-page one 2 ([pricing](https://shareglint.com/en/pricing)).

The comparison itself — pixelmatch or any image diff — stays in your script: we take the screenshots, we do not keep baselines or report differences.

## Checklist

1. Same browser, fonts and image for the baseline and every run; ideally one Docker image in CI.
2. Fixed viewport and device pixel ratio.
3. Animations, transitions and the caret off; fonts loaded; lazy content loaded or blocked.
4. Changing data frozen, replaced or covered.
5. Image sizes compared before pixels.
6. A low colour threshold and an explicit number of allowed pixels.
7. A diff image saved with every failure.

## FAQ

### How do I do visual regression testing with Puppeteer?

Take a baseline screenshot of each page or component and commit it. On every change, take the same screenshot with the same viewport, browser and fonts, compare the two with an image diff library such as pixelmatch, and fail when more pixels differ than you allow. Freeze animations, the caret, dates and other changing data first, or the test fails on its own.

### Why do my screenshots differ when nothing changed?

Something on the page changes between shots: a CSS animation, a blinking caret in a focused input, a timestamp, an ad or a lazy image. In our test the same page shot twice differed in 264 pixels; with animations, the caret and the timestamp frozen it differed in none.

### What pixelmatch threshold should I use?

Start with 0 or a very low value in a fixed environment and allow a few pixels instead. The default 0.1 is a per-pixel colour tolerance: in our test it reported 0 differing pixels for a button whose colour changed from #3366cc to #2f5fc4, while threshold 0 reported 2,674.

### Should I use Playwright's toHaveScreenshot instead?

If you already test with Playwright, yes: it stores baselines, disables animations and hides the caret by default, retries until two shots match, and can mask elements. Its default threshold is 0.2, so set threshold or maxDiffPixels deliberately. With Puppeteer you assemble the same pieces yourself.