Skip to content
CASE STUDY

Two codebases, two test suites, two gates.

Grumpy Old Gamers is a PC game price comparison site built and maintained by Axiom Testing Ltd. It's two separate deployments — a site and the data service behind it — and neither ships unless its own tests pass.

23,810
titles tracked with live prices
1,668
tests across both codebases
2
components, each gating its own deploy
68
test files across the two suites

npm run build runs the suite before it compiles anything, on both sides independently. A failing test doesn't produce a build, so it can't produce a deploy.

Figures measured on 31 August 2026, from the site's published build report, which ingests the data service's own report rather than restating it.

What it is

Grumpy Old Gamers compares PC game prices across storefronts and regions. The front end is a Next.js site; behind it sits a separate Python service that ingests pricing, runs discovery and serves the API the site reads.

They deploy independently, which is exactly why each needs its own gate. A green front end tells you nothing about whether the service feeding it still returns correct prices.

Why it's unforgiving

Prices are the product. A currency conversion applied twice, a regional price silently falling back to the wrong country, a discount computed against the wrong base — none of these crash. They just show a number that is wrong by a plausible amount.

Add eight regions, each with its own currency and formatting, and the number of ways to be quietly wrong multiplies. That's a test-design problem before it's a coding one.

How each part is tested

The site's unit suite covers the pricing and region logic it owns, and is the only one of the site's suites that gates the build. An end-to-end suite and an API-contract suite run separately in CI against a deployed build; neither is line-instrumented, so neither contributes to the coverage figures below. The data service runs two suites together under one coverage run — unit tests against its services, and an in-process end-to-end suite that drives the real application over HTTP, so routing, middleware and authentication are executed rather than mocked — and publishes the result for the site to read.

Site (vitest)
501 tests in 21 files · 70.49% statements
End-to-end (playwright)
94 tests in 8 files · does not gate the build
Real device
playwright (adb + CDP) · Pixel 9a, Android 16, Chrome · 11 tests in 4 files · does not gate the build · not continuously verified, measured 31 August 2026
API contract
newman (postman collection v2.1) · 92 assertions across 29 requests · does not gate the build
Data service (pytest)
1,167 tests in 47 files · 95.93% statements

Coverage on both sides counts every file in scope, tested or not, rather than only the files a suite happens to touch — so these are lower, and more honest, than a headline number. Each report states its own scope — for the coverage figures, and for the real-device suite, which is measured differently and says so — quoted here from the feed rather than described second-hand.

Site — coverage scope, from the published build report
Coverage of the server-side logic in lib/ only — price and currency selection, region resolution, affiliate URL construction, search, category mapping, and the data-service and IsThereAnyDeal response mappers. It does not cover the React components in components/, the App Router pages and API routes in app/, or the node:https bulk-price path and unused legacy helpers in lib/itad.ts, which together account for most of the uncovered lines. The end-to-end and API suites reported alongside this one do exercise components/, app/ and the API routes from the outside, but they are not line-instrumented and contribute nothing to this percentage.
Data service — coverage scope, from its own build report
Coverage of the entire application package: the ingestion and pricing logic, the FastAPI route handlers, the application entrypoint and the auth dependency. The scheduled sync workers and one-off maintenance scripts are not measured. The figures come from two suites that run together under one coverage run — unit tests against the services, and an in-process end-to-end suite that drives the real application object over HTTP, so routing, middleware, authentication and the response envelopes are executed rather than mocked. Neither suite touches the network; outbound sockets are blocked for the duration of the run, and the database and the third-party storefronts are the only doubles. coverage.py measures executable lines, so the line and statement figures are the same measurement reported under both names. A further HTTP contract suite (Postman/Newman) runs against the deployed service, including as a smoke test after every deploy; it is not measured by coverage.py and is not counted in the figures above.
Real device (Pixel 9a, Android 16, Chrome) — suite scope, 11 tests
End-to-end journeys driven against Chrome on a physically attached Android handset (Pixel 9a, Android 16) over adb and the Chrome DevTools Protocol, rather than an emulated viewport. It covers what emulation cannot: the real browser build, the real device pixel ratio and the on-device layout. Current specs cover the region switcher and its dropdown overflow, the paid-link disclosure as a touch user sees it (the badge tooltip never fires without a pointer, so the footnote carries the disclosure alone), cookie-consent dismissal and persistence, and price-table layout at real device width. These figures are NOT continuously verified: no CI runner has a handset, so this suite runs only on attached hardware and the numbers below are as of the measurement date given, not as of the latest commit. It does not gate deployment.
01 — THE GATE

Both sides block their own build

Each component runs its suite before it compiles, and the host runs that same command. A failing test produces no build, so there is nothing to deploy — the gate isn't a policy someone has to remember.

02 — THE BOUNDARY

The response mappers are tested, not trusted

Prices arrive from third-party APIs whose shapes change without notice. The code that maps those responses is covered on both sides, so a change upstream fails a test rather than surfacing as a wrong price.

03 — THE EVIDENCE

The service publishes its own results

The data service serves a build report of its own, which the site reads and republishes. The figures on this page can be checked against that source rather than taken on trust.

Where the prices come from

Published by the product itself, so this section can't drift from how the site actually behaves.

Price data
IsThereAnyDeal API v3 and the Steam storefront, ingested by a separate data service. No storefront is scraped directly.
Price freshness
Game pages revalidate every 24 hours and trigger a background price sync on view. Deals, new releases and top sellers revalidate hourly.
Regions priced
8 — GB, US, CA, DE, FR, NL, ES and IT, each in its own currency and number format.
Estimated prices
Currency-converted prices are marked as estimated, shown with a ~ prefix, and are never used as the headline lowest price when a real quote exists.
Affiliate disclosure
Some storefront links pay us, via affiliate networks (GOG, Humble Bundle) or the Epic Games Creator Program. Each is labelled "Affiliate" at the point of the link and carries rel="nofollow sponsored". Disclosed at https://www.grumpyoldgamers.net/affiliate-disclosure

A few of the pages

Each one is driven by the same priced dataset.

Grumpy Old Gamers is owned and operated by Axiom Testing Ltd (company no. 17267026) — we built it, we own it, and we maintain it. Our work on it is software testing, not price advice: displayed prices come from third-party sources and can change or lag.

One per slug returned by the data service, which is also one per URL in /game/sitemap.xml. Includes DLC, soundtracks and alternate editions: the upstream catalogue does not reliably distinguish them from base games, so this is a count of tracked pages, not of distinct games.

Shipping more than one moving part?

Tell us what you're building and we'll scope the coverage that fits it.

Start a project →