Engineering

Test Suites People Actually Trust

A flaky suite is worse than no suite, because it teaches the team to ignore red. How to build coverage that earns confidence and keeps it.

Purushottam Kumar Suman
Purushottam Kumar Suman
Founder & CEO, Drema AI
8 min read
Female engineer working on code in a contemporary office

The measure of a test suite is not coverage percentage. It is whether a green run gives the team enough confidence to release on a Friday afternoon, and whether a red run makes anyone stop what they are doing.

01

Flakiness is the fatal defect

A suite that fails randomly trains everyone to re-run it until it passes. From that moment the suite is decorative: it will catch a genuine regression and be dismissed as noise. Fixing flakiness therefore outranks adding coverage, always.

The moment a team starts re-running a red build, the suite has stopped working.

02

The three causes of flakiness

Almost every intermittent failure traces to timing assumptions, shared state between tests, or non-deterministic data. Waiting for a condition rather than a duration, isolating data per test, and freezing time and randomness fix the overwhelming majority. Adding a retry hides the symptom and preserves the bug.

TimingWait for conditions, never for durations
Shared stateFresh data per test, no ordering dependency
Non-determinismFreeze clocks, seed randomness
RetriesHide flakiness — and real intermittent bugs
03

Test at the level that catches real bugs

Unit tests around pure logic are fast and precise. Integration tests catch the things unit tests mock away — and mocked-away boundaries are exactly where bugs live. A modest number of tests against a real database usually finds more genuine defects than an exhaustive suite of mocked unit tests.

04

End-to-end: few, critical, reliable

Browser tests are slow and the most prone to flake, so spend them on the handful of journeys that cost money when broken — signup, checkout, the core workflow. Ten reliable end-to-end tests beat two hundred that half the team ignores.

05

Speed is a feature of the suite

A suite that takes forty minutes will be skipped under deadline pressure. Parallelise aggressively, run a fast subset on every commit and the full set before merge. Under ten minutes for the pre-merge suite is a target worth engineering for, because it changes behaviour.

06

Let failures write your tests

Every production bug should leave behind a test that would have caught it. This grows the suite along the axis of what actually breaks in your system, which no coverage target can direct you towards. Over a year it becomes a record of the ways your product fails, and it is worth more than any percentage.

0
Tolerated flaky tests
<10 min
Pre-merge suite target
1 test
Per production bug, permanently
Purushottam Kumar Suman
Written by
Purushottam Kumar Suman
Founder & CEO, Drema AI

Founder and CEO of Drema AI. Builds AI systems, SaaS platforms and industry software — and writes about what actually survives production.

CTA Background

Got a problem like this one?

Bring it to a call with a founder.You will get a straight answer, including when the answer is no.

View Our Work
AI-First Engineering
Secure & Scalable
Built to Deliver Impact