Quality assurance

Quality assurance and software testing

Find the defects before your users do. And before any of that: work out whether what you have is a testing problem at all, because often it is not.

Diagnose first

Is this actually a testing problem?

Three symptoms that look identical from the outside and have entirely different causes. Buying testing for the third one wastes your money and ours.

  • Bugs reach users that testing should have caught This is a coverage problem. There is testing, and it is not looking in the right places. Fixable by knowing which paths matter.
  • The same defect keeps coming back This is a regression problem. Each fix is real and nothing stops the next change undoing it. What is missing is a set of checks that run every release.
  • Something breaks every single release, somewhere new Not a testing problem. It points upstream, at how changes are made, reviewed and deployed. More testing reports it sooner; it will not reduce it. We say so rather than sell the wrong thing.
If it is the third one, start here
Five bars approaching a vertical line: four continue through to the released side, the third stops at the line and is marked stopped here. Beneath: one did not get through — which one, and why, is the finding, along with what was never checked at all.
Automated

Automate the checks that run every single release

Automation is an investment with a break-even point. A test that runs on every release for two years repays writing it many times over; a test written once for a feature nobody touches again never does.

So the honest rule is not “automate everything”. It is: automate the paths where a failure would be serious and the check is identical every time — logging in, taking a payment, saving the thing the business runs on.

Manual

Do by hand the things that need a person to judge

Anything where the question is “is this right?” rather than “did this run?”. Whether a layout is usable on a small screen, whether an error message makes sense, whether a new feature interacts strangely with an old one.

Exploratory testing — someone deliberately trying to break it, without a script — finds the class of defect no automated suite was ever written to look for, because nobody had thought of it yet.

Use this without hiring anyone

Before you release: a checklist you can run yourself

01

The paths that pay the bills

Walk through the two or three journeys the business actually depends on, end to end, as a real user with real data. Not the feature you just changed — the ones you did not.

02

One level of user below the top

Most testing is done by someone with full permissions. Log in as the most restricted role that exists and try the same journeys.

03

The wrong input

Empty required fields, an absurd number, a date in the past, a file of the wrong type, the same form submitted twice. How the software behaves when it is misused is the part users meet and specifications rarely cover.

04

A real phone, not a resized window

A narrow browser window is not a phone. Touch targets, the on-screen keyboard covering the field you are typing in, and a form that cannot be submitted because the button is off-screen.

05

What the last release broke

Re-check whatever was fixed in the previous two releases. The cheapest regression test available, and the one that needs no tooling at all.

06

The way back out

Cancel halfway. Hit the browser's back button mid-flow. Refresh on a confirmation page. A broken exit is how half-finished records get written.

07

Write down what you did not check

The single most useful item on this list. An untested area you know about is a risk; one you have forgotten about is an incident.

What an engagement produces

What you are left holding afterwards

  • A test plan tied to your actual risks

    Not a generic matrix of test types. The journeys that matter to your business, in priority order, with the reason each one is on the list.

  • Defects you can act on

    Each with the steps to reproduce it, what happened, what should have happened, and where it sits relative to everything else. A defect nobody can reproduce is not a finding.

  • A regression set that outlives us

    The checks that run on every release, in your repository, runnable by your team or the next supplier. It is yours, and it is the reason ongoing changes stay affordable.

  • A written statement of what was not covered

    The areas we did not test and why — out of scope, no test data, no access to a system. This is the item that protects you, because it turns an unknown risk into a decision you get to make.

Not sure whether you need testing or something upstream?

Describe what breaks and how often. Telling you the problem is not testing costs us a sale and saves you a quarter.