professional headshot photo of Mr Palumbo
NICHOLAS PALUMBO October 4, 2024

User Acceptance Testing, and What It Is Actually For

User acceptance testing is the check that software solves the problem it was built for. It is done by the people who will actually use the thing, not by the people who wrote it, and it happens on something that looks like production rather than a developer laptop.

I have not run a formal UAT program. This is a researched explainer, not a report from the trenches. I wrote it because UAT is one of those terms that gets used loosely to mean "the testing we do at the end", and the standard definitions are more specific and more useful than that. Sources are linked at the bottom.

The actual definition

The ISTQB glossary defines user acceptance testing as acceptance testing carried out by future users in a simulated operational environment, focusing on user requirements and needs. Acceptance testing more broadly is defined as formal testing with respect to user needs, requirements and business processes, conducted to determine whether a system satisfies the acceptance criteria and to let the customer decide whether to accept the system.

Two things in that definition do most of the work. It is carried out by future users, which rules out the development team doing it themselves. And its output is an accept or do not accept decision, not a bug list. A UAT that produces only a pile of tickets and no decision has not really finished.

UAT sits inside a family of related terms that get mixed up constantly: operational acceptance testing (OAT), which covers whether the operations team can actually run the thing, contract and regulation acceptance testing, and alpha and beta testing. The ISTQB entry for acceptance testing lists them.

It is not the same as QA

Unit tests, integration tests and system tests answer whether the software does what the spec said. UAT answers whether the spec was right. Those are genuinely different questions and passing one tells you very little about the other.

This is why UAT catches a specific class of problem that no amount of automated testing will find: a feature that works exactly as designed and is still wrong, because the design was based on a misunderstanding of how the work actually gets done.

Acceptance criteria come first

ISTQB defines acceptance criteria as the criteria a system must meet to be accepted by a user or customer. The important property is that they are written down before testing starts, and that they are specific enough that two people would agree on whether a given criterion passed.

"The checkout should be fast" is not an acceptance criterion. "A logged-in user can complete checkout in under 60 seconds without leaving the page" is one. If the criteria are written after the testing, or written vaguely enough that they can be argued either way, the accept decision collapses into whoever is most senior in the room.

In agile teams this usually shows up as acceptance criteria on the user story plus a team-wide definition of done. ISO/IEC TR 29119-6, the part of the software testing standard covering agile projects, is the formal treatment of that.

Running it

  • Pick testers who do the actual job. The value comes from people who know the business process well enough to notice when the software quietly makes it harder. A tester who has never done the task will click through the happy path and report that it works.
  • Give them realistic scenarios, not click scripts. A script tells you the button works. A scenario like "process a return for a customer who paid with two payment methods" tells you whether the workflow holds up.
  • Use a production-like environment with realistic data. Most of the interesting failures only appear at real data volumes or with the messy records that accumulate in a real system.
  • Make reporting an issue take under a minute. Testers are doing this on top of their real job. Friction in the reporting path means you silently lose the smaller findings, which are often the usability ones.
  • Separate defects from change requests. "This is broken" and "this is not what I wanted" need different handling. Mixing them is how a UAT turns into an unbounded second requirements phase.
  • End with an explicit decision. Accept, accept with known issues and a fix schedule, or do not accept. Write down which one, and who made the call.

How it usually goes wrong

  • It gets scheduled as a buffer. UAT is put in the plan as the last two weeks, then the build runs late and eats them. The testing that survives is a demo, not an acceptance test.
  • The testers are proxies. Business analysts or product managers stand in for real users because real users are busy. They tend to already share the team's assumptions, which is exactly the thing UAT is supposed to check.
  • There are no criteria. Without written criteria the accept decision becomes a vote on how tired everyone is.
  • Findings have nowhere to go. If there is no agreed path for a finding to become a fix before launch, testers work out quickly that reporting things is pointless.

Sources