Why We Test: Verification and Validation

Last updated on 2026-07-16 | Edit this page

Overview

Questions

  • Why do we test software?

Objectives

  • Understand what is testing
  • Understand the goals of testing

The Natural Skepticism

If you are new to Test-Driven Development, your brain is probably screaming some version of this right now:

“Okay, that was a cool trick, but why am I restructuring my entire mental workflow for this? Can’t I just print the output and look at it?”

It is a completely fair question. When we write small, single-file scripts, manual testing (running the code, pointing our eyeballs at the screen, and saying “Yep, looks right”) feels incredibly fast.

But manual verification has a massive shelf-life problem. The moment your project grows past one file, or you hand it to a collaborator, or you return to it after a six-month hiatus, manual testing breaks down.

To understand why we build automated test suites, we need to understand the two questions every scientist must answer about their software.

Verification vs. Validation

In software engineering—and especially in computational science—we draw a hard line between two related but fundamentally different concepts: Verification and Validation.

SH 

              +-----------------------------------+
              |        Computational Code         |
              +-----------------------------------+
                                |
             Is the math implemented correctly?
                                |
                [ YES: Code is VERIFIED ]
                                v
        Does the model match empirical physical reality?
                                |
                [ YES: Code is VALIDATED ]

1. Verification: “Are we building the product right?”

Verification is a check against your own specifications. It asks: Does the code do what we, the programmers, intended it to do? * Are the loops indexing correctly? * Is our rectangle overlap logic catching edge cases without throwing an error? * Are our units converted properly?

This is what software testing solves. A passing test suite means your code is verified.

2. Validation: “Are we building the right product?”

Validation is a check against the real world. It asks: Does our software model actually reflect empirical reality? * Even if our rectangle overlap code is 100% bug-free, does representing a complex biological cell as a simple 2D rectangle actually simulate cell collisions accurately? * Does our thermodynamic simulation accurately predict the physical state of the gas?

This is solved by scientific experimentation, field data, and peer review.

The Crucial Takeaway

A code can be perfectly verified while being completely invalid. Your tests can all be beautifully green, but if your underlying physical equations are wrong, your science is still wrong. However, you cannot validate an unverified model. If your code has silent indexing bugs, your physical simulations are meaningless noise. Verification is the prerequisite for validation.

Case Study: The “Supercooled Water War”

To see the real-world cost of skipped verification, we only have to look at one of the most famous computer simulation controversies in modern chemistry: the “Supercooled Water War.”

For years, researchers debated whether deeply supercooled water could spontaneously split into two distinct liquid phases of different densities. * The Claim: A highly respected Berkeley group published molecular dynamics papers claiming the answer was a definitive “No.” * The Catch: The group relied on a private, custom simulation code that wasn’t shared. * The Discovery: A Princeton team spent years trying to reproduce the Berkeley results. When they finally got a look at the Berkeley code years later, they found a critical, silent bug. An algorithmic shortcut in how the code sampled physical states meant the code wasn’t actually doing the math the scientists thought it was doing.

Because the code was never properly verified with rigorous unit tests, a silent software bug held back an entire scientific field for nearly a decade.

The Berkeley scientists had spent years validating their grand physical theories on code that was fundamentally unverified.

The High Cost of Manual Testing

If we agree we must verify our code, why can’t we just do it manually?

1. Humans are Terrible at Repetitive Tasks

If you modify your coordinate system in three weeks, are you really going to manually type in 15 different rectangle coordinates and eyeball the print statements to make sure you didn’t break anything?

No. You will test the two cases you care about right now, assume the rest are fine, and unknowingly push a silent regression bug to GitHub.

2. The Bug Cost Curve

Bugs are incredibly cheap to fix when they are caught immediately. They become exponentially more expensive—and devastating—the longer they live in your codebase.

  • Caught in the TDD Loop (Seconds): Cost is zero. You fix the typo, run pytest, and move on.
  • Caught before Merge (Hours/Days): Low cost. You fix it before your peers see it.
  • Caught post-publication (Months/Years): Disastrous. Retracted papers, corrupt public datasets, and lost credibility.
Discussion

Discussion: Your Scientific Baseline (5 mins)

Think about a computational tool, script, or model you currently use in your daily research.

  1. How do you currently check that this code is verified (doing what you expect)?
  2. What is the most terrifying silent bug that could exist in that codebase right now without you knowing?
  3. How would you design a simple automated test to catch that specific bug?
Key Points
  • Testing improves confidence about your code. It doesn’t prove that code is correct.
  • Testing is a vehicle for both software design and software documentation.
  • Creating and executing tests should not be an afterthought. It should be an activity that goes hand-in-hand with code development.
  • Tests are themselves code. Thus, test code should follow the same overarching design principles as functional code (e.g. DRY, modular, reusable, commented, etc).