Testing Strategy

100% code coverage is a starting point, not a finish line. A test suite can achieve full coverage while still missing critical bugs. Pithos uses a multi-layered testing strategy where each level catches bugs that the others miss.

The Problem with Coverage Alone

Consider this function. It looks simple, but a single test can achieve 100% line coverage while completely missing a critical edge case:

function divide(a: number, b: number): number {
  return a / b;
}

A single test achieves 100% coverage:

it("divides two numbers", () => {
  expect(divide(10, 2)).toBe(5); // ✅ 100% coverage
});

But what happens with divide(10, 0)? The test passes, coverage is complete, yet the function silently returns Infinity. Coverage tells you code executed, not that it works correctly.

The Four Levels

Each level of testing answers a different question:

Level	Prefix	Question Answered	What It Catches
Standard	(none)	"Does each line execute?"	Dead code, forgotten branches
Edge case	`[🎯]`	"Are all API contracts tested?"	Untested union types, undocumented behaviors
Mutation	`[👾]`	"If someone changes this code, will a test fail?"	Weak assertions, silent regressions
Property-based	`[🎲]`	"Does it work for _any_ valid input?"	Edge cases: `null`, `""`, `NaN`, huge arrays

Why This Order Matters

Standard tests establish that your code runs
Edge case tests ensure your API promises are kept
Mutation tests verify your tests are actually checking something
Property-based tests explore the input space you didn't think of

A Complete Example: `evolve`

The evolve function applies transformation functions to object properties. Here's how each testing level contributes:

// evolve({ a: 5 }, { a: x => x * 2 }) → { a: 10 }

Standard Tests - "Does it work?"

These tests verify the documented behavior:

it("applies transformation functions to properties", () => {
  const result = evolve({ a: 5, b: 10 }, { a: (x: number) => x * 2 });
  expect(result).toEqual({ a: 10, b: 10 });
});

it("handles nested transformations", () => {
  const result = evolve(
    { nested: { value: 5 } },
    { nested: { value: (x: number) => x + 1 } }
  );
  expect(result).toEqual({ nested: { value: 6 } });
});

it("preserves properties without transformations", () => {
  const result = evolve({ a: 1, b: 2 }, {});
  expect(result).toEqual({ a: 1, b: 2 });
});

At this point, coverage might be 100%. But are the tests solid?

Edge Case Tests - "Is the API contract complete?"

The evolve function accepts transformations as either:

A function that transforms the whole value
An object of nested transformations

Both branches must be tested explicitly:

it("[🎯] applies function transformation to object value", () => {
  // Tests the "transformation is a function" branch for object values
  const result = evolve(
    { nested: { value: 5 } },
    { nested: (obj: { value: number }) => ({ value: obj.value * 2 }) }
  );
  expect(result).toEqual({ nested: { value: 10 } });
});

The [🎯] prefix signals: "This test covers a specific API branch or documented behavior."

Mutation Tests - "Are my tests actually checking?"

Stryker modifies your code and checks if tests fail. If they don't, you have a surviving mutant: a bug your tests would miss.

// Stryker might change this:
if (transformation === undefined) { ... }
// To this:
if (transformation !== undefined) { ... }  // Mutant!

If no test fails, you need a targeted test:

it("[👾] handles undefined transformation for nested object", () => {
  const result = evolve(
    { nested: { value: 5 }, other: 10 },
    { other: (x: number) => x * 2 }
  );
  // This specifically tests the branch where transformation is undefined
  expect(result).toEqual({ nested: { value: 5 }, other: 20 });
});

The [👾] prefix signals: "This test exists to kill a specific mutant."

Property-Based Tests - "Does it work for any input?"

Instead of testing specific values, test invariants that should hold for all inputs:

import { it as itProp } from "@fast-check/vitest";
import { safeObject } from "_internal/test/arbitraries";

// Invariant: without transformations, object is preserved
itProp.prop([safeObject()])(
  "[🎲] preserves object when no transformations",
  (obj) => {
    expect(evolve(obj, {})).toEqual(obj);
  }
);

// Invariant: all keys are preserved
itProp.prop([safeObject()])("[🎲] preserves all keys", (obj) => {
  const result = evolve(obj, {});
  expect(Object.keys(result).sort()).toEqual(Object.keys(obj).sort());
});

// Independence: evolve returns a new object
itProp.prop([safeObject()])("[🎲] returns new object reference", (obj) => {
  const result = evolve(obj, {});
  if (Object.keys(obj).length > 0) {
    expect(result).not.toBe(obj);
  }
});

The [🎲] prefix signals: "This test uses random inputs to verify an invariant."

Property-based tests found bugs in Pithos that hundreds of manual tests missed, particularly around edge cases like empty objects, objects with Symbol keys, and deeply nested structures.

The Result

Running the tests shows all levels working together:

Vitest output showing Pithos evolve function tests with unit, property, and mutation test prefixes

Each test has a clear purpose. No redundancy, no gaps.

When to Use Each Level

Situation	Recommended Approach
New function	Start with standard tests, add property-based for invariants
Union types in API	Add `[🎯]` tests for each branch
Mutation score < 100%	Add targeted `[👾]` tests for surviving mutants
Complex input domain	Add `[🎲]` tests with appropriate arbitraries
Bug report	Write a failing test first, then fix

Tools

Tool	Purpose	Documentation
Vitest	Test runner with TypeScript support	Guide
Stryker	Mutation testing	Docs
fast-check	Property-based testing	Tutorials

Running Tests

Pithos provides several test commands depending on what you want to verify, from a quick full run to targeted mutation testing on a single file:

# Run all tests
pnpm test

# Run tests with coverage report
pnpm coverage

# Run mutation tests (entire project)
pnpm test:mutation

# Run mutation tests on a specific file
pnpm stryker run --mutate 'packages/pithos/src/arkhe/object/evolve.ts'

Coverage Goals

Metric	Target	Why
Code coverage	100%	Baseline: ensures all code executes
Mutation score	100%	Ensures tests actually verify behavior
Property-based tests	1-5 per function	Explores edge cases automatically
Edge case coverage	All union types + `@note`	Ensures API contracts are tested

The Philosophy

"A test that passes when the code is wrong is worse than no test at all: it gives false confidence."

Each prefix tells a story:

No prefix: "This is expected behavior"
[🎯]: "This covers a specific API branch"
[👾]: "This kills a specific mutant"
[🎲]: "This verifies an invariant holds for any input"

When you see a failing test, the prefix immediately tells you why that test exists and what kind of bug you're dealing with.

Error Handling — How Pithos handles errors at the design level
Best Practices — Validate at boundaries, trust the types

The Problem with Coverage Alone​

The Four Levels​

Why This Order Matters​

A Complete Example: evolve​

Standard Tests - "Does it work?"​

Edge Case Tests - "Is the API contract complete?"​

Mutation Tests - "Are my tests actually checking?"​

Property-Based Tests - "Does it work for any input?"​

The Result​

When to Use Each Level​

Tools​

Running Tests​

Coverage Goals​

The Philosophy​

Related