CUCaTS

Example tests check known cases. Property-based tests and fuzzing help explore inputs you have not listed by hand.

Properties and generators

A property describes something that should hold across many inputs:

from hypothesis import given, strategies as st

@given(st.lists(st.integers()))
def test_roundtrip(xs):
    assert decode(encode(xs)) == xs

Useful properties include round trips, idempotence, preserved invariants and agreement with a reference implementation. The functions above are placeholders for the code being tested.

Generators should include boundaries, duplicates and other important cases. Shrinking reduces a failing input to a smaller counterexample. Stateful testing checks sequences of operations against a model.

Coverage-guided fuzzing

Tools such as AFL++ and libFuzzer mutate inputs and keep cases that reach new code paths. A harness connects the generated input to your code.

Combine fuzzing with sanitizers to detect memory errors. Save useful inputs in a corpus so later runs build on earlier work.

Oracles

A test needs a way to recognise failure. This may be an expected result, a reference implementation, an invariant or a relationship between outputs.

For example, sorting a permuted input should give the same result. Internal consistency checks can catch invalid states even when the final output is hard to predict.

Mutation testing

Coverage shows which code ran. Mutation testing makes small changes to the code and checks whether tests catch them. Surviving changes can reveal missing assertions, though some are equivalent to the original program.

Determinism and flakes

Control clocks, random seeds and external services where practical. Record enough information to reproduce failures. Investigate flaky tests rather than repeatedly rerunning them until they pass.

Choosing the boundary

Unit tests exercise a small piece of behaviour. Integration tests check that components work together. End-to-end tests follow a complete path through the system. Use each where its boundary reveals a useful failure.

Keep assertions about observable results. A test that repeats the implementation can repeat its mistake. For a regression, preserve the input or sequence that originally exposed the bug.

Cases to cover

Empty input, one item and the smallest valid case.
Boundary values and values just outside the accepted range.
Duplicates, missing fields and unexpected ordering.
Failures from dependencies, cancellation and timeouts.
Repeated operations and cleanup after partial failure.

Not every function needs every case. Choose cases from the contract it promises.

Testing