Profiling and Performance

Measure before changing code. Use a representative workload and keep a baseline for comparison.

What time decomposes into

Elapsed time includes both running and waiting. User and system CPU time help distinguish computation from blocking, though multithreaded programs can accumulate more CPU time than elapsed time.

perf stat reports hardware counters. Low instructions per cycle may indicate stalls, but the cause needs further measurement. Supported systems also offer top-down analysis of frontend, backend and speculation costs.

Sampling and its biases

A CPU profile samples running code; it does not explain time spent waiting. Use blocking or off-CPU profiles for that.

Samples may be attributed slightly after the event that caused them. Inlining and missing frame pointers can also make stacks harder to read. Check the profiler's collection and unwinding settings.

Measurements that mislead

Use realistic input sizes and access patterns.
Ensure the benchmark consumes its results so the compiler cannot remove the work.
Allow for warmup, frequency changes and background activity.
Repeat measurements before trusting a small improvement.

Where the wins are

Check the algorithm first. Then examine memory access, allocation and contention. Contiguous data often uses caches better than pointer-heavy structures.

Changing layout or reducing allocation can help, but measure each change. Branchless code and specialised allocators are not automatically faster.

Amdahl, and what parallelism does not fix

The serial part of a workload limits its speedup. More threads can add contention around shared counters, locks and allocators.

For services, track latency percentiles as well as averages. Queueing, collection pauses and page faults may mainly affect the slowest requests.

A useful measurement record

Save the commit, build mode, hardware, input, concurrency and tool settings. Record elapsed time, memory and the measure you actually want to improve. Keep the baseline executable or its measurements so later comparisons use the same conditions.

Throughput and latency

Throughput is work completed per unit time. Latency is the time a piece of work takes. Improving one does not necessarily improve the other: batching may increase throughput while making individual requests wait.

Track the distribution of request times. Averages hide slow tails. Compare systems at the same load and include failed requests in the results.

Memory measurements

Allocated bytes, live heap and resident memory are different quantities. A program can allocate heavily while retaining little, or retain a large heap while allocating slowly.

Look for retained references when memory grows between repeated operations. Use allocation profiles for churn and heap snapshots for what stays alive.

Accepting a change

Measure the same workload before and after. Check that results are still correct and that the gain holds beyond one favourable input. Keep an optimisation when its measured benefit justifies its added complexity.