Learn C++ - Performance & Optimization

#Introduction

C++ is chosen for its performance — but performance doesn't come by itself. The same code can run hundreds of times slower just because of poor memory layout or wrong compiler flags. The good news: C++ gives you the tools to measure and optimize.

Episode 15 teaches you how to think about performance: measure first with profiling before changing anything, understand compiler optimization flags, realize how big an impact the cache has, and maintain a balance between fast code and code humans can read.

#Measure First with Profiling

#Profiling with perf

Optimizing without measurement is guessing. perf is a lightweight profiler built into Linux that measures CPU, cache misses, and branch mispredictions. Build with -pg or debug symbols, then profile:

g++ -g -O2 -std=c++20 app.cpp -o app
perf stat ./app
perf record -g ./app
perf report

perf stat ./app shows a summary — how long the execution took, how many cache misses. perf record -g ./app records call stacks, and perf report shows which functions are the hottest. Pay attention to the cache misses column: high numbers signal a memory locality problem.

#Valgrind for Memory and Timing

Valgrind checks for leaks and memory errors, and callgrind provides a more detailed profile without hardware support. Run:

valgrind --leak-check=full ./app

valgrind --leak-check=full ./app reports every leaked block along with its stack trace. Programs run much slower under Valgrind, so use it on debug builds. You can also use -fsanitize=address, already covered in episode 11, as a faster alternative.

#Compiler Optimization Flags

#Optimization Levels

C++ compilers have optimization levels that trade compilation time and binary size for speed: -O0 (no optimization, for debugging), -O1, -O2 (production default), -O3 (aggressive optimization), and -Os (size optimization). -march=native enables instructions available on your CPU:

g++ -O3 -DNDEBUG -march=native -std=c++20 app.cpp -o app

-O3 enables aggressive optimizations like inlining and vectorization. -DNDEBUG removes all asserts — important because assert is only active in debug builds. -march=native targets the local CPU, but don't use it for distributed binaries because the code won't run on older CPUs.

#Beware of Aggressive Optimization

-O3 and fast-math can change program behavior. -ffast-math assumes no NaN and ignores the sign of zero, which is valid for numerical simulation but can break code that depends on IEEE 754. Measure the impact; if -O2 is fast enough, you don't need -O3.

#Memory Layout and Cache Friendliness

#The Cache Dominates Performance

Reading from RAM is much slower than from the L1 cache. Programs that jump around in memory spend their time waiting for data. The key to cache friendliness is locality: sequential memory access, and data used together placed close together.

Contiguous containers like std::vector store elements adjacently, so sequential iteration uses the cache well. std::list, whose elements are scattered randomly, is actually slow to iterate even though insertion is O(1):

cat > cache.cpp <<'EOF'
#include <iostream>
#include <vector>
 
int main() {
    std::vector<std::vector<int>> grid(1000, std::vector<int>(1000, 1));
    long long total = 0;
 
    for (int i = 0; i < 1000; ++i) {
        for (int j = 0; j < 1000; ++j) {
            total += grid[i][j];
        }
    }
    std::cout << total << "\n";
}
EOF
g++ -O2 -std=c++20 cache.cpp -o cache
./cache

The loop grid[i][j] with row-first order reads memory contiguously — this is the cache-friendly row-major pattern. Reversing the order to grid[j][i] would make the program much slower because it jumps between distant rows.

#Data Layout

The order of fields in a struct affects its size and cache behavior. Arrange fields from largest to smallest to reduce padding. This rule matters for large struct arrays iterated constantly. sizeof(struct) reveals the real result — a small optimization with a big effect on hot data.

#Inline and Loop Unrolling

#Inline

Inlining copies a function body to the call site, eliminating call overhead. With -O2 and above, the compiler decides on its own which functions to inline. The inline keyword is only a suggestion; __attribute__((always_inline)) forces it. Rule of thumb: let the compiler choose — too much inlining bloats the instruction cache.

#Loop Unrolling

Loop unrolling duplicates the loop body so each iteration processes several elements, reducing condition-checking overhead. The compiler does this automatically at -O3; manual unrolling with a pragma is available if needed:

#include <iostream>
#include <vector>
 
#pragma GCC unroll 4
void jumlah(const std::vector<int>& v, long long& total) {
    for (int x : v) {
        total += x;
    }
}

#pragma GCC unroll 4 asks the compiler to open up the loop 4 times per iteration. Before using a pragma or other micro-optimizations, measure first — the compiler often already produces good code without intervention.

#The Readability vs Performance Tradeoff

#The Best Optimization Is the One You Don't Need

Lots of code is "optimized" prematurely — writing micro-optimizations before there's evidence of need. The result is complicated, hard-to-maintain code that often isn't even faster because the compiler already handled it. The principle the industry holds:

• Measure first: profile before changing anything.
• Optimize only the bottleneck: focus on the dominant hot functions.
• Keep quality: clear code is easier to optimize later.
• Document the reasoning: comment why an optimization was done.

There are two architectural decisions with more impact than micro-optimizations: choosing the right container (episode 8) and avoiding data copies with references and move semantics (episode 16). Make it a habit to start there.

#Conclusion

Here's what to take away:

• Profile first with perf or Valgrind before optimizing anything.
• -O2 is the production default; -O3 and -march=native as needed.
• Cache locality dominates performance; std::vector is more cache-friendly than std::list.
• Arrange struct fields from largest to smallest to reduce padding.
• Inlining and loop unrolling should be left to the compiler.
• Premature optimization is a bug source; focus on measured bottlenecks.

In the next episode, episode 16, we'll discuss modern C++ features — auto, range-based loops, and structured bindings, move semantics with rvalue references, std::optional, std::variant, std::any, and coroutines, as well as modules, concepts, and constexpr improvements.