How to Reach Peak Floating-Point Throughput on x86-64 Intel CPUs in C/C++

By the end of this page, you will understand why theoretical FLOPs-per-cycle numbers are difficult to achieve in practice, how latency, throughput, SIMD width, instruction scheduling, and loop unrolling interact, and what kind of benchmark is actually needed to approach peak floating-point throughput on Intel x86-64 CPUs.

Peak floating-point performance is not just about how fast a single instruction finishes. It depends on several lower-level CPU concepts working together.

The key ideas

1. Latency is not the same as throughput

• Latency = how many cycles one instruction takes to finish.
• Throughput = how often the CPU can start a new instruction of that type.

For example, a floating-point multiply may take 5 cycles of latency, but the CPU may still start one multiply every cycle if there are enough independent multiply chains.

2. Independent operations are required

This works:

a *= x;
b *= x;
c *= x;
d *= x;

because each variable is independent.

This does not allow full throughput:

a *= x;
a *= x;
a *= x;
a *= x;

because each operation depends on the previous result.

3. SIMD increases work per instruction

Scalar SSE instructions like addsd operate on one double. Packed SSE instructions like addpd operate on two doubles at once.

So if the CPU can issue one packed add per cycle, that is already 2 double-precision FLOPs per cycle.

4. Different execution ports matter

Many Intel CPUs have separate execution resources for floating-point add and multiply. In the best case, the CPU can issue:

• one packed add per cycle
• one packed multiply per cycle

With SSE packed doubles, that means:

• addpd = 2 FLOPs
• mulpd = 2 FLOPs
• total = 4 FLOPs/cycle

But that is only possible if:

• the CPU microarchitecture supports this issue pattern
• the compiler generates suitable code
• there are enough independent operations
• instruction ordering does not create bottlenecks
• loop overhead and branch overhead are small enough

5. Theoretical peak is a best-case microbenchmark

The advertised peak assumes an ideal loop that:

• stays entirely in registers
• has no memory bottleneck
• has no dependency stalls
• is heavily unrolled
• uses packed SIMD instructions
• is scheduled to match the CPU’s execution units

Your original code is close in spirit, but it uses scalar operations and relies on the compiler to schedule things ideally. That often falls short of hardware peak.

Why this matters in real programming

Understanding peak throughput helps with:

• writing realistic microbenchmarks
• understanding why vectorized code is much faster than scalar code
• interpreting profiler results correctly
• knowing when your code is CPU-bound vs memory-bound
• understanding why compilers sometimes reorder operations or unroll loops

In short: peak FLOPs/cycle is usually achieved only in carefully designed tight loops, not automatically in ordinary C/C++ source.

Think of the CPU as a small factory with multiple workstations.

• One station handles adds.
• One station handles multiplies.
• Each station can start a new job every cycle.
• But each job takes a few cycles to finish.

If you only give the factory one item at a time, workers sit idle waiting for the previous item to move forward.

If you give the factory many independent items, all stations stay busy.

Now add SIMD:

• A scalar instruction is like moving one box.
• A packed SSE instruction is like moving two boxes at once.

So the maximum output comes from:

1. feeding both stations continuously
2. making sure jobs are independent
3. packing more data into each instruction
4. reducing management overhead like branching and loop counters

That is why peak FLOPs/cycle is less about one fast instruction and more about keeping the whole pipeline busy.

Scalar vs packed SSE idea

At a high level, these operations differ like this:

Scalar style

double a = 1.0, b = 2.0;
a += b;
a *= b;

This works on one double at a time.

Packed style with intrinsics

#include <immintrin.h>

__m128d a = _mm_set_pd(2.0, 1.0);
__m128d b = _mm_set1_pd(3.0);

a = _mm_add_pd(a, b);
a = _mm_mul_pd(a, b);

This works on two doubles at once using SSE2.

A simple throughput-oriented example

The following example creates several independent SIMD chains so the CPU can overlap work:

#include <immintrin.h>
#include <stdio.h>

int main() {
    __m128d addv = _mm_set1_pd(1.0);
    __m128d mulv = _mm_set1_pd(1.0000001);

    __m128d s0 = _mm_set_pd(1.0, 2.0);
    __m128d s1 = _mm_set_pd(3.0, 4.0);
    __m128d s2 = _mm_set_pd(5.0, 6.0);
    __m128d s3 = _mm_set_pd(7.0, 8.0);

    for (int i = 0; i < 100000000; i++) {
        s0 = _mm_add_pd(s0, addv);
        s1 = _mm_mul_pd(s1, mulv);
        s2 = _mm_add_pd(s2, addv);
        s3 = _mm_mul_pd(s3, mulv);
    }

    double out[2];
    _mm_storeu_pd(out, s0);
    printf("%f %f\n", out[0], out[1]);
    return 0;
}

Why this is better than a single chain

This loop is better for throughput because:

• s0, s1, s2, and s3 are independent
• packed instructions process 2 doubles each
• the CPU can overlap adds and multiplies more easily

Why it still may not hit peak

Even this may not reach theoretical peak because:

• the compiler may not schedule instructions ideally
• the loop may need more unrolling
• the target CPU may have architectural limits
• the benchmark still includes loop control overhead

A more realistic benchmark pattern

A better benchmark usually uses:

• more independent accumulators
• explicit unrolling
• packed SIMD intrinsics or hand-written assembly
• minimal loads and stores inside the loop

Example:

#include <immintrin.h>

void kernel(long long iters) {
    __m128d addv = _mm_set1_pd(1.0);
    __m128d mulv = _mm_set1_pd(1.0000001);

    __m128d a0 = _mm_set1_pd(1.0);
    __m128d a1 = _mm_set1_pd(2.0);
    __m128d a2 = _mm_set1_pd(3.0);
    __m128d a3 = _mm_set1_pd(4.0);
    __m128d m0 = _mm_set1_pd(1.1);
    __m128d m1 = _mm_set1_pd(1.2);
    __m128d m2 = _mm_set1_pd(1.3);
    __m128d m3 = _mm_set1_pd(1.4);

    for (long long i = 0; i < iters; i++) {
        a0 = _mm_add_pd(a0, addv);
        a1 = _mm_add_pd(a1, addv);
        a2 = _mm_add_pd(a2, addv);
        a3 = _mm_add_pd(a3, addv);

        m0 = _mm_mul_pd(m0, mulv);
        m1 = _mm_mul_pd(m1, mulv);
        m2 = _mm_mul_pd(m2, mulv);
        m3 = _mm_mul_pd(m3, mulv);
    }

    volatile double sink[2];
    _mm_storeu_pd((double*)sink, _mm_add_pd(a0, m0));
}

This structure gives the compiler more freedom to fill the pipeline.

Consider this smaller example:

#include <immintrin.h>

void demo() {
    __m128d addv = _mm_set1_pd(1.0);
    __m128d mulv = _mm_set1_pd(2.0);

    __m128d a0 = _mm_set1_pd(10.0);
    __m128d a1 = _mm_set1_pd(20.0);
    __m128d m0 = _mm_set1_pd(3.0);
    __m128d m1 = _mm_set1_pd(4.0);

    for (int i = 0; i < 2; i++) {
        a0 = _mm_add_pd(a0, addv);
        a1 = _mm_add_pd(a1, addv);
        m0 = _mm_mul_pd(m0, mulv);
        m1 = _mm_mul_pd(m1, mulv);
    }
}

Before the loop

• a0 = [10.0, 10.0]
• a1 = [20.0, 20.0]
• m0 = [3.0, 3.0]
• m1 = [4.0, 4.0]
• addv = [1.0, 1.0]
• mulv = [2.0, 2.0]

First iteration

a0 = a0 + addv;  // [11.0, 11.0]
a1 = a1 + addv;  // [21.0, 21.0]
m0 = m0 * mulv;  // [6.0, 6.0]
m1 = m1 * mulv;  // [8.0, 8.0]

• Two packed adds happened.
• Two packed multiplies happened.
• Each instruction processed 2 doubles.

So this iteration performs:

• 2 add instructions × 2 doubles = 4 FLOPs
• 2 mul instructions × 2 doubles = 4 FLOPs
• total = 8 FLOPs

Second iteration

a0 = [12.0, 12.0]
a1 = [22.0, 22.0]
m0 = [12.0, 12.0]
m1 = [16.0, 16.0]

Again, the CPU can overlap these because the chains are independent:

• a0 depends only on prior a0
• a1 depends only on prior a1
• m0 depends only on prior m0
• m1 depends only on prior m1

Why independence matters

If all operations updated only one register, the next instruction would have to wait for the previous result. Multiple independent accumulators hide latency and increase throughput.

Even though this question is about a microbenchmark, the underlying idea shows up in real software.

Numerical computing

Scientific code often tries to keep arithmetic units busy in kernels such as:

• vector updates
• matrix multiplication
• polynomial evaluation
• simulation loops

These kernels are often manually unrolled or vectorized.

DSP and signal processing

Operations like filtering and transforms often use repeated multiply/add patterns. Throughput matters because the same small loop runs millions of times.

Graphics and physics

Game engines and physics solvers perform large numbers of floating-point operations on vectors. SIMD and instruction scheduling can matter a lot in hot loops.

ML and linear algebra

Although modern workloads often use AVX, AVX2, AVX-512, or GPUs, the same principle applies: peak performance depends on feeding vector units efficiently.

Benchmarking and performance engineering

When teams evaluate CPUs or compilers, they often write small kernels like this to measure:

• raw arithmetic throughput
• compiler vectorization quality
• instruction scheduling behavior
• differences across microarchitectures

In real projects, developers rarely write a benchmark whose only purpose is to maximize FLOPs. But they do use the same patterns inside performance-critical code.

Common patterns

Multiple accumulators

Instead of one running total:

double sum = 0.0;
for (int i = 0; i < n; i++) {
    sum += data[i];
}

developers may use several accumulators to reduce dependency chains:

double s0 = 0.0, s1 = 0.0, s2 = 0.0, s3 = 0.0;
for (int i = 0; i < n; i += 4) {
    s0 += data[i];
    s1 += data[i + 1];
    s2 += data[i + 2];
    s3 += data[i + 3];
}
double sum = s0 + s1 + s2 + s3;

This helps expose instruction-level parallelism.

Loop unrolling

Compilers or developers unroll loops to:

• reduce branch overhead
• expose more independent instructions
• improve scheduling opportunities

SIMD intrinsics

For critical kernels, codebases often use intrinsics such as:

• _mm_add_pd
• _mm_mul_pd
• _mm256_add_pd
• _mm256_mul_pd

This gives more direct control than plain scalar C++.

Guarding against optimization removal

Benchmarks need to ensure the compiler cannot delete the work. Common techniques:

• store final results to memory
• return a value derived from the computation
• use volatile only carefully

Architecture-specific tuning

Real codebases often provide several implementations:

• scalar fallback
• SSE2 version
• AVX2 version
• AVX-512 version

and pick the best one at compile time or runtime.

Important practical note

Real applications are often limited by memory bandwidth, cache misses, or data layout, not just raw FLOP throughput. Microbenchmarks reveal CPU capability, but application performance depends on the whole system.

1. Confusing latency with throughput

Beginners often think a 5-cycle multiply means only one multiply every 5 cycles.

That is incorrect on a pipelined CPU.

• Latency: result ready after 5 cycles
• Throughput: a new multiply may still start every cycle

2. Using dependent operations only

Broken benchmark pattern:

double x = 1.0;
for (int i = 0; i < n; i++) {
    x *= c;
}

Why it is bad:

• every iteration depends on the previous result
• the CPU cannot overlap many multiplies

Better:

double x0 = 1.0, x1 = 1.1, x2 = 1.2, x3 = 1.3;
for (int i = 0; i < n; i++) {
    x0 *= c;
    x1 *= c;
    x2 *= c;
    x3 *= c;
}

3. Measuring scalar code but expecting SIMD peak

If you use addsd and mulsd, you are only operating on one double per instruction. You cannot expect packed-SSE peak from scalar instructions.

4. Forgetting loop overhead

Very small loops pay for:

• incrementing the counter
• comparing the counter
• branching back

Unrolling helps reduce this overhead.

5. Assuming the compiler will schedule perfectly

Compilers do a good job, but not always the ideal job for a specific microarchitecture. Your generated assembly may be correct but not optimal for peak throughput.

6. Ignoring microarchitecture differences

Different Intel generations behave differently. A schedule that works well on one CPU may be suboptimal on another.

7. Letting the compiler optimize away the benchmark

If the result is unused, the compiler may remove work.

Broken example:

for (int i = 0; i < n; i++) {
    x += 1.0;
}

If x is never observed, this may disappear entirely.

8. Using a weak timing method

For very short benchmarks, coarse timers can distort results. Use a stable timing method and enough iterations to make the runtime significant.

Scalar vs packed example

// Scalar: 1 double per instruction
x = x + a;
y = y * b;

// Packed SSE2: 2 doubles per instruction
vx = _mm_add_pd(vx, va);
vy = _mm_mul_pd(vy, vb);

Throughput benchmark vs real application

A loop can be ideal for peak FLOPs but still not represent real application performance.

Core rules

• Latency = cycles until result is ready
• Throughput = new instructions started per cycle
• Peak FLOPs require high throughput, not just low latency
• Use independent accumulators to hide latency
• Use SIMD packed instructions to process multiple values per instruction
• Use loop unrolling to reduce branch overhead and improve scheduling

SSE2 double-precision basics

• addsd = scalar add of 1 double
• mulsd = scalar multiply of 1 double
• addpd = packed add of 2 doubles
• mulpd = packed multiply of 2 doubles

Theoretical SSE2 peak idea

If a CPU can issue per cycle:

• 1 packed add = 2 FLOPs
• 1 packed multiply = 2 FLOPs

Then peak is:

• 4 double-precision FLOPs/cycle

What you need to get close

• enough independent add chains
• enough independent multiply chains
• packed SIMD instructions
• good compiler scheduling or manual tuning
• minimal memory traffic
• enough loop iterations for stable timing

Common benchmark pattern

for (...) {
    a0 = _mm_add_pd(a0, addv);
    a1 = _mm_add_pd(a1, addv);
    m0 = _mm_mul_pd(m0, mulv);
    m1 = _mm_mul_pd(m1, mulv);
}

Warning signs

• only one accumulator
• scalar instructions only
• loop not unrolled
• result optimized away
• timing includes setup work
• expecting one CPU’s schedule to match another CPU’s behavior

Practical takeaway

To approach peak FLOPs/cycle on x86-64 Intel CPUs, think in this order:

1. vectorize
2. add independent chains
3. unroll
4. inspect assembly
5. measure carefully

Why does a 5-cycle multiply not limit me to one multiply every 5 cycles?

Because modern CPUs are pipelined. A multiply can take 5 cycles to finish, but the CPU may still start a new multiply every cycle if the operations are independent.

Why doesn't my scalar C++ loop reach the theoretical peak?

Because scalar instructions process only one double per instruction, and the compiler may not schedule operations ideally. Peak numbers usually assume packed SIMD instructions and carefully structured loops.

Do I need assembly to reach peak performance?

Not always. Intrinsics and a good compiler can get close. But for a strict microbenchmark of theoretical peak, hand-written assembly gives the most control.

Why do independent accumulators help so much?

They break dependency chains. That lets the CPU work on several operations in flight at once instead of waiting for one chain to finish.

Is 4 FLOPs per cycle always possible on Intel x86-64?

No. It depends on the exact microarchitecture, the instruction set in use, and whether the code can keep the relevant execution units busy.

Why does loop unrolling help floating-point throughput?

It reduces branch overhead and exposes more instructions to the CPU scheduler, which improves instruction-level parallelism.

Is packed SSE enough on modern CPUs?

For older or SSE2-focused discussions, yes. But many modern CPUs support AVX and AVX2, which process even more doubles per instruction and raise the peak further.

Why can compiler choice change the result so much?

Different compilers make different decisions about vectorization, instruction ordering, register allocation, and loop unrolling.

• Instruction latency — related because peak throughput depends on hiding latency.
• Instruction throughput — the central concept behind FLOPs per cycle.
• SIMD — packed instructions increase work per instruction.
• SSE2 intrinsics — practical way to write packed double-precision code in C/C++.
• AVX and AVX2 — newer vector instruction sets that extend the same idea.
• Loop unrolling — reduces overhead and exposes more independent operations.
• Instruction-level parallelism — explains why independent chains matter.
• Compiler auto-vectorization — relevant when expecting C/C++ code to become SIMD automatically.
• Microbenchmarking — important for measuring theoretical CPU limits correctly.
• CPU execution ports — helps explain why adds and multiplies may run in parallel on some architectures.