How to Reach Peak Floating-Point Throughput on x86-64 Intel CPUs in C/C++

By the end of this page, you will understand why theoretical FLOPs-per-cycle numbers are difficult to achieve in practice, how latency, throughput, SIMD width, instruction scheduling, and loop unrolling interact, and what kind of benchmark is actually needed to approach peak floating-point throughput on Intel x86-64 CPUs.

Peak floating-point performance is not just about how fast a single instruction finishes. It depends on several lower-level CPU concepts working together.

The key ideas

1. Latency is not the same as throughput

• Latency = how many cycles one instruction takes to finish.
• Throughput = how often the CPU can start a new instruction of that type.

For example, a floating-point multiply may take 5 cycles of latency, but the CPU may still start one multiply every cycle if there are enough independent multiply chains.

2. Independent operations are required

This works:

a *= x;
b *= x;
c *= x;
d *= x;

because each variable is independent.

This does not allow full throughput:

a *= x;
a *= x;
a *= x;
a *= x;

because each operation depends on the previous result.

3. SIMD increases work per instruction

Scalar SSE instructions like addsd operate on one double. Packed SSE instructions like addpd operate on two doubles at once.

So if the CPU can issue one packed add per cycle, that is already 2 double-precision FLOPs per cycle.

4. Different execution ports matter

Many Intel CPUs have separate execution resources for floating-point add and multiply. In the best case, the CPU can issue:

Think of the CPU as a small factory with multiple workstations.

• One station handles adds.
• One station handles multiplies.
• Each station can start a new job every cycle.
• But each job takes a few cycles to finish.

If you only give the factory one item at a time, workers sit idle waiting for the previous item to move forward.

If you give the factory many independent items, all stations stay busy.

Now add SIMD:

• A scalar instruction is like moving one box.
• A packed SSE instruction is like moving two boxes at once.

So the maximum output comes from:

1. feeding both stations continuously
2. making sure jobs are independent
3. packing more data into each instruction
4. reducing management overhead like branching and loop counters

That is why peak FLOPs/cycle is less about one fast instruction and more about keeping the whole pipeline busy.

Scalar vs packed SSE idea

At a high level, these operations differ like this:

Scalar style

double a = 1.0, b = 2.0;
a += b;
a *= b;

This works on one double at a time.

Packed style with intrinsics

#include <immintrin.h>

__m128d a = _mm_set_pd(2.0, 1.0);
__m128d b = _mm_set1_pd(3.0);

a = _mm_add_pd(a, b);
a = _mm_mul_pd(a, b);

This works on two doubles at once using SSE2.

A simple throughput-oriented example

The following example creates several independent SIMD chains so the CPU can overlap work:

#include <immintrin.h>
#include <stdio.h>

int main() {
    __m128d addv = _mm_set1_pd(1.0);
    __m128d mulv = _mm_set1_pd(1.0000001);

    __m128d s0 = _mm_set_pd(1.0, 2.0);
    __m128d s1 = _mm_set_pd(3.0, 4.0);
    __m128d s2 = _mm_set_pd(, );
    __m128d s3 = _mm_set_pd(, );

     ( i = ; i < ; i++) {
        s0 = _mm_add_pd(s0, addv);
        s1 = _mm_mul_pd(s1, mulv);
        s2 = _mm_add_pd(s2, addv);
        s3 = _mm_mul_pd(s3, mulv);
    }

     out[];
    _mm_storeu_pd(out, s0);
    (, out[], out[]);
     ;
}

Consider this smaller example:

#include <immintrin.h>

void demo() {
    __m128d addv = _mm_set1_pd(1.0);
    __m128d mulv = _mm_set1_pd(2.0);

    __m128d a0 = _mm_set1_pd(10.0);
    __m128d a1 = _mm_set1_pd(20.0);
    __m128d m0 = _mm_set1_pd(3.0);
    __m128d m1 = _mm_set1_pd(4.0);

    for (int i = 0; i < 2; i++) {
        a0 = _mm_add_pd(a0, addv);
        a1 = _mm_add_pd(a1, addv);
        m0 = _mm_mul_pd(m0, mulv);
        m1 = _mm_mul_pd(m1, mulv);
    }
}

Before the loop

• a0 = [10.0, 10.0]
• a1 = [20.0, 20.0]
• m0 = [3.0, 3.0]
• m1 = [4.0, 4.0]
• addv = [1.0, 1.0]
• mulv = [2.0, 2.0]

First iteration

a0 = a0 + addv;  // [11.0, 11.0]
a1 = a1 + addv;  // [21.0, 21.0]
m0 = m0 * mulv;  
m1 = m1 * mulv;  

Even though this question is about a microbenchmark, the underlying idea shows up in real software.

Numerical computing

Scientific code often tries to keep arithmetic units busy in kernels such as:

• vector updates
• matrix multiplication
• polynomial evaluation
• simulation loops

These kernels are often manually unrolled or vectorized.

DSP and signal processing

Operations like filtering and transforms often use repeated multiply/add patterns. Throughput matters because the same small loop runs millions of times.

Graphics and physics

Game engines and physics solvers perform large numbers of floating-point operations on vectors. SIMD and instruction scheduling can matter a lot in hot loops.

ML and linear algebra

Although modern workloads often use AVX, AVX2, AVX-512, or GPUs, the same principle applies: peak performance depends on feeding vector units efficiently.

Benchmarking and performance engineering

When teams evaluate CPUs or compilers, they often write small kernels like this to measure:

• raw arithmetic throughput
• compiler vectorization quality
• instruction scheduling behavior
• differences across microarchitectures

In real projects, developers rarely write a benchmark whose only purpose is to maximize FLOPs. But they do use the same patterns inside performance-critical code.

Common patterns

Multiple accumulators

Instead of one running total:

double sum = 0.0;
for (int i = 0; i < n; i++) {
    sum += data[i];
}

developers may use several accumulators to reduce dependency chains:

double s0 = 0.0, s1 = 0.0, s2 = 0.0, s3 = 0.0;
for (int i = 0; i < n; i += 4) {
    s0 += data[i];
    s1 += data[i + 1];
    s2 += data[i + 2];
    s3 += data[i + 3];
}
double sum = s0 + s1 + s2 + s3;

This helps expose instruction-level parallelism.

Loop unrolling

Compilers or developers unroll loops to:

• reduce branch overhead
• expose more independent instructions
• improve scheduling opportunities

SIMD intrinsics

For critical kernels, codebases often use intrinsics such as:

• _mm_add_pd

1. Confusing latency with throughput

Beginners often think a 5-cycle multiply means only one multiply every 5 cycles.

That is incorrect on a pipelined CPU.

• Latency: result ready after 5 cycles
• Throughput: a new multiply may still start every cycle

2. Using dependent operations only

Broken benchmark pattern:

double x = 1.0;
for (int i = 0; i < n; i++) {
    x *= c;
}

Why it is bad:

• every iteration depends on the previous result
• the CPU cannot overlap many multiplies

Better:

double x0 = 1.0, x1 = 1.1, x2 = 1.2, x3 = 1.3;
for (int i = 0; i < n; i++) {
    x0 *= c;
    x1 *= c;
    x2 *= c;
    x3 *= c;
}

3. Measuring scalar code but expecting SIMD peak

If you use addsd and mulsd, you are only operating on one double per instruction. You cannot expect packed-SSE peak from scalar instructions.

4. Forgetting loop overhead

Very small loops pay for:

Core rules

• Latency = cycles until result is ready
• Throughput = new instructions started per cycle
• Peak FLOPs require high throughput, not just low latency
• Use independent accumulators to hide latency
• Use SIMD packed instructions to process multiple values per instruction
• Use loop unrolling to reduce branch overhead and improve scheduling

SSE2 double-precision basics

• addsd = scalar add of 1 double
• mulsd = scalar multiply of 1 double
• addpd = packed add of 2 doubles
• mulpd = packed multiply of 2 doubles

Theoretical SSE2 peak idea

If a CPU can issue per cycle:

• 1 packed add = 2 FLOPs
• 1 packed multiply = 2 FLOPs

Then peak is:

• 4 double-precision FLOPs/cycle

What you need to get close

• enough independent add chains
• enough independent multiply chains
• packed SIMD instructions
• good compiler scheduling or manual tuning
• minimal memory traffic
• enough loop iterations for stable timing

Common benchmark pattern

for (...) {
    a0 = _mm_add_pd(a0, addv);
    a1 = _mm_add_pd(a1, addv);
    m0 = _mm_mul_pd(m0, mulv);
    m1 = _mm_mul_pd(m1, mulv);
}

Why does a 5-cycle multiply not limit me to one multiply every 5 cycles?

Because modern CPUs are pipelined. A multiply can take 5 cycles to finish, but the CPU may still start a new multiply every cycle if the operations are independent.

Why doesn't my scalar C++ loop reach the theoretical peak?

Because scalar instructions process only one double per instruction, and the compiler may not schedule operations ideally. Peak numbers usually assume packed SIMD instructions and carefully structured loops.

Do I need assembly to reach peak performance?

Not always. Intrinsics and a good compiler can get close. But for a strict microbenchmark of theoretical peak, hand-written assembly gives the most control.

Why do independent accumulators help so much?

They break dependency chains. That lets the CPU work on several operations in flight at once instead of waiting for one chain to finish.

Is 4 FLOPs per cycle always possible on Intel x86-64?

No. It depends on the exact microarchitecture, the instruction set in use, and whether the code can keep the relevant execution units busy.

Why does loop unrolling help floating-point throughput?

It reduces branch overhead and exposes more instructions to the CPU scheduler, which improves instruction-level parallelism.

Is packed SSE enough on modern CPUs?

For older or SSE2-focused discussions, yes. But many modern CPUs support AVX and AVX2, which process even more doubles per instruction and raise the peak further.

Why can compiler choice change the result so much?

Different compilers make different decisions about vectorization, instruction ordering, register allocation, and loop unrolling.

• Instruction latency — related because peak throughput depends on hiding latency.
• Instruction throughput — the central concept behind FLOPs per cycle.
• SIMD — packed instructions increase work per instruction.
• SSE2 intrinsics — practical way to write packed double-precision code in C/C++.
• AVX and AVX2 — newer vector instruction sets that extend the same idea.
• Loop unrolling — reduces overhead and exposes more independent operations.
• Instruction-level parallelism — explains why independent chains matter.
• Compiler auto-vectorization — relevant when expecting C/C++ code to become SIMD automatically.
• Microbenchmarking — important for measuring theoretical CPU limits correctly.
• CPU execution ports — helps explain why adds and multiplies may run in parallel on some architectures.