Question

Fast Entity Framework Bulk Inserts: Batching, AddRange, and Transactions

csharpsqlentity-framework

Question

How can I insert a large number of records efficiently with Entity Framework when an active TransactionScope is involved?

In my scenario, I may need to insert more than 4,000 records. The operation can take longer than the default transaction timeout of 10 minutes, which could cause the transaction to fail and leave the work incomplete.

My current process is:

  1. Create a TransactionScope.
  2. Instantiate the DbContext.
  3. Open the database connection.
  4. Loop through the records, insert one record, and call SaveChanges() for each record.

Is there a more efficient approach? The TransactionScope and DbContext are inside using statements, and the connection is closed in a finally block.

Short Answer

Entity Framework insert performance is often limited by calling SaveChanges() once per row. You will learn why batching work into fewer database saves is faster, when AddRange helps, how transactions affect large imports, and when a true bulk-insert tool is appropriate.

Concept

Entity Framework tracks entity objects in memory and converts their changes into SQL commands when you call SaveChanges() or SaveChangesAsync().

Calling SaveChanges() for every record is usually slow because each call may involve:

  • Detecting changes in tracked entities.
  • Creating and sending database commands.
  • Waiting for a database round trip.
  • Participating in transaction work and writing transaction-log records.

For thousands of rows, thousands of small saves create unnecessary network and database overhead. A better default is to add many entities, then save them together—or save in batches of a practical size.

AddRange is convenient for adding a collection to the context, but it does not automatically guarantee one SQL INSERT statement. The exact SQL batching behavior depends on the Entity Framework version and database provider. Its main benefit is expressing that a collection is being added; the important performance improvement is avoiding a SaveChanges() call per item.

For very large imports, regular EF insertion can still be too slow because EF tracks every object and generates normal insert commands. Database-native bulk operations, such as SQL Server bulk copy, can transfer rows much more efficiently. They may bypass EF's normal tracking, validation, and lifecycle behavior, so they must be used deliberately.

A long-running transaction also holds locks and increases transaction-log pressure. Increasing a timeout can be valid for an operation that must be atomic, but it is not the first performance fix. Reduce the amount of work inside the transaction first, then decide whether the entire import truly needs to succeed or fail as one unit.

Mental Model

Think of SaveChanges() as sending a delivery truck to a warehouse.

If you call it once for every record, you send 4,000 trucks, each carrying one box. Each trip has startup, travel, and unloading costs.

If you collect 500 records and save them together, you send far fewer trucks with fuller loads. A database bulk-copy operation is more like loading boxes directly onto a freight train: it is efficient, but it does not inspect and handle every box in the same way Entity Framework does.

Take Quiz

Syntax and Examples

Add all entities first and save once when the collection is reasonably sized and fits comfortably in memory.

using var context = new AppDbContext();

var orders = new List<Order>
{
    new Order { CustomerName = "Ava", Total = 25.00m },
    new Order { CustomerName = "Noah", Total = 40.00m }
};

context.Orders.AddRange(orders);
context.SaveChanges();

AddRange marks each Order as Added. SaveChanges() then sends the pending inserts to the database.

For a larger input, process records in batches. This limits memory used by the change tracker and keeps each save operation manageable.

const int batchSize = 500;

using var context = new AppDbContext();

foreach (var batch in incomingOrders.Chunk(batchSize))
{
    context.Orders.AddRange(batch);
    context.SaveChanges();
    context.ChangeTracker.Clear();
}

Chunk is available in .NET 6 and later. ChangeTracker.Clear() is available in EF Core and detaches tracked entities after a successful batch. This is useful when the entities will not be used again.

For older .NET versions, build each batch manually with a List<T> and save when it reaches the chosen batch size.

Step by Step Execution

Consider this EF Core batch insert:

const int batchSize = 2;
var products = new[]
{
    new Product { Name = "Keyboard" },
    new Product { Name = "Mouse" },
    new Product { Name = "Monitor" }
};

using var context = new AppDbContext();

foreach (var batch in products.Chunk(batchSize))
{
    context.Products.AddRange(batch);
    context.SaveChanges();
    context.ChangeTracker.Clear();
}

Execution sequence:

  1. Chunk(2) produces two groups: [Keyboard, Mouse] and [Monitor].
  2. The first AddRange marks Keyboard and Mouse as Added.
  3. The first SaveChanges() inserts those two pending entities. EF and the provider may combine commands internally, but this is provider-dependent.
  4. ChangeTracker.Clear() removes those saved entities from EF's memory tracking.
  5. The second iteration adds Monitor.
  6. The second SaveChanges() inserts it.
  7. The context is disposed at the end of the using scope.

If these batches are not wrapped in one outer transaction, a failure in a later batch does not undo earlier successful batches. Whether that is acceptable is a business requirement, not just a performance choice.

Real World Use Cases

  • CSV or spreadsheet import: Insert customer, product, or inventory rows uploaded by an administrator.
  • Data migration: Move records from a legacy system into a new application database.
  • Event ingestion: Store batches of telemetry, audit events, or integration messages.
  • Nightly synchronization: Import new catalog items or pricing data from an external supplier.
  • Initial setup data: Load a large but controlled set of reference data during deployment.

In each case, choose between one all-or-nothing transaction and independently committed batches based on whether partial progress is acceptable.

Take Quiz

Real Codebase Usage

A common application-level pattern is to validate input before opening a transaction, then keep the transaction focused on database work.

var validOrders = incomingOrders
    .Where(order => !string.IsNullOrWhiteSpace(order.CustomerName))
    .ToList();

if (validOrders.Count == 0)
{
    return;
}

using var context = new AppDbContext();
using var transaction = context.Database.BeginTransaction();

try
{
    foreach (var batch in validOrders.Chunk(500))
    {
        context.Orders.AddRange(batch);
        context.SaveChanges();
        context.ChangeTracker.Clear();
    }

    transaction.Commit();
}
catch
{
    transaction.Rollback();
    throw;
}

Common production practices include:

  • Validate before the transaction: Parsing files, calling remote services, and expensive calculations should normally happen before locks are held.
  • Use a context per import unit: A short-lived DbContext avoids an ever-growing change tracker.
  • Log batch progress: Record batch number, row count, duration, and failures for diagnosability.
  • Make retries safe: Use a natural key, unique constraint, or import identifier so retrying does not create duplicate rows.
  • Use bulk tools for high volume: For very large loads, teams often use a provider-specific bulk-copy API or a vetted bulk-insert library. Verify how it handles generated keys, constraints, triggers, and transactions.

SaveChanges() already uses a transaction for its own set of changes when needed. An explicit transaction or TransactionScope is most useful when several SaveChanges() calls, contexts, or other database operations must commit together.

Common Mistakes

Saving each entity inside the loop

foreach (var order in incomingOrders)
{
    context.Orders.Add(order);
    context.SaveChanges(); // Slow for a large import
}

This creates a database save operation for every item. Add multiple entities before saving.

Keeping thousands of entities tracked forever

foreach (var batch in incomingOrders.Chunk(500))
{
    context.Orders.AddRange(batch);
    context.SaveChanges();
    // The context continues tracking every previous entity.
}

For a long import, tracking can consume memory and make change detection slower. In EF Core, call context.ChangeTracker.Clear() after each successful batch when those objects are no longer needed. Another option is a fresh context per batch.

Treating a larger timeout as the performance solution

A longer timeout may stop a timeout exception, but the operation still holds a transaction open for longer. First reduce round trips, batch work, and consider bulk copy. Increase timeout only when the required all-or-nothing operation still legitimately needs more time.

Assuming AddRange means one database command

AddRange adds entities to EF's tracking state. SQL batching is determined by EF version, provider, configuration, generated columns, and other details. Measure against the actual database.

Committing batches when atomicity is required

Independent batches improve recovery and reduce transaction duration, but they allow partial imports. Use one transaction only when the business rule requires all rows to succeed or all rows to be rolled back.

Opening or closing the connection for every row

Let EF manage the connection in normal cases, or open it once around the full database operation if there is a specific reason. Repeated connection work inside a row loop adds overhead.

Disabling change detection without understanding the consequences

Some EF versions allow configuration changes related to automatic change detection. This can help in carefully measured scenarios, but it is easy to forget to restore the setting or to save incomplete changes. Prefer batching and measuring first.

Comparisons

ApproachBest forAdvantagesTrade-offs
SaveChanges() per rowVery small, simple operationsStraightforwardMany round trips; poor performance at scale
AddRange then one SaveChanges()Small to medium collectionsSimple; one unit of workContext tracks every entity; can use substantial memory
Batched AddRange and SaveChanges()Large imports with manageable partial progress or one outer transactionFewer saves; controlled memoryRequires batch sizing and failure strategy
One explicit transaction around batchesImports that must be atomicAll batches commit or roll back togetherLong transaction can hold locks and hit timeouts
Separate transaction per batchRestartable imports where partial progress is acceptableShorter locks; easier recoveryEarlier batches remain committed after a later failure
Provider bulk copy / bulk libraryVery large, throughput-focused loadsOften fastestMay bypass EF tracking and require special handling

TransactionScope is an ambient transaction mechanism. context.Database.BeginTransaction() is an explicit transaction created from a particular EF database connection. Both can be valid; use the simpler option that matches the resources involved. If only one context and one database are used, an explicit context transaction is often easier to reason about.

Take Quiz

Cheat Sheet

// Good default for a modest collection
context.Entities.AddRange(entities);
context.SaveChanges();
// EF Core: batch a large collection
foreach (var batch in entities.Chunk(500))
{
    context.Entities.AddRange(batch);
    context.SaveChanges();
    context.ChangeTracker.Clear();
}
// One atomic transaction across multiple batches
using var transaction = context.Database.BeginTransaction();
try
{
    // Add and save batches here
    transaction.Commit();
}
catch
{
    transaction.Rollback();
    throw;
}

Rules of thumb:

  • Do not call SaveChanges() once per row for a large import.
  • Choose a batch size by measuring; 500 is a starting point, not a universal rule.
  • Keep non-database work outside a transaction.
  • Clear tracking or use a new context for long-running EF Core imports.
  • Use bulk copy when normal EF batching is not fast enough.
  • Decide explicitly whether partial success is allowed before choosing transaction scope.

FAQ

Is AddRange faster than calling Add in a loop?

It is clearer for collections, but the largest improvement usually comes from calling SaveChanges() fewer times. Both approaches mark entities as added; actual SQL batching depends on the EF version and provider.

Should I call SaveChanges() once or use batches?

Save once for a modest collection that comfortably fits in memory. Use batches for large or unbounded input so the context does not keep tracking every entity.

What batch size should I use for Entity Framework inserts?

There is no universal number. Start with a few hundred rows, such as 500, then measure with realistic data, indexes, network conditions, and database load.

Does Entity Framework automatically use a transaction for SaveChanges()?

For relational providers, SaveChanges() normally uses transaction behavior appropriate to saving its changes. Use an explicit transaction when multiple saves must act as one atomic unit.

Should I increase the TransactionScope timeout?

Only if the import must remain one transaction and is still too slow after reducing unnecessary work. Longer transactions can hold locks longer and increase resource use.

Is SQL bulk copy always the best choice?

It is often faster for very large SQL Server imports, but it can bypass normal EF tracking and may need extra work for identity values, validation, triggers, and related data.

Do I need to manually open and close the EF connection?

Usually no. EF opens the connection when needed and closes it afterward. Manually managing it is mainly useful for specific advanced workflows or when coordinating operations.

Take Quiz

Mini Project

Description

Build a batch importer that receives product records, validates them before writing, and inserts valid products in manageable Entity Framework Core batches. This mirrors a common CSV or supplier-feed import workflow.

Goal

Insert valid products efficiently while keeping the EF Core change tracker from growing across the entire import.

Requirements

Validate that each product has a non-empty name and a non-negative price. Insert only valid products. Save products in batches of 500 records. Clear EF Core tracking after each successful batch. Print the number of inserted and rejected products.

Take Quiz

Keep learning

Related questions

View more