Stack Overflow Blog

Your .NET parallel loop can become serial in a one-CPU container

8.5内容质量

TL;DR · AI 摘要

容器单CPU限制使.NET并行循环退化为串行,处理时间翻倍,需显式配置并发度。

核心要点

  • 单CPU容器使200记录处理时间从42.1秒增至83.6秒。
  • Parallel.ForEachAsync默认使用Environment.ProcessorCount,需设置MaxDegreeOfParallelism。
  • Little's Law计算显示100记录/秒需39并发,但单CPU限制仅允许1个。

结构提纲

按章节快速跳转。

  1. 单CPU容器使.NET并行循环处理时间翻倍,从42.1秒增至83.6秒。

  2. Parallel.ForEachAsync默认使用Environment.ProcessorCount,单CPU限制下仅允许1个并发。

  3. 双CPU处理200记录耗时42.1秒,单CPU版本耗时83.6秒。

  4. ProcessorCount基于容器CPU限制,-1默认值未移除并发限制。

  5. 需显式设置ParallelOptions.MaxDegreeOfParallelism并结合Little's Law计算。

  6. Go 1.25和JVM已支持容器检测,但需代码调整避免I/O串行化。

思维导图

用一张图看清主题之间的关系。

查看大纲文本(无障碍 / 无 JS 友好)
  • 容器CPU限制与.NET并发控制
    • 问题现象
      • 处理时间翻倍(42.1s→83.6s)
    • 核心机制
      • ProcessorCount基于容器限制
      • -1默认值未移除限制
    • 解决方案
      • 显式设置MaxDegreeOfParallelism
      • 应用Little's Law计算并发需求

金句 / Highlights

值得收藏与分享的关键句。

#.NET#容器化#并发控制#性能优化#Stack Overflow
打开原文

Your .NET parallel loop can become serial in a one-CPU container - Stack Overflow

November 4, 2026

Your .NET parallel loop can become serial in a one-CPU container

CPU-based defaults can leave an I/O-heavy loop processing one record at a time. Raising the limit helps, but each downstream service still needs a concurrency budget it can handle.

[

Same loop, twice the time

In my lab, changing a container's CPU limit from two CPUs to one almost doubled the time needed to process 200 records: 42.1 seconds became 83.6 seconds. These are medians from five runs of each configuration. The two-CPU version processed two records at once; the one-CPU version processed one. Both wrote all 200 records.

I think this kind of slowdown is easy to miss: the batch still finishes and every record gets written. Without a baseline for how long it should take, the extra waiting can look like ordinary service latency.

Each record needed three enrichment calls followed by a write to a system of record. I used this familiar pattern:

code
await Parallel.ForEachAsync(records, cancellationToken, async (record, ct) =>
{
    await ProcessAsync(record, ct);
});

Why the default becomes one

Parallel.ForEachAsync without ParallelOptions allows at most Environment.ProcessorCount concurrent operations. Passing new ParallelOptions { CancellationToken = ct } leaves MaxDegreeOfParallelism at its initial value, -1 . For ForEachAsync, that means ProcessorCount again with the default scheduler. Setting -1 explicitly does not remove the limit.

The cancellation-token-only options are what I'd be most likely to overlook. An explicit options object looks reassuring, but here it still leaves concurrency to a CPU-based default.

On Linux and macOS, and on Windows since .NET 6, ProcessorCount returns the smallest applicable value among the machine's logical processor count, the process's CPU affinity count, and its CPU utilization limit rounded up to a whole number. A one-CPU limit therefore gives 1; a half-CPU limit also gives 1. This value is fixed when the runtime starts and stays unchanged for the process's lifetime.

Go 1.25 uses the cgroup CPU limit, rather than the CPU request, when choosing its default GOMAXPROCS; the JVM has enabled container detection by default on supported Linux platforms. That does not mean Go or Java serializes I/O loops. The concern is code that turns a CPU-derived value into an I/O concurrency limit.

How much work can be waiting?

A record can spend most of its lifetime waiting for other services. During an asynchronous wait, it does not need a CPU to itself. How many records you need in flight depends on how fast you want to process them and how long each one waits. Little's law relates these averages: items in flight equal throughput multiplied by time per item.

For illustration, suppose three enrichment calls take 360 ms altogether and the write takes another 30 ms. At 100 records per second, a 0.39-second record lifetime calls for about 39 records in flight. This is an illustrative calculation, not a lab measurement. The one-CPU default permits only one.

A larger loop can overload a service

Setting a larger MaxDegreeOfParallelism lets more records make progress while others wait. But it also allows more calls to every dependency. An enrichment API might tolerate dozens of concurrent requests while the system of record, often an older component in the chain, can process only a few.

A single limit has no way to express that difference. If the write service becomes slow, records stay in its write stage longer. More writes overlap, and the service may start queuing or rejecting requests even though the enrichment API still has capacity.

Putting one SemaphoreSlim around all outbound calls limits their combined concurrency. It still leaves the distribution uncontrolled. A slow dependency holds permits longer and can occupy much of the shared pool. Thirty permits across all calls do not guarantee that a service with capacity for six sees no more than six.

Concurrency and rate also need separate controls. A semaphore caps calls in flight; it does not enforce an API's per-minute quota, including an LLM provider's quota. Little's law helps estimate the concurrency needed for a target average rate, but enforcing a time-based quota requires a rate limiter.

Set a budget for each dependency

I want the enrichment API and the system of record to have independent limits. The outer loop can then allow enough records in flight to keep both services busy. This helper gives each dependency its own semaphore:

code
public sealed class DependencyBudgets(int enrichmentLimit, int recordsLimit)
{
    private readonly SemaphoreSlim _enrichment = new(enrichmentLimit, enrichmentLimit);
    private readonly SemaphoreSlim _records = new(recordsLimit, recordsLimit);
    public Task<T> CallEnrichmentAsync<T>(Func<CancellationToken, Task<T>> call, CancellationToken ct)
        => GateAsync(_enrichment, call, ct);
    public Task<T> CallRecordsAsync<T>(Func<CancellationToken, Task<T>> call, CancellationToken ct)
        => GateAsync(_records, call, ct);
    private static async Task<T> GateAsync<T>(SemaphoreSlim gate, Func<CancellationToken, Task<T>> call, CancellationToken ct)
    {
        await gate.WaitAsync(ct);
        try { return await call(ct); }
        finally { gate.Release(); }
    }
}

This is a trimmed version of PerDependencyStrategy in the lab's src/Lab.Runner/ConcurrencyStrategy.cs (strategy C4), which also runs the outer loop.

Each wrapper acquires the relevant permit, invokes the supplied call, and releases the permit in finally. Keep one set of budgets shared by the work it is meant to constrain. Creating a new set for each record would give each record its own allowance.

Choose the limits from the services' capacities. Ask their owners how many simultaneous requests they can handle. If that is unknown, increase concurrency in steps in a test environment and watch latency and rejections. A service whose latency starts climbing at a particular load needs a budget that respects that capacity.

Work out how many calls need to be in flight to reach your target rate, and check whether the service can handle that many. Use each dependency's own service time in that calculation. If the desired concurrency exceeds its capacity, reduce the throughput target; capacity divided by service time gives a rough upper bound for that dependency.

Make these limits explicit configuration and validate them at startup. Monitor calls in flight for each service so you can compare the configured allowance with actual behavior.

I don't want a container resize to quietly change how many calls I send to another service. If that limit matters to the service, I want it written down in the configuration.

Slower calls mean a growing backlog

When a service slows down, a fixed concurrency budget reduces the rate at which records get through it. The budget prevents the client from responding by adding more simultaneous calls to that service. Work waits upstream instead. That is backpressure, and the growing backlog needs monitoring too.

Use separate time limits for calling the service and waiting for permission to call it. Start the service-call timeout after acquiring the permit, so local queue time does not consume it. Also give permit acquisition its own deadline. The lab starts call timeouts after acquisition, but does not implement a deadline for the permit wait. The helper above accepts a cancellation token; it does not create either timeout.

Comparing four strategies

I built a small lab ( code and preregistered hypotheses ). Its two simulated dependencies run in a separate container without a CPU limit. The enrichment API tolerates many concurrent calls. The system of record processes six requests at once, allows six more to wait, and returns HTTP 503 beyond that. A Docker runner limited to one CPU processes 200 records, each with three enrichment calls and one write.

  • C1 uses the runtime default.
  • C2 allows 32 records in flight, with no dependency limiter.
  • C3 allows 32 records in flight and shares 30 permits across outbound calls.
  • C4 allows 32 records in flight, with 24 enrichment permits and six write permits.

C3 and C4 have the same total allowance of 30 permits. I committed the hypotheses and thresholds to the repository before the first run, then ran each combination five times. One degraded scenario raises enrichment's median latency to 1.2 seconds, with about 5% of calls hanging for eight seconds, beyond the client's two-second timeout. The other raises write time from 30 ms to 150 ms.

C1 followed the CPU setting: enrichment never had more than one call in flight at one CPU or more than two at two CPUs. The opening timings come from this comparison.

With writes slowed from 30 ms to 150 ms, C2, C3 and C4 delivered similar useful throughput, but the rejection counts were very different. C2 accepted 34.35 records/s with 64 HTTP 503 rejections; C3 accepted 34.01 records/s with 57; C4 accepted 35.08 records/s with none. These are medians of five runs. Accepted records/s counts only records whose writes the system of record accepted.

Across these runs, both C2 and C3 offered up to 14 concurrent writes; their median peaks were 13. C4 never offered more than six, as configured, and the system of record accepted all 200 writes in each run. C4's median completion time was about 5.7 seconds, compared with 4.2 seconds for C3, while its useful throughput was the same or slightly higher. If I had compared only completion times, I'd have picked C2 or C3 over C4. The rejected writes change my answer.

With both services healthy, C4 was the slowest of C2, C3 and C4: 58.38 accepted records/s, versus 65.32 for C3 and 67.86 for C2. The configuration reserves six permits for writes, leaving only 24 for enrichment, where healthy records spend most of their time. C2 and C3 each received a few 503 responses in two of five healthy runs; C4 received none.

I'd accept that roughly 11–14% reduction in healthy throughput when reliable writes matter more than finishing the batch sooner. If rejected writes are cheap to recover and latency is the priority, I'd want the measurements to justify that cost.

When only enrichment slowed down, C4 achieved 0.82 times C3's accepted throughput. C3 could allocate all 30 permits to enrichment; C4 could use only 24. About a fifth of records were incomplete in each of C2, C3 and C4, consistent with enrichment timeouts. Those records were written and marked incomplete, never complete.

What these results support

These are simulated services. Their timings and capacities describe the model, not a real system. My rule of thumb is to protect the dependency I can overload first, then see how much throughput I can get within that limit. I'd raise its budget only after measurements show the service can take it.

What to check in your own loop

  • Inspect the default concurrency before relying on it for I/O-heavy work.
  • Find each dependency's capacity and typical service time.
  • Give each dependency its own budget, with enough records in flight to use it.
  • Configure limits explicitly and validate them at startup.
  • Monitor calls in flight per dependency alongside CPU and memory.
  • Start call timeouts after permit acquisition and set a separate permit-wait deadline.
  • Repeat your measurements under the CPU limit your containers will actually have.

AI use disclosure: I used Anthropic's Claude for research, the outline, an initial draft of this article, and most of the lab code and the script that draws the two figures, written from my specification. I specified the experiments, directed and reviewed each stage of the lab, and ran it on my machine with Claude Code. I rewrote and edited the prose, including the evaluative passages, and checked every number about the lab runs, in the text and in both figures, against the experiment output with Claude Code.

]

These articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International license.

creativecommons.org/licenses/by-sa/4.0/deed.en