Should I limit MaxDegreeOfParallelism because of memory concerns?

Question

I am using Parallel.For to start a large number of jobs (say 1000). This works well, however each job is also quite memory intensive, and from what I can tell Prallel For starts a much higher number of parallel jobs that I would expect.

Running on old home my dev box with 4 cores, I see 400+ ongoing jobs:

This might be fine, however each of these jobs is running a relatively memory intensive algorithm. Therefore the memory usage of the program is high, and I suspect the performance is now impeded due to memory swapping

Currently I am not using any ParallelOptions, just running with the defaults. But I am wondering if I should be adjusting MaxDegreeOfParallelism to keep memory usage from exploding. Or am I overthinking this, and Parallel already takes something way smarter in account?

Answer 1

But I am wondering if I should be adjusting MaxDegreeOfParallelism to keep memory usage from exploding. Or am I overthinking this, and Parallel already takes something way smarter in account?

If you don't provide the ParallelOptions then default ones will be used which have MaxDegreeOfParallelism set to -1, i.e.:

If it is -1, there is no limit on the number of concurrently running operations (with the exception of the ForEachAsync method, where -1 means ProcessorCount).

So parallel the limitations will be provided by task scheduler used, and if none provided the default one (TaskScheduler.Default) will just post everything to the thread pool which can allocate up to ThreadPool.GetAvailableThreads(out int workerThreads, out int completionPortThreads) threads and AFAIK it considers mainly available CPUs, threads and CPU load (see this answer) and memory would not be taken into account at least directly (though in case of extreme memory usage GC can consume a lot of CPU affecting the monitored resources).

So in short - you will need to test your actual workloads and adjust accordingly.

Answer 2

Yes, you should definitely limit the MaxDegreeOfParallelism to a reasonable value like Environment.ProcessorCount, not only because of memory usage considerations but also because most likely you want a consistent behavior across subsequent Parallel executions. The default MaxDegreeOfParallelism is -1, which means unlimited parallelism, and in practice saturates the ThreadPool. A saturated ThreadPool creates one new thread every second, which means that the effective degree of parallelism of the Parallel operation increases over time. After the completion of the Parallel loop the ThreadPool is no longer saturated, and starts terminating superfluous threads at about the same rate (1/sec). So the effective degree of parallelism of the next Parallel operation will depend on the duration of the previous Parallel operation, and on how much time has passed after the completion of the previous Parallel operation. Basing the behavior of your program on such random non-deterministic factors is unlikely to be your intention.

I have posted here an experimental demonstration that an unconfigured Parallel execution uses all the available ThreadPool threads, and keeps asking for more. You could also take a look at this answer, for a more detailed description of the inner workings of the Parallel class in respect to the MaxDegreeOfParallelism option.