Discussion related to cado-nfs

[Patrick Lauer <patrick@gentoo.org>]

Hello happy people,

I've stumbled across a strange interaction in sqrt that might be 
considered a bug, or maybe it's just a lack of documentation.

The origin of this is that there's two strategies for parallel 
processing: classical threads, and OpenMP.

With OpenMP disabled everything works as expected. But with OpenMP 
enabled the omp-accelerated parts try to use $(nproc) threads, and the 
previous parallelism implementation then runs by default $(nproc) 
threads, each of which is OpenMP-accelerated.

The result is that instead of N threads it starts N^2 threads, and 
memory usage goes weird. For comparison:

sqrt/sqrt -poly RSA-150.poly -prefix RSA-150.dep -dep 0 -t N -side0 
-side1 -gcd

-t 1:
Total usage: time 379s (cpu), 61s (wct) ; memory 7197MiB, peak 7365MiB
-t 2:
Total usage: time 774s (cpu), 77s (wct) ; memory 13518MiB, peak 13902MiB
-t 4:
Total usage: time 1912s (cpu), 124s (wct) ; memory 22558MiB, peak 23327MiB

As you can see the memory usages spikes quite nicely, leading to an OOM 
quite easily. -t 24 needs around 80-90GB RAM for this case, and is very 
unpleasant.

As a workaround it's best to always set tasks.sqrt.threads=1

But this leaves open the question what the expected behaviour is - 
should the 'old' threading implementation be disabled with OpenMP in 
use? Should the OpenMP bits honour a command line config for parallelism?

I'm not sure what to do here, but I hope that at least this helps people 
avoiding accidental OOM.

Have fun,

Patrick