Re: [Starpu-devel] Performance profiling of tile low rank cholesky (HiCMA) with StarPU on distributed memory systems

[Hatem Ltaief <hatem.ltaief@kaust.edu.sa>]

Hi Samuel,

> Hello,
>
> Hatem Ltaief, le jeu. 26 avril 2018 14:57:42 +0000, a ecrit:
>> The profiles indicate that most of the time is being spent into internal 
>> StarPU
>> functions:
>> || 42.6% | 79,366.5 | 851.5 | 1.1% | _starpu_get_worker_task
>> || 2.2% | 4,111.3 | 114.7 | 2.8% | 
>> _starpu_handle_pending_node_data_requests
>> || 1.9% | 3,523.9 | 145.1 | 4.1% | _starpu_cpu_worker
>> || 1.9% | 3,507.2 | 130.8 | 3.7% | _starpu_cpu_driver_run_once
>> || 1.9% | 3,469.6 | 128.4 | 3.7% | _starpu_may_pause
>
> Note that these are "idle" functions actually, i.e. StarPU is just
> looking for things to do due to lack of parallelism.  This is not a sign
> of overhead.

The threads are indeed running out of work due to strong scaling mode of 
operation.

>
>> We reduced then the number of worker threads per node to 16 and 8: we 
>> recover
>> our parallel efficiency up to a decent level!
>
> But did it achieve the computation faster?
YES

So this is the question: why does it run much faster with smaller number of 
threads per node?

>
> In other words, you may just not have enough parallelisme to feed all
> these threads.
>
>> we would expect threads to stay idle and not consume resources.
>
> To provide good latency, StarPU does not leave resources idle, they take
> work to be done as quickly as possible.
>
> Samuel


________________________________
This message and its contents including attachments are intended solely for 
the original recipient. If you are not the intended recipient or have 
received this message in error, please notify me immediately and delete this 
message from your computer system. Any unauthorized use or distribution is 
prohibited. Please consider the environment before printing this email.