[Starpu-devel] Strange behaviour using GPUs

[Xavier Lacoste <xavier.lacoste@inria.fr>]

Hello,

I tried to compare ParSEC and StarPU using full machines (mirage) on PaStiX.
We achieve to have good scaling with ParSEC by statically scheduling task on 
GPUs.
So, I tried to reproduce the same scheduling with StarPU 1.1.0rc1 and 
obtained strange behavious.
My GPUs take all there time FetchingInput and PushingOutputs.

The output trace can be seen here : 
http://img401.imageshack.us/img401/5213/1pb.png
Or, on plafrim cluster : 
9 CPUs + 3 GPUs
/lustre/lacoste/rsync/log_mirage_AUDI_dmda+LLT+flop+cmin20+frat8+distFlop+selfCopy+fxt_9_3_4231_20130709_120332_crit2_0.5GPUmem.trace
10 + 2 :
/lustre/lacoste/rsync/log_mirage_AUDI_dmda+LLT+flop+cmin20+frat8+distFlop+selfCopy+fxt_10_2_4231_20130709_120332_crit2_0.5GPUmem.trace
11 + 1 :
/lustre/lacoste/rsync/log_mirage_AUDI_dmda+LLT+flop+cmin20+frat8+distFlop+selfCopy+fxt_11_1_4231_20130709_120332_crit2_0.5GPUmem.trace
12 + 0 :
/lustre/lacoste/rsync/log_mirage_AUDI_dmda+LLT+flop+cmin20+frat8+distFlop+selfCopy+fxt_12_0_4231_20130709_120332_crit2_0.5GPUmem.trace

The results are good with 12 CPUs + 0, adding 1 or 2 GPUs bring no loss, but 
no gain either, adding three slows down the run time by a factor 10.

Do you have any idea of what can explain this behaviour ?

With a dynamic scheduling, GPUs are usefull with a small number of CPUs but 
not with the whole machine. 
(My cost model may not be accurate enough for the moment... 
http://img515.imageshack.us/img515/2348/njz.png)

Thanks,

XL.