Re: [Starpu-devel] bad performance scaling behavior of cholesky decomposition

[Samuel Thibault <samuel.thibault@ens-lyon.org>]

Hello,

Sorry for the long delay, May is actually a busy month for us teachers
:)

Mario Tacconi, le Fri 11 May 2012 16:55:02 +0200, a écrit :
> I've able to run the Cholesky decomposition example (cholesky_tile_tag
> binary file) given in the StarPU without any problem and I've got a
> reasonable performance profile when using 6 CPU cores with a single GPU
> which is comparable with what is reported in various StarPU related
> publications.
> Problems arise when I try to run the Cholesky decomposition over 12 CPU
> cores and 2 Tesla GPU. I was expecting a nearly linear scaling whilst
> the 12core + 2GPU calculation takes about
> the same amount of time of the 6 cores + 1GPU calculation. Hence, no
> performance scaling at all!

That could be odd, or maybe not. Our cholesky factorization is really
not advanced, and suffers from lack of parallelism, so it is not
necessarily surprising that it does not scale ; maybe you could rather
give a try at Magma 1.1 or later revision, which is the state of the
art for cholesky, and supports using StarPU for exploiting hybrid
machines. Only using GPUs might also be a sign that data transfer are
too costly compared to letting a lot of computations happen on the
CPUs. Actually on my tests on our latest machine with 9 cores and 3
gpus, no computation is scheduled on CPUs.

> Clearly, I'm doing something wrong, but I can't figure out what could
> be. Any help/suggestion will be greatly appreciated.

What you have shown looks right. Maybe you could check whether
the performance models look fine, using starpu_perfmodel_display
or starpu_perfmodel_plot. Also, check the bandwidth shown by
starpu_machine_display.

Samuel