Re: [Starpu-devel] StarPU on multiple communicator

[Nathalie Furmento <nathalie.furmento@labri.fr>]

Hello,

I tried a simple application which splits in 2 communicators a group of 4 MPI 
processes,
and a communication is made between the rank 0 and the rank 1 of each new sub 
communicator.

This works fine with the branch 1.1 of StarPU. However it fails with the 
trunk version of
StarPU. Its new communication engine assumes MPI_COMM_WORLD to be the 
communicator for all
communications. I am going to have a look at how it can be fixed. I will keep 
you informed.

Cheers,

Nathalie

On Jan 23, 16:40, Xavier Lacoste wrote:
> Hello,
> 
> I am using PaStiX in Jorek (A Tokamak simulation code).
> 
> Jorek uses sub-communicators to factorize different matrices on different 
> communicators (i.e. P0 and P1 factorize mat1 while P2 and P3 factorize 
> mat2). The two matrices have exactly the same shape, so the same shape 
> block columns (my data unit in starpu), identified with the same tags, but 
> these data are in fact different as they belong to different matrices (mat1 
> and mat2).
> 
> As far as I understand this cannot work in StarPU today ( I saw Nathalie 
> who agrees with me on that ;) ).
> 
> P0 and P2 (resp. P1 and P3) will register their data with exactly the same 
> TAG. Thus we have two different data with the same TAG which could be a 
> problem.
> 
> An other issue is that the rank of the data is set using the 
> sub-communicator rank which may not be what is expected by StarPU.
> 
> All this results in StarPU hanging in a deadlock, certainly waiting for 
> communications...
> 
> I hope I am not too confusing in my explanations.
> 
> I'll try a hack in my code to fix that: use global rank when registering 
> data and retrieve the number of communicators (here my communicators have 
> the same size hopefully) and add an offset to the TAGS.
> 
> I think supporting communicator in StarPU would be a great feature.
> 
> Regards,
> 
> XL.
> 
> 
> ----------------------------------------
> Xavier Lacoste
> xavier.lacoste@inria.fr
> INRIA Bordeaux Sud-Ouest
> 200, avenue de la Vieille Tour
> 33405 Talence Cedex
> Tél : +33 (0)5 24 57 40 69
> 
> 
> 
> 
> 
> 
> 
> 

> _______________________________________________
> Starpu-devel mailing list
> Starpu-devel@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/starpu-devel


-- 
----------------------------------------------------------------------------
Dr Nathalie Furmento                         Ingénieur de recherche CNRS
Centre de Recherche INRIA Bordeaux Sud-Ouest http://www.labri.fr/~furmento/
200 avenue de la vieille Tour                Tel: +33 (0)5 24 57 41 20
33405 Talence cedex, FRANCE                  Fax: +33 (0)5 24 57 40 41
----------------------------------------------------------------------------