[Starpu-devel] MPI datatypes for user-defined interfaces

[Marc Sergent <marc.sergent@inria.fr>]

Hello,

I am working on a user interface for StarPU for compressible matrices, 
which is in fact an extension of the matrix interface of StarPU (i.e the 
code of my user interface which handles non-compressed matrices is 
exactly the same as the code of the matrix interface of StarPU). I want 
to use it in Chameleon to perform distributed computation, so I provided 
pack/unpack functions to the interface.

I tried to validate the performance of my user interface on a specific 
test case which uses non-compressed matrices (dpotrf, N=65536, NB=512, 4 
computing nodes with 8 CPUs + 2GPUs on each node) by comparing it to the 
matrix interface of StarPU currently used by Chameleon, and I confirmed 
that the performance I obtained with my user interface is significantly 
worse than the one I obtained with the matrix interface of StarPU. I 
also confirmed that the performance obtained on a single node test case 
(N=32768, NB=512, 8 CPUs + 2 GPUs) is similar.

After checking the code, I realized that the matrix interface has 
predefined functions to use an optimized MPI dataype for MPI 
communications, while the data decribed with user-defined interfaces are 
always communicated using the MPI_BYTE datatype combined with 
pack/unpack function calls.

It seems that it is not possible to give a user-defined function for 
defining an MPI datatype for a data described by a user-defined 
interface to StarPU-MPI. Is that a feature you would want to implement 
in StarPU-MPI ?

Thanks in advance,
Regards,

Marc Sergent

---
Marc Sergent
Ph.D Student at Inria Bordeaux Sud-Ouest
STORM Team
Phone: (+33|0) 5 24 57 40 71