[Starpu-devel] Question about MPI usage

[Xavier Lacoste <xl64100@gmail.com>]

Hi,

I'm porting PaStiX multicore StarPU code to MPI StarPU.
I register on each node local data and halo data (data that willl be read 
used to update local data or that will be updated using local data).
halo data is registered using NULL pointer :

    starpu_matrix_data_register(&(Lhalo_handle[itercblk]), 0,
                                (uintptr_t)NULL,
                                (uint32_t)HCBLK_STRIDE(itercblk),
                                (uint32_t)HCBLK_STRIDE(itercblk),
                                HCBLK_COLNBR(itercblk),
                                sizeof(PASTIX_FLOAT));
    starpu_data_set_rank(Lhalo_handle[itercblk], HCBLK_OWNER(itercblk));
    starpu_data_set_tag(Lhalo_handle[itercblk],  HCBLK_GCBLK(itercblk));

On the proc that possess the data, the same data is registered with an 
allocated area (same owner/tag) :

    starpu_matrix_data_register(&(L_handle[itercblk]), 0,
                                (uintptr_t)SOLV_COEFTAB(itercblk),
                                (uint32_t)SOLV_STRIDE(itercblk),
                                (uint32_t)SOLV_STRIDE(itercblk),
                                CBLK_COLNBR(itercblk),
                                sizeof(PASTIX_FLOAT));
    starpu_data_set_rank(L_handle[itercblk], SOLV_PROCNUM);
    starpu_data_set_tag(L_handle[itercblk], UPDOWN_LOC2GLOB(itercblk)); 

When some tasks are submitted i get this error :

[starpu][_starpu_mpi_irecv_data_func][assert failure] Invalid pointer to 
receive data

simple: starpu_mpi.c:487: _starpu_mpi_irecv_data_func: Assertion `req->ptr' 
failed.

When I look into starpu_mpi.c:1422 i found that it is due to 
starpu_data_get_local_ptr() returns a NULL pointer:

                                        if (found_req->user_datatype == 0)
                                        {
                                                found_req->count = 1;
                                                found_req->ptr = 
starpu_data_get_local_ptr(found_req->data_handle);
                                                fprintf(stdout, "%s:%d 
found_req->ptr %p\n", __FILE__, __LINE__, found_req->ptr);
                                        }

I'm certainly doing an error somewhere, do you have any clue ? Is this the 
NULL pointer i'm giving to starpu_matrix_data_register() ?
I don't want to allocate this halo data before i need it... It should only 
update my local data and be deleted, I was hopping that StarPU was going to 
do that...

Any Idea of what I'm doing wrong ?

Regards,

XL.

PS: As we already gave all informations to starpu_mpi_task_build(), wouldn't 
it be possible to only give the task to starpu_mpi_task_post_build() ? That 
would make the code lighter :)