Re: [Starpu-devel] Question about MPI usage

[Xavier Lacoste <xl64100@gmail.com>]

Hello,

Thanks, this is due to a read only array that I only register on the node 
which will execute the task.
Do I need to register it on every node ? Is there an option to tell StarPU 
that this buffer is already on the node that will execute the kernel ?

Thanks, 

XL.


Le 3 févr. 2014 à 16:36, Nathalie Furmento <nathalie.furmento@labri.fr> a 
écrit :

> Hello,
> 
> That means the node has received some data for which no MPI recv has been 
> posted.
> 
> Cheers,
> 
> Nathalie
> 
> On Jan 31, 17:59, Xavier Lacoste wrote:
>> Hello again,
>> 
>> An other question,
>> 
>> I have factorized all my panels (counted it by hand on a small case) -- so 
>> I must have performed all my updates -- and I get this error during the 
>> starpu_wait_for_all() call :
>> 
>> [starpu][_starpu_mpi_progress_thread_func][assert failure] Number of copy 
>> requests left is not zero
>> 
>> (The other MPI process is just waiting for something)
>> 
>> Any clue what it can come from ?
>> Does that mean that I have received handles that I still need to use ?
>> 
>> Regards,
>> 
>> XL.
>> 
>> Le 31 janv. 2014 à 10:53, Xavier Lacoste <xl64100@gmail.com> a écrit :
>> 
>>> Nathalie,
>>> 
>>> Thanks for your fast answer, I forgot what was this parameter for when I 
>>> copy/paste the registering code.
>>> 
>>> Sorry, should have read the manual again ;)
>>> 
>>> XL.
>>> 
>>> Le 31 janv. 2014 à 10:50, Nathalie Furmento <nathalie.furmento@labri.fr> 
>>> a écrit :
>>> 
>>>> Xavier,
>>>> 
>>>> When you want to register a data that will be allocated only when it is 
>>>> used for the first time, you need to set the "home node", the 2nd 
>>>> parameter of starpu_xxx_data_register to -1.
>>>> 
>>>> Cheers,
>>>> 
>>>> Nathalie
>>>> 
>>>> On 31/01/2014 10:46, Xavier Lacoste wrote:
>>>>> Hi,
>>>>> 
>>>>> I'm porting PaStiX multicore StarPU code to MPI StarPU.
>>>>> I register on each node local data and halo data (data that willl be 
>>>>> read used to update local data or that will be updated using local 
>>>>> data).
>>>>> halo data is registered using NULL pointer :
>>>>> 
>>>>>   starpu_matrix_data_register(&(Lhalo_handle[itercblk]), 0,
>>>>>                               (uintptr_t)NULL,
>>>>>                               (uint32_t)HCBLK_STRIDE(itercblk),
>>>>>                               (uint32_t)HCBLK_STRIDE(itercblk),
>>>>>                               HCBLK_COLNBR(itercblk),
>>>>>                               sizeof(PASTIX_FLOAT));
>>>>>   starpu_data_set_rank(Lhalo_handle[itercblk], HCBLK_OWNER(itercblk));
>>>>>   starpu_data_set_tag(Lhalo_handle[itercblk],  HCBLK_GCBLK(itercblk));
>>>>> 
>>>>> On the proc that possess the data, the same data is registered with an 
>>>>> allocated area (same owner/tag) :
>>>>> 
>>>>>   starpu_matrix_data_register(&(L_handle[itercblk]), 0,
>>>>>                               (uintptr_t)SOLV_COEFTAB(itercblk),
>>>>>                               (uint32_t)SOLV_STRIDE(itercblk),
>>>>>                               (uint32_t)SOLV_STRIDE(itercblk),
>>>>>                               CBLK_COLNBR(itercblk),
>>>>>                               sizeof(PASTIX_FLOAT));
>>>>>   starpu_data_set_rank(L_handle[itercblk], SOLV_PROCNUM);
>>>>>   starpu_data_set_tag(L_handle[itercblk], UPDOWN_LOC2GLOB(itercblk));
>>>>> 
>>>>> When some tasks are submitted i get this error :
>>>>> 
>>>>> [starpu][_starpu_mpi_irecv_data_func][assert failure] Invalid pointer 
>>>>> to receive data
>>>>> 
>>>>> simple: starpu_mpi.c:487: _starpu_mpi_irecv_data_func: Assertion 
>>>>> `req->ptr' failed.
>>>>> 
>>>>> When I look into starpu_mpi.c:1422 i found that it is due to 
>>>>> starpu_data_get_local_ptr() returns a NULL pointer:
>>>>> 
>>>>>                                       if (found_req->user_datatype == 0)
>>>>>                                       {
>>>>>                                               found_req->count = 1;
>>>>>                                               found_req->ptr = 
>>>>> starpu_data_get_local_ptr(found_req->data_handle);
>>>>>                                               fprintf(stdout, "%s:%d 
>>>>> found_req->ptr %p\n", __FILE__, __LINE__, found_req->ptr);
>>>>>                                       }
>>>>> 
>>>>> I'm certainly doing an error somewhere, do you have any clue ? Is this 
>>>>> the NULL pointer i'm giving to starpu_matrix_data_register() ?
>>>>> I don't want to allocate this halo data before i need it... It should 
>>>>> only update my local data and be deleted, I was hopping that StarPU was 
>>>>> going to do that...
>>>>> 
>>>>> Any Idea of what I'm doing wrong ?
>>>>> 
>>>>> Regards,
>>>>> 
>>>>> XL.
>>>>> 
>>>>> PS: As we already gave all informations to starpu_mpi_task_build(), 
>>>>> wouldn't it be possible to only give the task to 
>>>>> starpu_mpi_task_post_build() ? That would make the code lighter :)
>>>>> _______________________________________________
>>>>> Starpu-devel mailing list
>>>>> Starpu-devel@lists.gforge.inria.fr
>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/starpu-devel
>>>> 
>>> 
>> 
> 
> -- 
> ----------------------------------------------------------------------------
> Dr Nathalie Furmento                         Ingénieur de recherche CNRS
> Centre de Recherche INRIA Bordeaux Sud-Ouest http://www.labri.fr/~furmento/
> 200 avenue de la vieille Tour                Tel: +33 (0)5 24 57 41 20
> 33405 Talence cedex, FRANCE                  Fax: +33 (0)5 24 57 40 41
> ----------------------------------------------------------------------------
> 
> _______________________________________________
> Starpu-devel mailing list
> Starpu-devel@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/starpu-devel