Re: [starpu-devel] Fw: Question about Properly Unregister Data for a Custom Interface

[Samuel Thibault <samuel.thibault@inria.fr>]

Xinbo Li, le mar. 22 avril 2025 19:28:39 +0000, a ecrit:
> Sending this question to the mailing list for the record.

And sending the answer too.

> I do have an allocate_data_on_node method implemented as 
> static starpu_ssize_t rkmatrix_allocate_data_on_node(void *data_interface,
> unsigned node)
> {
>     std::cout << "Allocate Data called.\n";
>     struct starpu_rkmatrix_interface *rkmatrix_interface =
>         (struct starpu_rkmatrix_interface *) data_interface;
> 
>     // prkmatrix rkmat = rkmatrix_interface->rkmat;
> 
>     /* Allocate memory for the rkmatrix struct */
>     prkmatrix rkmat_new = NULL;
>     double *a_real = NULL; double *a_imag = NULL;
>     double *b_real = NULL; double *b_imag = NULL;
>     starpu_ssize_t requested_memory_a = 0, requested_memory_b = 0;
>     starpu_ssize_t rk_struct_size = sizeof(rkmatrix);
> 
>     rkmat_new = (prkmatrix) starpu_malloc_on_node(node, rk_struct_size); // 
> a
> new rkmatrix pointer
>     if (!rkmat_new)
>         goto failure;
> 
>     *rkmat_new = *rkmatrix_interface->rkmat; // copy the meta data (data
> pointers will also be
>     // copied but will be reset shortly)
> 
>     rkmatrix_interface->rkmat = rkmat_new;

So this is allocated in rkmatrix_allocate_data_on_node, and
not in rkmatric_data_register, so it should be released in
rkmatrix_released_data_on_node, and not in rkmatric_data_unregister.

>     /* Allocate memory for the data members in rkmat */
>     requested_memory_a = rkmatrix_interface->rkmat->rows * 
> rkmatrix_interface->
> rkmat->k * sizeof(rkmatrix_interface->rkmat->a[0]);
>     requested_memory_b = rkmatrix_interface->rkmat->cols * 
> rkmatrix_interface->
> rkmat->k * sizeof(rkmatrix_interface->rkmat->b[0]);
> 
>     a_real = (double*) starpu_malloc_on_node(node, requested_memory_a);
>     if (!a_real)
>         goto failure;
>     a_imag = (double*) starpu_malloc_on_node(node, requested_memory_a);
>     if (!a_imag)
>         goto failure;
>     b_real = (double*) starpu_malloc_on_node(node, requested_memory_b);
>     if (!b_real)
>         goto failure;
>     b_imag = (double*) starpu_malloc_on_node(node, requested_memory_b);
>     if (!b_imag)
>         goto failure;
>    
>     /* update the data properly in consequence */
>     rkmatrix_interface->rkmat->a = a_real;
>     rkmatrix_interface->rkmat->a_imag = a_imag;
>     rkmatrix_interface->rkmat->b = b_real;
>     rkmatrix_interface->rkmat->b_imag = b_imag;
> 
>     return 2*(requested_memory_a+requested_memory_b) + rk_struct_size;
> 
> failure:
>     return -ENOMEM;
> }
> 
> Would this method be called on the nodes who do not have the data after the
> registration somehow?

StarPU calls it when it needs to have the data allocated somewhere, and
calls release when it's not useful any more, or on registration, before
calling the unregister_data_handle.

> Also, the reason I implemented an unregistration method and called it
> explicitly is because calling starpu_data_unregister will always seg fault 
> for
> some reason, so I thought it might be easier to have a explicit call to the
> unregistration method.

I don't understand what you mean. It's starpu_data_unregister
which shall call the unregister_data_handle method. Not calling
starpu_data_unregister means you'll not synchronize anything, leak all
starpu internal buffers etc.

Samuel