Re: [Starpu-devel] Processor grid and data distribution

[Hatem Ltaief <hatem.ltaief@kaust.edu.sa>]

Hi Sam,

Please find attached a cartoon of what we would like to achieve with fine-grained tasks. We would like to use the data distribution, which increase data locality and engenders less data movement (halo exchanges) across the NICs.

I just found out about the StarPU stencil example, which I believe has pretty much what we would like to achieve.

In particular, the function assign_blocks_to_mpi_nodes should do it.

Thanks!

Hatem

Attachment: data-distrib.pdf
Description: data-distrib.pdf

On Oct 18, 2018, at 5:24 PM, Samuel Thibault <samuel.thibault@inria.fr> wrote:

Hello,

Hatem Ltaief, le jeu. 18 oct. 2018 09:43:08 +0000, a ecrit:

Is there a way to specify an arbitrary processor grid and data distribution in an MPI-based application? How?
In fact, are the data structure and the way it is distributed via processor grid completely uncoupled?

I'm not sure what you mean by processor grid.

The way it is usually done is to set for each data the MPI node it
resides on with starpu_mpi_data_register or starpu_mpi_data_set_rank.
By default each task gets executed on the node which owns
the data which is written to by the task.  One can use
starpu_mpi_node_selection_register_policy to set this to another policy,
or use STARPU_EXECUTE_ON_NODE to explicitly specify the node which shall
execute the task. The data location and the task execution location are
thus completely uncoupled.

Samuel