Re: [Starpu-devel] Mixing starpu_task_insert() and starpu_mpi_task_insert()

[Samuel Thibault <samuel.thibault@inria.fr>]

Hello,

Mirko Myllykoski, le mar. 21 août 2018 15:31:39 +0200, a ecrit:
> Are there any dangers in mixing starpu_task_insert() and
> starpu_mpi_task_insert() functions in a same code? I assumed that
> this is ok as long as I make sure that a node that inserts a task
> using the starpu_task_insert() function owns the related data
> handles (or it at least has a valid local copy of the data).

That should be fine, yes, except for cache tricks.

> In summary:
>  Initial: Matrix A is distributed among all nodes
>  Step 1. All nodes: "MPI insert" a set of copy task:
>       B (rank X, RW) <- part of A (any rank, R)
>  Step 2. Node X: Insert a set of compute tasks:
>       part of B (RW) <- C_1 (R), ...
>  Step 3. All nodes: "MPI insert" a set of copy task:
>       part of A (any rank, RW) <- B (rank X, R)

That looks correct, except something like: nodes other than X do not
know that X has modified B, and thus they believe that they still have a
valid copy in their cache, and won't expect to be receiving B again from
X.

>  Outcome: Node X has "unexpected messages" that are part of A.

Which is not the scenario mentioned above, but I guess it's something
similar to that.

Perhaps you could, on all nodes, flush the cache for B between
starpu_mpi_task_insert and starpu_task_insert calls. That should
properly separate out notions of what data is where.

Samuel