Re: [Starpu-devel] hwloc CUDA plugin missing

[Alfredo Buttari <alfredo.buttari@enseeiht.fr>]

I do not know if there is an easy way to figure out whether a GPU is closer to another GPU or to another core. As far as I understand this information (essentially the existence of an nvlink interconnect between GPUs) is not provided by hwloc. Without this information, I believe the fill_tree does not have much choice.

If I get it right, the bindid for a CUDA worker is the CPU id of the core that drives the GPU and the binding of workers is done (by default) in a round-robin way in topology.c . As a result, in the output in my previous email the CUDA workers are bound to cores 20-23 which is a suboptimal choice looking at the topology of the machine; a better choice would be, for example, to bind them to cores 0, 1, 12, 13. Is the idea to leave it completely to the user to specify the binding or is there something better that can be done automatically?

In this respect, I have tried to set the STARPU_WORKERS_CPUID to have a better binding but that didn't really work; I did

export STARPU_WORKERS_CPUID="0 1 12 13 2-11 14-23"

but the output was exactly the same as the one in my previous email.

On Thu, Feb 7, 2019 at 4:30 PM Samuel Thibault <samuel.thibault@inria.fr> wrote:

Alfredo Buttari, le jeu. 07 févr. 2019 15:46:51 +0100, a ecrit:
> I'm trying to figure out how proxylists are built in the lws scheduler

Apparently it is just following starpu_tree, which is initialized by
_fill_tree in src/core/workers. That does not seem to be taking into
account CUDA workers, so that'd need fixing.

Samuel

--

-----------------------------------------
Alfredo Buttari, PhD
CNRS-IRIT
2 rue Camichel, 31071 Toulouse, France
http://buttari.perso.enseeiht.fr