Re: [Starpu-devel] Starpu stats about disk, OOC, and swapping

[Samuel Thibault <samuel.thibault@inria.fr>]

Amani Alonazi, le lun. 14 janv. 2019 17:26:35 +0300, a ecrit:
> Where does it appear as memory node? I see node 0 and node 1 but I can't see
> disk in the stats output.

It should. How do you enable OOC exactly? For instance,

STARPU_BUS_STATS=1 STARPU_DISK_SWAP=/tmp STARPU_DISK_SWAP_SIZE=200  
./examples/basic_examples/vector_scal

gives

Data transfer statistics:
*************************
        0 -> 1  0.00 MB 0.00MB/s        (transfers : 0 - avg -nan MB)
        1 -> 0  0.00 MB 0.00MB/s        (transfers : 0 - avg -nan MB)

I don't have a GPU, so here node 1 is the disk.

In StarPU 1.3 the output will contain the node name:

        Disk 0 -> NUMA 0        0.0000 GB       0.0000 MB/s     (transfers : 
0 - avg -nan MB)
        NUMA 0 -> Disk 0        0.0000 GB       0.0000 MB/s     (transfers : 
0 - avg -nan MB)

> Speaking of traces, does it take the time spent on GPUs?

It should, yes.

> I don't see it in my fxt output traces. 

Could you post them so we can see what it looks like?

> The execute_on_all_wrapper time sometimes increases dramatically, any idea 
> why?

That could be just the CUBLAS initialization that happens badly
for some CUDA-ish reason which we can't do anything about. You
could check the timing of the starpu_cublas_init() call in
./runtime/starpu/control/runtime_control.c

Samuel