Re: [Starpu-devel] Starpu stats about disk, OOC, and swapping

[Hatem Ltaief <hatem.ltaief@kaust.edu.sa>]

Dear Samuel,
Thanks for helping out.
We are struggling in understanding the performance obtained with OOC feature 
from starpu, as opposed to straight calls to fwrite/fread. OOC/starpu is much 
faster than our naive implementation based on these aforementioned IO system 
calls.
Does starpu allocate separate threads to perform the OOC data movement? Does 
the IO operation turn out to be done in parallel? How does it differ from 
standard fwrite and fread system calls?
IO is new for us and we appreciate your help and expertise in this matter.
Thanks,
Hatem

Sent from my iPhone

> On Jan 15, 2019, at 13:13, Samuel Thibault <samuel.thibault@inria.fr> wrote:
> 
> Amani Alonazi, le lun. 14 janv. 2019 22:06:03 +0300, a ecrit:
>> On Mon, Jan 14, 2019 at 9:31 PM Samuel Thibault 
>> <[1]samuel.thibault@inria.fr>
>> wrote:
>> 
>>    Amani Alonazi, le lun. 14 janv. 2019 21:24:33 +0300, a ecrit:
>>> Hopefully last question :-) I'm using STARPU_DISK_SWAP_BACKEND=unistd.
>>    How can
>>> I analyze how much data transferred to disk during OOC computation? Shall
>>    I
>>> assume that anything exceeds the sustained BW that it runs at  the
>>    sustained
>>> BW?
>>> 
>>> I clearly notice the extra time required in executing each tasks with big
>>    /
>>> smaller STARPU_LIMIT_CPU_MEM. I just want to quantify the amount of
>>    transfer in
>>> /out to disk.
>> 
>>    I'm not sure which precise information do you want.  Do you mean you
>>    want to know *during the execution* the amount of data transferred so
>>    far?
>> 
>>    We can easily add a function to get this.
>> 
>> Yes, I would like to know during the execution.
> 
> The function is actually already available, it's
> starpu_bus_get_profiling_info(). You can look at
> _starpu_profiling_bus_helper_display_summary() in
> src/profiling/profiling_helpers.c to have a usage example.
> 
> Samuel