Re: [Starpu-devel] StarPU+SimGrid: FetchingInput computation

[Luka Stanisic <luka.stanisic@inria.fr>]

Hi,

Could you please try just to comment out the src/datawizard/malloc.c:148:

    MSG_process_sleep((float) dim * 0.000650 / 1048576.);

and then rerun your experiments.

After discussing with Samuel, we have the impression that SimGrid is wrongly simulating the cost of allocation for CUDA, although you do not have any GPUs on your machine. We will correct this in StarPU code.

Best,
Luka

On 14/12/2016 11:38, Luka Stanisic wrote:

Hi Mirko,

Strangely, the difference between Native and SimGrid does seem to come from FetchingInput on workers and/or Allocating on memory node (see the attached table with some stats). Other discrepancies in state durations are less relevant or a product of the previous two. In fact, I am not even sure if maybe longer Allocating is actually responsible for longer corresponding FetchingInputs. I will inspect the code in more details and come to you ASAP.

As duration of your tasks (process_window, left_update, right_update) is predicted quite accurately, I believe that once we solve this Allocating/FetchingInput issue, you will be able to get faithful simulation predictions for more cores and more complicated inputs. In the future, if your tasks are too complex and simple linear regression models arent sufficient, you might want to switch to more advanced multiple linear regression models, currently available only in StarPU trunk (see examples/mlr/mlr.c and trunk documentation for more details), but that will be the part of the next StarPU release.

Best,
Luka

On 14/12/2016 10:22, Mirko Myllykoski wrote:

Hi Luka,

Just to clarify my previous email. I do not doubt SimGrid's ability to accurately predict execution times in general. However, the numerical code I am developing has some features which I believe will make the simulation harder (large number of small tasks, complicated data dependencies, varying sensitivity to parameter value changes, etc).

Best Regards,
Mirko

On 2016-12-14 09:52, Mirko Myllykoski wrote:

Hi Luka,

Here are the two paje traces you requested:

https://dl.dropboxusercontent.com/u/1521774/paje.trace.tar.gz
https://dl.dropboxusercontent.com/u/1521774/paje-simgrid.trace.tar.gz

I must say that the way I am using the starpu_perfmodel::size_base
field is a bit unorthodox. That's why I ran the simulation twice, once
with the size_base field and once without it. My ultimate intention is
to autotune my code using an external black box software. However, the
execution time may vary from a few seconds to hours depending on the
input data and various parameters. I hope that SimGrid would help with
this problem by saving countless CPU hours.

Right now, I am trying to figure out whether this idea is feasible. I
realized that a linear regression model can predict the codelet
execution times quite accurately provided that the input data does not
change too much and parameters are kept constants. These regression
models cannot be used to autotune anything but if SimGrid fails the
predict the total execution time in this simple case, then I can be
quite sure that this overall idea does not work and I should try
something else instead.

Best Regards,
Mirko

On 2016-12-13 16:40, Luka Stanisic wrote:

Hi Mirko,

Indeed, I was wondering if your platform has any GPUs, but as you said
it is a simple 4 cores machine. Adding more CPUs or GPUs in future
shouldnt be a problem.

You are right, SimGrid shouldnt include any significant fetching time
to the simulation since everything is running with shared memory.
However, appearance of FetchingInput state in the traces is possible,
since StarPU is passing through many parts of the code. Still, the
duration of FetchingInput should be negligible.

Could you please share two paje.trace traces (one for real execution
and one for SimGrid), so I can try to understand better what is
happening? If the traces are big (>100MB), it might be better to run
your application with smaller problem size (if possible).


Also from what I have seen, you are using STARPU_REGRESSION_BASED or
STARPU_NL_REGRESSION_BASED performance models for your codelets,
right? Is this something that you need for your application?
Personally, I have never tried to simulate applications using these
models, although I dont see any reason why it shouldnt work. The
starpu_perfmodel::size_base field is actually used by these models,
more information is available here:
http://starpu.gforge.inria.fr/doc/html/OnlinePerformanceTools.html


So my first guess is that you are somehow using codelet perfmodels and
their size_base incorrectly (or there is an unknown bug in StarPU or
StarPU+SimGrid code), which makes simulation longer than expected.
Then, in the traces this is manifested as long FetchingInputs,
although fetching inputs have nothing to do with the actual problem.


Best regards,
Luka
On 13/12/2016 14:34, Mirko Myllykoski wrote:

Hi Luka,

and thank you for your reply.

I performed the same experiment twice, once with the size_base field included and once without it. I erased the samples directory before each experiment and gave it a few rounds to calibrate properly (STARPU_CALIBRATE=1). Here are the corresponding sample folders:

https://dl.dropboxusercontent.com/u/1521774/sampling_with_size_base.tar.gz https://dl.dropboxusercontent.com/u/1521774/sampling_without_size_base.tar.gz In this case, the error seems to be about 35%.

As I mentioned in my previous email, the code is shared memory only (at the moment). I performed the experiment on my local machine (quad-core i5) but my plan is to move on to a bigger machine (28 or 42 cores per node) and distributed memory once everything works.

I don't quite understand why SimGrid would include any fetching time to the simulation since everything is running in shared memory.

Best Regards,
Mirko

On 2016-12-12 18:21, Luka Stanisic wrote:

Hello Mirko,

Indeed, 50% prediction error is quite big and it suggests that
something is probably not correctly configured. Could you please send
us a compressed version of you ".starpu/sampling" folder, the one from
which simulation will read the performance models. This can help us
get the first ideas of the machine and application you are trying to
simulate.

To answer your question, the fetching time is computed based on the
size of the data being transfered, latency and bandwidth of the link
(in machine.platform.xml file) and the possible contention due to
other transfers occurring in parallel.

Best regards,
Luka

On 07/12/2016 12:50, Mirko Myllykoski wrote:

Hi,

my name is Mirko Myllykoski and I work as a PostDoc researcher for the NLAFET project at Umeå University.

I am currently implementing a (shared memory) numerical software using StarPU and I am trying to simulate my code using SimGrid. However, I noticed that the simulated execution time is way off (about 50%). I checked the generated FxT traces using vite and it seems that SimGrid introduces too much fetching time (State: FetchingInput) to the simulation.

How is this fetching time being computed? My performance models include the starpu_perfmodel::size_base data field and I guess that information is somehow used to compute the fetch time.

Best Regards,
Mirko Myllykoski
_______________________________________________
Starpu-devel mailing list
Starpu-devel@lists.gforge.inria.fr
http://lists.gforge.inria.fr/mailman/listinfo/starpu-devel

_______________________________________________
Starpu-devel mailing list
Starpu-devel@lists.gforge.inria.fr
http://lists.gforge.inria.fr/mailman/listinfo/starpu-devel