Re: [Starpu-devel] Suggestions about creating starpu threads

[Fangli Pi <hpcfapix@hlrs.de>]

Hi Andra,

Thanks very much for your suggestions.

1. No problem, can be done today.
2. I don't know how the doc is generated and where should I put the 
instructions, could you give me an example?
3. Do you mean to change configure.ac? Is it the only file to change?
4. We have stable releases on github for the two external tools, but a stable 
package is not generated. Is a stable package would be better? I can try to 
make it if it is necessary.

Best regards,
Fangli

----- Original Message -----
From: "Andra Hugo" <andra.hugo@inria.fr>
To: "Fangli Pi" <hpcfapix@hlrs.de>
Cc: "starpu-devel" <starpu-devel@lists.gforge.inria.fr>
Sent: Monday, November 7, 2016 9:25:53 PM
Subject: Re: Suggestions about creating starpu threads

Hi Fangli,

I am currently working on the power branch. I have a few things that you may 
probably be able to help if you have some time.

1. I've seen that you have a folder demo, could you integrate that one in 
starpu examples?
2. You have a few instructions on how to use the tool, could you integrate 
them in the starpu doc?
3. You are updating the compile flags manually in src/Makefile.am, could you 
do it through the autotools? (we do this with basically all the external 
libraries, you can take hwloc as an example)
4. Will the 2 external tools, the server and the agent, still exist on github 
for some time now? Do you plan a release or a stable package for it?

Thank you,
Andra

----- Mail original -----
> De: "Fangli Pi" <hpcfapix@hlrs.de>
> À: "Andra Hugo" <andra.hugo@inria.fr>
> Envoyé: Mercredi 19 Octobre 2016 11:59:59
> Objet: Re: Suggestions about creating starpu threads
> 
> Hi Andra,
> 
> That is totally fine for me. Thanks also for making it clear with me about
> the release process.
> 
> Please tell me what need to be adjusted when you use it for the use cases.
> 
> Besides, what can be done now from my side to assist you? If there are any,
> please feel free to contact me.
> 
> Best regards,
> Fangli
> 
> ----- Original Message -----
> From: "Andra Hugo" <andra.hugo@inria.fr>
> To: "Fangli Pi" <hpcfapix@hlrs.de>
> Cc: "Samuel Thibault" <samuel.thibault@inria.fr>, "starpu-devel"
> <starpu-devel@lists.gforge.inria.fr>, "dennis hoppe" <dennis.hoppe@hlrs.de>
> Sent: Wednesday, October 19, 2016 11:45:20 AM
> Subject: Re: Suggestions about creating starpu threads
> 
> Hi Fangli,
> 
> So sorry I got caught up with different deadlines.
> 
> I suggest integrating it as a branch of the trunk of starpu sources in order
> to pass it regularly through the starpu testing system, and then from there
> it will start its road towards the release.
> 
> We might need to adjust it on the way to the release in order to match the
> use cases we might have for power measurements. Would that work for you?
> 
> I can work on this during the week of 7-11 of November. Would that work for
> you if I indicate after the mearge how does the final setup looks like?
> Would that match your deadlines?
> 
> So sorry again for the delay and thank you for getting back to me on this
> matter!
> Andra
> 
> ----- Mail original -----
> > De: "Fangli Pi" <hpcfapix@hlrs.de>
> > À: "Andra Hugo" <andra.hugo@inria.fr>
> > Cc: "Samuel Thibault" <samuel.thibault@inria.fr>, "starpu-devel"
> > <starpu-devel@lists.gforge.inria.fr>, "dennis hoppe"
> > <dennis.hoppe@hlrs.de>
> > Envoyé: Mercredi 19 Octobre 2016 11:10:29
> > Objet: Re: Suggestions about creating starpu threads
> > 
> > Hi Andra,
> > 
> > Sorry for bothering you again. But it has been months since your last
> > reply.
> > 
> > Now my EU project is ending, at this moment, it would be great to know if
> > my
> > starpu extension implementation could be possibly accepted and merged into
> > your next release sooner or later.
> > 
> > For your convenience, I give the link to my source repo here:
> > http://gitlab.excess-project.eu/hpcfapix/starpu-ex-1-2-0rc5
> > 
> > Please contact me in case of any related problems and questions.
> > 
> > Best regards,
> > Fangli
> > 
> > 
> > 
> > ----- Original Message -----
> > From: "Andra Hugo" <andra.hugo@inria.fr>
> > To: "Fangli Pi" <hpcfapix@hlrs.de>
> > Cc: "Samuel Thibault" <samuel.thibault@inria.fr>, "starpu-devel"
> > <starpu-devel@lists.gforge.inria.fr>
> > Sent: Friday, June 17, 2016 9:57:45 AM
> > Subject: Re: Suggestions about creating starpu threads
> > 
> > Hi Fangli,
> > 
> > Thank you! Yes will take a look!
> > 
> > Best,
> > Andra
> > 
> > ----- Mail original -----
> > > De: "Fangli Pi" <hpcfapix@hlrs.de>
> > > À: "Andra Hugo" <andra.hugo@inria.fr>
> > > Cc: "Samuel Thibault" <samuel.thibault@inria.fr>, "starpu-devel"
> > > <starpu-devel@lists.gforge.inria.fr>
> > > Envoyé: Jeudi 16 Juin 2016 16:19:01
> > > Objet: Re: Suggestions about creating starpu threads
> > > 
> > > Hi Andra,
> > > 
> > > During these two weeks I have updated my starpu extension part according
> > > to
> > > the reviews from you and Samuel and a preliminary assessment about our
> > > monitoring client-server communication's overhead is done.
> > > 
> > > Please check these updates in the following repo:
> > > http://gitlab.excess-project.eu/hpcfapix/starpu-ex-1-2-0rc5
> > > 
> > > There is one clarification file "EX_ChangLog.md" with more details and a
> > > figure showing the assessment result.
> > > 
> > > Please feel free to contact me in case of any possible comments.
> > > 
> > > Best regards,
> > > Fangli
> > > 
> > > ----- Original Message -----
> > > From: "Andra Hugo" <andra.hugo@inria.fr>
> > > To: "Fangli Pi" <hpcfapix@hlrs.de>
> > > Cc: "Samuel Thibault" <samuel.thibault@inria.fr>, "starpu-devel"
> > > <starpu-devel@lists.gforge.inria.fr>
> > > Sent: Monday, May 30, 2016 10:12:30 AM
> > > Subject: Re: Suggestions about creating starpu threads
> > > 
> > > Hi,
> > > 
> > > That sounds good!
> > > 
> > > Andra
> > > 
> > > ----- Mail original -----
> > > > De: "Fangli Pi" <hpcfapix@hlrs.de>
> > > > À: "Andra Hugo" <andra.hugo@inria.fr>, "Samuel Thibault"
> > > > <samuel.thibault@inria.fr>
> > > > Cc: "starpu-devel" <starpu-devel@lists.gforge.inria.fr>
> > > > Envoyé: Lundi 30 Mai 2016 09:04:22
> > > > Objet: Re: Suggestions about creating starpu threads
> > > > 
> > > > Hi,
> > > > 
> > > > Sorry for the delay of reply.
> > > > 
> > > > Fangli
> > > > 
> > > > ----- Original Message -----
> > > > From: "Andra Hugo" <andra.hugo@inria.fr>
> > > > To: "Fangli Pi" <hpcfapix@hlrs.de>
> > > > Cc: "starpu-devel" <starpu-devel@lists.gforge.inria.fr>, "Samuel
> > > > Thibault"
> > > > <samuel.thibault@inria.fr>
> > > > Sent: Wednesday, May 25, 2016 6:46:11 PM
> > > > Subject: Re: Suggestions about creating starpu threads
> > > > 
> > > > Hi,
> > > > 
> > > > Yes indeed we should first measure if this is actually a problem 
> > > > before
> > > > going
> > > > into unnecessary modifications.
> > > > 
> > > > Sure, I will try to do the assessment of the effect on performance.
> > > > 
> > > > Of course as long as we use the tools just for training it should not
> > > > be
> > > > a
> > > > problem. However, as far as I know RAPL bases their values on a model
> > > > (IPC,
> > > > cache misses, frequency etc.) they don't do actual hardware 
> > > > measurement
> > > > of
> > > > the voltage. So as Sam says, depending on what we executed before hand
> > > > the
> > > > frequency may change and affect the measurements. But I suppose this
> > > > can
> > > > happen anyway. Do you do any approximations for this?
> > > > 
> > > > Right now I don't have any approximations according to the frequency
> > > > change,
> > > > but I can have a look at it and try if it is possible.
> > > > 
> > > > So these measurements should be desactivated if STARPU_CALIBRATE = 0,
> > > > right?
> > > > 
> > > > According to one of your reviews, I have added one environment to
> > > > activate
> > > > and deactivate the power measurements, named as 
> > > > STARPU_POWER_PROFILING,
> > > > which determines if ex_starpu_init() is called or not.
> > > >  
> > > > Andra
> > > > 
> > > > > ----- Mail original -----
> > > > > > De: "Samuel Thibault" <samuel.thibault@inria.fr>
> > > > > > À: "Fangli Pi" <hpcfapix@hlrs.de>
> > > > > > Cc: "Andra Hugo" <andra.hugo@inria.fr>, "starpu-devel"
> > > > > > <starpu-devel@lists.gforge.inria.fr>
> > > > > > Envoyé: Mercredi 25 Mai 2016 16:33:42
> > > > > > Objet: Re: Suggestions about creating starpu threads
> > > > > > 
> > > > > > Fangli Pi, on Wed 25 May 2016 16:30:23 +0200, wrote:
> > > > > > > After all, the performance during training is not so important,
> > > > > > > right?
> > > > > > 
> > > > > > Hurting the performance of the application is not very important
> > > > > > indeed,
> > > > > > as long as the performance of the tasks is preserved.
> > > > > > 
> > > > > > That said, if it really hurts the performance of the application,
> > > > > > that
> > > > > > could potentially change the energy consumption. If it's not too
> > > > > > bad
> > > > > > it's fine, but it'd be useful to assess it.
> > > > > > 
> > > > > > Samuel
> > > > > > 
> > > > >
> > > > --
> > > > High Performance Computing Center (HLRS)
> > > > University of Stuttgart
> > > > Nobelstr. 19
> > > > D-70569 Stuttgart, Germany
> > > > 
> > > > phone: ++49-711-685-60442
> > > >
> > > --
> > > High Performance Computing Center (HLRS)
> > > University of Stuttgart
> > > Nobelstr. 19
> > > D-70569 Stuttgart, Germany
> > > 
> > > phone: ++49-711-685-60442
> > >
> > --
> > High Performance Computing Center (HLRS)
> > University of Stuttgart
> > Nobelstr. 19
> > D-70569 Stuttgart, Germany
> > 
> > phone: ++49-711-685-60442
> >
> --
> High Performance Computing Center (HLRS)
> University of Stuttgart
> Nobelstr. 19
> D-70569 Stuttgart, Germany
> 
> phone: ++49-711-685-60442
>
-- 
High Performance Computing Center (HLRS)   
University of Stuttgart                    
Nobelstr. 19                               
D-70569 Stuttgart, Germany

phone: ++49-711-685-60442