Re: [Starpu-devel] StarPU's limitations

[Chris Hennick <christopherhe@trentu.ca>]

On 21 February 2012 05:59, Samuel Thibault <samuel.thibault@ens-lyon.org> wrote:

- We don't need to add new drivers that often.

But will that remain the case as HPC branches into quantum computing, neuromorphic computing and whatever else may be needed at yottascale?

You mean machine failover?  Well, yes, and I believe we'd need to
rethink quite a bit the structure of StarPU if it had to cope with tasks
which failed.

Isn't recovering from machine failure a fairly basic requirement at petascale and above? That was my understanding.

>   • It can't optimize the choice of scheduling policy or performance models

>     based on the trade-off between accuracy and overhead.

Do you mean e.g. single precision vs double precision?  A problem I see
is that this involves the application quite a bit, making the interface
more complex. But that could be interesting to have a look at for e.g.
cg/gmres etc.

No, I mean e.g. pheft vs pgreedy, or STARPU_HISTORY_BASED vs STARPU_NL_REGRESSION_BASED. For example, if we can eliminate 5 GFLOPS of idle time by using a more accurate performance model and more sophisticated scheduling algorithm, but the algorithm changes require an extra 10 GFLOPS of overhead, then the change isn't worthwhile. 

 

> nor for optional best-effort tasks that aren't always
>     worth their power cost.

I've been wondering about possibly cancelling tasks for some time. The
difficult part here is how to trade "worth" and "power cost". How to
quantify "worth"? :)

Most corporate and government users probably have accountants who can figure that out.

- have a scheduling "window": make starpu_submit block when 'n' tasks
have already been submitted, to avoid filling memory with myriads of
tasks when the computation is huge (expected to last months).

Could that be solved in practice by simplifying the representation of recurring tasks, and/or by moving the long-term queue to a database (which would also help with failure recovery)?