Re: [Starpu-devel] Drivers, schedulers: where should sched_mutex be locked ?

[Cyril Roelandt <cyril.roelandt@inria.fr>]

On 06/27/2012 07:10 PM, Samuel Thibault wrote:
> Cyril Roelandt, le Wed 27 Jun 2012 16:43:39 +0200, a écrit :
> > The attached patch locks sched_mutex at two different places, rather
> > than once in the driver:
> > - in _starpu_pop_task(), before calling _starpu_pop_local_task();
> > - in the driver, before calling _starpu_block_worker().
>
> That won't fly, because what is needed is that the driver decision to
> sleep has to be atomic with the scheduler discover of no available task.
> Failing that, a driver can go to sleep just after the scheduler has
> pushed a task to be executed.

I see.

> Composition is nice on the paper, but locks screw it up most of the time
> :)

:-/

The thing is, writing schedulers seems to be quite hard for people who 
do not know the internals of StarPU. Harder than just implementing the 
push() and pop() functions :)

Even for us, the lack of modularity make things really hard. Andra has 
to implement both a "heft with work stealing" and a "heft with prio" 
schedulers, and I do not think it can be done in a clean way. I mean, we 
could modify heft so that it pushes tasks to internal queues, from which 
workers pop their tasks when needed (just like in the work stealing 
policy), but well, that involves a lot of copy/paste and it will be hard 
to maintain. Plus it won't allow users to disable part of the features 
dynamically.

I do not really know how we could make the whole thing more modular. Any 
ideas ?

Cyril.