Re: [Starpu-devel] A possible bug in StarPU 1.3.xx

[Olivier Aumage <olivier.aumage@inria.fr>]

Hi Mirko,

Most of the work about clustering has been done as part of the Ph.D thesis of 
Terry Cojean. This work is still pending to be integrated into the master 
branch of StarPU, however. At this time, the clustering support available in 
the public releases of StarPU is only an early version of Terry's work.

Thus, the status of these features is the following:
- pheft parallel tasks: experimental and mostly deprecated
- peager parallel tasks: experimental
- clustering: experimental, work-in-progress

The .tar in attachment contains an example on how to use the clustering 
support currently included in public releases of StarPU. The shell script in 
the .tar shows the configure settings to use in order to build StarPU with 
clustering support.

Best regards,
--
Olivier

Attachment: clustering.tar.gz
Description: GNU Zip compressed data

> Le 28 sept. 2018 à 09:42, Mirko Myllykoski <mirkom@cs.umu.se> a écrit :
> 
> Hi,
> 
> Should I consider parallel task policies to be a deprecated feature of 
> StarPU? If so, what is the recommended way to implement parallel tasks? I 
> read about "clustering" but the documentation of that feature appears to be 
> very limited.
> 
> - Mirko
> 
> On 2018-09-28 09:29, Olivier Aumage wrote:
>> Hi Mirko,
>> The issue is actually the "pheft" policy selected for the sched
>> context being created. The "pheft" policy has always been experimental
>> and has not received much work for quite a long time.
>> The "peager" policy is also experimental but in a reasonably stable
>> state. When replacing the "pheft" policy by "peager"  (or one of the
>> sequential task policies) the program completes for me.
>> Best regards,
>> --
>> Olivier Aumage
>>> Le 27 sept. 2018 à 14:45, Mirko Myllykoski <mirkom@cs.umu.se> a écrit :
>>> Hi,
>>> I believe that I have found a bug from StarPU 1.3.xx. I am
>>> trying to create a scheduling context that contains a set of
>>> CPU and CUDA workers. The code functions perfectly with
>>> StarPU 1.2.6 but leads to a segmentation fault with StarPU
>>> 1.3.xx. I have isolated the problem to the following code
>>> lines:
>>> ========== CODE BEGINS ==========
>>> #include <starpu.h>
>>> int main()
>>> {
>>>   if (starpu_init(NULL) != 0)
>>>       abort();
>>>   // get CPU workers
>>>   int cpu_worker_ids[STARPU_NMAXWORKERS];
>>>   int cpu_worker_count = starpu_worker_get_ids_by_type(
>>>       STARPU_CPU_WORKER, cpu_worker_ids, STARPU_NMAXWORKERS);
>>>   // get CUDA workers
>>>   int gpu_worker_ids[STARPU_NMAXWORKERS];
>>>   int gpu_worker_count = starpu_worker_get_ids_by_type(
>>>       STARPU_CUDA_WORKER, gpu_worker_ids, STARPU_NMAXWORKERS);
>>>   // create a parallel scheduling context and add all CPU
>>>   // workers to it
>>>   unsigned parallel_ctx = starpu_sched_ctx_create(
>>>       cpu_worker_ids, cpu_worker_count, "parallel_cxt",
>>>       STARPU_SCHED_CTX_POLICY_NAME, "pheft", 0);
>>>   // add CUDA workers, if any, to the parallel scheduling
>>>   // context
>>>   if (0 < gpu_worker_count)
>>>       starpu_sched_ctx_add_workers(   // <=== *** LINE 21 ***
>>>           gpu_worker_ids, gpu_worker_count, parallel_ctx);
>>>   starpu_sched_ctx_delete(parallel_ctx);
>>>   starpu_shutdown();
>>> }
>>> ==========  CODE ENDS  ==========
>>> GDB backtrace:
>>> ========== GDB BEGINS  ==========
>>> (gdb) bt
>>> #0  0x00007ffff7846d29 in lws_select_victim (ws=0x555555a14250, 
>>> sched_ctx_id=0, workerid=3) at sched_policies/work_stealing_policy.c:773
>>> #1  0x00007ffff7845ff8 in ws_pop_task (sched_ctx_id=0) at 
>>> sched_policies/work_stealing_policy.c:567
>>> #2  0x00007ffff782a2f0 in _starpu_pop_task (worker=0x7ffff7ba31c8 
>>> <_starpu_config+5640>) at core/sched_policy.c:871
>>> #3  0x00007ffff78629aa in _starpu_get_worker_task (worker=0x7ffff7ba31c8 
>>> <_starpu_config+5640>, workerid=3, memnode=0) at 
>>> drivers/driver_common/driver_common.c:377
>>> #4  0x00007ffff7915750 in _starpu_cpu_driver_run_once 
>>> (cpu_worker=0x7ffff7ba31c8 <_starpu_config+5640>) at 
>>> drivers/cpu/driver_cpu.c:303
>>> #5  0x00007ffff7915dd8 in _starpu_cpu_worker (arg=0x7ffff7ba31c8 
>>> <_starpu_config+5640>) at drivers/cpu/driver_cpu.c:407
>>> #6  0x00007ffff6f756db in start_thread (arg=0x7fffbbfff700) at 
>>> pthread_create.c:463
>>> #7  0x00007ffff74b288f in clone () at 
>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
>>> ==========  GDB ENDS   ==========
>>> Or, if I set STARPU_SCHED=prio, I get:
>>> ========== GDB BEGINS  ==========
>>> (gdb) bt
>>> #0  0x00007ffff783c59e in _starpu_sched_ctx_list_remove_elt 
>>> (list=0x7ffff7ba25f8 <_starpu_config+2616>, rm=0x5555558979c0) at 
>>> core/sched_ctx_list.c:262
>>> #1  0x00007ffff783c6ad in _starpu_sched_ctx_list_move 
>>> (list=0x7ffff7ba25f8 <_starpu_config+2616>, sched_ctx=1, prio_to=1) at 
>>> core/sched_ctx_list.c:293
>>> #2  0x00007ffff78390d3 in set_priority_on_notified_workers 
>>> (workers=0x7fffffffd9b0, nworkers=1, sched_ctx_id=1, priority=1) at 
>>> core/sched_ctx.c:2099
>>> #3  0x00007ffff78356d9 in add_notified_workers (workerids=0x7fffffffd9b0, 
>>> nworkers=1, sched_ctx_id=1) at core/sched_ctx.c:1228
>>> #4  0x00007ffff7835fe7 in starpu_sched_ctx_add_workers 
>>> (workers_to_add=0x7fffffffd9b0, nworkers_to_add=1, sched_ctx_id=1) at 
>>> core/sched_ctx.c:1316
>>> #5  0x0000555555554a54 in main () at main.c:21
>>> ==========  GDB ENDS   ==========
>>> Best Regards,
>>> Mirko Myllykoski
>>> _______________________________________________
>>> Starpu-devel mailing list
>>> Starpu-devel@lists.gforge.inria.fr
>>> https://lists.gforge.inria.fr/mailman/listinfo/starpu-devel