[Starpu-devel] [LU factorisation: gdb debug output]

[Maxim Abalenkov <maxim.abalenkov@gmail.com>]

Dear all,

I’m on a mission to implement a custom task-based LU factorisation by means of StarPU. I’m encountering the following error:

  test: datawizard/data_request.c:150: _starpu_create_data_request: Assertion `handling_node == 0 || _starpu_memory_node_get_nworkers(handling_node)’ failed

Please find the output of the gdb `thread apply all bt` command attached. I will continue debugging. But any hints will be highly appreciated. Thank you and have a good day!

—

Best wishes,

Maxim

(gdb) thread apply all bt

Thread 5 (Thread 0x7fffedcdc700 (LWP 11308)):
#0  0x00007ffff73ce519 in starpu_memory_get_total (node=4127119393) at 
datawizard/memory_manager.c:151
#1  0x00007ffff73d3627 in starpu_memchunk_tidy (node=0) at 
datawizard/memalloc.c:966
#2  0x00007ffff73c343f in __starpu_datawizard_progress (memory_node=0, 
may_alloc=1, push_requests=1) at datawizard/datawizard.c:42
#3  0x00007ffff73c3501 in _starpu_datawizard_progress (memory_node=0, 
may_alloc=1) at datawizard/datawizard.c:69
#4  0x00007ffff7442345 in _starpu_cpu_driver_run_once 
(cpu_worker=0x7ffff76d0df8 <_starpu_config+4440>) at 
drivers/cpu/driver_cpu.c:235
#5  0x00007ffff74429ef in _starpu_cpu_worker (arg=0x7ffff76d0df8 
<_starpu_config+4440>) at drivers/cpu/driver_cpu.c:361
#6  0x00007ffff355508a in start_thread () from /usr/lib/libpthread.so.0
#7  0x00007ffff328c42f in clone () from /usr/lib/libc.so.6

Thread 4 (Thread 0x7fffee4dd700 (LWP 11307)):
#0  0x00007ffff73b6213 in _starpu_exponential_backoff (worker=0x7ffff76d09d0 
<_starpu_config+3376>) at drivers/driver_common/driver_common.c:328
#1  0x00007ffff73b7632 in _starpu_get_worker_task (worker=0x7ffff76d09d0 
<_starpu_config+3376>, workerid=2, memnode=0) at 
drivers/driver_common/driver_common.c:436
#2  0x00007ffff744237a in _starpu_cpu_driver_run_once 
(cpu_worker=0x7ffff76d09d0 <_starpu_config+3376>) at 
drivers/cpu/driver_cpu.c:243
#3  0x00007ffff74429ef in _starpu_cpu_worker (arg=0x7ffff76d09d0 
<_starpu_config+3376>) at drivers/cpu/driver_cpu.c:361
#4  0x00007ffff355508a in start_thread () from /usr/lib/libpthread.so.0
#5  0x00007ffff328c42f in clone () from /usr/lib/libc.so.6

Thread 3 (Thread 0x7fffeecde700 (LWP 11306)):
#0  _starpu_get_worker_struct (id=1) at ./core/workers.h:462
#1  0x00007ffff7349c7d in _starpu_worker_mutex_is_sched_mutex (workerid=1, 
mutex=0x7ffff76d06c0 <_starpu_config+2592>) at ./core/workers.h:533
#2  0x00007ffff7349da5 in starpu_pthread_mutex_unlock_sched 
(mutex=0x7ffff76d06c0 <_starpu_config+2592>) at common/thread.c:618
#3  0x00007ffff73b7554 in _starpu_get_worker_task (worker=0x7ffff76d05a8 
<_starpu_config+2312>, workerid=1, memnode=0) at 
drivers/driver_common/driver_common.c:433
#4  0x00007ffff744237a in _starpu_cpu_driver_run_once 
(cpu_worker=0x7ffff76d05a8 <_starpu_config+2312>) at 
drivers/cpu/driver_cpu.c:243
#5  0x00007ffff74429ef in _starpu_cpu_worker (arg=0x7ffff76d05a8 
<_starpu_config+2312>) at drivers/cpu/driver_cpu.c:361
#6  0x00007ffff355508a in start_thread () from /usr/lib/libpthread.so.0
#7  0x00007ffff328c42f in clone () from /usr/lib/libc.so.6

Thread 2 (Thread 0x7fffef4df700 (LWP 11305)):
#0  0x00007ffff31ca860 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff31cbec9 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff31c30bc in __assert_fail_base () from /usr/lib/libc.so.6
#3  0x00007ffff31c3133 in __assert_fail () from /usr/lib/libc.so.6
#4  0x00007ffff73bfafd in _starpu_create_data_request (handle=0x555555a2f750, 
src_replicate=0x555555a2f848, dst_replicate=0x555555a2f890, handling_node=1, 
mode=STARPU_W, ndeps=0, is_prefetch=0, 
    is_write_invalidation=0) at datawizard/data_request.c:150
#5  0x00007ffff73bbfff in _starpu_create_request_to_fetch_data 
(handle=0x555555a2f750, dst_replicate=0x555555a2f890, mode=STARPU_W, 
is_prefetch=0, async=0, callback_func=0x0, callback_arg=0x0)
    at datawizard/coherency.c:634
#6  0x00007ffff73bc751 in _starpu_fetch_data_on_node (handle=0x555555a2f750, 
node=1, dst_replicate=0x555555a2f890, mode=STARPU_W, detached=0, 
is_prefetch=0, async=0, callback_func=0x0, callback_arg=0x0)
    at datawizard/coherency.c:754
#7  0x00007ffff73bc8eb in fetch_data (handle=0x555555a2f750, node=1, 
replicate=0x555555a2f890, mode=STARPU_W) at datawizard/coherency.c:782
#8  0x00007ffff73bd75e in _starpu_fetch_task_input (j=0x555555a5e580) at 
datawizard/coherency.c:972
#9  0x00007ffff744197c in execute_job_on_cpu (j=0x555555a5e580, 
worker_task=0x555555a5d900, cpu_args=0x7ffff76d0180 <_starpu_config+1248>, 
rank=0, perf_arch=0x7ffff76d01c8 <_starpu_config+1320>)
    at drivers/cpu/driver_cpu.c:71
#10 0x00007ffff744282e in _starpu_cpu_driver_run_once 
(cpu_worker=0x7ffff76d0180 <_starpu_config+1248>) at 
drivers/cpu/driver_cpu.c:300
#11 0x00007ffff74429ef in _starpu_cpu_worker (arg=0x7ffff76d0180 
<_starpu_config+1248>) at drivers/cpu/driver_cpu.c:361
#12 0x00007ffff355508a in start_thread () from /usr/lib/libpthread.so.0
#13 0x00007ffff328c42f in clone () from /usr/lib/libc.so.6

Thread 1 (Thread 0x7ffff7fa6c00 (LWP 11301)):
#0  0x00007ffff355b38d in pthread_cond_wait@@GLIBC_2.3.2 () from 
/usr/lib/libpthread.so.0
#1  0x00007ffff734395f in _starpu_barrier_counter_wait_for_empty_counter 
(barrier_c=0x7ffff76e7b60 <_starpu_config+97984>) at 
common/barrier_counter.c:44
#2  0x00007ffff739bd5f in _starpu_wait_for_all_tasks_of_sched_ctx 
(sched_ctx_id=0) at core/sched_ctx.c:1312
#3  0x00007ffff7356039 in 
_starpu_task_wait_for_all_in_ctx_and_return_nb_waited_tasks (sched_ctx=0) at 
core/task.c:887
#4  0x00007ffff7355fc8 in 
_starpu_task_wait_for_all_and_return_nb_waited_tasks () at core/task.c:874
#5  0x00007ffff7355ffa in starpu_task_wait_for_all () at core/task.c:880
---Type <return> to continue, or q <return> to quit---
#6  0x00007ffff7b5a150 in plasma_zgetrf (m=16, n=16, pA=0x555555a19bd0, 
lda=16, iPiv=0x555555a13eb0) at compute/zgetrf.c:152
#7  0x000055555557d2f1 in test_zgetrf (param=0x7fffffffa5c0, run=true) at 
test/test_zgetrf.c:122
#8  0x0000555555574c43 in run_routine (name=0x7fffffffb052 "zgetrf", 
pval=0x7fffffffa5c0, run=true) at test/test.c:501
#9  0x00005555555746a3 in test_routine (name=0x7fffffffb052 "zgetrf", 
pval=0x7fffffffa5c0, test=true) at test/test.c:397
#10 0x0000555555574225 in main (argc=5, argv=0x7fffffffad18) at 
test/test.c:250

Maxim Abalenkov \\ maxim.abalenkov@gmail.com
+44 7 486 486 505 \\ http://mabalenk.gitlab.io