Re: [Starpu-devel] [LU factorisation: gdb debug output]

[Maxim Abalenkov <maxim.abalenkov@gmail.com>]

Dear Samuel,

I hope all is well with you. I need your help please. I have successfully implemented a panel-wise version of the LU factorisation in StarPU using the "multiple views" approach. However, multiple views create a drawback of using twice as much memory compared to the tiles approach (when the matrix is divided into square or rectangular blocks). The memory for the tiles and the memory for the panel views.

Now instead of using a panel view by allocating explicit StarPU handles (and memory), I would like to create temporary panel “containers” by means of StarPU data mode arrays. I know how to prepare the arrays for the task insertion routine (core_starpu_zgetrf_pnl), but I struggle to unpack the data of tiles inside the CPU kernel function (core_starpu_cpu_zgetrf_pnl). Another complication is that I pass two kinds of data to the task insertion function: data mode array with matrix tiles and a vector of pivots:

STARPU_DATA_MODE_ARRAY,    panel_desc,    A.mt,
STARPU_W,                                    hPiv,

How do I unpack these two pieces of data inside the CPU kernel function? Also would you be able to comment on my approach please? Is it a good day to utilise StarPU? Maybe there is a better mechanism for my case? Thank you and have a good evening ahead!

—

Best wishes,

Maxim

Attachment: core_zgetrf_pnl.c
Description: Binary data

Maxim Abalenkov \\ maxim.abalenkov@gmail.com
+44 7 486 486 505 \\ http://mabalenk.gitlab.io

On 21 Dec 2017, at 19:16, Maxim Abalenkov <maxim.abalenkov@gmail.com> wrote:

Hello Samuel,

Thank you very much for your help and advice. I came up with an alternative solution. Instead of modifying the same codelet, I have created two separate codelets. Each one is responsible for its own operation (tile2panel, panel2tile view switch). It works well.

—

Best wishes,

Maxim

Maxim Abalenkov \\ maxim.abalenkov@gmail.com
+44 7 486 486 505 \\ http://mabalenk.gitlab.io

On 20 Dec 2017, at 15:28, Samuel Thibault <samuel.thibault@inria.fr> wrote:

Maxim Abalenkov, on mer. 20 déc. 2017 14:53:23 +0100, wrote:

I believe I know where the problem comes from. Please see the routines in the attached file. They switch the “view” from blocks (tiles) of the matrix to panels (columns of tiles). I have adopted the lines 25–29 and 73–78 from the StarPU example on multiple views. I think it was a mistake.

No, that is fine, it should be working, because STARPU_MAIN_RAM is 0,
not 1.

What is wrong, however, is modifying a codelet after having submitted
tasks using it before they are finished. In the case at stake, it
shouldn't be a problem, though, since you are rewriting the same thing
in it.

But to be sure, you could use, for this thread

Thread 2 (Thread 0x7fffef4df700 (LWP 11305)):
#0  0x00007ffff31ca860 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff31cbec9 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff31c30bc in __assert_fail_base () from /usr/lib/libc.so.6
#3  0x00007ffff31c3133 in __assert_fail () from /usr/lib/libc.so.6
#4  0x00007ffff73bfafd in _starpu_create_data_request (handle=0x555555a2f750, src_replicate=0x555555a2f848, dst_replicate=0x555555a2f890, handling_node=1, mode=STARPU_W, ndeps=0, is_prefetch=0,
   is_write_invalidation=0) at datawizard/data_request.c:150
#5  0x00007ffff73bbfff in _starpu_create_request_to_fetch_data (handle=0x555555a2f750, dst_replicate=0x555555a2f890, mode=STARPU_W, is_prefetch=0, async=0, callback_func=0x0, callback_arg=0x0)
   at datawizard/coherency.c:634
#6  0x00007ffff73bc751 in _starpu_fetch_data_on_node (handle=0x555555a2f750, node=1, dst_replicate=0x555555a2f890, mode=STARPU_W, detached=0, is_prefetch=0, async=0, callback_func=0x0, callback_arg=0x0)
   at datawizard/coherency.c:754
#7  0x00007ffff73bc8eb in fetch_data (handle=0x555555a2f750, node=1, replicate=0x555555a2f890, mode=STARPU_W) at datawizard/coherency.c:782
#8  0x00007ffff73bd75e in _starpu_fetch_task_input (j=0x555555a5e580) at datawizard/coherency.c:972
#9  0x00007ffff744197c in execute_job_on_cpu (j=0x555555a5e580, worker_task=0x555555a5d900, cpu_args=0x7ffff76d0180 <_starpu_config+1248>, rank=0, perf_arch=0x7ffff76d01c8 <_starpu_config+1320>)
   at drivers/cpu/driver_cpu.c:71
#10 0x00007ffff744282e in _starpu_cpu_driver_run_once (cpu_worker=0x7ffff76d0180 <_starpu_config+1248>) at drivers/cpu/driver_cpu.c:300
#11 0x00007ffff74429ef in _starpu_cpu_worker (arg=0x7ffff76d0180 <_starpu_config+1248>) at drivers/cpu/driver_cpu.c:361
#12 0x00007ffff355508a in start_thread () from /usr/lib/libpthread.so.0
#13 0x00007ffff328c42f in clone () from /usr/lib/libc.so.6

gdb> thread 2
gdb> frame 10
gdb> p cl
gdb> p *cl

to check that it is your starpu_codelet_switch codelet, and that it has
the expected content according to your code. If not, you can use watch
to check what is modifying it.

Samuel