[Starpu-devel] Performance issue of StarPU on Intel self-hosted KNL

[Mawussi Zounon <mawussi.zounon@manchester.ac.uk>]

Dear all,

Recently we developed a new version of the PLASMA library fully based on the OpenMP task-based
runtime system. Our benchmark on both regular Intel Xeon (Haswell and Broadwell in the experiment) and Intel KNL, showed that the new OpenMP PLASMA has a performance comparable to the old version based on QUARK. This motivated us to extend the experiment to StarPU.
To this end, on one hand we used KSTAR to generate a StarPU version of PLASMA. On another hand we developed another version of PLASMA (restricted to a few routines) based StarPU.
It is important to note that the algorithms are the same; we simply replaced the task-based runtime system. Below are our findings:

1. On regular Intel Xeon architectures, PLASMA_OpenMP (OMP)  PLASMA_KSTAR (KSTAR), and PLASMA_HAND_WRITTEN_STARPU (STARPU) have comparable performance, except for very small size matrices where our hand written StarPU version of PLASMA is outperformed by the generic KSTAR.
   
2. On the Intel Self-hosted KNL (68 cores), both our own STARPU version and KSTAR are significantly slow compared to OMP.  But again our KSTAR and our StarPU version exhibited difference performance behaviour.
   

I am wondering whether you can provide us with some hints or guidance to improve the performance of StarPU on the Intel KNL architecture. There might be some configuration options I missed. In addition I will be happy if you can help us to understand why our StarPU version seemed more penalized for small size matrices while KSTAR seems to be doing relatively  better.

Below some performance charts of dgemm and Cholesky (dpotrf) to illustrate our observations:

For the experiments on KNL, the matrices have been allocated in the MCDRAM.

I am looking forward to hearing from you.

Best regards,

--Mawussi