[Starpu-devel] Problem with performance model calibration

[Mirko Myllykoski <mirkom@cs.umu.se>]

Hi,

I believe that I have a some type of problem with performance model 
calibration. I am using StarPU 1.3.3 but I have experienced similar 
problems with other StarPU version as well.

My code [1] is basically a mix of
 * non-trivial CPU-only tasks and
 * trivial CPU+GPU GEMM tasks (starneig_{left,right}_gemm_update).

The GEMM and starneig_push_bulges tasks should have quite consistent 
performance but the other tasks are expected to be more noisy since they 
contain iterative components.

I am using linear regression models (STARPU_REGRESSION_BASED) and all 
"expensive" codelets (including the CPU-only codelets) have have "size 
base" functions implemented. The GEMM codelets use the flop count as a 
"size base". I have also experimented with non-linear regression models 
(STARPU_NL_REGRESSION_BASED) and multiple-regression models (can be 
enabled with the STARNEIG_ENABLE_MRM CMake option). It appears that MRMs 
perform even worse than the linear models.

The ***main problem*** is that the performance is very unpredictable. 
This happens even if I first calibrate the models by setting 
STARPU_CALIBRATE=1 and running the code though a range of different 
problem sizes. The calibration data from one of my experiments can be 
found from [2].

My first ***question*** is: Is is normal that many rows in the 
calibration files have mainly zeros in them?

The machine consists of two 14-core Skylake CPUs and two V100 GPUs. If 
you are interested in replicating the numbers, then you can run:

$ ./starneig-test --experiment schur --init hessrand --n N --hooks

where N is the problem size. The models [2] were calibrated with 
N-values ranging from 1000 to 50000. A machine with 64GB memory can 
handle N ~ 30000 - 35000.

Best regards,
Mirko Myllykoski

[1]: https://github.com/NLAFET/StarNEig/tree/v0.1.2
[2]: https://drive.google.com/open?id=1Aj1-tAJ59dFDHcP-wozLBy5INGKQcbfi

--
Mirko Myllykoski, Ph.D.
Senior research engineer, Department of Computing Science
Application expert, HPC2N
Umeå University

By sending an email to Umeå University, the University processes your 
personal data. For more information, please read: https://www.umu.se/en/gdpr