Accéder au contenu.
Menu Sympa

starpu-devel - [starpu-devel] Late data transfers

Objet : Developers list for StarPU

Archives de la liste

[starpu-devel] Late data transfers


Chronologique Discussions 
    < Chronologique >      < Discussions >
  • From: XAVIER LACOSTE <xavier.lacoste@eviden.com>
  • To: "starpu-devel@inria.fr" <starpu-devel@inria.fr>
  • Subject: [starpu-devel] Late data transfers
  • Date: Wed, 7 Aug 2024 13:23:57 +0000
  • Accept-language: fr-FR, en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=eviden.com; dmarc=pass action=none header.from=eviden.com; dkim=pass header.d=eviden.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lKtEfHmkh5VxKSRUGTNf2wnCeQmjrpc3H8l4ktZf9Bs=; b=vefRs3CsFr/FHeq0Hg8HigiK1jHZLLyJfJEoUmeeSNyIumu3wykR6/qCRzdoZ/4VdMsl7NbsbqJTNXAsddB36EJnHG6Zyh684wqKCRFDqbyOIJ8BuM4ZNKbMtaylxQn6U0Iroy1VjIP5EVMHO74Il/NcicGxjqvfZjK5xbV5RJcHZH5Q+vBL26qTAM3/af9gNsJLXyug4MDP2Kh1TYt5oRdATmayFoWJ/uksTiMj1v8aZHTpXlkO8739F3GO/2DnYmBF5l2WN2yyle0iaMpr2Y19mBXCJ4H8jhcYBFjlTfC60A8peY3J6dSVlWW43CqBYoqbTqNLyY3C3kV/PPKN2Q==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ZWXBDgENo0iFfD+eg/FDddfiSkBH7bZKH8sQlza+1HM6YRVUYptr0yvTC9R8waCaw8mDdaMrJPrn/xHMijDQaniol4ECK6AK8wdE76HPVWOl+lulbczgMlw+POfGHyLSVtGl6AySbclWs0d62SrHFQtC9K0/MUSzzlrucIdes/q8axe0zoYRv9+gL54ZAmlwGbBtx1FG+jwSAocrC7IhFJolAGc/JbeSLFSniK+wDEQ9U6F6wGzSoPhaCOpjmBPnqMC2Q0K5YgyzEGP5LwmXtbI87c5Ba/7DQI5914eOh6of6ji2u56OoHz0si58IZcwKD8zopQIp33JP3u7w/Ek7Q==
  • Authentication-results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=xavier.lacoste@eviden.com; spf=Pass smtp.mailfrom=xavier.lacoste@eviden.com; spf=Pass smtp.helo=postmaster@smarthost1.eviden.com
  • Ironport-sdr: 66b37575_o2cEVArQafWlTRflHWBhCJQTKY/EBAhd42KGIyBKBU9Q0Qw 2262SeqFN/GXYtgy/u+CnlVEk2wRKbvpofOImXw==
  • Msip_labels: MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_Enabled=True;MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_SiteId=7d1c7785-2d8a-437d-b842-1ed5d8fbe00a;MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_SetDate=2024-08-07T13:23:57.669Z;MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_Name=Eviden For Internal Use - All Employees;MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_ContentBits=0;MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_Method=Standard;
Hello,

I'm curious about why my data transfer do not occur directly after the task producing the data.
One the attached trace of a LU factorization, GETRF (black) and TRSMs (blue) are executed on CPU  except for the first ones in row and column and GEMMs are all batched on the GPU (except the first one wich is on the GPU but not batch).
Here are some remarks I can't explain on the trace:
  • The TRSMs on host wait until the block is sent to GPU to start (while they could start just after it).
  • data transfers from host to device do not start immediatly after GETRF nor after each TRSM.
  • data transfer from device to host, before next GETRF do not occurs immediatly after the tiny GEMM on the GPU (just after the TRSMs on the GPU)
Data are allocated as starpu_data_matrix_t by starpu, on the GPU, initalized on the CPU then prefetched on GPU using an empty task.
I use 2 streams on the CUDA device because there are small idle time between each kernel....

Any advice is welcome,

Regards,

XL

Attachment: Capture d’écran 2024-08-07 152326.jpg
Description: Capture d’écran 2024-08-07 152326.jpg


Archives gérées par MHonArc 2.6.19+.

Haut de le page