Objet : Developers list for StarPU
Archives de la liste
- From: XAVIER LACOSTE <xavier.lacoste@eviden.com>
- To: "starpu-devel@inria.fr" <starpu-devel@inria.fr>
- Subject: RE: [starpu-devel] Late data transfers
- Date: Wed, 7 Aug 2024 13:56:48 +0000
- Accept-language: fr-FR, en-US
- Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=eviden.com; dmarc=pass action=none header.from=eviden.com; dkim=pass header.d=eviden.com; arc=none
- Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ct9f+tBOaUt7SUM1kl9rAGdyKbN9adYsbUfQRPFrrBw=; b=RBOuwOfoFeC5W/veSPEgkmkdfmiK5o55ji7lIQjtkzdip493LihD94SthhNY+n8FHw3tZrYTl3ZXwGeduSGA/ZGVV0+0fVUlJFnWq5NOyD3127T5HVKJHp2iTWi6af62H1cgFB4Vb9uTooV6F6JLFm+eAHr03NZMzY8Sz56a0VwrKSYa/9uG2cqEAhZ05Rek4QgZzcHnhF+98DQ5URkBW1r5MxrqPQbse7iOgpmJlwbDwVO1R3G8SQwbnEVKjeC+qF4sxWfnQtxOHNvStRJ8YmCgp0/f33TS8iteIWJ7gdn6uwgTkdvkKnbJR5wQP8WtI81T77a2dibde8WgRvNeFA==
- Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=s9QEY35dLU5rgTvrSrt4BVkyRUM3DV0C2f7jREJhCLpmTlR201QjY82xvlxweg+8IMwQ4HF+dsdmx7iLoQEEfeq6hXbwuP0R1e1TVFUgPI3yu4w68dYT8bCwAxmRkwRNC0I/faqtA429oMrgt5tNbOFS+EJSI7yr9y+xTVrbEkdhsQzlxtXTqAsC21ciu2Zi1vo4FF9lLJRKGlEvDyXwMEVzEYw4LAx7bBpt7i57lhS0frNu9C8kIbQFeRBB0tUkzLn2WaLcFp6ax4MZQDyra3MK+KGodmeq8RP4tYVlkEWSXxth35gdvDbIJZgIk4ZINC02wd1pkIYzKmK/zMR2vw==
- Authentication-results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=xavier.lacoste@eviden.com; spf=Pass smtp.mailfrom=xavier.lacoste@eviden.com; spf=Pass smtp.helo=postmaster@smarthost4.eviden.com
- Ironport-data: A9a23:X5oV+KvgJEBPkc1FMSMxP6UHrufnVJJVMUV32f8akzHdYApBs4E2e luraxnfZ67dN3y1JY5G3L7G8hkAu8SHn98wQFE4/yo3RnkV95KfCYjBd0urYHLNdZ2fRR4/v 55BNoXNds0+F3SN/Rz1bOW8piAmjP+FG+akWIYoVswQqSpMEU/N3jo6w7di2NMAbaGFPj6xV boewiG1EF6g0jF5dHoS5Mpvwzs+taqu6TlEsFZkOasX7AOFxiNLV85GfvzrfiOlE4dfRbO0G +iak+7npWrS8h4nW4ioybr2IxcAGuDfZFbS0SNbM0TObnmugwRrukpsHKBHNh0O1F1l5uxM9 eihlaBcaC95b/SRlbUQDEhUHngiM6Efpe/NenPh7pfKkB2bLSLgyNxjXRo8VWE60rYuUDsWr ZT0CxhUM0vd37ruqF6fYrMx7ij2BJCzZOvzglk5kXeDS6xOranrG83i/cVf0Co7mvdAFPPfY 9txQTd0ZXwsWTUWUrssIMx4xLbAakXXKWUC9ArF/fVvuQA/8SQouFTTGIqNEjC1bZ4N9qqoj jqu13j0BBgcKOuexVKtmp57rrKncYvTAer+JZXgnhJYqAX7Kl87VHX6YWCGTcyR0SZSbT7wx 3s8oULCpYBqnKCio0KUsxeQ+BZotTZEMzZc/nFTBKhgBcM47i7AblXoQAKtZ/QXmegIWxYv9 2bQgtTWKxl9tbGTSkiCo+L8QTOaYUD5LEcZYCYeUQsC+MW/5owtjx7CSJBoF6vdYt/dSWm2n GDM9XF41+h7Yc0jj81X+XjXiiytq4KPSgc+4R7LU0qo8AJ1aYjjbIutgbTexa8ffN3JEwHQ5 xDoneDA4swhFq+fzhXOQcFSDLi3vc29LCTT1AsH854JrGj9pST9Iei8+gpWP1tgKN4ZPDPkf kLXkRhA4Ydaenqsd65+JYyrY/nG1oD7EN3/SvDTdcYSJJVvdQOK+mdlYkv4M33RfFYEoIwYB 4mxdOeVVGsdNopC3j+6QeQnzup+rswh/l/7SZf+xhWh9LORYn+JVLsIWGdiiMhlsctoRy2Jr 75i29u29vlJbAHpSgftmbP/wHgRKHEhGZDxsNALMOOdIwZrHicqDPq5LVIdl25Nwf89egTgp CnVtqpkJLzX3iyvxeKiMSALVV8XdcwjxU/XxAR1VbpS51AtYJy08IAUfIYtcL8s+YRLlKEuF ahaJ5ncXaQXG1wrHgjxi7Gg9+SOkzz23Gqz09aNO2JXk2NIG1yXprcIgCO1pHhm4tWLWTsW+ OHwiFmFHfLvtixrEMHNIPy1xlW6sDAGkfkaYqc7CoQ7RakYy6Azc3aZpqZue6kkcEyfrhPEj Fz+KUlD+oHwT3odrYOhaVas9dn5SoOT3yNyQgHm0FpBHXKLpzvzmtIdALrgkPK0fDqcxZhOr N59l5nUWMDrVn4T22alO+c0lfAN9JH0qqVEzw9pOnzOYh75QvljO3SKl40H/KFE2rYT60P8V 1Ot6+tqH+yDGPrkN1oNeysjTOCIjs8PlhfosP8aHUTd5Q1MxoShb3l8BRe3pRJ4EKpUK6Igm Ocoh94X4Vewiz0sKde3sRpX/GWtcF0GdbkWhrNCCaC6mAYH80xwUa7EOADI/bWKTctGCWgxA zqunKGZra9t9knDVHsSFHb2wutWg6oVii1K1FMvI1eomMLPo+0egS1q7jU8SzpKwiV90+5cP nZhM2t3L/6s+whEqddiXWf2PS18Hzycp1LMzmUWmF3jT0WHUnLHKEs/M72v+GEb62dtQShJz oqHyWrKUSfYQ++p53EcAXVakv3EScB90ibgm8r9RsSMIMQcUArf26SrYTIFlgvjDcYPn3b4n OhN/ttrSKjFJCUV8rwaCY6b6Owqcyq6Bld+GNNvwKBYOlvnWmCW+SOPIEWPaM9yN6T00UunO fdPeONLdTqDjRirkB5KK5IIEbFOmNwR2OEjYZLufG4PjKuepGFmsbXW7SnPu1UoSNRPz+c4F NnYSAiGGG3Nh3EOw27ph+tHM1qeftMrSlDd3ue00eNRDLMFkrhmXn8T24uOnUe+EVVY7TeLm z2bfI7Q7ehp6bo0rrvWCq8ZWjmFc4LiZtqH4CWYkopob+qWFezsqglMiF3sHzoODIsrQ94ty Ii86o/m7njk4oQzfXvSwaSaNq9z4s63YupbH+T3IFRengqAQMXc2AQCyU/pNa12lM5h2ef/S zuacMeQcfsna+VZzlBRaAlcFE85IIbzZaHCuyi8jqqtDj4w7A/5F+6kpETZNTxjSiw1OpPFG lDVvdSq7Ytmt4hiPkIPKMxnJJ5aG2Xdf5UaWef/jhSiNVnwsGi+4uPjsTEC9QD0DmK1FZemw JDdGTn7WheAmIDJ69B7t4B3uh4lMypgsLNhYmMc5N9EpDSoB0EWLekmEMsnC7MFtgfQxZ3Hd DX2Q296MhrEXBNAagfZzNDveiy9F94+EI71CRJx9nzFdhrsIp2LBYVQ0xtJ4lB0S2PF9/6mI 9RPwU/AFEG97b8xTNlC+8Hhp/lsw87b4Xc6+UrdtcjWKDRGCJUo0E1RJiZ8ZRblIersynqSf XMUQFpaSn6VUUTySMZsW0BEESEj4Q/A8W8aUjet8v3+5aOg0+xy+N/uMbrS05oCTvgwCpwge HfVf1aJskeqgiE9mK1wt9wQ1Po+TbrBG8WhN6bsSDEDh6z6uCxtI8oGmjFJV804vhJWF1TGj DS3/nwiHwK/JVtM3KGNgxA8k36rvqng0xmS5OI+mdPHrfD958PccgK2zAXgNc22prbis0heB jwVaS59ZnWI4SD8q2AWWusz/zS67QM5TBEokRzEirv/jBagRWkbH7dku63//8wE62VKn+24a 47Iu+jQO+ubsuen0h7lkNofJLNgqt/t3+jFZ6DBpHpWKzd76G83nxtV5Fp3osb+TCyagWg87 /A=
- Ironport-hdrordr: A9a23:kWI8D64Z7+QPMVuTUQPXwd+CI+orL9Y04lQ7vn2ZFiY5TiXIra qTdaogviMc0AxhIU3I6urwQJVoIEmsvKKdjrNhWItKMDOJhILsFvAF0WKA+UypJ8SdzJ846U 4IScEXZLLN5BpB/KHHCWKDc+rIteP3kpxA8t2urEuFIzsKV4hQqyNCTiqLGEx/QwdLQbI/CZ qn/8JC4xawZHgNacy/J38dG8zOvcfCmp7KaQMPQ0dP0njFsRqYrJrBVzSI1BYXVD1ChZ8k7G j+igT8oomzrv2hzRfY9mnLq7BbgsHoxNdvDNGFzuIVNjLvoAC1Y5kJYczIgBkF5MWUrHo6mt jFpBkte+5p7WnKQ22zqRzxnyH9zTcH8RbZuBOlqEqmhfa8aCMxCsJHi44cWADe8VAcsNZ117 8O936FtqBQEQjLkE3Glpf1vlBR5wSJSEgZ4K4uZk9kIMgjgXhq3M4iFXZuYdY99eTBmcUa+a dVfYXhDb1tACinhjjizxNSKZqXLzkO9169MzU/k93Q3D5MkH9jyUwEgMQZg3cb7Zo4D4JJ/u LeL81T5cVzp+ItHNBA7d06ML6KI32IRQiJPHOZIFzhGq1CM3XRq4Tv6LFw4O2xYpQHwJY7hZ yECTpjxBgPUlOrDdfL0IxA8xjLTmn4VTPxyttG75w8vrHnXrLkPSCKVVhrmcq9pPcUBNHdRp +ISeVrKu6mKXGrFZdC3gX4VZUXIX4CUNcNstJ+QF6KqtKjEPyYigUaSoeiGFPAK0dbZoqkOA p+YNHaHrQ+0mm7HnnlnRPWR3ThPkTi4JMYKtmqw9Qu
- Ironport-phdr: A9a23:s5BS7hA4GTlaFmmNJAyAUyQUv0kY04WdBeb1wqQuh78GSKm/5ZOqZ BWZua41yg6RFtuKo9t/yMPo8InYGlY8qa6bt34DdJEeHzQksu4x2zIaPcieFEfgJ+TrZSFpV O5LVVti4m3peRMNQJW2aFLduGC94iAPERvjKwV1Ov71GonPhMiryuy+4ZLebgtUiDanZb5+M Bq6oATVu8ILnYZsN6E9xwfGrXdPZ+lY32RkK06NkR3h/Mmw5plj8ypRu/Il6cFNVLjxcro7Q 7JFEjkoKng568L3uxbNSwuP/WYcXX4NkhVUGQjF7Qr1UYn3vyDnq+dywiiaPcnxTbApRTSv6 rpgRRH0hCsbMTMy7WfagdFygq1GuhKsvxxxzZDJboGJOvRwfa3dctEGSmVORctRTTBNDp+mY ocTE+YNIOBVpJT9qVsUqhu+ABGhC+LzxzBHhXH40rE02PknHwzawgEuBs8Avmrbot7oM6oTX vq6zLfJzTrddfNawzP955LUfRAmuvGMQax/fNHPxkkxFgPKk0ufqYr/MzyJ0eQNtnGW4ux9X u2gl2ApsRt+oiSzxsgykInJgJoYx0zY+Cllzog4Id61RVNlbNO5HpVduS+XOohoTs4hTWxlu SI3x7IbtZC0fCUHyIgrywPcZvGbfYaE/xztWeKRLDp+mXlre6q/ig6x/ES80OHxVMq53ExXo iZYiNXBtXAA2wTN5sWGVvdw8EKs1SuT2w/J6+xJJF04mbbFJ5Mk3LI9mZgevEHGEyPrmUj7i rKde1sg+ui18OTnfqvppoWBOY91iwDxLLwjltC5DO8lKAYBRXKb9v651LD7/U32XrFKjvoun 6nCtZDVP94bprK5Aw9SyYYj9xmyAy2439QGnHkIMUxKeBSdg4f1PFHOJej0Dfa5g1uyjDdm3 /PLMqH7DpjPLXXPiqrtcatn50NfyAc/1dVf6IhVCrEFLvLzQEjxtNnAAx42KQO03efnB8tn1 o8EXWKAGLOWMKTJsV+U+O0iOPeDZJEUuDngNfcl4eTigmM+mV8YZaWp24AYZ2iiHvt6O0WZf WbsgtAZHGsXpgY+VvDliEWeUT5PYHa/R7k85i0hB4K8FYfDW5mhj6Gf0yelAJJWYntLCkuME Hftb4WLQe0AaCOUIs97kzwLT6KtS4E71ULmiAivgaF7J/DM52gUuIzu0PB04ffSnFc873Y8W 9+B2nuVUiR4k30FQxc32rp+qApz0AHQ/7J/hqloPOJyyskBdi4REqXg4qQuG9nsWw/bONOAQ 1G9XtiODC8wQt83hdQJZhAuSJ2Zkhnf0n/yUPcunLuRCclsmkq99334JsInjm3DyLFklV4tB M1GKWyhgKd7sQnVHY/A1UuDxO6xbapJ+inL+S+YyHaW+llCWVtgWLvEUGpZYkLdqs/l62vPU b6iBLJhOQxEmoaZMqUfUtTylh1dQev7ftHXYma/gWC1UAaB2LKBdszucm4axznQIEUakgwe+ zCNMg1tTjy5rTfmBSd1XUnqf1uq8eR6ryajSVQoygiRc0B7/6Gw8wQJhfuHV69V06wFtCon7 T5zGT5RxvrwDNyN70pkdaRYO5Ym5UtfkHjevEp7N4ChKKZrghgfdR52tgXgzUc/DIIIisUso H4wqWg6YauFzFNMcS+Z1pHsK/XWLGf15hWmd6/R3BnXzt+X/q4F7PlwpU/kuUmlEU8r8nMv1 NcwsTPU9pzUDQcIF5b4WUEs6xVSqq/bbiQ7oYjT0DwkMKW5tCPDx8N8HPEsmXPCN59UNKKJE hO3EtVPV5Dod7xswgHvPk9XWYIavLQ5NM6nafacjauiPeI62SmjkXwC+od2lESF6yt7TOfMm ZcD2fCRmAWdBFKexB+stN76nYdcaHQcBG26nGL0DZBQa7c0eY8ADnyyLuW62NRyipirUHldv g3GZRtOyIqydByeYkaolx1Zy0UauzqjkCC/1CB5uzw3r6aS2mrFxOGoJ39lciZbAWJli1nrO 429idsXCVOpYwYenxyg/U/mxqJfqcyTNkHrSFxTN2jzJmBmCO6rs6aaJtRI49UuuDlWV+K1Z RabTKT8ql0UyXGrE2xbzTE9Pzak3/ex1wxzlmOQNjB3pXHeYt197RrB5dDRQ7ha2T9OSCRji DbRD0SxJJH5pZPLx9Ga66buCiqoTdVLfDPuzJ+cuSfehyUiGhC5k/2p25XmHQU8zS7nxoxvX CTMogz7Z9qj3KC7POR7O0hwUQ+krZUqXNgn1Ntt1/RykTAAi56Y/GQKizL2ONRfg+flaWYVA CQMyJjT6RTk30tqKjSIwZj4XzOT2JgEBZHyb2UI1yY6980PBr2T6ekOjSJro1ei6wbYav9mg jo1xOUv5HMcxeoOvUB+q0fVSqBXBkReMSH2wl6Y5s6/ob8RYG+kdqOs2WJzg9GtAbfEqQZZE iWcGN9qDWp76cNxN0jJ2Xv459T/edXeWtkUswWdjxbKi+UGYIJ0jPcBgjBrfH7spXBwgfBul gRghNvp2erPY3Uo5q+yBQRUcyH4d99GsC+4lr5Qx46Xx9z9R8Q6XGhVGsKxEbTxTm5V7qyCV U7GESVg+C7KQvyCQ0nGsRsg9i2RVMvyf3CPeCtJlZM9FF/EfQoP2EdPBH07hsJrRljsm5ynK R4/tmxWvQedyFMEy+RjMwTzXzXEvAnuci0zVJWUMBtR6EdF+lvRNsudqOl0Gmkd95ml5mRhM 0Sjbh9TRSEMU02AXRX4O6W2oMPH+K6eD/a/KP3HZfOPr/ZfXrGG38Dn3oxj9jeKfsKBWxsqR +U8wVZGVGtlFt7xgDAGUTAXkDnRM4iWuhaw9ysxpce6uPjmQwPg44KTBqAaYY0ppEDw2P7FZ 7PYjT0cS34Qzp4WwH7U1LUTlEUfjS1jbXjlELgNszLMULOFmqJTCE1TYCdyOc1UqqMki1kVf 5GLzIqskO4l3ZtXQx9fWFfsm9+kf5kPKmC5bhbcAVqTca6BLnvNyt32ZqW1TftRiv9Vvlu+o 2X+cQerMzKdmj3uTx3qP/tLiXTRLBVAuYynNBJkAm/4UN/OZwO6PNJxyzYxxPdn4xGCfX5ZK jV6f05X+/eI6jhEh/xkB2FbxmFgJPOYmyGC87WeJ4gXvfZtRC9zkqgJhRZyg6sQ5yZCSvtvn SLUpdM7uFCqnN6EzT9/WQZPoDJG18qb+F9vMqLD+txcSG7JqVgTuH6IBU1A9L4HQpX//rpdw d/VmOfvJSdeppjKqNAECZGcIYrCOX4le3IB+RbPCQIfUT6sLn2DwUFDkf2f8DueqZ1o8/AEf bIDVL5XWVpzHfQfWBwN9DMqLYl2Uj4i17WciZxRjUc=
- Ironport-sdr: 66b37d25_+KnUZohlVLiI9opV08UB7m5UW6OUDyeCTm6uf1BaqqHFvYK DHNv7PEvxNikJvXmCFvo/3HKMDGVDz7u4kn2UZw==
- Msip_labels: MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_Enabled=True;MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_SiteId=7d1c7785-2d8a-437d-b842-1ed5d8fbe00a;MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_SetDate=2024-08-07T13:56:48.069Z;MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_Name=Eviden For Internal Use - All Employees;MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_ContentBits=0;MSIP_Label_ecb69475-382c-4c7a-b21d-8ca64eeef1bd_Method=Standard;
If I add prefecthes after node execution the holes are filled but the final time stay the same :


De : XAVIER LACOSTE
Envoyé : mercredi 7 août 2024 15:23
À : starpu-devel@inria.fr <starpu-devel@inria.fr>
Objet : Late data transfers
Envoyé : mercredi 7 août 2024 15:23
À : starpu-devel@inria.fr <starpu-devel@inria.fr>
Objet : Late data transfers
Hello,
I'm curious about why my data transfer do not occur directly after the task producing the data.
One the attached trace of a LU factorization, GETRF (black) and TRSMs (blue) are executed on CPU except for the first ones in row and column and GEMMs are all batched on the GPU (except the first one wich is on the GPU but not batch).
Here are some remarks I can't explain on the trace:
-
The TRSMs on host wait until the block is sent to GPU to start (while they could start just after it).
-
data transfers from host to device do not start immediatly after GETRF nor after each TRSM.
-
data transfer from device to host, before next GETRF do not occurs immediatly after the tiny GEMM on the GPU (just after the TRSMs on the GPU)
Data are allocated as starpu_data_matrix_t by starpu, on the GPU, initalized on the CPU then prefetched on GPU using an empty task.
I use 2 streams on the CUDA device because there are small idle time between each kernel....
Any advice is welcome,
Regards,
XL

- [starpu-devel] Late data transfers, XAVIER LACOSTE, 07/08/2024
- RE: [starpu-devel] Late data transfers, XAVIER LACOSTE, 07/08/2024
- Re: [starpu-devel] Late data transfers, Samuel Thibault, 22/08/2024
- Re: [starpu-devel] Late data transfers, Samuel Thibault, 22/08/2024
- RE: [starpu-devel] Late data transfers, XAVIER LACOSTE, 28/08/2024
- Re: [starpu-devel] Late data transfers, Samuel Thibault, 28/08/2024
- RE: [starpu-devel] Late data transfers, XAVIER LACOSTE, 28/08/2024
- RE: [starpu-devel] Late data transfers, XAVIER LACOSTE, 07/08/2024
Archives gérées par MHonArc 2.6.19+.