Objet : Developers list for StarPU
Archives de la liste
- From: Konstantin Sozykin <Ko.Sozykin@skoltech.ru>
- To: Samuel Thibault <samuel.thibault@inria.fr>
- Cc: "starpu-devel@inria.fr" <starpu-devel@inria.fr>, Aleksandr Mikhalev <Al.Mikhalev@skoltech.ru>, Roman Schutski <R.Schutski@skoltech.ru>, "Georgii Ovchinnikov" <G.Ovchinnikov@skoltech.ru>
- Subject: Re: [starpu-devel] Customization/Optimization for Scheduling Methods
- Date: Tue, 18 Jul 2023 16:01:15 +0000
- Accept-language: ru-RU, en-US
- Authentication-results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=Ko.Sozykin@skoltech.ru; spf=Pass smtp.mailfrom=Ko.Sozykin@skoltech.ru; spf=None smtp.helo=postmaster@srv-ksmg-01.skoltech.ru
- Ironport-data: A9a23:CiGQiq9xBIdrAHOBHNcjDrUDznWTJUtcMsCJ2f8bNWPdYAuXwgSz/ BJCHDXCJLXTIlJBSKl0aYXn8kIO7JbcztYyHlRp+ChjFysU8saVCNjEchyvZi3CI5XPQRM85 M5FNoKcd8ptHyCDr0b1YuDv8iBxiq3TLlaQ5aOt1gVZHGeIHw9w0kw58wJAvrNVvDSZP++sk Y+j/JyPaAKvhGUpbz9Esvnd9Bo0sP6rtj4Ws1FvbvsTlVKPzHN94LA3dfjgdSOgKmV3NrfnH LiblOnREkfxpUpF5gaNy+6jGqEyaueOe1LI0BK6YoD66jBavCs+z60nA/QVbEZTml2hkst4o Dl3ncXYpTwBYOuUwYzxbzECS3slZfIfqOedSZSCmZX7I3PuIyOEL8pGVxle0b0woo5fHWxI/ PoEHzEBBjjrazWendpX4sE17igSBJGD0LE34hmM/hmFZRoSeq0vdo2RjTNuMJjcsegVdRrWT 5JxhTOC930saTUXUrscIMpWcOtFGhAT2tCXwb6YjfNf3oTd8OB++JHRGd+Pe9qrefxqpRyjp k322Eb0AihPYbRzyRLdmp6trvTVg2b4Q4cdUeXhsOVph1mPy2cTTgcLPbe5iaDm2gjkAIIZc hZIvHN18cDe92TzJjX5dz6/5kWDrA8fWtx4CO497B2AwaqS+Rvx6m0sF2cRMIZ37JVpLdAs/ kaGlYjpLAYxiZqEa3ut/62KrQK1NidAeAfuYgdfF1dduoG9yG0ptTrTTtdpGbOxkvXnGDT62 TnMrS4khrxVg9Rj/76n9Erfnnegq4bESiYu9wDPVySk6Bl4bcirfeSVBUPz9uxeaYWCS1bE5 CFCgMuZ6PwDBpHLjjHlrPgx8K+BzsqsNWfdh39TJqYarQ+t1E6EX41Q2WQrTKt2CfosdTjsa U7VnApe4p5PIXenBZObharsUazGKoC9TLzYuuDogslmP8UuKF/dlM16TRXPgTmyyCDAhIlmY c/DGftAG0r2Hoxd8VJarc841qQgwT9WKYj7HMqkk0/7uVZyTFuSUrwCLzOzgg0R97ma5QbO9 tkab5LM0BZYVPHzaSmR65N7wbE2wZoTWM+eRy9/LLbrzu9a9IcJUaS5LVQJINQNokitvr2Ul kxRo2cBoLYFuVXJKB+RdldoY671UJB0oBoTZHJ8bAjxiyZ4Pd7zsc/zkqfbm5F6roSPKtYpF pE4lzmoW5yjtxycp2xBPfERUqQ5K03Daf2y09qNO2JjJ8c4GWQlC/fmZQCn8SAVEietqcY6y 4BMJSuGKafvsz9KVZ6MANr2lgPZlSZGwopaARCXSuS/jW2wrOCG3QSq1KRmSyzNQD2frganO /G+UUtJ9bmX+NdvmDQL7Ijdx7qU/yJFNhIyNwHmAXyebEE2J0LzkdUSY/XCZj3HSmL/9YOrY Ogfnbm2M+QKkBwO+8BwGqpih/B2rdb+ha5o/iI9FlXybnOvFuxBJFuC1pJxraFj/OJSlja3f UOtweNkH4u1FvnrK3MrHzp9XN+/jakVvhLw8cUKJF7L4X4r3bifDmRXERq+qA1cC7pXIop/k Pwtl/AU4ifimxYRE82nixpM/D+mNU0wUKQAt7AbDrT0iwEt9EpwXJzEBgLy47CNc99pMHR2E gSLhaHHuatQ9nDCf1U3C3LJ++hX3rYKhzxn034AIA6vtufep/pqwiBUzys7fj5VwjpDzeh3H GphbG9xBKeW+gZXlNpxZH+tFy5BFS+m1BTIkXVRr1LgTm6sSmDpB08+M7zU/EkmrkRtTgIC9 7SckGvYQTLmef/q5RQLWGlnla3Hbcdw/Qj8isyYD5y7P50lUwHE3I6qR0Q18iXCP+1gpXf6t dFL/flxY5LVLSQ/gbM2IKjE2KUySCKrHn1jQ/Zg9/grR0XjZCO45jOSDmycZ8gXO+LA0XW8O pZhO8lJahWa/wSNpwA9GqQjDeJVnvkoxdxaYZLtBzcMnIW+pwpTkqD713bBlko0ZdRxgOABK o/1XBCTIFy63Hd7tTfEk5hZBzCef9IBWjzZ4Mm01+c4T7Q4r+BmdBAJ4Iue5nm6HlNuwEOJg VnleaTT8u1FzLZslavKFoFoJV29CfH3ZdSy3DGDifZ8RvKRDp6WrCIQkEftACpON7hIW9hXq 6WEgOSq4Gz75oQJQ0LrsLjfMZkR/sijfvtlAuSuJllgoCaycsvN4RwCxmOGFaJ0gO5tvveAe Q/pR/azJPg0WshczkJ7cyJxMQgQIIWpY7bCpRGSleWtCB8c2lbbMtmHz3vYN25GfSk1OrnVK Az9i9Cx7P928aVOAx4lAal9IplaeVXMZ4ovR+fThxK5UFa6owqllOP5tBwC7TrrNCG1IPzi6 8iYeimkJQWAhq7Y6fp47apwh0QzJ1Rgi7ASekk9xYZHuwqiBjRbEdVHYIQ0MbAKoCncz5qiW SrsalElAiDDXThpVxXwzdDgfwWHDNw1JdbLCW01znyQdhuJKtuMMJl5+gdkxkVGSD/p4eWkC NMZo3POZ0n7htkjQOsI/fW0jNt23v6QlDpC5Un5lNe0GBoER6kD0Ht6BgdWSCjbCIf3mV7WI XQuD3VxKK1hpZUdze47E5KUJP0YgN8r5y4yc2KM29PQ/t3CivZDz/njNurzlKceBCjPyHjiW luvL1ZhIUjPspDQhUftk8kqhqtoBPmLWNKnREMmbRNHhLm+swzLIOtb9Rfij6geFMp3CFDak iWh6Hh4HlntxIW9HlGJ4V1hxq+dmU7gw90EYMATaNMGfdEEIwDlRiWX
- Ironport-hdrordr: A9a23:fyuZVqA1tRwT1MnlHej9tMeALOsnbusQ8zAXPhhKOGRomszxra CTdYcgpGTJYVcqKQMdcLW7UpVoLkmskaKdjbN+AV7mZniehILKFvAW0WKB+UyGJ8SWzIc0vs oNHJSWSueAa2SS5vyKgzVQZuxQpeVvh5rY59s2oU0NcShaL4Nbw0NQDRuaKFR/QE1nApU+fa DsrbsM1nfQNQVZH4rLRwhELquz5ayP5dbbSC9DJSQKrCGIkDOT8bb/VzyVwxYZXlp0sP4f2F mAsTbUoo2lrvGA0xfZk0nO8p9RhNztjvtFGcCWgMQNNzP2ziaua4IJYcz3gBkF5MOUxBIDl8 TNmQ4pO4BJ427cF1vF6SfF6k3D6hprw3j+yWWAiX+mm9HwWS4hYvAx875xQ1/60WZllNZg0J hT12bciJZNFxPHm03Glq/1fiAvtHCZ5VIkjO4LjnxEUIsYZvt7pZYS5ipuYc89NRO/yroKVM FoEcHE5fZNcFOcYTT4uWZoqebcGUgbL1OleGRHksuJ0yFWnGp5z0wXgOgSg3YH+fsGOsB5zt WBEJ5F0JlIU88ObKJmBecHT4+MAGTLRhLKPl+DOFiPLtBAB1v977zMxJtwzOa2eKUIzIEymJ POOWko4lIaSgbNM+mhmLVN7xD2WW24GRzw18A23ekrhpTMAJjQFQPGYFcyk/G4q/FaLsrFV/ O0IZ4TLOTkKQLVaNl09jy7eaN3Bz03V9AUgM8yXxavpc3MQ7ea5NDzQbLwHp7CVR4iR2PEG3 MGGBj1Ocla5EyzR3nixDjXVHWFQD2qwbtAVIrh28ZW4I8RNrdUugxQrEiwoviRLzJDt6AtFX EOfY/Po+eYn02ZuU7O8mVSIxJYSmJY/bTnXhpx1F83GnKxU4wmlZG2fn1T5mKOIltFR97beT Qv+WhfyOaUFbi/gQgkFtKcK2qfy0EPrG+XVf4n6/y+zPagWKkdINIPX7FwLxzCE1hLlR1xqG FFATV0GnP3J3fNs4GKyLQSGeHhd8Jhmm6QUJhpgEOal3+5i4UGQWYaYD60TdTSuxsnWiZP72 cB2pM3sf6joxKAbU8ymv4xPlEJQG6LALRAZT71Prl8q/TIQiVfCVyXgDDfqzxbQBuwy6xOvB 21EcTeQ4CwPrMUgAEk7o/6tFFveiGUeF9xdXBgrZc4FWPIvB9IoJ+2T7v20XGcLlME3+EDMC rYe38TJR5j3Nq6zQOSg3KDEnohr69eT9D1HfAgab2W0m6xIIeIka0AAvBV+YtoL9DovKsKVv +SYAWYNS7/EKco3AOSz0xVRhVcuT0ni/+t0wHh7my1xmQ2D/bJL1RiWr0BI9aTq3LvT/HNy5 likdcp1NHAQVnZe5qF062SdTJZNR/JoWm4SYgT2O9pVIwJxcdOI6U=
- Ironport-phdr: A9a23:6/o4kh2HMxzFeOMXsmDOlA0yDhhOgF0UFjAc5pdvsb9SaKPrp82kY BaBo6kxxw6QFazgqNt8w9LMtK7hXWFSqb2gi1slNKJ2ahkelM8NlBYhCsPWQWfyLfrtcjBoV J8aDAwt8H60K1VaF9jjbFPOvHKy8SQSGhLiPgZpO+j5AIHfg9q22uyo9JDebApEiT6gbbhvM Bi4sALdu9UMj4B/MKgx0BzJonVJe+RS22xlIE+Ykgj/6Mmt4pNt6jxctP09+cFOV6X6ZLk4Q qdDDDs6KWA15dbkugfFQACS+3YTSGQWkh5PAwjY8BH3W4r6vyXmuuZh3iSRINb7Rq4oVzu88 6hrSQfoiCYZOD4/7GHXkdF7gKZCrB68uxBz34vYbYeIP/R8Y6zdZ8sXS2pfUMtPSiJBHo2yY YgBD+UDPOZXs4byqkALrReiBwmhHv/jxiNJi3LwwKY00/4hEQbD3AE4GtwOt3LUrM7pNKgIV +C10KjIzTPeZP1L3jf86YjJchMgofGKQLl9ds/fyU41Gw3YiVWRqIzkPzCU1uQLqGWb9OlgV eWri247tQ5+vjivyt02ionUn48YzE3P+iplzogvP9K4VFJ7bsC+EJtWryyUN4p7TMMsTW9ot ig3y7wLtYCncCUEx5kr2QPSZ+Cbf4WG/B/tVOicLCp8iX94eL+ymxm8/Eqvx+PzWcS530pHo yxYmdfCsXAN0gbc6smBSvZl/0eh3yyP1wHJ5e1eL0A0j6zWIIMizL4ojpcfrEbOEyzslEnog qKbdV8o9vW25+noeLnqu5GROoBshg3jMKkjmdazDfo2PwUNRWSX5Pqw2bL58UHkT7hGkPs7m bTDvp/AP8QUvKu5DhdV0ok97xa/CC+r0M8CnXkZNV5FewmHj43mO13XPPz0EO2zg0+vkDdsw /DGIqPuDYnNLnTZirfhYK5x609ayAUt0dBS/49YB7MCLf7pREP8ut/VAgUkPwCqzOvrEshx2 40RVG6XB6+WKqLSsVuG5uI1JOmMYZcYtjP6K/Q/+vDuin44mF8efamo25sac2u4HvdhI0Wdf XrjnMsOEWcQsgokUezmkFuCXiVLaHaoQ608/i07CJ6hDYrbWIyhmKaB0zujHp1KemBGDUiBE Wvod4WFQPgDcj+SIst9kjEfVLmhSooh2gq0tADg0bZnL+zU+jcZtZ35ztR16ffTmQko9TNoF 8Sdz32NT2Zsk2MTWz85xrp/oVRgyluazKd3neJYGsJX5/5SSQc7NZvcz/RgBNzoWwLBeM2JS Fe8Tdm8DzExVIF5/9hbWE94BtimxizD3iCjHrsJ3+iQDZkp/6aaw3jwLc9gz2ru1a87jlBgT NEZZkO8gasq1A/VHJLAlVmYke6RcrYN3y/LvDOgyiy1u1NJUw92eb7GWnQDa0DW687ksBCRB 4SyAKgqZ1MSgfWJLbFHP5iw1Q0urJbLPd3fZzj0gGKsHVOSwbjKaoP2emIb1SGbCU4ekglV8 2zVfRMmCHKHpGTTRCdrCUqpe1nlpMp57ki8XF43zgeidE5m06a48R5Tn+HPA+gL0Oc8sTw64 y5xAE772tvXD9SaoA80X6IaWtQv8VRG00rCtAt5IpGrL+Z4mw1WaBx56mXp0Rg/EYBciY4qo XctmRJ1Mr6d2Uhdeimwxoruf77ML2a3pEnpcK3R3k3T29bQ4LpnBO0QjVLlsUnpE0Mj9y8iy NxJyz6G4Z6MCgMOUJX3W0Jx9h5gpricbDNvr4XTnWZhN6W5qFqgk5ogGfclxxC8ftxeLLLMF Qn8FNcfDtSvL+pikkagbxYNNuRfvKAuOMbue/yD0a+tdOFu+VDuxUdAprt6wFOP9ixUUe3P2 4oIyPHexBbGHzbwgVG9s9zmzJhebGJ3fCL3wizlCYhNI6xqKN9UWSH0eJ3xn48iwce2By09l hbrHV4N1c63dADHalX82VcVzkELuTm9niD+yTVokjYvp67Z3SrUwu2kegBUXwwDDGRkk1roJ pC5yt4AW039JSsu0iCi91f5xq5zv6d4K3XeSkcOZTW8fAQAGuOg86GPZcJC8sZikiIRa+Ouc FObTJbmoBwezy7oHy1E2XpoElPi8oW8lBt8hmWHKX91p3eMYsB8yyDU49nETOJQ1D4LLMVho QHeHUP0f9yg/NHO0ozGrvj7TWWqEJtabSjsy4qE8iq9/2xjRxOlzbi/ndjuEA5y1iGetZEid iKOlxHuc4rq0YyhMuVjYkRjBBnm9oJ2F5p/nY05mJwLkSFF1tPPpipByD2vd4wAher3dx9vD XYTzsTQ4RT51UErNX+Py4/jFz2czsZne9imczYT0yM54dpNDfTxjvQMli90r1yk6APJNKEiz nFHkqtosSRGxbhS5FlIrG3VGL0ZEEhGMDa5kh2J64r7t6BLfCO1drP20kNinNenBbXEowdGW X+/dI1xeE04psh5Ll/I12X+r4/+f9yFJ/EU8CaTgQbDju59M5E2kuYHjiohI3+37hhHg6Yry Adj2523pt3NCGIrx6WrEx1ZMBXuZMQX5jzmjeBEg4zFuuLnVoUkETINUpzySPuuGz9HrvXrO TGFFzgko2uaE77STkePrV1rpHXVH9W3JmmacTMHmM56SkDXdyk9yEgEGS83lZkjGkW2ydz9J Q1nsysJ6AewqwMQmLs5al+mCTyZ/l72LG1uENCeNEYEt14SoR2OYYrHtL41RnoEm//p5A2Vd j7CNlkOVDpSHBbdWRa5ZObm5MGcobHHVqzkdKuIOO/S77UFBr/AxIrzgNI3onDWbpXJZz87V 6FeuAILXGglSZ2Gy3NVEHxRzniQKZTG/F+94nEl/pH5qay1Hluwuc3WUeE3U50n+ha9hbqPO ryLnCg/Mjte2ogF1DnS1fAe2lUWli1jJSK1H/Idsy/WQaTcl7URBBkeDkE7fMpO5KYh0gQfI tbVzM7z17l+g7g0DB9EU1flh82kN9cSLSegMlLcCUCHOa7APyCD1cjzZenUpaR4quJSulXwv D+aFxWmJTGfj3zzUBvpN+hQjSadNRgYuYencx8rB3KxBNThIga2NtN6l1hUifU9m2/KOGgAM DN9b1IFr7ue6jldi+l+HGoJ52RsLO2NkSKUp+fCLZNevfxuCyVy3+VUhRZyg6NS9z1BTedpl TH6tcRw5lK9l+bKkmMhSxlJripGj4LNp1gjcaTV+59cWGrVqRIA6WLDbnZC79BhC9Dpp+VR0 o2Wy/K1cmwStYuOoY1FXJSHTaDPeGAsOhfoBjPOWQ4MTDrxcHram1QYivaZsHucspk9rJHo3 psIUL5SElIvRZZ4QgxoGsIPJJBvU3Yqi7me2YQr4jyFqwvAR8RclovEUfaODP7vbiyEx+ohB VNA0fbjIIIfO5eukVRlcUV/lZ/WFlD4RspT5y56awBx+BwL63t5SnE+0EKjcB7nsxpxXbak2 xUxjAV5e+Ek8jzhtkw2KlT9ryw1iEAtmN/hjFh5lRbqL6C0QYxdC2zuqhppWnsaax1vc0ixh klrcW+dAalSiL98eGtrzhXB68Mn8RF0Vq5DZAMdzvHRfeh6iDxh
- Ironport-sdr: 64b6b74f_rcKzr5ieJj3x8rNC6QYeyRnbl+dIOU3MuzQw6cdYDWecSBd 0m+Qao114I5j8CGoTNOBgGyDO82Q8+4PJtteSrQ==
Dear Samuel,
Thanks for the response. We have a lot of additional questions. But lets discuss step by step.
The current questions are following:
- Although mps-lp files are readable, We would like understand and exact lp formulation for further optimization. Could you give reference where this linear programs for scheduling described in detail.
- Are the functions in src/profiling/bound.c ready for use? Currently, we experience problems with integrating bound calculations even in dgemm example (attached).
- Yes we have followed graph test policy example. However, this example exposes only the dependency graph of the tasks (and not the size of the data transfers ), and most functions in it are not public in StarPU 1.4.0. Is it ready for use?
Many thanks!
От: Samuel Thibault <samuel.thibault@inria.fr>
Отправлено: 17 июля 2023 г. 10:46:04
Кому: Konstantin Sozykin
Копия: starpu-devel@inria.fr; Aleksandr Mikhalev; Roman Schutski; Georgii Ovchinnikov
Тема: Re: [starpu-devel] Customization/Optimization for Scheduling Methods
Отправлено: 17 июля 2023 г. 10:46:04
Кому: Konstantin Sozykin
Копия: starpu-devel@inria.fr; Aleksandr Mikhalev; Roman Schutski; Georgii Ovchinnikov
Тема: Re: [starpu-devel] Customization/Optimization for Scheduling Methods
Hello,
Konstantin Sozykin, le ven. 14 juil. 2023 14:52:26 +0000, a ecrit:
> we discovered that the default schedulers can't handle our custom
> operations.
What custom operation are not handled?
> Consequently, we are considering developing our own optimization- or
> even a learning-based scheduler. How reasonable is this, given the
> current development of StarPU?
StarPU was actually made for people to create their own schedulers, so
that is entirely reasonable :)
> Firstly and most importantly, how can we extract the full DAG? Are the *.rec
> and *.dot files sufficient for this purpose?
They can be used, yes, but that's offline.
> Can we access the full DAG in online mode?
You can use the graph module, see the graph_test policy example.
https://files.inria.fr/starpu/doc/html_web_extensions/HowToDefineANewSchedulingPolicy.html#GraphScheduling
> We have located functions for exporting linear programs (*.lp and *.mps files)
> to determine theoretical bounds for execution time. However, as far as we
> understand, these are only for measuring performance and not for optimizing
> scheduling. Is our understanding correct?
Indeed, they only provide performance bounds.
Samuel
Konstantin Sozykin, le ven. 14 juil. 2023 14:52:26 +0000, a ecrit:
> we discovered that the default schedulers can't handle our custom
> operations.
What custom operation are not handled?
> Consequently, we are considering developing our own optimization- or
> even a learning-based scheduler. How reasonable is this, given the
> current development of StarPU?
StarPU was actually made for people to create their own schedulers, so
that is entirely reasonable :)
> Firstly and most importantly, how can we extract the full DAG? Are the *.rec
> and *.dot files sufficient for this purpose?
They can be used, yes, but that's offline.
> Can we access the full DAG in online mode?
You can use the graph module, see the graph_test policy example.
https://files.inria.fr/starpu/doc/html_web_extensions/HowToDefineANewSchedulingPolicy.html#GraphScheduling
> We have located functions for exporting linear programs (*.lp and *.mps files)
> to determine theoretical bounds for execution time. However, as far as we
> understand, these are only for measuring performance and not for optimizing
> scheduling. Is our understanding correct?
Indeed, they only provide performance bounds.
Samuel
Attachment:
mult.zip
Description: mult.zip
- [starpu-devel] Customization/Optimization for Scheduling Methods, Konstantin Sozykin, 14/07/2023
- Re: [starpu-devel] Customization/Optimization for Scheduling Methods, Samuel Thibault, 17/07/2023
- Re: [starpu-devel] Customization/Optimization for Scheduling Methods, Konstantin Sozykin, 18/07/2023
- Re: [starpu-devel] Customization/Optimization for Scheduling Methods, Samuel Thibault, 17/07/2023
Archives gérées par MHonArc 2.6.19+.