Re: [Starpu-devel] StarPU Fortran support

[Olivier Aumage <olivier.aumage@inria.fr>]

Dear Matthias,

I made some experiments using PGI 17.10 over the last few days. CUDA Fortran
kernels compile and link properly with StarPU. However, CUDA aborts the 
execution
of kernels with the following error: "misaligned address (74)", related to 
Fortran
arrays passed to the kernels. 

Basically, the code generated by the PGI CUDA compiler does not seem to 
accept arrays
allocated on the device by a third party code (StarPU, in this case). I have 
not found 
sufficient details in the PGI CUDA Fortran user guide so far on how to 
implement
interoperability between PGI CUDA kernels and third party software. Thus, 
StarPU + CUDA Fortran is not currently supported.

I am attaching to this email one of the test cases I unsuccessfully attempted 
to run using PGI 17.10. I have experimented some other strategies as well, 
such 
as using the 'c_devptr' in the host wrapper to the CUDA kernel, with the same 
result.

Section "4.8. Memory Management" of the latest PGI CUDA user guide revision 
indicates
that pointer management on the device is still experimental somehow. Thus, the
situation may evolve in the future.

Best regards,
--
Olivier

Attachment: starpu_fortran_cuda.tar.gz
Description: GNU Zip compressed data

> Le 23 nov. 2017 à 18:00, Matthias Lieber <matthias.lieber@tu-dresden.de> a 
> écrit :
> 
> Dear Olivier,
> 
> thanks for your reply. It would be great of course if the CUDA kernels 
> themselves
> could be written in Fortran, too. Is it possible to use PGI's CUDA Fortran 
> with
> StarPU? Or do you know any other method to use kernels written in Fortran 
> with
> StarPU?
> 
> Best wishes
> Matthias
> 
> Olivier Aumage wrote on 2017-11-01, 15:35:
>> Dear Matthias,
>> There are two ways to use StarPU for a Fortran application:
>> ** Using the Native Fortran API:
>> You can have a look at examples in the STARPU/examples/native_fortran/ 
>> directory.
>> For instance, to use a CUDA kernel with the example nf_vector.f90, you can 
>> replace
>> the following lines:
>> %-------------------
>>         ! add a CPU implementation function to the codelet
>>         call fstarpu_codelet_add_cpu_func(cl_vec, 
>> C_FUNLOC(cl_cpu_func_vec))
>> %-------------------
>> with:
>> %-------------------
>>         ! add a CUDA implementation function to the codelet
>>         call fstarpu_codelet_add_cuda_func(cl_vec, C_FUNLOC(... CUDA 
>> kernel launch function ...))
>> %-------------------
>> and replace every remaining occurrence of cpu/CPU with cuda/CUDA.
>> If you want to use both CPU dans CUDA kernels for the same codelet, you can
>> remove the call to fstarpu_codelet_set_where() and add both kernels to the 
>> same codelet:
>> %-------------------
>>         call fstarpu_codelet_add_cpu_func(cl_vec, 
>> C_FUNLOC(cl_cpu_func_vec))
>>         call fstarpu_codelet_add_cuda_func(cl_vec, C_FUNLOC(... CUDA 
>> kernel launch function ...))
>> %-------------------
>> ** Using C marshaling functions:
>> The example in the STARPU/examples/fortran90/ directory uses the C 
>> marshaling
>> approach, where all interactions between the Fortran application with 
>> StarPU
>> (both the application calls to StarPU and the StarPU calls to application
>> kernels) are done through a custom C layer. Here, the use of CUDA kernels 
>> is
>> exactly the same as for a C/C++ application.
>> The native Fortran API of StarPU is more convenient, but it requires a
>> Fortran compiler compliant with ISO/IEC TS 29113:2012
>> (see https://www.iso.org/standard/45136.html). You can try to build the
>> examples in StarPU/Examples/native_fortran to check if your compiler is ok.
>> Recent versions of GNU GFortran and Intel Fortran compilers are known to
>> build these examples successfully.
>> The C marshaling way works with any Fortran compiler supporting 
>> 'iso_c_binding',
>> but it requires more coding.
>> Best regards,
> 
> 
>