[Starpu-devel] OpenCL 64bit Double errors

[Jeff Hand <jeffrey.hand@gmail.com>]

Tripped over this issue on a number of systems.  This time decided to take a closer look.

Error: "Instruction 'ld' requires SM 1.3 or higher, or map_f64_to_f32 directive"

Test Example Output:

===============================================================
./complex
[starpu][initialize_eager_center_policy] Warning: you are running the default eager scheduler, which is not very smart. Make sure to read the StarPU documentation about adding performance models in order to be able to use the dmda or dmdas scheduler instead.
[starpu][_starpu_opencl_driver_init] Warning: reducing STARPU_OPENCL_PIPELINE to 0 because blocking drivers are not enabled (and simgrid is not enabled)
[starpu][_starpu_opencl_driver_init] Warning: reducing STARPU_OPENCL_PIPELINE to 0 because blocking drivers are not enabled (and simgrid is not enabled)
[starpu][_starpu_opencl_compile_or_load_opencl_from_string] Compilation output
ptxas application ptx input, line 51; error   : Instruction 'ld' requires SM 1.3 or higher, or map_f64_to_f32 directive
ptxas application ptx input, line 52; error   : Instruction 'st' requires SM 1.3 or higher, or map_f64_to_f32 directive
ptxas application ptx input, line 57; error   : Instruction 'ld' requires SM 1.3 or higher, or map_f64_to_f32 directive
ptxas application ptx input, line 58; error   : Instruction 'st' requires SM 1.3 or higher, or map_f64_to_f32 directive
ptxas fatal   : Ptx assembly aborted due to errors
ptxas application ptx input, line 51; warning : Double is not supported. Demoting to float
[starpu][_starpu_opencl_compile_or_load_opencl_from_string] Error: Failed to build program executable!
[starpu][_starpu_opencl_compile_or_load_opencl_from_string] clBuildProgram: -42 - clGetProgramBuildInfo: -2
[starpu] Unexpected value: <1> returned for starpu_opencl_load_opencl_from_file
[starpu][abort][main()@interface/complex.c:98]
Aborted (core dumped)
===============================================================

The error is warning that 64bit double isn't supported by the compiler.  Which is true:
===============================================================

  Device Name                                     Quadro NVS 295
  Device Vendor                                   NVIDIA Corporation
  Device Vendor ID                                0x10de
  Device Version                                  OpenCL 1.0 CUDA
  Driver Version                                  340.65
  Device OpenCL C Version                         OpenCL C 1.0
  Device Type                                     GPU
  Device Profile                                  FULL_PROFILE
  Device Topology (NV)                            PCI-E, 03:00.0
  Max compute units                               1
  Max clock frequency                             1300MHz
  NVIDIA Compute Capability                       1.1
  Max work item dimensions                        3
  Max work item sizes                             512x512x64
  Max work group size                             512
  Preferred work group size multiple              32
  Warp size (NV)                                  32
  Preferred / native vector sizes                
    char                                                 1 / 1      
    short                                                1 / 1      
    int                                                  1 / 1      
    long                                                 1 / 1      
    half                                                 0 / 0        (n/a)
    float                                                1 / 1      
    double                                               0 / 0        (n/a)
===============================================================

After reviewing the complex.c file, it does run "can_execute" function.  If this test would have executed in cuda, it wouldn't have executed because of "Old card does not support doubles" checking.

This system actually has 2 Nvidia cards.  One that does support 64bit and the other doesn't.

At this time, there is no intent on running 64bit floating point.

Options:

1. Ignore the failures. 

Less than ideal solution.

2. Added a #def to turn the tests off

This would work at a source level, but the packaged version of the examples wouldn't auto detect capability.

3. Check if all OpenCL devices are capable of 64bit double.  If not, quietly exit test.

Probably the best solution.  If this is the preferred solution, I'm inclined to see if we could come up with a patch. 

Thanks,

Jeff