Re: [Starpu-devel] StarPU 1.3 does not purge GPU memory

[Mirko Myllykoski <mirkom@cs.umu.se>]

Hi,

I should add that my StarPU 1.2.6 build has been compiled with 
--disable-cuda-memcpy-peer and patched with 
8c35781b0ac517304ac2f2461b243b4447c38ab3 (Enable suballocator for pinned 
main RAM which is expensive).

 - Mirko

On 2019-01-16 16:04, Mirko Myllykoski wrote:
> Hi,
>
> I have noticed that the StarPU 1.3 nightly snapshots have a problem
> where the library does not purge the GPU memory when it is necessary.
> Instead, the ptr field in the interface ends up begin a NULL pointer.
> The problem can be reproduced with the following example code:
>
> #include <unistd.h>
> #include <starpu.h>
>
> // more loop iterations are required when the GPU has a lot of memory
> #define ITER 100
>
> static void kernel(void *buffers[], void *cl_args)
> {
>     printf("Pointer is %lu\n", (uintptr_t) 
> STARPU_MATRIX_GET_PTR(buffers[0]));
> }
>
> static struct starpu_codelet codelet = {
>     .name = "codelet",
>     .cuda_funcs = { kernel },
>     .nbuffers = 1,
>     .modes = STARPU_SCRATCH
> };
>
> int main()
> {
>     if (starpu_init(NULL) != 0)
>         return 1;
>
>     starpu_data_handle_t handle[ITER];
>
>     for (int i = 0; i < ITER; i++) {
>         starpu_matrix_data_register(
>             &handle[i], -1, 0, 10000, 10000, 1000, sizeof(double));
>         starpu_task_insert(&codelet, STARPU_SCRATCH, handle[i], 0);
>     }
>
>     starpu_task_wait_for_all();
>
>     for (int i = 0; i < ITER; i++)
>         starpu_data_unregister(handle[i]);
>
>     starpu_shutdown();
>
>     return 0;
> }
>
> StarPU 1.2.6 outputs the following:
>
> Pointer is 140366414086144
> Pointer is 140366332297216
> ...
> Pointer is 140363469684736
> Pointer is 140363387895808
> [starpu][_starpu_memory_reclaim_generic] Not enough memory left on
> node CUDA 0. Your application data set seems too huge to fit on the
> device, StarPU will cope by trying to purge 908 MiB out. This message
> will not be printed again for further purges
> Pointer is 140366495875072
> Pointer is 140366414086144
> ....
> Pointer is 140365023674368
> Pointer is 140363387895808
> Pointer is 140364941885440
>
> StarPU 1.3 nightly snapshots output the following:
>
> Pointer is 140073297248256
> Pointer is 140073215459328
> ...
> Pointer is 140070434635776
> Pointer is 140070352846848
> Pointer is 140070271057920
> Pointer is 140070189268992
> Pointer is 0
> Pointer is 140070107480064
> Pointer is 0
> Pointer is 140073297248256
> Pointer is 0
> Pointer is 140073215459328
> ...
> Pointer is 140071170736128
> Pointer is 0
> Pointer is 140071088947200
> Pointer is 0
> Pointer is 140071007158272
> Pointer is 0
>
>  - Mirko Myllykoski