[Starpu-devel] StarPU 1.3 does not purge GPU memory

[Mirko Myllykoski <mirkom@cs.umu.se>]

Hi,

I have noticed that the StarPU 1.3 nightly snapshots have a problem 
where the library does not purge the GPU memory when it is necessary. 
Instead, the ptr field in the interface ends up begin a NULL pointer. 
The problem can be reproduced with the following example code:

#include <unistd.h>
#include <starpu.h>

// more loop iterations are required when the GPU has a lot of memory
#define ITER 100

static void kernel(void *buffers[], void *cl_args)
{
    printf("Pointer is %lu\n", (uintptr_t) 
STARPU_MATRIX_GET_PTR(buffers[0]));
}

static struct starpu_codelet codelet = {
    .name = "codelet",
    .cuda_funcs = { kernel },
    .nbuffers = 1,
    .modes = STARPU_SCRATCH
};

int main()
{
    if (starpu_init(NULL) != 0)
        return 1;

    starpu_data_handle_t handle[ITER];

    for (int i = 0; i < ITER; i++) {
        starpu_matrix_data_register(
            &handle[i], -1, 0, 10000, 10000, 1000, sizeof(double));
        starpu_task_insert(&codelet, STARPU_SCRATCH, handle[i], 0);
    }

    starpu_task_wait_for_all();

    for (int i = 0; i < ITER; i++)
        starpu_data_unregister(handle[i]);

    starpu_shutdown();

    return 0;
}

StarPU 1.2.6 outputs the following:

Pointer is 140366414086144
Pointer is 140366332297216
...
Pointer is 140363469684736
Pointer is 140363387895808
[starpu][_starpu_memory_reclaim_generic] Not enough memory left on node 
CUDA 0. Your application data set seems too huge to fit on the device, 
StarPU will cope by trying to purge 908 MiB out. This message will not 
be printed again for further purges
Pointer is 140366495875072
Pointer is 140366414086144
....
Pointer is 140365023674368
Pointer is 140363387895808
Pointer is 140364941885440

StarPU 1.3 nightly snapshots output the following:

Pointer is 140073297248256
Pointer is 140073215459328
...
Pointer is 140070434635776
Pointer is 140070352846848
Pointer is 140070271057920
Pointer is 140070189268992
Pointer is 0
Pointer is 140070107480064
Pointer is 0
Pointer is 140073297248256
Pointer is 0
Pointer is 140073215459328
...
Pointer is 140071170736128
Pointer is 0
Pointer is 140071088947200
Pointer is 0
Pointer is 140071007158272
Pointer is 0

 - Mirko Myllykoski