Tensorflow low GPU utilization

Question

I am running windows 10, core i7-8700 cpu, gtx geforce 1660 ti GPU. When training models, gpu utilization is very low (5-10% at max, sometimes lower). Even is network is five layers. CPU utilization on the other hand is 30% and above.

Answer 1

Please check the following:

1. The CUDA and CuDNN versions match. According to the statistics, it may very well use the CPU instead of GPU while training. You can try to see if your GPU is available below option 2.

2. If the former is solved, you may want to increase the batch_size, in case there is a very small batch size. It may be the case that TensorFlow pre-allocates a small amount of GPU to your training.

For step 1, in order to verify that the video card is both available and used, make use of the next lines of code:

import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))

tf.debugging.set_log_device_placement(True)

# Create some tensors
a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
c = tf.matmul(a, b)

print(c)

The print should contain(along the result) the following information:

Executing op MatMul in device /job:localhost/replica:0/task:0/device:GPU:0