Cupy error : cudaErrorIllegalAddress - an illegal memory access was encountered

921 Views Asked by At

Training a feed forward network on GPU using Chainer. After training some batches, getting aerror "CUDARuntimeError: cudaErrorIllegalAddress: an illegal memory access was encountered".

only 1.5GB out of 11GB gpu memory is in use. Once the error is encountered, attempt to create a small cupy array too fails with same error.

Chainer : v6.5.0 Cupy : v6.5.0

0

There are 0 best solutions below