An approximately 4.3 GB memory map I want to take the log of without loading the whole thing into memory. Is there a way to assign cl that minimizes amount of memory used?
import numpy
import psutil
print(psutil.virtual_memory().available) #32156053504
cc = numpy.memmap(r'data//cc.dat',mode='r',dtype=float)
print(psutil.virtual_memory().available) #32156049408 #very little memory used
cl = numpy.log(cc) + 1
print(psutil.virtual_memory().available) #23610658816 #8.6 GB used
I want to take slices of cc and cl which normally wouldn't require loading all of cc.dat into memory.