Hibernate lazy loading LOBs. How to unload processed objects to stop memory consuming?

Question

Please somebody help me with a memory issue while processing records with LOBs! So I have the following situation: I have two tables, and one of them is a table with some data. One of the columns of the table has "one-to-many" mapping to the second table. The second table keeps millions of photo images that are stored in LOBs. I use the "lazy loading" method to access the images. I need to iterate the first table's records to get photos and other information and send them to some SOAP-gate. Every time new data falls to the tables so I need the process to be restarted periodically. So I did and it works well for some amount of time but then suddenly stops with an exception like this:

июн 08, 2021 3:23:44 PM org.hibernate.engine.jdbc.connections.internal.DriverManagerConnectionProviderImpl$PoolState stop
INFO: HHH10001008: Cleaning up connection pool [jdbc:postgresql://x.x.x.x:yyyy/SomeDbInstance]
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.awt.image.DataBufferByte.<init>(DataBufferByte.java:92)

That is my app consumed out all the memory and stopped execution with this error.

So my question: is it possible to clean up processed entries to release the memory they start to consume after they got loaded?

Answer 1

Lazy loading allow you to defer loading something into memory until you need it. It doesn't prevent you from loading too many things into memory and running out of heap. You need to think about your design for how you load the objects as needed, process them and then release the references so that the GC can free space in the heap.

Other things to consider:

• use a profiler so you can understand how your app is using and consuming heap at runtime
• use the JVM gc flags to watch gc performance at runtime - are you releasing references so the GC can perform collection? How long is the collection taking? Are you creating new instances faster than the GC is able to free space in the heap?
• is your heap sized appropriately to work with the size of data you're attempting to process?

Answer 2

Thank you, Kevin for your suggestions! I started Java Visual VM, used profiler, heap dump and monitor to analyze the executing process of my service. So, I found many heavy instances of LOBs hanging in the heap. And, yes, I have to invent method to sweep them out. And I seemed to get it!

for (PhotoContext pc:tr.getPhotoContext()) {

    . . .

    photo.setPhotoType(photoTypeMapping(pc.getInd()));
    GregorianCalendar gk = new GregorianCalendar();
    gk.setTime(pc.getDatetime());
    photo.setDatetime(DatatypeFactory.newInstance().newXMLGregorianCalendar(gk));
    photo.setPhoto(Arrays.copyOf(pc.getPhoto(), pc.getPhoto().length));
    photos.add(photo);

    sourceDao.getSession().evict(pc);
    pc.setPhoto(null);
}

I had to detach the photo entry and than set it's photo field (byte[]) to null. Maybe that is not particularly elegant but it works.