How to chunk List<Object> in Netty

Question

When sending file, you can do ctx.writeAndFlush(new ChunkedFile(new File("file.png")));.

how about a List<Object>?

The list contains String and bytes of image.

from the documentation there's ChunkedInput() but I'm not able to get the use of it.

UPDATE

let's say in my Handler, inside channelRead0(ChannelHandlerContext ctx, Object o) method where I want to send the List<Object> I've done the following

@Override
protected void channelRead0(ChannelHandlerContext ctx, Object o) throws Exception {

   List<Object> msg = new ArrayList<>();

   /**getting the bytes of image**/
   byte[] imageInByte;
   BufferedImage originalImage = ImageIO.read(new File(fileName));
   // convert BufferedImage to byte array
   ByteArrayOutputStream bAoS = new ByteArrayOutputStream();
   ImageIO.write(originalImage, "png", bAoS);
   bAoS.flush();
   imageInByte = baos.toByteArray();
   baos.close();

   msg.clear();
   msg.add(0, "String"); //add the String into List
   msg.add(1, imageInByte); //add the bytes of images into list

   /**Chunk the List<Object> and Send it just like the chunked file**/
   ctx.writeAndFlush(new ChunkedInput(DONT_KNOW_WHAT_TO_DO_HERE)); //

}

Answer 1

Just implement your own ChunkedInput<ByteBuf>. Following the implementations shipped with Netty you can implement it as follows:

public class ChunkedList implements ChunkedInput<ByteBuf> {
    private static final byte[] EMPTY = new byte[0];
    private byte[] previousPart = EMPTY;
    private final int chunkSize;
    private final Iterator<Object> iterator;

    public ChunkedList(int chunkSize, List<Object> objs) {
        //chunk size in bytes
        this.chunkSize = chunkSize;
        this.iterator = objs.iterator();
    }


    public ByteBuf readChunk(ChannelHandlerContext ctx) {
        return readChunk(ctx.alloc());
    }

    public ByteBuf readChunk(ByteBufAllocator allocator) {
        if (isEndOfInput())
            return null;
        else {
            ByteBuf buf = allocator.buffer(chunkSize);
            boolean release = true;
            try {
                int bytesRead = 0;
                if (previousPart.length > 0) {
                    if (previousPart.length > chunkSize) {
                        throw new IllegalStateException();
                    }
                    bytesRead += previousPart.length;
                    buf.writeBytes(previousPart);
                }
                boolean done = false;
                while (!done) {
                    if (!iterator.hasNext()) {
                        done = true;
                        previousPart = EMPTY;
                    } else {
                        Object o = iterator.next();
                        //depending on the encoding
                        byte[] bytes = o instanceof String ? ((String) o).getBytes() : (byte[]) o;
                        bytesRead += bytes.length;
                        if (bytesRead > chunkSize) {
                            done = true;
                            previousPart = bytes;
                        } else {
                            buf.writeBytes(bytes);
                        }
                    }
                }
                release = false;
            } finally {
                if (release)
                    buf.release();
            }
            return buf;
        }
    }

    public long length() {
        return -1;
    }

    public boolean isEndOfInput() {
        return !iterator.hasNext() && previousPart.length == 0;
    }

    public long progress() {
        return 0;
    }

    public void close(){
        //close
    }
}

In order to write ChunkedContent there is a special handler shipped with Netty. See io.netty.handler.stream.ChunkedWriteHandler. So just add to your downstream. Here is the quote from documentation:

A ChannelHandler that adds support for writing a large data stream asynchronously neither spending a lot of memory nor getting OutOfMemoryError. Large data streaming such as file transfer requires complicated state management in a ChannelHandler implementation. ChunkedWriteHandler manages such complicated states so that you can send a large data stream without difficulties.