Creating a zip response without actually creating all the files in the machine

Question

The main endpoint in my program manipulates lot's of data and eventually creates Zip file, which contains parent directory with a few sub directories and a few files(splitted between the sub directories). After sending the zip back to the client, all the content which was created during the process will be deleted, as their only job was to be created in order to initiate the zip file. The main goal is to send the zip file as a response to the user.

In terms of performance, is there a way to execute this logic without actually creating the folder & all it's sub folders and files for each invocation of the endpoint?

Answer 1

Yes, you can do it better. Instead of manipulating physical data on your hard disk, you can do it totally on memory.

As @Olivier mentioned in the comment section, you need an output stream.

For example, you have a controller which returns a .txt file as a zip:

@PostMapping(value = "/zip", produces = "application/zip")
public ResponseEntity<StreamingResponseBody> zip() {
    return ResponseEntity
        .ok()//
        .header("Content-Disposition", "attachment; filename=\"my.zip\"")//
        .body(out -> generateZip(out));
}

To generate a zip file on the given OutPutStream in response by in-memory data:

public void generateZip(OutputStream outPutStream) {
    String data = "Test data \n123\n456";
    String fileNameInZip = "abc.txt";
    try (ZipOutputStream zipOutput = new ZipOutputStream(outPutStream)) {
        ZipEntry zipEntry = new ZipEntry(fileNameInZip);
            zos.putNextEntry(zipEntry);

            ByteArrayInputStream bais = new ByteArrayInputStream(data.getBytes());
            // one line, able to handle large size?
            //zos.write(bais.readAllBytes());

            // play safe
            byte[] buffer = new byte[1024];
            int len;
            while ((len = bais.read(buffer)) > 0) {
                zos.write(buffer, 0, len);
            }

            zos.closeEntry();
    }
}

Of course, this generateZip function is just an example of demonstrating the zipping operation and writing it on the output stream.

You need to implement your logic in generateZip.

You can read more about zip operations such as multiple files in one file zipping, etc. here.

Answer 2

@Soroush Shemshadi Thank you for your help. Managed to solve this thing. For anyone who will need this in the future -

Controller:

    @SneakyThrows
@PostMapping("/generate2")
public ResponseEntity<byte[]> generate() {
    return ResponseEntity
            .ok()
            .header(HttpHeaders.ACCESS_CONTROL_EXPOSE_HEADERS, HttpHeaders.CONTENT_DISPOSITION)
            .header("Content-Disposition", "attachment;filename=CompressedZip.zip")
            .contentType(MediaType.parseMediaType("application/zip"))
            .body(guestService.generateZip());

}

Logic:

    public byte[] compressInMemoryToZip(List<ZipMemoryModel> memoryFiles) throws IOException {
    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
    ZipOutputStream zipOutputStream = new ZipOutputStream(byteArrayOutputStream);
    try {
        for (ZipMemoryModel memoryFile : memoryFiles) {
            ZipEntry zipEntry = new ZipEntry(memoryFile.getFilePath() +memoryFile.getFileName());
            zipOutputStream.putNextEntry(zipEntry);
            zipOutputStream.write(memoryFile.getFileContent().getBytes());
            zipOutputStream.closeEntry();
        }
    } finally {
        zipOutputStream.close();
    }
    return byteArrayOutputStream.toByteArray();
}

ZipMemoryModel:

@Data

@AllArgsConstructor public class ZipMemoryModel {

private String fileName;
private String filePath;
private String fileContent;

}

Invocation with dummy data:

    public byte[] generateZip() throws IOException {
    return zipUtils.compressInMemoryToZip(List.of(new ZipMemoryModel("fileName.ending", "path/", "content")));
}