Download files as stream without downloading complete files into memory

Question

I want to download files as stream from a remote server and make them zip stream and return it to frontend as stream without waiting all files to complete downloads.

Is it possible in python and framework of python ?? I tried as below but its not working i am using Django framework:

 zip_buffer = io.BytesIO()

 with zipstream.ZipFile(zip_buffer,"w", zipfile.ZIP_DEFLATED, False) as zf:
     url = "https://download-file/zy"
     r = requests.get(url, stream=True)
     zip_inf = zipstream.ZipInfo(file_name)
     zf.write_iter(zip_inf, r.content)
response = StreamingHttpResponse(zip_buffer.getvalue(), content_type='application/octet-stream')
            response['Content-Disposition'] = 'attachment; filename=zip-files.zip'
            return response

Answer 1

I tried by using aiozipstream and fast api for streamingResponse. this solution worked for me.

 files = []
 urls = ['url1','ur2']
 for url in urls:
    files.append({'stream': file_content_generator(url), 'name': url.split('/')[-1]})
 zf = ZipStream(files, chunksize=32768)

 response = StreamingResponse(zf.stream(), media_type='application/zip')
    return response

def file_content_generator(url):
     with httpx.stream('GET', url, timeout=None) as r:
          yield from r.iter_bytes()