Right now, I am using fine-uploader as my front-end and python-flask as my backend. What I need to do is that I need to send large files from the front end to my server. I can do this easily without concurrent chunking. But once I turn it on, the file gets corrupted. I think this is because when you concurrently chunk, I would guess part 7 could get added before part 5 gets added. Again, non-concurrent chunking is OK since it is sequential.
I need to know if there is some sort of temporary, global variable or way that I can store the chunk parts temporarily. I tried redis but unfortunately when I get the data from the redis, it shows a decoding error because probably redis tries to turn it into a string when I just need it to be bytes.
If all else fails, I'll just go to my last resort, which is to put the parts into their own little files then open them later on to combine them one by one.
Here is my code for your reference. It still has the redis methods on it.
def upload_temp_file_part(redis, request):
try:
# Remember the paramName was set to 'file', we can use that here to grab it
file = request.files['qqfile']
uuid = request.form['qquuid']
part = request.form['qqpartindex']
offset = request.form['qqpartbyteoffset']
key_content = 'file_content_' + uuid + part
key_offset = 'file_offset_' + uuid + part
value_content = file.stream.read()
value_offset = offset
logging.info("Setting part " + part + " of " + uuid)
redis.set(key_content, value_content)
redis.set(key_offset, value_offset)
except Exception as e:
logging.error(e)
def combine_temp_file_part(redis, request):
try:
uuid = request.form['qquuid']
total_parts = request.form['qqtotalparts']
save_path = os.path.join(os.getenv('UPLOAD_FOLDER_TEMP'), uuid)
with open(save_path, 'ab') as f:
for index in range(0, int(total_parts)):
key_content = 'file_content_' + uuid + str(index)
key_offset = 'file_offset_' + uuid + str(index)
logging.info("Get part " + str(index) + " of " + uuid)
value_content = redis.get(key_content)
value_offset = redis.get(key_offset)
if value_content is None or value_offset is None:
pass
# Throw Error
logging.info("Placing part " + str(index) + " of " + uuid)
f.seek(value_offset)
f.write(value_content)
redis.delete(value_offset)
redis.delete(value_content)
except Exception as e:
logging.error(e)