I am trying to write a huge output file the quickest possible on two different files. I have created my lists list1 and list2, and I am now trying to write those in a respective file.
I started to increase the buffer size by doing so:
rand_file = io.FileIO("test1.txt", 'w')
writer = io.BufferedWriter(rand_file,buffer_size=500000000)
for x in mylist1:
writer.write(x.encode())
writer.flush()
rand_file = io.FileIO("test2.txt", 'w')
writer = io.BufferedWriter(rand_file,buffer_size=500000000)
for x in mylist2:
writer.write(x.encode())
writer.flush()
but now I wanted to change those "for loops" with list comprehension but I cannot make it work.
if I do the following:
writer.write(x for x in mylist1)
I do have the following error "TypeError: a bytes-like object is required, not 'generator'". Note I also tried writer.write(x.encode() for x in mylist1), with the same result.
I then wanted to use the map function like so:
map(writer.write, mylist1)
This time, no error are raised but my output files are empty.
What am I doing wrong here ?