Small question, but high priority in my situation. How shuffle is apply with tf.data.experimental.make_csv_dataset. For exemple, I want to load 20000 .csv (each is data of a different city) in my dataset:
dataset = tf.data.experimental.make_csv_dataset(
file_pattern='*.csv',
num_parallel_reads=4,
batch_size=512,
num_epochs=10,
label_name='streamflow',
select_columns=keep_columns,
shuffle_buffer_size=10000,
header=True,
field_delim=','
)
1- shuffle the orther of read of the csv list only
2- shuffle the orther of sample inside each .csv read
3- shuffle all sample of all csv like it was an single big csv
Thank for your time