Fast way to read and merge(concat) multiple text files into a new file in c++

84 Views Asked by At

My problem

  • I have multiple text files(depends on user), can be 100 or 1000
  • Can be of variable size 1kb to 100mb
  • I want to read all of them and merge them into a new file after some formatting.
  • dont worry about formatting part

my code works fine for small problems but app crashes at 200-300mb files(can be multiple of these), how can i make this work. the answer can be in multiple range for eg algo 1 for small file size and another for larger, im open to suggestions

This function reads a file

string lance::getFileContents(string path) {
    ifstream fs(path);
    fs.seekg(0, std::ios::end);
    size_t size = fs.tellg();
    string buffer(size, ' ');
    fs.seekg(0);
    fs.read(&buffer[0], size);
    fs.close();
    return buffer.c_str();
}

now i am iterating over each file and storing its text in "out" variable

string out = "";
for (auto& chapterPath : filePath) {
    out += lance::getFileContents(chapterPath);
}

i then write it to a file using ofstream

im new to c++ so i appologise for any bad practice.

0

There are 0 best solutions below