Split ifstream in n streams?

489 Views Asked by At

i have problems with an ifstream. I want to split the ifstream in n parts.

For example n = 3:

  1. Ifstream includes first 1/3 of the file.
  2. Ifstream includes second 1/3 of the file.
  3. Ifstream includes third 1/3 of the file.
    std::ifstream in("test.txt");
    std::vector<std::string> v1;
    std::vector<std::string> v2;
    std::vector<std::string> v3;

    //first 1/3 of file
    read(in, v1);
    //second 1/3 of file
    read(in, v2);
    //third 1/3 of file
    read(in, v3);

    read(in, v){

         std::string line {""};
         while(getline(in, line)){
              v.pushback(line);
         }
    }

2

There are 2 best solutions below

0
Mandy007 On

You can read and push all lines in a vector and then split the vector in 3 part, for example:

std::string s;
while(!in.eof() && getline(in, s)) v1.push_back(s);
int size = v1.size(), itv2 = 0, itv3 = 0, chunk = v1.size()/3;
for(unsigned i = size-1; i >= size/3; --i, v1.pop_back()) 
    (i > chunk*2)? v3[chunk-itv3++] = v1[i] : v2[chunk-itv2++] = v1[i]; 

And well now if you wanna do this for n partitions you can do something like that:

//n must be defined before use
std::vector<std::vector<std::string> > vChunks(n+1);
std::vector<std::string> v;
std::string s;
while(!in.eof() && getline(in, s)) v.push_back(s);
int size = v.size(), chunk = v.size()/n, r = v.size()%n;
vChunks[n].resize(r);
for(int i = 0; i < n; i++)
    vChunks[i].resize(chunk);
for(int i = v.size()-1, it =1; it <= r; it++, --i, v.pop_back())
    vChunks[n][r-it] = v[i];
for(int i = v.size()-1; i >= 0; --i, v.pop_back())
    vChunks[(i%chunk == 0)? (i-1)/chunk : i/chunk][i%chunk] = v[i];

Where vChunks the first n partitions have the number of lines between n dimensions and in n + 1 has dimension the rest of the last lines if it is not divisible by n the total number of lines

0
Ben Voigt On

@Mandy007 showed you a simple way to do it, by pre-reading all the content into memory.

The "clean" way to do it would be to define a streambuf-derived class that delegates read requests through to the underlying istream but manipulates the seek position and end-of-file indication to make it look like the region of the file is a complete stream.

This is how customization works in the iostream library... the stream classes themselves are not polymorphic, all behavior comes from the streambuf instance.