How to use istream to read formatted input with empty field

523 Views Asked by At

I would like to read a file with specified format with std::istream or fscanf().

Each line of the file is consist of several fields. The fields may be a char, a float or a integer. Each field has fixed width and may be empty. Is there a way to stop std::istream from ignoring the empty fields?

=================================================================

The following is the detailed description.

I am writing an program parsing pdb-style files. Part of the format follows the following format: Record Format

COLUMNS        DATA  TYPE    FIELD        DEFINITION
------------------------------------------------------------------------------------
 1 -  6        Record name   "ATOM  "
 7 - 11        Integer       serial       Atom  serial number.
13 - 16        Atom          name         Atom name.
17             Character     altLoc       Alternate location indicator.
18 - 20        Residue name  resName      Residue name.
22             Character     chainID      Chain identifier.
23 - 26        Integer       resSeq       Residue sequence number.
27             AChar         iCode        Code for insertion of residues.
31 - 38        Real(8.3)     x            Orthogonal coordinates for X in Angstroms.
39 - 46        Real(8.3)     y            Orthogonal coordinates for Y in Angstroms.
47 - 54        Real(8.3)     z            Orthogonal coordinates for Z in Angstroms.
55 - 60        Real(6.2)     occupancy    Occupancy.
61 - 66        Real(6.2)     tempFactor   Temperature  factor.
77 - 78        LString(2)    element      Element symbol, right-justified.
79 - 80        LString(2)    charge       Charge  on the atom.

And here is part of the actual input:

ATOM      1  N   MET A   0      24.512   8.259  -9.688  1.00 33.83           N  
ATOM      2  CA  MET A   0      24.523   9.740  -9.865  1.00 32.90           C  
ATOM      3  C   MET A   0      25.889  10.228 -10.330  1.00 31.90           C  
ATOM      4  O   MET A   0      26.886   9.516 -10.198  1.00 32.07           O  
ATOM      5  CB  MET A   0      24.143  10.414  -8.560  1.00 34.34           C  
ATOM      6  CG  MET A   0      24.891   9.880  -7.378  1.00 35.66           C  
ATOM      7  SD  MET A   0      24.111  10.428  -5.871  1.00 38.66           S  
ATOM      8  CE  MET A   0      24.454  12.221  -5.988  1.00 36.36           C  
ATOM      9  N   VAL A   1      25.922  11.435 -10.891  1.00 30.10           N  
ATOM     10  CA  VAL A   1      27.161  12.020 -11.393  1.00 27.92           C  
ATOM     11  C   VAL A   1      27.260  13.522 -11.114  1.00 26.21           C  
ATOM     12  O   VAL A   1      26.304  14.278 -11.304  1.00 26.54           O  
ATOM     13  CB  VAL A   1      27.312  11.769 -12.919  1.00 27.99           C  
ATOM     14  CG1 VAL A   1      28.557  12.455 -13.466  1.00 27.68           C  
ATOM     15  CG2 VAL A   1      27.395  10.282 -13.189  1.00 28.05           C  

As you may notice, some of the fields are empty. For example, altLoc(alternative location) at column 17 is optional, and charge at column 79-80 is usually missing.

Sometimes the fields are not separated, as the name, altLoc and resName field may form something like CG1AVAL, which is actually CG1, A and VAL.

I am trying to implement the program with C++. I tried both operator>> and fscanf but failed to find a solution to read the input into a struct Atom.

struct Atom
{
    std::string recordName{ 6 };
    int serial;
    std::string name{ 4 };
    char altLoc;
    std::string resName{ 3 };
    char chainID;
    int resSeq;
    char iCode;
    double x;
    double y;
    double z;
    double occupancy;
    double tempFactor;
    char segment;
    char element;
    char charge;
};

setw(size_t n) does not work as I expected, and since I have to dual with 41.9GB of input, performance is important, so I prefer not to add too much overhead like getline() the input in a string and then parse that string. Here is my failed attempt:

ChainReader & ChainReader::operator>>(Atom & atom)
{
    *stream //stream is a pointer to stream, which is a member of ChainReader
        >> std::setw(4) >> atom.recordName
        >> std::setw(7) >> atom.serial
        >> std::setw(5) >> atom.name
        >> std::setw(1) >> atom.altLoc
        >> std::setw(3) >> atom.resName
        >> std::setw(2) >> atom.chainID
        >> std::setw(4) >> atom.resSeq
        >> std::setw(1) >> atom.iCode
        >> std::setw(11) >> atom.x
        >> std::setw(8) >> atom.y
        >> std::setw(8) >> atom.z
        >> std::setw(6) >> atom.occupancy
        >> std::setw(6) >> atom.tempFactor
        >> std::setw(10) >> atom.segment
        >> std::setw(2) >> atom.element
        >> std::setw(2) >> atom.charge;
    return *this;
}

Update:

Come up with a solution with getline and string parsing. Works, still testing for performance.

ChainReader & ChainReader::operator>>(Atom & atom)
{
    std::string line;
    line.reserve(80);
    std::getline(*stream, line);
    atom.recordName = line.substr(0, 4);
    atom.serial = std::stoi(line.substr(6, 5));
    atom.name = line.substr(12, 4);
    atom.altLoc = line[16];
    atom.resName = line.substr(17, 3);
    atom.chainID = line[21];
    atom.resSeq = std::stoi(line.substr(22, 4));
    atom.iCode = line[26];
    atom.x = std::stod(line.substr(30, 8));
    atom.y = std::stod(line.substr(38, 8));
    atom.z = std::stod(line.substr(46, 8));
    atom.occupancy = std::stod(line.substr(54, 6));
    atom.tempFactor = std::stod(line.substr(60, 6));
    atom.segment = line[75];
    atom.element = line[77];
    atom.charge = line[79];
    return *this;
}

Update 2: The program reads 276231 chains in ASTRAL data set and calculates dihedral angles along the chains. Takes totally 10237134 ms, or 3 hours. The data set is about 42GB, so the performance is totally acceptable.

0

There are 0 best solutions below