Efficiency of parsing a string in C vs C++

[Benny the Guard]

I have a comma delimited file to parse up. So I figured I will go line by line and parse them up. Now efficiency is the top priority here. So I figure I can use the old-school C approach or use std::string. The way I see it I could (assuming infile is a valid std::ifstream)

char buffer [4096];
while (!infile.eof () && !infile.bad ())
    {
    // Get the next line in the file    
    infile.getline (buffer, 4095);
 
    // If fail bit is set it means we did not read the entire
    // line in. So we need to skip it, but the rest of the line is
    // still on the stream. So clear the error and contnue reading
    // until the end of the line, skipping it all.
    if (infile.fail ())
        {
        while (infile.fail ())
            {
            infile.clear ();
            infile.getline (buffer, 4095);
            }
        continue;
        }
 
    // If the string is empty or all whitepace skip it.
    char *str = buffer;
    while (*str != '\0' && isspace (*str))
        str++;
 
    if (strlen (str) == 0)
        continue;
 
    char *tokstr = strtok (str, ",");
    while (tokstr)
        {
        // Do something with the data
        tokstr = strtok (NULL, ",");
        }
    }
 

or

string buffer;
while (!infile.eof () && !infile.bad ())
    {
    getline (infile, buffer);
 
    // Don't worry about fail bit, string is dynamic and can read all lines
 
    string::iterator stritr = buffer.begin ();
    while (isspace (*stritr))
        stritr++;
 
    if (stritr == buffer.end ())
        continue;
    if (stritr != buffer.begin ())
        buffer.erase (buffer.begin (), stritr)
 
    int startpos = 0;
    int endpos = buffer.find_first_of (",");
    while (startpos != npos)
        {
        valuestr = buffer.substr (startpos, endpos);
        // Do something with the data
 
        startpos = endpos + 1;
        endpos = buffer.find_first_of (",", startpos);
        }
    }

The string version is a little nicer on memory since there is no real cap on the line size whereas the first approach caps it at 4095 and requires a bit more logic to deal with longer lines (not to mention not importing them). However, I have a strong feeling the first one is much faster. Any thoughts on which is faster? Any optimizations you see?

[weaknessforcats]

The STL templates are faster than the C functions.

P J Plauger, who wrote the STL templates says they were optimized for speed and that If you think you can do this faster, then think three times.

Operations on STL containers with millions of strings are much faster than C code.

[Benny the Guard]

Cool. My concern of speed was in the line:

valuestr = buffer.substr (startpos, endpos);

Simply because its allocatin/dealloc temporary memory. But the other speed enhancements probably outweigh it.

[ravenspoint]

Perhaps you can avoid the copy - it depends on what "do something with the data" involves.

Let's suppose you want to covert it to an integer

// replace comma with null

buffer[ endpos ] = '\0';

// convert

int i = atoi( (const char*) &buffer[ startpos ] );

All very tricky, of course, and of dubious value. Are you SURE this is the bottleneck? Have you run a profile?

James