Regular Expressions -- count lines with a specific pattern in a flat file

[Sathyaish]

I have a CSV file like so:

"HDR",200606291 33932,"9845","9 083","0010"
1,"3","00000000 0690","000007", "rsM4hJXR5Ik0O8 RWghjtDBlUVAOZq 7tO","BAR","001 0","","",20. 00
2,"3","00000000 0691","000007", "65Xbp5dMcDFflP JnxWCrsJtV1jzcU jgd","BAR","001 0","","",20. 00
3,"3","00000000 0692","000007", "SEjcf3eDA7hWmw GrNsLWoCWt1Geyh 4GN","BAR","001 0","","",20. 00
4,"3","00000000 0693","000007", "MJMkrp/kRMMGimeZo1uFOJ zeDTVeOkFU","BA R","0010","","" ,20.00
5,"3","00000000 0694","000007", "fDIBFgockQHhN+ eVQxEBqqrJfZ78r oja","BAR","001 0","","",20. 00
......and so on...

Each file has about a million records or more. Instead of iterating
through each line and counting line breaks, and ignoring header and
footer records and counting only data records, I thought of writing a
regex pattern for the same. Here's what I've written to count only data
records, i.e rows that start with a number followed by a comma and then
any othe text and ending with a line break.

numRecords = System.Text.Reg ularExpressions .Regex.Matches( ret,
"(?m)^[0-9]{1, 6}*$",
System.Text.Reg ularExpressions .RegexOptions.M ultiline).Count ;

I get a zero match collection count.

[Marc Gravell]

So using that form of .Matches means that you have to load the entire string
at once? Bet that's fast... ;-p Especially with the lookaround ...

However, it makes sense that it fails:

^[0-9]{1,6}*$

says newline, then "between 1 and 6 digits" "zero or more times" then end of
line, with nothing else; well the commas and quotes seem to get in the way?
Did you mean

^[0-9]{1-6},.*$

which is new line, "between 1 and 6 digits", comma, "zero-ormore chars
except newline", end of line

However, for performance I would still suggest using line by line,
stream-based, reading, and also re-using a single Regex instance (ideally
precompiled):

Regex re = new Regex("[0-9]{1-6},.*",RegexOpt ions.Compiled);
int count = 0;
using(StreamRea der reader = File.OpenText(p ath)) {
while(!reader.E ndOfStream) {
string line = reader.ReadLine ();
if(!string.IsNu llOrEmpty(line) && re.IsMatch(line ))
count++;
}
}

Marc

[Marc Gravell]

Sorry - typo by me: I meant {1,6} (as per your original example); likewise
"^[0-9]{1,6},.*$" in the example code - although given we don't care about
the rhs it may also work with just "^[0-9]{1,6},".

Marc