Process File in Chunks - StreamReader

[das]

I posted this on ado.net, which seems like the wrong group - so here it
goes!

Hi All,
I want to process a file in chunks, meaning I have a big file of
50K rows. I want to process 5K rows at a time.

I am using StreamReader object and looping through each row and process

it, and as I process it I create an XML file and send it to DB to be
processed.
while(oSR.Peek( ) > -1)
{
// 1. Create Xml from these lines
}
// 2. Call DB Functions.

How do I read only certain number of lines at a time, complete my steps

1 and 2 and come back and read another 5K lines and so on....
any ideas gurus?
thanks

[Peter Bromberg [C# MVP]]

This is kind of hokey, and untested, but should give the idea:
StringBuilder sb=null;
while(oSR.Peek( ) > -1)
{
sb=new StringBuilder() ;
for(int i=0;i<5000;i++)
{
if(oSr.Peek()>0-1)
sb.Append(oSR.R eadLine());

}
ProcessMyXml(sb .ToString();)

}

-- where ProcessMyXml accepts the string containing the 5000 lines,
you can Split ('\r') on the newline to get into a string[] array for
processing.
and Split again on each row if it has delimited info in it.

Peter
--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"das" wrote:

I posted this on ado.net, which seems like the wrong group - so here it
goes!

Hi All,
I want to process a file in chunks, meaning I have a big file of
50K rows. I want to process 5K rows at a time.

I am using StreamReader object and looping through each row and process

it, and as I process it I create an XML file and send it to DB to be
processed.
while(oSR.Peek( ) > -1)
{
// 1. Create Xml from these lines
}
// 2. Call DB Functions.

How do I read only certain number of lines at a time, complete my steps

1 and 2 and come back and read another 5K lines and so on....
any ideas gurus?
thanks

[das]

thank you, I will test this and reply back.