473,327 Members | 2,094 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,327 software developers and data experts.

Working with large txt files using streams

hi all,

I am getting an error when i process 1 MB txt file using StreamReader and i
am reading char by char using streamReader.Read() method at some point it is
throwing an exception System.ComponentModel.win32Exception:The handle is
invalid.
Any help for this problem or any kind of informaiton to process large files
the file size can be 10 MB.

thanks in advance
Syed

Nov 17 '05 #1
6 1872
Hi Syed,

Just a thought, is the text file UTF8 encoded? If not, specify another encoding when creating the StreamReader.
On Sat, 16 Jul 2005 10:05:02 +0200, syed javid"" <sy*******@discussions.microsoft.com> wrote:
hi all,

I am getting an error when i process 1 MB txt file using StreamReader and i
am reading char by char using streamReader.Read() method at some point it is
throwing an exception System.ComponentModel.win32Exception:The handle is
invalid.
Any help for this problem or any kind of informaiton to process large files
the file size can be 10 MB.

thanks in advance
Syed



--
Happy coding!
Morten Wennevik [C# MVP]
Nov 17 '05 #2
Hi Morten,

The text file is having arabic data, but i tested the streamReader with
UTF8 encoding and still the same error occurs.

Thanks
"Morten Wennevik" wrote:
Hi Syed,

Just a thought, is the text file UTF8 encoded? If not, specify another encoding when creating the StreamReader.
On Sat, 16 Jul 2005 10:05:02 +0200, syed javid"" <sy*******@discussions.microsoft.com> wrote:
hi all,

I am getting an error when i process 1 MB txt file using StreamReader and i
am reading char by char using streamReader.Read() method at some point it is
throwing an exception System.ComponentModel.win32Exception:The handle is
invalid.
Any help for this problem or any kind of informaiton to process large files
the file size can be 10 MB.

thanks in advance
Syed



--
Happy coding!
Morten Wennevik [C# MVP]

Nov 17 '05 #3
Hi Syed,

If you don't specify an encoding when creating the StreamReader, it will try to decode the file as UTF8. Try specifying Encoding.Unicode or Encoding.Default. If both fail I'm afraid I'm out of ideas and you might want to start a new thread with your question.
On Sun, 17 Jul 2005 07:56:01 +0200, syed javid"" <sy*******@discussions.microsoft.com> wrote:
Hi Morten,

The text file is having arabic data, but i tested the streamReader with
UTF8 encoding and still the same error occurs.

Thanks
"Morten Wennevik" wrote:
Hi Syed,

Just a thought, is the text file UTF8 encoded? If not, specify another encoding when creating the StreamReader.
On Sat, 16 Jul 2005 10:05:02 +0200, syed javid"" <sy*******@discussions.microsoft.com> wrote:
> hi all,
>
> I am getting an error when i process 1 MB txt file using StreamReader and i
> am reading char by char using streamReader.Read() method at some point it is
> throwing an exception System.ComponentModel.win32Exception:The handle is
> invalid.
> Any help for this problem or any kind of informaiton to process large files
> the file size can be 10 MB.
>
> thanks in advance
> Syed
>
>
>
>
>
>


--
Happy coding!
Morten Wennevik [C# MVP]


--
Happy coding!
Morten Wennevik [C# MVP]
Nov 17 '05 #4
hmm. 1MB is not large, so don't think that is the issue. Could you post a
small but complete code sample that shows the error?

--
William Stacey [MVP]

"syed javid" <sy*******@discussions.microsoft.com> wrote in message
news:53**********************************@microsof t.com...
hi all,

I am getting an error when i process 1 MB txt file using StreamReader and
i
am reading char by char using streamReader.Read() method at some point it
is
throwing an exception System.ComponentModel.win32Exception:The handle is
invalid.
Any help for this problem or any kind of informaiton to process large
files
the file size can be 10 MB.

thanks in advance
Syed


Nov 17 '05 #5
Hi William ,
i am sending u the text file data and code to process the text data.
Note : All this processing is done in windows service and checked with all
the encodings (Default,UTF8,unicode). i really don't known where i am going
wrong.
------------------------------------------------------
V|173848|1|4001938833|14250316|dsfsdfs|sdfsdfsdfsd |1000174076|0|sdfsdfsdfsdf|6674544|2330310451|1426 0601|sdfsdf||TUR|3|1|||asdsd|1||4001938833|16-03-1425||||asdas|sdfsdf||sdfds|Umit|Muhriz||Cicekli|T UR|asdas|1982/11/22||1|asdasd|1|140946|2005/06/30|asdasd|2006/06/29|1|||233|Umit Muhriz Cicekli||||||
V|173848|1|4001938833|14250316|dsfsdfs|sdfsdfsdfsd |1000174076|0|sdfsdfsdfsdf|6674544|2330310451|1426 0601|sdfsdf||TUR|3|1|||asdsd|1||4001938833|16-03-1425||||asdas|sdfsdf||sdfds|Umit|Muhriz||Cicekli|T UR|asdas|1982/11/22||1|asdasd|1|140946|2005/06/30|asdasd|2006/06/29|1|||233|Umit Muhriz Cicekli||||||
V|173848|1|4001938833|14250316|dsfsdfs|sdfsdfsdfsd |1000174076|0|sdfsdfsdfsdf|6674544|2330310451|1426 0601|sdfsdf||TUR|3|1|||asdsd|1||4001938833|16-03-1425||||asdas|sdfsdf||sdfds|Umit|Muhriz||Cicekli|T UR|asdas|1982/11/22||1|asdasd|1|140946|2005/06/30|asdasd|2006/06/29|1|||233|Umit Muhriz Cicekli||||||
V|173848|1|4001938833|14250316|dsfsdfs|sdfsdfsdfsd |1000174076|0|sdfsdfsdfsdf|6674544|2330310451|1426 0601|sdfsdf||TUR|3|1|||asdsd|1||4001938833|16-03-1425||||asdas|sdfsdf||sdfds|Umit|Muhriz||Cicekli|T UR|asdas|1982/11/22||1|asdasd|1|140946|2005/06/30|asdasd|2006/06/29|1|||233|Umit Muhriz Cicekli||||||
V|173848|1|4001938833|14250316|dsfsdfs|sdfsdfsdfsd |1000174076|0|sdfsdfsdfsdf|6674544|2330310451|1426 0601|sdfsdf||TUR|3|1|||asdsd|1||4001938833|16-03-1425||||asdas|sdfsdf||sdfds|Umit|Muhriz||Cicekli|T UR|asdas|1982/11/22||1|asdasd|1|140946|2005/06/30|asdasd|2006/06/29|1|||233|Umit Muhriz Cicekli|||||||

-------------------------------------------------------
// Reading the text file using stream reader
//Open the file using stream reader

ObjStreamReader = new StreamReader(e.FullPath,System.Text.Encoding.UTF8) ;

char c = ' ';
while (ObjStreamReader.Peek() > 0)
{
// Reading the line char by char
c = (char) ObjStreamReader.Read();
if (c != '\n')
{
strLine += c;
}
else
{
// building the Xml by splitting the strLine and
sending to the queue.
}
//Going to next line
} // end of while loop

thanks
syed

"William Stacey [MVP]" wrote:
hmm. 1MB is not large, so don't think that is the issue. Could you post a
small but complete code sample that shows the error?

--
William Stacey [MVP]

"syed javid" <sy*******@discussions.microsoft.com> wrote in message
news:53**********************************@microsof t.com...
hi all,

I am getting an error when i process 1 MB txt file using StreamReader and
i
am reading char by char using streamReader.Read() method at some point it
is
throwing an exception System.ComponentModel.win32Exception:The handle is
invalid.
Any help for this problem or any kind of informaiton to process large
files
the file size can be 10 MB.

thanks in advance
Syed



Nov 17 '05 #6
syed javid <sy*******@discussions.microsoft.com> wrote:
i am sending u the text file data and code to process the text data.
Note : All this processing is done in windows service and checked with all
the encodings (Default,UTF8,unicode). i really don't known where i am going
wrong.


Could you post a short but complete program which demonstrates the
problem?

See http://www.pobox.com/~skeet/csharp/complete.html for details of
what I mean by that.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 17 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

12
by: Martin Dieringer | last post by:
I am trying to split a file by a fixed string. The file is too large to just read it into a string and split this. I could probably use a lexer but there maybe anything more simple? thanks m.
6
by: Gaurav | last post by:
Hello, I am using visual c++ 6 and i am having problems with string to work. ******** Here is the program project.cpp********* #include <iostream.h> #include <string> #include "stdafx.h"
4
by: Dag Sunde | last post by:
Just wondering if anyone have looked into this? How to split up ones JavaScript library? A lot of very specific (and small) .js files, or a few larger files. I'm thinking about load-time...
57
by: Bing Wu | last post by:
Hi all, I am running a database containing large datasets: frames: 20 thousand rows, coordinates: 170 million row. The database has been implemented with: IBM DB2 v8.1
5
by: David | last post by:
Hi all: I am processing a 3D bitmaps(essentially ~1024 2D bitmaps with a size of 1MB each). If I want read large amount of radom data from this series, how could I buffer the file to get...
2
by: chis2k | last post by:
I need help figuring out how to get the MD5 hash of large files. I cannot go about the traditional way of reading the entire file into a byte array, because then the huge file is placed in memory,...
0
by: Ramin Dalkouhi | last post by:
Hello, We are currently working on a serialization task which imports data into memory streams. At first, we had problems with a very high amount of data as the memory management of such memory...
1
by: AWHK | last post by:
Hi, I have a customer which want to upload LARGE files via a web-service and stream it into a database while it is uploading. I don't want to go out of my way and develop a large HTTP App....
0
by: satish | last post by:
Hi, I have an issue ..i have a text which has the information of 4 test files and they are placed in the order 4 --- this number says (total no of text files) File1 1370 -- Memory of the text...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.