473,320 Members | 1,724 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Text parser

I am trying to write a parser for textfiles that can be very, very
large. What would be the best solution to do this? Would it be better to
store the entire textfile into an array and work with the array, or
would it be better to just loop through the file using the StreamReader?

I'm not experienced with VB.NET at all, but have extensive knowledge of
VBA. I already got one of my applications ported from VBA to VB.NET and
this works (almost) fine, but the next project requires me to read very
large textfiles. (They can become >100MB.)

The idea behind the file is to split it into .csv-files for various
sections contained in the master file. I also want to create a summary
textfile containing maximums, minimums and various averages/calculations.

*************

Another question: I've created and build a test-application, but when I
try to run this program on a network-path, the application crashes with
the following error:

EventType : clr20r3
P1: test.exe P2: 1.0.0.0 P3: 461e3ed7
P4: test P5: 1.0.0.0 P6: 461e3ed7
P7: d P8: c6 P9: system.invalidoperationexception

I think this has something to do with the way Visual Basic 2005 Express'
ClickOnce works, and that I'm not using the correct settings. These are
my settings on the Security-tab of the Project, with which my
application fails to run from network-paths:

Enabled "Enable ClickOnce Security Settings"-checkbox
Selected "This is a full trust application"-radiobutton

Is there a way to set my application so that it will work from network
paths as well as local drives?

(The working folder for Visual Basic 2005 Express is also located on a
network location. Can this also be a part of the problem?)

Regards, CoRrRan
Apr 16 '07 #1
4 3211
CoRrRan,

If you can use CSV files you can use SQL SeverExpress in my idea as well.

http://msdn.microsoft.com/vstudio/express/sql/

CorNotRunning

"CoRrRan" <CoRrRan~NOSPAM~@~NOSPAM~gmail.comschreef in bericht
news:46***********************@dreader21.news.xs4a ll.nl...
>I am trying to write a parser for textfiles that can be very, very large.
What would be the best solution to do this? Would it be better to
store the entire textfile into an array and work with the array, or
would it be better to just loop through the file using the StreamReader?

I'm not experienced with VB.NET at all, but have extensive knowledge of
VBA. I already got one of my applications ported from VBA to VB.NET and
this works (almost) fine, but the next project requires me to read very
large textfiles. (They can become >100MB.)

The idea behind the file is to split it into .csv-files for various
sections contained in the master file. I also want to create a summary
textfile containing maximums, minimums and various averages/calculations.

*************

Another question: I've created and build a test-application, but when I
try to run this program on a network-path, the application crashes with
the following error:

EventType : clr20r3
P1: test.exe P2: 1.0.0.0 P3: 461e3ed7
P4: test P5: 1.0.0.0 P6: 461e3ed7
P7: d P8: c6 P9: system.invalidoperationexception

I think this has something to do with the way Visual Basic 2005 Express'
ClickOnce works, and that I'm not using the correct settings. These are my
settings on the Security-tab of the Project, with which my application
fails to run from network-paths:

Enabled "Enable ClickOnce Security Settings"-checkbox
Selected "This is a full trust application"-radiobutton

Is there a way to set my application so that it will work from network
paths as well as local drives?

(The working folder for Visual Basic 2005 Express is also located on a
network location. Can this also be a part of the problem?)

Regards, CoRrRan

Apr 16 '07 #2
here is a nice class i have used:

An enumerable class to read large text files in a For Each loop:

http://www.dotnet2themax.com/ShowCon...7-56bba526534f

some interesting comments between the code snippets, a 2.0 version at the
bottom


I am trying to write a parser for textfiles that can be very, very
large. What would be the best solution to do this? Would it be better
to store the entire textfile into an array and work with the array, or
would it be better to just loop through the file using the
StreamReader?

I'm not experienced with VB.NET at all, but have extensive knowledge
of VBA. I already got one of my applications ported from VBA to VB.NET
and this works (almost) fine, but the next project requires me to read
very large textfiles. (They can become >100MB.)

The idea behind the file is to split it into .csv-files for various
sections contained in the master file. I also want to create a summary
textfile containing maximums, minimums and various
averages/calculations.

*************

Another question: I've created and build a test-application, but when
I try to run this program on a network-path, the application crashes
with the following error:

EventType : clr20r3
P1: test.exe P2: 1.0.0.0 P3: 461e3ed7
P4: test P5: 1.0.0.0 P6: 461e3ed7
P7: d P8: c6 P9: system.invalidoperationexception
I think this has something to do with the way Visual Basic 2005
Express' ClickOnce works, and that I'm not using the correct settings.
These are my settings on the Security-tab of the Project, with which
my application fails to run from network-paths:

Enabled "Enable ClickOnce Security Settings"-checkbox Selected "This
is a full trust application"-radiobutton

Is there a way to set my application so that it will work from network
paths as well as local drives?

(The working folder for Visual Basic 2005 Express is also located on a
network location. Can this also be a part of the problem?)

Regards, CoRrRan

Apr 16 '07 #3
Hmm, I think you're right that putting it all in a database would be
better, but unfortunately I have got zero knowledge on database
design/implementation and just starting to get a bit comfortable with
programming in VB 2005. I guess I'll have to get myself up to speed with
databases too one day.

Thanks for answering Cor.

CoRrRan

Cor Ligthert [MVP] wrote:
CoRrRan,

If you can use CSV files you can use SQL SeverExpress in my idea as well.

http://msdn.microsoft.com/vstudio/express/sql/

CorNotRunning

"CoRrRan" <CoRrRan~NOSPAM~@~NOSPAM~gmail.comschreef in bericht
news:46***********************@dreader21.news.xs4a ll.nl...
>I am trying to write a parser for textfiles that can be very, very large.
What would be the best solution to do this? Would it be better to
store the entire textfile into an array and work with the array, or
would it be better to just loop through the file using the StreamReader?

I'm not experienced with VB.NET at all, but have extensive knowledge of
VBA. I already got one of my applications ported from VBA to VB.NET and
this works (almost) fine, but the next project requires me to read very
large textfiles. (They can become >100MB.)

The idea behind the file is to split it into .csv-files for various
sections contained in the master file. I also want to create a summary
textfile containing maximums, minimums and various averages/calculations.

*************

Another question: I've created and build a test-application, but when I
try to run this program on a network-path, the application crashes with
the following error:

EventType : clr20r3
P1: test.exe P2: 1.0.0.0 P3: 461e3ed7
P4: test P5: 1.0.0.0 P6: 461e3ed7
P7: d P8: c6 P9: system.invalidoperationexception

I think this has something to do with the way Visual Basic 2005 Express'
ClickOnce works, and that I'm not using the correct settings. These are my
settings on the Security-tab of the Project, with which my application
fails to run from network-paths:

Enabled "Enable ClickOnce Security Settings"-checkbox
Selected "This is a full trust application"-radiobutton

Is there a way to set my application so that it will work from network
paths as well as local drives?

(The working folder for Visual Basic 2005 Express is also located on a
network location. Can this also be a part of the problem?)

Regards, CoRrRan

Apr 17 '07 #4
Ah, this looks indeed promising. Thanks for pointing me in that
direction Jay!

Regards, CoRrRan

Jay Parzych wrote:
here is a nice class i have used:

An enumerable class to read large text files in a For Each loop:

http://www.dotnet2themax.com/ShowCon...7-56bba526534f
some interesting comments between the code snippets, a 2.0 version at
the bottom


>I am trying to write a parser for textfiles that can be very, very
large. What would be the best solution to do this? Would it be better
to store the entire textfile into an array and work with the array, or
would it be better to just loop through the file using the
StreamReader?

I'm not experienced with VB.NET at all, but have extensive knowledge
of VBA. I already got one of my applications ported from VBA to VB.NET
and this works (almost) fine, but the next project requires me to read
very large textfiles. (They can become >100MB.)

The idea behind the file is to split it into .csv-files for various
sections contained in the master file. I also want to create a summary
textfile containing maximums, minimums and various
averages/calculations.

*************

Another question: I've created and build a test-application, but when
I try to run this program on a network-path, the application crashes
with the following error:

EventType : clr20r3
P1: test.exe P2: 1.0.0.0 P3: 461e3ed7
P4: test P5: 1.0.0.0 P6: 461e3ed7
P7: d P8: c6 P9: system.invalidoperationexception
I think this has something to do with the way Visual Basic 2005
Express' ClickOnce works, and that I'm not using the correct settings.
These are my settings on the Security-tab of the Project, with which
my application fails to run from network-paths:

Enabled "Enable ClickOnce Security Settings"-checkbox Selected "This
is a full trust application"-radiobutton

Is there a way to set my application so that it will work from network
paths as well as local drives?

(The working folder for Visual Basic 2005 Express is also located on a
network location. Can this also be a part of the problem?)

Regards, CoRrRan

Apr 17 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Nebojsa Topolscak | last post by:
Hello, I wrote application which doesn't use graphic at all. One part of application needs Parser class from javax.swing.text.html.HTMLEditorKit. The ridiculous fact is when I try to...
0
by: Himanshu Garg | last post by:
Hello, I am using HTML::Parser to extract text from html pages from http://bbc.co.uk/urdu/ However the encoding of the input text seems to change to some unknown encoding in the output. The...
27
by: Eric | last post by:
Assume that disk space is not an issue (the files will be small < 5k in general for the purpose of storing preferences) Assume that transportation to another OS may never occur. Are there...
1
by: Eileene Cordoves | last post by:
hi i'm a newbie in xml and we're using org.apache.xerces.parsers.SAXParser. anyone know what the invalid characters in xml are? one of the value in the parsed xml is '<space><space>1', we...
1
by: google | last post by:
It would seem that when I assign an HTML entity to a form text input using "inline" javascript that it will display properly. But when trying to set it via a function call, the entity text shows...
2
by: Mattias Thuresson | last post by:
I want to convert a text based protocol into XML, using C#, how do I do that in the best maner ? Can I use XSLT in any way ? or do I have to make an parser that converts the file manually ? An...
3
by: matofarides | last post by:
Hello all! I was wondering whether there was a way, using any XML parser (preferably xerces), to create an XML file by reading, line by line a text file. Thank you, Marios Tofarides
9
by: Alex Buell | last post by:
I have a small text file which consist of the following data: ]] And the code I've written is as follows: ]] The trouble is, I can't work out why it goes into an infinite loop reading the...
2
by: David Virgil Hobbs | last post by:
Loading text strings containing HTML code into an HTML parser in a Javascript/Jscript I would like to know, how one would go about loading a text string containing HTML code, so as to be able to...
13
by: sonald | last post by:
Hi, Can anybody tell me how to change the text delimiter in FastCSV Parser ? By default the text delimiter is double quotes(") I want to change it to anything else... say a pipe (|).. can anyone...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.