I must read a very heavy-weight text plain file (usually .txt
extension) )and replace a given character with another given character
in all text inside the file. My application was working pretty well
with this below shown code (code placed in a buttonclick event after
selecting the file in a normal OpenFileDialog):
---------------------------------
System.IO.StreamReader sr = new System.IO.StreamReader(@"C:\1.txt");
string all = sr.ReadToEnd();
sr.Close();
all = all.Replace(",",".");
System.IO.StreamWriter sw = new System.IO.StreamWriter(@"C:\1.txt",
false);
sw.Write(all);
sw.Flush();
sw.Close();
MessageBox.Show("Done!");
---------------------------------
The problem is that user now needs to read over 500MBytes .txt files.
Incredible and hellish but true!
User now runs the application and after selecting .txt file with
OpenFileDialog, application starts processing but inmediatelly crashes
or hangs, I don't remember.
I need a good performance way for reading so heavy .txt files. Reading
the file in different steps would do it? I mean, reading in arrays of
bytes or similar, with MemoryStream object..........
Any help will be greatly appreciated. 13 3915
<ml*******@gmail.com> wrote: [replacing characters in a 500 MB text file] [...] string all = sr.ReadToEnd();
This means you're reading the entire file into memory at once.
Instead, try having a StreamReader and a StreamWriter open at the same
time. Read a line from one file and write it to the other file, then
delete the first file when done and replace it with the second one.
You will only use the memory taken up by one line, rather than
slurping up 500 MB out of nowhere.
P.
Hi,
Below is a better solution:
string sourceFile = @"C:\1.txt";
string tempTarget = System.IO.Path.GetTempFileName();
StreamReader reader = new System.IO.StreamReader( sourceFile );
StreamWriter writer = new StreamWriter( tempTarget )
string line;
while ( (line = reader.ReadLine() )
writer.WriteLine( line.Replace( ",",".") )
reader.Close();
writer.Close();
File.Delete( source);
File.Copy( target, source);
cheers,
--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation
<ml*******@gmail.com> wrote in message
news:11**********************@f14g2000cwb.googlegr oups.com... I must read a very heavy-weight text plain file (usually .txt extension) )and replace a given character with another given character in all text inside the file. My application was working pretty well with this below shown code (code placed in a buttonclick event after selecting the file in a normal OpenFileDialog):
--------------------------------- System.IO.StreamReader sr = new System.IO.StreamReader(@"C:\1.txt"); string all = sr.ReadToEnd(); sr.Close(); all = all.Replace(",","."); System.IO.StreamWriter sw = new System.IO.StreamWriter(@"C:\1.txt", false); sw.Write(all); sw.Flush(); sw.Close(); MessageBox.Show("Done!"); ---------------------------------
The problem is that user now needs to read over 500MBytes .txt files. Incredible and hellish but true!
User now runs the application and after selecting .txt file with OpenFileDialog, application starts processing but inmediatelly crashes or hangs, I don't remember.
I need a good performance way for reading so heavy .txt files. Reading the file in different steps would do it? I mean, reading in arrays of bytes or similar, with MemoryStream object..........
Any help will be greatly appreciated.
He probably wants to do a File.Move instead of a File.Copy for that last
one.
"Ignacio Machin ( .NET/ C# MVP )" <ignacio.machin AT dot.state.fl.us> wrote
in message news:Og****************@TK2MSFTNGP12.phx.gbl... Hi,
Below is a better solution:
string sourceFile = @"C:\1.txt"; string tempTarget = System.IO.Path.GetTempFileName();
StreamReader reader = new System.IO.StreamReader( sourceFile ); StreamWriter writer = new StreamWriter( tempTarget )
string line; while ( (line = reader.ReadLine() ) writer.WriteLine( line.Replace( ",",".") ) reader.Close(); writer.Close(); File.Delete( source); File.Copy( target, source);
cheers,
-- Ignacio Machin, ignacio.machin AT dot.state.fl.us Florida Department Of Transportation
<ml*******@gmail.com> wrote in message news:11**********************@f14g2000cwb.googlegr oups.com...I must read a very heavy-weight text plain file (usually .txt extension) )and replace a given character with another given character in all text inside the file. My application was working pretty well with this below shown code (code placed in a buttonclick event after selecting the file in a normal OpenFileDialog):
--------------------------------- System.IO.StreamReader sr = new System.IO.StreamReader(@"C:\1.txt"); string all = sr.ReadToEnd(); sr.Close(); all = all.Replace(",","."); System.IO.StreamWriter sw = new System.IO.StreamWriter(@"C:\1.txt", false); sw.Write(all); sw.Flush(); sw.Close(); MessageBox.Show("Done!"); ---------------------------------
The problem is that user now needs to read over 500MBytes .txt files. Incredible and hellish but true!
User now runs the application and after selecting .txt file with OpenFileDialog, application starts processing but inmediatelly crashes or hangs, I don't remember.
I need a good performance way for reading so heavy .txt files. Reading the file in different steps would do it? I mean, reading in arrays of bytes or similar, with MemoryStream object..........
Any help will be greatly appreciated.
Hi,
Yes, you are right :)
cheers,
--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:2I********************@giganews.com... He probably wants to do a File.Move instead of a File.Copy for that last one.
"Ignacio Machin ( .NET/ C# MVP )" <ignacio.machin AT dot.state.fl.us> wrote in message news:Og****************@TK2MSFTNGP12.phx.gbl... Hi,
Below is a better solution:
string sourceFile = @"C:\1.txt"; string tempTarget = System.IO.Path.GetTempFileName();
StreamReader reader = new System.IO.StreamReader( sourceFile ); StreamWriter writer = new StreamWriter( tempTarget )
string line; while ( (line = reader.ReadLine() ) writer.WriteLine( line.Replace( ",",".") ) reader.Close(); writer.Close(); File.Delete( source); File.Copy( target, source);
cheers,
-- Ignacio Machin, ignacio.machin AT dot.state.fl.us Florida Department Of Transportation
<ml*******@gmail.com> wrote in message news:11**********************@f14g2000cwb.googlegr oups.com...I must read a very heavy-weight text plain file (usually .txt extension) )and replace a given character with another given character in all text inside the file. My application was working pretty well with this below shown code (code placed in a buttonclick event after selecting the file in a normal OpenFileDialog):
--------------------------------- System.IO.StreamReader sr = new System.IO.StreamReader(@"C:\1.txt"); string all = sr.ReadToEnd(); sr.Close(); all = all.Replace(",","."); System.IO.StreamWriter sw = new System.IO.StreamWriter(@"C:\1.txt", false); sw.Write(all); sw.Flush(); sw.Close(); MessageBox.Show("Done!"); ---------------------------------
The problem is that user now needs to read over 500MBytes .txt files. Incredible and hellish but true!
User now runs the application and after selecting .txt file with OpenFileDialog, application starts processing but inmediatelly crashes or hangs, I don't remember.
I need a good performance way for reading so heavy .txt files. Reading the file in different steps would do it? I mean, reading in arrays of bytes or similar, with MemoryStream object..........
Any help will be greatly appreciated.
Thanks guys, specially Ignacio.
I'll try next days but really seems to be a good solution. If I have
any more problems, I'll return to the post.
Thanks again.
Hi again, the "while" statement does not compile because the while
comparison must return true and we are doing an assigment statement.
ReadLine() returns string and must be bool. Besides, I cannot do
while (reader.ReadLine() != String.Empty) because the file can contain
blank lines and this way we will go out of the while loop.
Any other idea?
Thanks.
It should be
while ( (line.reader.ReadLine()) != null)
Another suggestion, however - rather than having manual calls to Close,
you should use "using" statements round both the reader and the writer.
That way they automatically get closed whether an exception occurs or
not.
Jon
Hi guys,
The correct code, perfectly working, although I haven't tried with 500
MBytes .txt files:
-----------------------------------------------------------------------
string sourceFile = textBox1.Text;
string tempTarget = System.IO.Path.GetTempFileName();
StreamReader reader = new System.IO.StreamReader(sourceFile);
StreamWriter writer = new StreamWriter(tempTarget);
string line = String.Empty;
while ( (line = reader.ReadLine()) != null)
{
writer.WriteLine(line.Replace(txt1.Text,txt2.Text) );
}
reader.Close();
writer.Close();
File.Delete(sourceFile);
File.Move(tempTarget,sourceFile);
----------------------------------------------------------------------------
I'm not able to do it with "using" clause :(
Thanks Jon.
<ml*******@gmail.com> wrote: Hi guys,
The correct code, perfectly working, although I haven't tried with 500 MBytes .txt files:
----------------------------------------------------------------------- string sourceFile = textBox1.Text; string tempTarget = System.IO.Path.GetTempFileName(); StreamReader reader = new System.IO.StreamReader(sourceFile); StreamWriter writer = new StreamWriter(tempTarget);
string line = String.Empty; while ( (line = reader.ReadLine()) != null) { writer.WriteLine(line.Replace(txt1.Text,txt2.Text) ); } reader.Close(); writer.Close(); File.Delete(sourceFile); File.Move(tempTarget,sourceFile); ----------------------------------------------------------------------------
I'm not able to do it with "using" clause :(
You should be able to:
using (StreamReader reader = new StreamReader (sourceFile))
{
using (StreamWriter writer = new StreamWriter (tempTarget))
{
string line;
while ( (line=reader.ReadLine()) != null)
{
writer.WriteLine (line.Replace(txt1.Text, txt2.text);
}
}
}
File.Delete(sourceFile);
File.Move(tempTarget,sourceFile);
--
Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Congratulations for this simple but really good snipplet of code Jon.
Nested "Using" structure seems to be very comfortable and as you said,
you don't have to worry about closing neither the reader nor the
writer.
Thanks very much.
Regards,
Sorry to dissapoint you but my programming style goes more towards
Jon's style. Code looks like much more estructured and clearer for me.
I think in the example you only are "saving" two brackets, don't you?
One of my own programming rules: "It's better to have a clear and
well-structured code (brackets and so on...) in spite of having a
little bit more lines of code".
Thanks anyway. I didn't know you couldn't nest two "using" loops
without any brackets between them.
Regards.
Lonifasiko <ml*******@gmail.com> wrote: Sorry to dissapoint you but my programming style goes more towards Jon's style. Code looks like much more estructured and clearer for me. I think in the example you only are "saving" two brackets, don't you?
It depends. You can easily get really heavily indented code with no
benefit in some situations - it's quite easy to end up with four nested
brackets without much in between.
I very rarely use the version with no extra brackets, but occasionally
it does improve readability. After all, you're treating the whole
contents as a single block.
--
Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
> Sorry to dissapoint you but my programming style goes more towards Jon's style. Code looks like much more estructured and clearer for me. I think in the example you only are "saving" two brackets, don't you?
Thats ok, no dissapointment. What ever make you more productive.
One of my own programming rules: "It's better to have a clear and well-structured code (brackets and so on...) in spite of having a little bit more lines of code".
I would disagree with that. It is more clear, IMO, to do it the other way
as you see right away where they come in scope and leave scope at the same
place. For two streams it probably makes no difference. But for 3 or 4 (or
more), using the method I showed can really make it cleaner IMO. Your not
just saving the brackets, but your eliminating all the unnessasary nesting
and indents which can get distracting. And if your stacking streams in a
pipeline (i.e. file to compression to encryption to networkstream), it
really looks good. The end result is the same, but naturally everyone has
different tastes.
--
William Stacey [MVP] This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: jamait |
last post by:
Hi all,
I m trying to read in a text file into a datatable...
Not sure on how to split up the information though, regex or substrings...?
sample:
Col1 Col2 ...
|
by: Neil Patel |
last post by:
I have a log file that puts the most recent record at the bottom of the file. Each line is delimited by a \r\n
Does anyone know how to seek to the end of the file and start reading backwards?
|
by: Sabin Finateanu |
last post by:
Hi I'm having problem reading a file from my program and I think it's from a
procedure I'm using but I don't see where I'm going wrong. Here is the code:
public bool AllowUsage()
{
...
|
by: haibhoang |
last post by:
I have a Windows Service that is trying to parse a large (> 1Gig) text
file. I am keep getting OutOfMemoryException exception. Here is the
code that's having problem:
using (StreamReader...
|
by: Gustaf |
last post by:
I'm converting a program I made in Python once to C#, and while I'm at
it, I want to do some performance improvements. The program takes a set
of mailbox files (in readable formats, like Eudora and...
|
by: leticia larrosa |
last post by:
Hi,
I try to read a file that have 8 bit character, but contain some
character
whose code is more than 128 (spanish character).
When I read this file using ASCII (Dim oRead As StreamReader =...
|
by: jeff M via .NET 247 |
last post by:
I'm still having problems reading EBCDIC files. Currently itlooks like the lower range (0 to 127) is working. I have triedthe following code pages 20284, 20924, 1140, 37, 500 and 20127.By working I...
|
by: Drew Berkemeyer |
last post by:
Hello,
I'm using the following code to read a text file in VB.NET.
Dim sr As StreamReader = File.OpenText(strFilePath)
Dim input As String = sr.ReadLine()
While Not input Is Nothing...
|
by: James Wong |
last post by:
Dear all,
I'm using StreamReader to read a text file containing BIG-5 data and found
that no matter which encoding method in StreamReader's construction
parameter, the BIG-5 contents become...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM).
In this month's session, the creator of the excellent VBE...
|
by: MeoLessi9 |
last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
|
by: DolphinDB |
last post by:
Tired of spending countless mintues downsampling your data? Look no further!
In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
|
by: Aftab Ahmad |
last post by:
Hello Experts!
I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
|
by: Aftab Ahmad |
last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below.
Dim IE As Object
Set IE =...
|
by: ryjfgjl |
last post by:
ExcelToDatabase: batch import excel into database automatically...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM).
In this month's session, we are pleased to welcome back...
|
by: jfyes |
last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
|
by: PapaRatzi |
last post by:
Hello,
I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
| |