473,406 Members | 2,217 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,406 software developers and data experts.

.txt to memory

Hi,

I have a file that contain 100K lines. I want to charge all these lines
to my application memory. I've been using a StreamReader object to read
line by line and process to my app memory, but it is so slow, more than
1.5 min.

How can I speed it up? Does passing from .TXT to XML would make the
transfer faster?

Any idea would be appreciated.

Thanks

Marty
Nov 21 '05 #1
4 993
if you aren't worried about memory constraints then set you variable =
stream.readtoend() to get all the text at once. see if that speeds things up
a bit. and, no, xml markup will actually increase your file size...
"Marty" <xm******@hotmail.com> wrote in message
news:Mbded.31768$cr4.16629@edtnps84...
| Hi,
|
| I have a file that contain 100K lines. I want to charge all these lines
| to my application memory. I've been using a StreamReader object to read
| line by line and process to my app memory, but it is so slow, more than
| 1.5 min.
|
| How can I speed it up? Does passing from .TXT to XML would make the
| transfer faster?
|
| Any idea would be appreciated.
|
| Thanks
|
| Marty
Nov 21 '05 #2
"Marty" <xm******@hotmail.com> schrieb:
I have a file that contain 100K lines. I want to charge all these lines
to my application memory. I've been using a StreamReader object to read
line by line and process to my app memory, but it is so slow, more than
1.5 min.

How can I speed it up? Does passing from .TXT to XML would make the
transfer faster?


I doubt that. Maybe it's your processing code that makes reading the file
slow...

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>
Nov 21 '05 #3
Finally I used the ReadBlock and it was the best handling for my needs.
I realize that I was able to get greater speed of execution with less
access/refresh of the GUI, then it get pretty fast to process 100000s of
text lines.

Thanks for your help guys.

Marty

Herfried K. Wagner [MVP] wrote:
"Marty" <xm******@hotmail.com> schrieb:
I have a file that contain 100K lines. I want to charge all these lines
to my application memory. I've been using a StreamReader object to read
line by line and process to my app memory, but it is so slow, more than
1.5 min.

How can I speed it up? Does passing from .TXT to XML would make the
transfer faster?

I doubt that. Maybe it's your processing code that makes reading the file
slow...

Nov 21 '05 #4
Finally I used the ReadBlock and it was the best handling for my needs.
I realize that I was able to get greater speed of execution with less
access/refresh of the GUI, then it get pretty fast to process 100000s of
text lines.

Thanks for your help guys.

Marty

Herfried K. Wagner [MVP] wrote:
"Marty" <xm******@hotmail.com> schrieb:
I have a file that contain 100K lines. I want to charge all these lines
to my application memory. I've been using a StreamReader object to read
line by line and process to my app memory, but it is so slow, more than
1.5 min.

How can I speed it up? Does passing from .TXT to XML would make the
transfer faster?

I doubt that. Maybe it's your processing code that makes reading the file
slow...

Nov 21 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Andreas Suurkuusk | last post by:
Hi, I just noticed your post in the "C# memory problem: no end for our problem?" thread. In the post you implied that I do not how the garbage collector works and that I mislead people. Since...
4
by: Frank Esser | last post by:
I am using SQL 8 Personal edition with sp2 applied. I set the max server memory to 32MB and leave the min server memory at 0. When my application starts hitting the database hard the memory usage...
4
by: Franklin Lee | last post by:
Hi All, I use new to allocate some memory,even I doesn't use delete to release them. When my Application exit, OS will release them. Am I right? If I'm right, how about Thread especally on...
9
by: Mike P | last post by:
I know everything about reference counting and making sure you don't have large objects lying around. I have also profiled my app with multiple tools. I know about the fact GC collects memory but...
22
by: xixi | last post by:
hi, we are using db2 udb v8.1 for windows, i have changed the buffer pool size to accommadate better performance, say size 200000, if i have multiple connection to the same database from...
14
by: Alessandro Monopoli | last post by:
Hi all, I'm searching a PORTABLE way to get the available and total physical memory. Something like "getTotalMemory" and it returns the memory installed on my PC in bytes, and...
1
by: Nick Craig-Wood | last post by:
I've been dumping a database in a python code format (for use with Python on S60 mobile phone actually) and I've noticed that it uses absolutely tons of memory as compared to how much the data...
5
by: kumarmdb2 | last post by:
Hi guys, For last few days we are getting out of private memory error. We have a development environment. We tried to figure out the problem but we believe that it might be related to the OS...
1
by: Jean-Paul Calderone | last post by:
On Tue, 22 Apr 2008 14:54:37 -0700 (PDT), yzghan@gmail.com wrote: The test doesn't demonstrate any leaks. It does demonstrate that memory usage can remain at or near peak memory usage even after...
5
by: cham | last post by:
Hi, I am working on c++ in a linux system ( Fedora core 4 ), kernel version - 2.6.11-1.1369_FC4 gcc version - 4.0.0 20050519 ( Red Hat 4.0.0-8 ) In my code i am creating a vector to store...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.