473,320 Members | 2,054 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

How to efficiently find a string in multiple text files?

The text files are not only the .txt files, but also all ascii files.

Thx.
Jan 3 '06 #1
5 3410
Buddhist,

For me a txt file is the same as an ascii file. (a classic name for a text
file)

Therefore can you make it more clear for us what you mean by this.

Be aware that finding strings in compressed or hassled files is not as easy
to do as in a txt file.

Cor
Jan 3 '06 #2
O, what I described was a little confusing and didn't make sense.

Actually, I want to ask how to searh a string in lots of text files quickly.
I implemented this by using System.IO(directory and file), read each line
and checked whether there was string desired, then recorded line numbers.
But its performance is very low, especially while processing lots of files.
So I am doubting whether directly using System.IO can meet my performance
requirement. Do I need to do many optimization by myself? And how?

Thx.
"Cor Ligthert [MVP]" <no************@planet.nl> дÈëÏûÏ¢
news:%2***************@TK2MSFTNGP15.phx.gbl...
Buddhist,

For me a txt file is the same as an ascii file. (a classic name for a text
file)

Therefore can you make it more clear for us what you mean by this.

Be aware that finding strings in compressed or hassled files is not as
easy to do as in a txt file.

Cor

Jan 3 '06 #3
Buddhist,
O, what I described was a little confusing and didn't make sense.

Actually, I want to ask how to searh a string in lots of text files
quickly.
I implemented this by using System.IO(directory and file), read each line
and checked whether there was string desired, then recorded line numbers.
But its performance is very low, especially while processing lots of
files.
So I am doubting whether directly using System.IO can meet my performance
requirement. Do I need to do many optimization by myself? And how?

That is in fact the only method I know as well, you can probably slightly
improve it by doing some multithreading with ony two threads, not more
(where the advantage is so low that I would not do it). (Finding the string
in an array can than be done seperated from retrieving the array).
Mutltithreading to read more files is without any sense and slows only your
application.

Have as well a look to this message from Jon some days ago.

http://groups.google.com/group/micro...149199c66b64bI hope this helps,Cor

Jan 3 '06 #4
Thx your help.

But I am dubious about the example by Jon.
Is checking each char in a huge char array very fast?
I'm afraid not.

Or there is sth I haven't understood.

"Cor Ligthert [MVP]" <no************@planet.nl> дÈëÏûÏ¢
news:u6**************@TK2MSFTNGP12.phx.gbl...
Buddhist,
O, what I described was a little confusing and didn't make sense.

Actually, I want to ask how to searh a string in lots of text files
quickly.
I implemented this by using System.IO(directory and file), read each line
and checked whether there was string desired, then recorded line numbers.
But its performance is very low, especially while processing lots of
files.
So I am doubting whether directly using System.IO can meet my performance
requirement. Do I need to do many optimization by myself? And how?

That is in fact the only method I know as well, you can probably slightly
improve it by doing some multithreading with ony two threads, not more
(where the advantage is so low that I would not do it). (Finding the
string in an array can than be done seperated from retrieving the array).
Mutltithreading to read more files is without any sense and slows only
your application.

Have as well a look to this message from Jon some days ago.

http://groups.google.com/group/micro...149199c66b64bI
hope this helps,Cor

Jan 4 '06 #5
Buddhist,

Sorry for Strings is the Visual Basic Find the fastest.

http://groups.google.com/group/micro...5c33cc87237dbf

See this test we did already a long while ago.

What we have tested is the Visual Basic find for finding strings twice as
fast as any other method.

Cor
Jan 4 '06 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: hokiegal99 | last post by:
How do I say: x = string.find(files, 'this', 'that', 'the-other') currently I have to write it like this to make it work: x = string.find(files, 'this') y = string.find(files, 'that') z =...
14
by: Klaus Neuner | last post by:
Hello, I need to gather information that is contained in various files. Like so: file1: ===================== foo : 1 2 bar : 2 4
1
by: Xah Lee | last post by:
suppose you want to do find & replace of string of all files in a directory. here's the code: ©# -*- coding: utf-8 -*- ©# Python © ©import os,sys © ©mydir= '/Users/t/web'
5
by: Jim | last post by:
Hello, I am working on a small windows application for a client, and as one of the functions they want a search that will let them enter a search string, then search a directory for all flies...
0
by: Xah Lee | last post by:
Interactive Find and Replace String Patterns on Multiple Files Xah Lee, 2006-06 Suppose you need to do find and replace of a string pattern, for all files in a directory. However, you do not...
2
by: rengask | last post by:
I got the code to find and replace within an open text file. ------------------ Private Sub cmdFile_Click() Dim strTemp As String txtFile = "" dlg.FileName = "*.*" dlg.ShowOpen ...
7
by: random guy | last post by:
Hi, I'm writing a program which creates an index of text files. For each file it processes, the program records the start and end positions (as returned by tellg()) of sections of interest,...
9
by: | last post by:
I am interested in scanning web pages for content of interest, and then auto-classifying that content. I have tables of metadata that I can use for the classification, e.g. : "John P. Jones" "Jane...
9
by: Jasper | last post by:
Hi, I have multiple data files which need parsing in realtime so high performance is *crucial*. I dont have a format definition, but from what I can see there is a hierarchy of data. Each...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.