473,804 Members | 3,312 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Problem with ifilters, regex and large text files

1 New Member
Hello,

I am currently designing a console app that will run and search our network for any files that contain 16 digit numbers. I'm having to utilise iFilters to properly index each file, which was gratefully based on the great article I found on CodeProject.

The problem I'm having is that it basically seems to hang after a certain amount of time and after some debugging, it seems to always happen on the same file. This file is a ~40MB text file and is the second or third one that it has processed. I've included the code snippet that is doing the ifilter processing below.

using (TextReader reader = new FilterReader(f) )
{
char[] buffer = new char[512 * 512];
int position = 0;
int charsRead = 0;

while ((charsRead = reader.Read(buf fer, 0, buffer.Length)) > 0)
{
string contentToProces s = new string(buffer, 0, charsRead);

//Check it against the provided regular expression
if (regexp.IsMatch (contentToProce ss))
{
//Update the global counter
matchCounter++;

//Write the filename out to a log
writeFileName(f );

//Skip over the remainign data chunks as we have found a match
goto skipChunk;
}
position += charsRead;
}
skipChunk: ;
}

I do not doubt that I am doing something fairly silly as I'm kinda self taught at C#, but any constructive help would be greatly appreciated. Any abuse at being rubbish I can get for free from my wife when I get home! :)
Mar 3 '09 #1
0 1373

Sign in to post your reply or Sign up for a free account.

Similar topics

5
27209
by: Blue Ocean | last post by:
In short, it's not working right for me. In long: The program is designed to read numbers from an accumulator and speak them out loud. Unfortunately, the class that contains the method to read off large numbers is only for integers. My intention is to split a String across the Regex of ".". However, this code does not work: private void doRealValueOf(String text) {
3
2064
by: jty202 | last post by:
I encounter a problem. I have three files: index.aspx index.aspx.vb HTMLContentParser.vb (doesn't have the class WebForm1 I put all three file in the same directory. when I ran index.aspx it has an error
0
1001
by: kunal | last post by:
I am developing a web based application in which i give an option to the registered users of the applications an option to upload files as attachments directly in the blob field of the database. Now Just before the upload of the file occurs i need to extract text from the files and store it as a column in the database. Now i searched a lot About IFilters and came up with a lots of solutions. I just need to detect the extension of the...
0
1719
by: Greg | last post by:
Not sure if this is best place for this problem, but here it is. I have a project that is simply a C# class that interfaces with an IFilter. This is so I can retreive the text from Word docs. I'm able to use this DLL without any problems within my test windows app, but not within my windows service (that's when I receive the casting exception). Here's the code (sorry it's long): VB Function within windows app and windows service:
5
5108
by: Chris | last post by:
How Do I use the following auto-generated code from The Regulator? '------------------------------------------------------------------------------ ' <autogenerated> ' This code was generated by a tool. ' Runtime Version: 1.1.4322.2032 ' ' Changes to this file may cause incorrect behavior and will be lost if ' the code is regenerated. ' </autogenerated>
5
1595
by: John Blogger | last post by:
(I don't know if it is the right place. So if I am wrong, please point me the right direction. If this post is read by you masters, I'm honoured. If I am getting a mere response, I'm blessed!) Hi, I'm a newbie regular expression user. I use regex in my Python programs. I have a strange
4
2648
by: Chris | last post by:
Hi Everyone, I am using a regex to check for a string. When all the file contains is my test string the regex returns a match, but when I embed the test string in the middle of a text file a match is never returned. The string that I give to the regex is one that contains the entire contents of a text file. I'm using the multi-line option and I've also tried stripping out the VbCr
12
3411
by: Julian | last post by:
Hi, I am having problems with a function that I have been using in my program to read sentences from a 'command file' and parse them into commands. the surprising thing is that the program works fine on some computers and not so fine on others. I tried debugging and cannot make any sense of it. I narrowed it down to the seekg function and made this simple program which (from what I understand) does not seem to be working as expected in all...
15
50272
by: morleyc | last post by:
Hi, i would like to remove a number of characters from my string (\t \r \n which are throughout the string), i know regex can do this but i have no idea how. Any pointers much appreciated. Chris
0
9708
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10589
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
10327
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10085
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9161
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7625
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
1
4302
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
3828
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2999
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.