473,473 Members | 2,185 Online
Bytes | Software Development & Data Engineering Community
Create Post

Home Posts Topics Members FAQ

Fast Index a File System?

Hi Everybody,

I am trying to build a tree of my entire file system. The tree
includes attributes like file size and last modified. My current
implementation is multi threaded using the threadpool, but it takes
around 10 minutes to complete. I don't understand why it would take
so long because I think everything I want is in the Master File
Table. Is there a faster implementation?

Thanks,
James
Dec 24 '07 #1
2 2642
Eps
james wrote:
Hi Everybody,

I am trying to build a tree of my entire file system. The tree
includes attributes like file size and last modified. My current
implementation is multi threaded using the threadpool, but it takes
around 10 minutes to complete. I don't understand why it would take
so long because I think everything I want is in the Master File
Table. Is there a faster implementation?

Thanks,
James
I am kinda doing a similar thing, indexing my mp3 collection, the
approach I have gone for is to store all the meta information in a sql
lite database. All the program does when it starts up is check whether
each file is there (disable or delete the record from the db if it is
not), its very quick to load (i am using it on ~10,000 plus files).

The downside is that you need to populate the db to begin with which in
my case only took about five minutes, I think I should be able to run
that in a separate thread and still have my app useful, maybe you do the
same ?.

Anybody elses thoughts on how to deal with this type of situation would
be most welcome.

--
Eps
Dec 24 '07 #2
Have you considered using the Windows Desktop Search API? It indexes
your files for you (and you can write custom filters for files if there
isn't a preexisting one) and you can query the results programmatically.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

"james" <ja********@gmail.comwrote in message
news:a3**********************************@q77g2000 hsh.googlegroups.com...
Hi Everybody,

I am trying to build a tree of my entire file system. The tree
includes attributes like file size and last modified. My current
implementation is multi threaded using the threadpool, but it takes
around 10 minutes to complete. I don't understand why it would take
so long because I think everything I want is in the Master File
Table. Is there a faster implementation?

Thanks,
James

Dec 24 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: SoftComplete Development | last post by:
AlphaTIX is a powerful, fast, scalable and easy to use Full Text Indexing and Retrieval library that will completely satisfy your application's indexing and retrieval needs. AlphaTIX indexing...
8
by: Neil | last post by:
I have a very puzzling situation with a database. It's an Access 2000 mdb with a SQL 7 back end, with forms bound using ODBC linked tables. At our remote location (accessed via a T1 line) the time...
22
by: Marc Mones | last post by:
Hello, I'working with IBM DB2 V8.1 and CLI/ODBC. I've got a problem with the following statement: ******************************************************************************** SELECT...
6
by: Cable | last post by:
Hello, I am hoping that someone can answer a question or two regarding file access. I have created an app that reads an image from a file then displays it (using OpenGL). It works well using...
2
by: Jon Lapham | last post by:
I have a table that stores TEXT information. I need query this table to find *exact* matches to the TEXT... no regular expressions, no LIKE queries, etc. The TEXT could be from 1 to 10000+...
6
by: Yi Xing | last post by:
Hi, I need to read specific lines of huge text files. Each time, I know exactly which line(s) I want to read. readlines() or readline() in a loop is just too slow. Since different lines have...
18
by: B D Jensen | last post by:
Hello! I have an big table with 50Gb data, written som functions that cleanup data and want to do something like this insert into newtable select id, func1(col1), func2(col2) from oldtable; ...
3
by: Michael Bacarella | last post by:
The id2name.txt file is an index of primary keys to strings. They look like this: 11293102971459182412:Descriptive unique name for this record\n 950918240981208142:Another name for another...
3
by: vikas000000a | last post by:
Hi All, I am new to this forum as far as posting a question is concerned, although I have quite frequently visiting this site as a guest. My quertion relates to preforming fast searched in a very...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
1
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
0
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The...
0
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.