473,320 Members | 2,094 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Help with GetFiles

Hi! I am about to create a small application that needs to process some
files stored in a directory.

GetFiles() returns a string array, OK, but I'm expecting to find MANY (over
150.000) files in the directory and I'm worried about getting a memory
overflow, since I expect something like... 15 letters per name in the
average.

Is there any way of accessing the files one by one like it was possible in
C++ with the _movefirst, _movenext, and _finddata_t type instead of storing
all names in memory?

PS: Sorry about my poor english!
Nov 15 '05 #1
4 2172
Check out this link...
http://www.knowdotnet.com/articles/r...orysearch.html

You could change this so you used a numeric index instead and only loop
through x at a time instead of all of them...
"EL OSO" <no***@nospam.com> wrote in message
news:OP**************@TK2MSFTNGP10.phx.gbl...
Hi! I am about to create a small application that needs to process some
files stored in a directory.

GetFiles() returns a string array, OK, but I'm expecting to find MANY (over 150.000) files in the directory and I'm worried about getting a memory
overflow, since I expect something like... 15 letters per name in the
average.

Is there any way of accessing the files one by one like it was possible in
C++ with the _movefirst, _movenext, and _finddata_t type instead of storing all names in memory?

PS: Sorry about my poor english!

Nov 15 '05 #2
My understanding of that example is that it will just store the results of
GetFiles() in a temporary variable, and be no differant than:

string[] files = Directory.GetFiles("path");
foreach(string file in file);

Unfortunatly, there doesn't seem to be any native way to enumerate the files
without using PInvoke and FindFirstFile, FindNextFile, and FindClose.

- Pete

"William Ryan [eMVP]" <do********@comcast.nospam.net> wrote in message
news:eI**************@TK2MSFTNGP10.phx.gbl...
Check out this link...
http://www.knowdotnet.com/articles/r...orysearch.html

You could change this so you used a numeric index instead and only loop
through x at a time instead of all of them...
"EL OSO" <no***@nospam.com> wrote in message
news:OP**************@TK2MSFTNGP10.phx.gbl...
Hi! I am about to create a small application that needs to process some
files stored in a directory.

GetFiles() returns a string array, OK, but I'm expecting to find MANY

(over
150.000) files in the directory and I'm worried about getting a memory
overflow, since I expect something like... 15 letters per name in the
average.

Is there any way of accessing the files one by one like it was possible in C++ with the _movefirst, _movenext, and _finddata_t type instead of

storing
all names in memory?

PS: Sorry about my poor english!


Nov 15 '05 #3
AFAIK, that's correct, it's just if you use numeric iteration, you control
the subset a little eaiser.
"AirPete" <x@x.x> wrote in message
news:o9*****************@newsread2.news.pas.earthl ink.net...
My understanding of that example is that it will just store the results of
GetFiles() in a temporary variable, and be no differant than:

string[] files = Directory.GetFiles("path");
foreach(string file in file);

Unfortunatly, there doesn't seem to be any native way to enumerate the files without using PInvoke and FindFirstFile, FindNextFile, and FindClose.

- Pete

"William Ryan [eMVP]" <do********@comcast.nospam.net> wrote in message
news:eI**************@TK2MSFTNGP10.phx.gbl...
Check out this link...
http://www.knowdotnet.com/articles/r...orysearch.html

You could change this so you used a numeric index instead and only loop
through x at a time instead of all of them...
"EL OSO" <no***@nospam.com> wrote in message
news:OP**************@TK2MSFTNGP10.phx.gbl...
Hi! I am about to create a small application that needs to process some files stored in a directory.

GetFiles() returns a string array, OK, but I'm expecting to find MANY (over
150.000) files in the directory and I'm worried about getting a memory
overflow, since I expect something like... 15 letters per name in the
average.

Is there any way of accessing the files one by one like it was
possible in C++ with the _movefirst, _movenext, and _finddata_t type instead of

storing
all names in memory?

PS: Sorry about my poor english!



Nov 15 '05 #4
I understand your concern, but I don't think you need to worry about it too
much.

Even with as many files as you're talking about, it's not going to consume
all that much memory.

150.000 files * 30 bytes (2 bytes per char) = 4.500.000 bytes

That's almost 4.3 megabytes. That's not too terribly much memory.
I do agree though, that the .NET framework should have exposed a funciton
that returns an IEnumerator wrapping a Win32 call to FindFirstFile,
FindNextFile, etc.

You could write such an enumerator if you wanted using P/Invoke, if you
don't mind calling unmanaged code. This will depend on your security and
portability requirements, though.
Anyway, sometimes you don't need to be able to index a collection of file
names...but when you do, the .NET way is very nice. Still, I could see
where the IEnumerator option would have been nice (imagine an IEnumerator
for recusive searching!)

--Matthew W. Jackson

"EL OSO" <no***@nospam.com> wrote in message
news:OP**************@TK2MSFTNGP10.phx.gbl...
Hi! I am about to create a small application that needs to process some
files stored in a directory.

GetFiles() returns a string array, OK, but I'm expecting to find MANY (over 150.000) files in the directory and I'm worried about getting a memory
overflow, since I expect something like... 15 letters per name in the
average.

Is there any way of accessing the files one by one like it was possible in
C++ with the _movefirst, _movenext, and _finddata_t type instead of storing all names in memory?

PS: Sorry about my poor english!

Nov 15 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: S. Han | last post by:
I'm using Directory.GetFiles to enumerate files in a directory. The problem is if you have to enumerate all files + subdirectories recursively, it takes too much memory, and it fails. Is there...
11
by: al jones | last post by:
I'm using filesystem.getfiles - and so far it's working correctly *however* I'd sure like to be able to pass it, as the last parameter, the extensions (plural) for which I'm looking. I assumed...
1
by: Hawk | last post by:
I was writing my own wildcard compare algorithm and making a custom GetFiles routine when I came across what I think is a bug in .net's own GetFiles. My routine was coming up with different...
13
by: Lance | last post by:
Hi All, I'm working on a program that requires searching multiple drives for multiple file types and cataloging them based on certain geospatial attributes. All together, there are hundreds of...
1
by: jobs | last post by:
Say I only have a single file I want to get information on. How can I adapt this code? Dim dr As DataRow Dim fi As FileInfo Dim dir As New DirectoryInfo(filename) Dim dt As New DataTable...
6
by: Kyote | last post by:
I'm trying to make, what I thought, would be a simple application. This application will help me organize files I have a lot of, on my computer(ebooks, pictures, etc..). I'm currently dealing with...
3
by: Michael Jackson | last post by:
In my .NET 2.0 VS 2005 VB application, I'm using Directory.GetFiles(path) to get all the files in the directory. However, I'm getting an error regarding "Illegal character in Path", even though I...
0
by: tshad | last post by:
I am trying to do multiple Directory.GetFiles and append the results to one array that I will process. I tried this: string strFiles; strFiles = Directory.GetFiles(SemaSettings.InputFilePath,...
1
by: innes | last post by:
Hi, I'm trying to test permission to access the file system, using FileIOPermission, but can't seem to get it to work. See my sample app below, for an app that tries to demand a permission to...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.