473,320 Members | 2,073 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Need some fast code for some string array logic

I'm trying to come up with the best (fastest, as this code will be run
often) way to accomplish the following:

I have a dynamic string array X (as one grabbed from a GetFiles() or
GetDirectories() function call), which could have anywhere from 0 to
maybe 250 elements in it. I also have a statically-defined string array
Y which currently has 309 elements in it, but may grow over time. I
need to look through X and remove all elements that match any element in
Y. This can be accomplished through some nested loops and copying of
string arrays, but the inefficiency of that just doesn't sit right with me.

Is there anything in the .NET classes and functions that can make this
easier? Would it help if the arrays are known to be sorted? Does
anyone have any more creative algorithms that can do this quickly? Any
help would be much appreciated, thank you.
Regards,
David P. Donahue
dd******@ccs.neu.edu
Nov 16 '05 #1
3 1434
Depending on usage patterns it might be better to store the info you are
pruning in a hashtable where the key and value are the same (or where the
value is the FileInfo object, if you need all that data along for the ride).
The speed of the hashed lookup would likely more than overcome the casting
overhead.

You can always copy the final result to an array and Array.Sort it if you
need it in a particular order.

Better still, under CLR 2.0 you could use Dictionary<string> instead of a
Hashtable, and then you won't have the casting penalty either.

Obviously your mileage may vary -- I would put together a few proof of
concept variations and benchmark them.

--Bob

"David P. Donahue" <dd******@ccs.neu.edu> wrote in message
news:uI**************@TK2MSFTNGP09.phx.gbl...
I'm trying to come up with the best (fastest, as this code will be run
often) way to accomplish the following:

I have a dynamic string array X (as one grabbed from a GetFiles() or
GetDirectories() function call), which could have anywhere from 0 to maybe
250 elements in it. I also have a statically-defined string array Y which
currently has 309 elements in it, but may grow over time. I need to look
through X and remove all elements that match any element in Y. This can
be accomplished through some nested loops and copying of string arrays,
but the inefficiency of that just doesn't sit right with me.

Is there anything in the .NET classes and functions that can make this
easier? Would it help if the arrays are known to be sorted? Does anyone
have any more creative algorithms that can do this quickly? Any help
would be much appreciated, thank you.
Regards,
David P. Donahue
dd******@ccs.neu.edu

Nov 16 '05 #2
Hi,

Well, you could store Y sorted , that will help a little , even maybe store
it in a b-tree will further improve the efficiency of the search.

The constrain you have is the X array , as you have no control over it. so
you will have to do a linear search on it so you should concentrate your
optimization in Y instead.

Don't think you have much more options here :(

cheers,

--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation
"David P. Donahue" <dd******@ccs.neu.edu> wrote in message
news:uI**************@TK2MSFTNGP09.phx.gbl...
I'm trying to come up with the best (fastest, as this code will be run
often) way to accomplish the following:

I have a dynamic string array X (as one grabbed from a GetFiles() or
GetDirectories() function call), which could have anywhere from 0 to maybe
250 elements in it. I also have a statically-defined string array Y which
currently has 309 elements in it, but may grow over time. I need to look
through X and remove all elements that match any element in Y. This can
be accomplished through some nested loops and copying of string arrays,
but the inefficiency of that just doesn't sit right with me.

Is there anything in the .NET classes and functions that can make this
easier? Would it help if the arrays are known to be sorted? Does anyone
have any more creative algorithms that can do this quickly? Any help
would be much appreciated, thank you.
Regards,
David P. Donahue
dd******@ccs.neu.edu

Nov 16 '05 #3
I guess I need to stop thinking so 1-dimensionally :) Hashtables make
the whole thing that much easier, thanks!
Regards,
David P. Donahue
dd******@ccs.neu.edu

Bob Grommes wrote:
Depending on usage patterns it might be better to store the info you are
pruning in a hashtable where the key and value are the same (or where the
value is the FileInfo object, if you need all that data along for the ride).
The speed of the hashed lookup would likely more than overcome the casting
overhead.

You can always copy the final result to an array and Array.Sort it if you
need it in a particular order.

Better still, under CLR 2.0 you could use Dictionary<string> instead of a
Hashtable, and then you won't have the casting penalty either.

Obviously your mileage may vary -- I would put together a few proof of
concept variations and benchmark them.

--Bob

"David P. Donahue" <dd******@ccs.neu.edu> wrote in message
news:uI**************@TK2MSFTNGP09.phx.gbl...
I'm trying to come up with the best (fastest, as this code will be run
often) way to accomplish the following:

I have a dynamic string array X (as one grabbed from a GetFiles() or
GetDirectories() function call), which could have anywhere from 0 to maybe
250 elements in it. I also have a statically-defined string array Y which
currently has 309 elements in it, but may grow over time. I need to look
through X and remove all elements that match any element in Y. This can
be accomplished through some nested loops and copying of string arrays,
but the inefficiency of that just doesn't sit right with me.

Is there anything in the .NET classes and functions that can make this
easier? Would it help if the arrays are known to be sorted? Does anyone
have any more creative algorithms that can do this quickly? Any help
would be much appreciated, thank you.
Regards,
David P. Donahue
dd******@ccs.neu.edu


Nov 16 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
by: javuchi | last post by:
I just want to share some code with you, and have some comments and improvements if you want. This header file allocates and add and delete items of any kind of data from a very fast array: ...
46
by: Bruce W. Darby | last post by:
This will be my very first VB.Net application and it's pretty simple. But I've got a snag in my syntax somewhere. Was hoping that someone could point me in the right direction. The history: My...
7
by: ashjas | last post by:
Hello, I have run this logic on turboc++ 3.0 and it is working fine on it but its not running on msvs2008 c++. i am not able to assign the value like this *temp=*main,where main and temp are char...
0
maxamis4
by: maxamis4 | last post by:
Does anyone know how to create a multi column array?
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.