Searching in Hashes

[Gangreen]

There is a hash in my program.
The problem is I need to search in the hash, but there are some keys that are identical, but I need to get the associated value of all these keys.

when I do something like

$value = $hash{ "theKey" };

obviously I'm only getting the value associated to the first key called "theKey". How can I get the others? Some sort of loop?

thanks

[minowicz]

it's the keys I'm comparing. It's to expand the search I made with the help of this topic by adding functionality like searching for:

'keyword 1' AND 'keyword2'

so both keywords need to be in the string representing the key.

also an 'keyword 1' OR 'keyword2'

and also searching for one word in keys (given the fact the keys are sentences: i.e "Harry Potter and ..." )

I was just hoping there would be a more efficient way than just check each value and compare...becau se that won't give hashes an advantage in speed against arrays.

OK, so to some extent, you really want to be able to query all sorts of possible matching situations against the data. Eventually, yes, you are going to end up wanting an SQL-like syntax to express searches. So you really need to stop and ask yourself how much farther your search semantics are going to evolve and if you're going to pass a point where the code your writing would have been better served with an SQL database. For simple searches, you just don't need one, but the more complex and varried your searches are going to become, the more of a payoff you're going to get from the investment in going to the effort.

That said, I don't think that what you're trying to do so far has quite reached that point.

At this point though I think you're best off keeping your original data in a hash with the product ID as the key and building arrays that index into that hash for the various searches you expect to be most common.

Assuming your inventory is kept in a hash that looks something like this:

my %inventoryDB = (
    DVD-432 => ['Harry Potter and the Prisoner of Azkaban'. '25.00', '5'],
    BOOK-432 => ['Harry Potter and the Prisoner of Azkaban', '15.00' '7'],
);
 

You'd build a title index that looked something like this:

my %titleIDX = (
    'Harry Potter and the Prisoner of Azkaban' => ['DVD-432', 'BOOK-432'],
);
 

And you could even go so far as to build a keywords index by iterating through titles and other such things to create indexes like:

my %keywordIDX = (
    'harry' => ['DVD-432', 'BOOK-432'],
    'potter' => ['DVD-432', 'BOOK-432'],
    'prisoner' => ['DVD-432', 'BOOK-432'],
    'azkaban' => ['DVD-432', 'BOOK-432'],
);
 

Though such an index would get rather large and do so quickly. (You'd end up doing better to start using something like Lucene at some point going down this path.)

Constructs such as these allow you to do things like grep or pattern match across the keys of the hashes. Depending on how you intend to update the information in the database, you could even build arrays of the keys ahead of time just to avoid repetitive calls to keys().

my @titleKEY = keys %titleIDX;
 

You'd then need to regenerate indexes and arrays with lists of keys either on every update that would change them, or on a periodic basis, understanding that they'd be out of date until such an update occurred. (This is largely a function of how 'live' the data is supposed to be.)

Very quickly this would lend itself to a bit of object orientation in which you'd presumably have objects for both individual inventory items and for the inventory database itself. Then the various instance data accessors would would have code that would understand when an update would require a re-index and some part of the indexes, potentially triggering a cascade of re-indexes along the way.

The upshot is that this all gets very complex in an effort to save you from simply doing:

foreach my $key (keys %hash) {
    if ($hash{$key} =~ /foo/) {
        found it
    }
}
 

I guess what I'm saying is that this last little code snippet may not be ideal, but it is always up to date with whatever is in your hash, and it is really simple to code. You don't need to worry about complex re-indexing schemes and so on.

If you really have a need for making this simple to do, simple to extend, AND high performance (so much so that you're looking att using multiple arrays with common numerical indexes rather than hashes for performance reasons) then I'm finally willing to admit that you should go ahead and start using an SQL database now.

[eWish]

minowicz,

What would be the main reason you would suggest the OP move towards using a database? Also, from the standpoint of performance, what gain(s) does the database have to offer vs the use of multiple hashes and/or arrays?

--Kevin

[minowicz]

minowicz,

What would be the main reason you would suggest the OP move towards using a database? Also, from the standpoint of performance, what gain(s) does the database have to offer vs the use of multiple hashes and/or arrays?

--Kevin

Actually, I'm largely opposed to the idea of moving this to a database unless one of two conditions are true:

1. The total size of the database is large enough that it will not readily fit into memory when all of the various indexes and such are considered along with the original data.
2. The search semantics required are so elaborate, varied, and/or changing through the application life cycle that the OP is likely to have to reinvent the wheel of SQL.

There are probably solutions for either of these situations that would not require a move to an SQL database, but the original question was a fairly simple case. New requirements in the follow-on seem to be revealing the likelihood of #2 above.

One simple possibility, particularly given the original data format would be DBD::RAM. More speed and/or a smaller memory footprint could probably be had from DBD::DBM or possibly DBD::SQLite or DBD::SQLite2 all listed on CPAN. Moving to something like MySQL or PostgreSQL are other possibilities if the situation warrants it.

Again, my contention is that this all *could* be done in Perl, but if there are going to be more complex searches, changing search needs, the need to keep the data 'live; for both searches and updates, complex rules for updating only the indexes that need to be updated for a given change, and so on... Well, there comes apoint when it is wise to be lazy and rely on systems that others have already built that do such things and do them very well.