Efficient lookup in list of dictionaries

[David Pratt]

Hi. I like working with lists of dictionaries since order is preserved
in a list when I want order and the dictionaries make it explicit what
I have got inside them. I find this combination very useful for storing
constants especially. Generally I find myself either needing to
retrieve the values of constants in an iterative way (as in my
contrived example below). Perhaps even more frequent is given one value
is to look up the matching value in a dict (contained in the list) and
then obtain the value of another element in the same dictionary.

Most of these lists are generally small so not normally a big deal but
I have one a list that contains about 100 or more dictionaries with
several elements and am thinking there is likely a more efficient way
of doing the lookup. For example if I have say 5000 insertions to do
into a database but have to check my constants since they effect what
is inserted (and it is doing a lookup each time before an insertion is
made), it is likely adding much to my processing time. At this point I
do not want store the constants themselves in a database table. I
usually use the lists by just importing them when needed.

Can someone advise a more efficient lookup when using lists of
dictionaries. Many thanks.

Regards
David

TEST_CONSTANTS = [
{'color':'red', 'shape':'octagon'},
{'color':'yellow', 'shape':'triangle'},
{'color':'green', 'shape':'circle'}]

def getShapeForColor(color):
shape = None
for test_constant in TEST_CONSTANTS:
if test_constant['color'] == color:
shape = test_constant['shape']
return shape

[Rob E]

> Hi. I like working with lists of dictionaries since order is preserved

in a list when I want order and the dictionaries make it explicit what
I have got inside them. I find this combination very useful for storing
constants especially. Generally I find myself either needing to
retrieve the values of constants in an iterative way (as in my
contrived example below). Perhaps even more frequent is given one value
is to look up the matching value in a dict (contained in the list) and
then obtain the value of another element in the same dictionary.

Instead of doing this in an iterative way -- create a dictionary that
points to the correct dictionary, i.e., who's key is the search key and
whose value is the dictionary (or list of dicts if more than one). This
approach is basically accelerating the lookup by creating a special index.

Another way of doing this is to create a class that works like an
"ordered" dictionary. Once can do this with a little programming -- for
example by putting your data in a list and creating a dict that indexes
the records of the list -- i.e. the key is the index and the value is the
index of the list. This is pretty much how databases work.

Rob

[bruno at modulix]

David Pratt wrote:
(snip)

Can someone advise a more efficient lookup when using lists of
dictionaries. Many thanks.
TEST_CONSTANTS = [
{'color':'red', 'shape':'octagon'},
{'color':'yellow', 'shape':'triangle'},
{'color':'green', 'shape':'circle'}]

COLOR_INDEX = dict([(item['color'], item) for item in TEST_CONSTANT])
SHAPE_INDEX = dict([item['shape'], item) for item in TEST_CONSTANT])

def getShapeForColor(color):
return COLOR_INDEX.get(color, {'shape':None})['shape']

def getColorForShape(shape):
return SHAPE_INDEX.get(color, {'color': None})['color']

This of course assume that there are no duplicate colors nor shapes.

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"