473,325 Members | 2,816 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,325 software developers and data experts.

efficient updating of nested dictionaries

I have a dictionary that looks like this
MY_DICT[KEY_X][KEY_Y][KEY_Z]=FOO

I am having a problem updating this with a simple
MY_DICT.update(NEW_DICT) as update doesn't seem to care about getting
into the inner dicts.
Getting the keys of each and iterating through and updating each one is
terribly slow as the number of keys gets bigger and bigger.
What is the bst way to update my nested dicts?

Jul 18 '05 #1
13 5413
omission9 wrote:
I have a dictionary that looks like this
MY_DICT[KEY_X][KEY_Y][KEY_Z]=FOO

I am having a problem updating this with a simple
MY_DICT.update(NEW_DICT) as update doesn't seem to care about getting
into the inner dicts.
Getting the keys of each and iterating through and updating each one is
terribly slow as the number of keys gets bigger and bigger.
What is the bst way to update my nested dicts?


Make a table whose rows are (KEY_X, KEY_Y, KEY_Z, FOO). If the table is
large use MySQL or some other database. For small or medium sized tables
try "http://members.tripod.com/~edcjones/MultiDict.py".
Jul 18 '05 #2
The following is probably too dependent on the data type of the keys,
but it may be suitable in some programs. It's certainly not a general
solution for all cases. Others will have much better ideas, but here
goes anyway ...

You may want to use a non-nested dict with a 'superkey' composed of the
concatenation of the three keys, seperated by some delimiter.
use MY_DICT[KEY_X+'_'+KEY_Y+'_'+KEY_Z]=FOO

Then you could use update().You would just have to do some pre- and
post-processing of the keys. i.e. splitting or joining the 'superkey' by
the delimiter you choose.

Although, that's probably kind of lame - I bet others will have much
better suggestions. I'm interested in how other people do this too.
Rich
On Sun, 2004-01-25 at 21:33, omission9 wrote:
I have a dictionary that looks like this
MY_DICT[KEY_X][KEY_Y][KEY_Z]=FOO

I am having a problem updating this with a simple
MY_DICT.update(NEW_DICT) as update doesn't seem to care about getting
into the inner dicts.
Getting the keys of each and iterating through and updating each one is
terribly slow as the number of keys gets bigger and bigger.
What is the bst way to update my nested dicts?


Jul 18 '05 #3
omission9 wrote:
I have a dictionary that looks like this
MY_DICT[KEY_X][KEY_Y][KEY_Z]=FOO

I am having a problem updating this with a simple
MY_DICT.update(NEW_DICT) as update doesn't seem to care about getting
into the inner dicts.
Getting the keys of each and iterating through and updating each one is
terribly slow as the number of keys gets bigger and bigger.
What is the bst way to update my nested dicts?

So far I have found this on the internet:
def rUpdate(self,targetDict,itemDict):
valtab=[]
for key,val in itemDict.items():
if type(val)==type({}):
newTarget=targetDict.setdefault(key,{})
self.rUpdate(newTarget,val)
else:
targetDict[key]=val

However, this does not seem to handle the fact that each dict has
multiple keys. :( So far the modification I have made to make it work
right have failed. Any ideas?

Jul 18 '05 #4
> Although, that's probably kind of lame - I bet others will have much
better suggestions. I'm interested in how other people do this too.
Rich


String concatenation is not that lame, but I'd use tuples:
MY_DICT[(KEY_X, KEY_Y, KEY_Z)] = FOO

Tuples save on string operations.

- Josiah
Jul 18 '05 #5
omission9 <om*******@invalid.email.info> wrote in message news:<H%*****************@nwrddc02.gnilink.net>...
I have a dictionary that looks like this
MY_DICT[KEY_X][KEY_Y][KEY_Z]=FOO

I am having a problem updating this with a simple
MY_DICT.update(NEW_DICT) as update doesn't seem to care about getting
into the inner dicts.
Getting the keys of each and iterating through and updating each one is
terribly slow as the number of keys gets bigger and bigger.
What is the bst way to update my nested dicts?


Use a tuple
MY_DICT[(KEY_X,KEY_Y,KEY_Z)]=FOO

unless you have a particular reason to use these nested dicts :)
Jul 18 '05 #6
omission9 <om*******@invalid.email.info> wrote in message news:<H%*****************@nwrddc02.gnilink.net>...
I have a dictionary that looks like this
MY_DICT[KEY_X][KEY_Y][KEY_Z]=FOO

I am having a problem updating this with a simple
MY_DICT.update(NEW_DICT) as update doesn't seem to care about getting
into the inner dicts.
Getting the keys of each and iterating through and updating each one is
terribly slow as the number of keys gets bigger and bigger.
What is the bst way to update my nested dicts?


Use Tuples

MY_DICT[(KEY_X,KEY_Y,KEY_Z)]=FOO

Unless for some you need to use nested dicts :)
Jul 18 '05 #7
omission9 <om*******@invalid.email.info> wrote in message news:<H%*****************@nwrddc02.gnilink.net>...
I have a dictionary that looks like this
MY_DICT[KEY_X][KEY_Y][KEY_Z]=FOO

I am having a problem updating this with a simple
MY_DICT.update(NEW_DICT) as update doesn't seem to care about getting
into the inner dicts.
Getting the keys of each and iterating through and updating each one is
terribly slow as the number of keys gets bigger and bigger.
What is the bst way to update my nested dicts?


Use Tuples

MY_DICT[(KEY_X,KEY_Y,KEY_Z)]=FOO

Unless for some you need to use nested dicts :)
Jul 18 '05 #8
Josiah Carlson <jc******@nospam.uci.edu> wrote in
news:bv**********@news.service.uci.edu:
Although, that's probably kind of lame - I bet others will have much
better suggestions. I'm interested in how other people do this too.
Rich


String concatenation is not that lame, but I'd use tuples:
MY_DICT[(KEY_X, KEY_Y, KEY_Z)] = FOO

Tuples save on string operations.


I would omit the extra parentheses here, but its a style thing.

MY_DICT[KEY_X, KEY_Y, KEY_Z] = FOO

(Note to original poster: I'd also turn off caps-lock)
Jul 18 '05 #9
This is untested code but i think it should work.(fingers crossed)
Btw i doubt this will be fast though.

def rec_update(mydict, newdict):
presentKeysPairs = [(key,value)
for (key, value) in newdict.items()
if mydict.has_key(key)]
newKeysPairs = [(key,value)
for (key, value) in newdict,items()
if not mydict.has_key(key)]
for key, newValue in presentKeysPairs:
currentValue = mydict[key]
if isisntance(newValue, dict):
mydict[key] = rec_update(newValue)
else:
mydict[key] = newValue
mydict.update(dict(newKeysPairs))
return mydict

regards

ps. why can't you simply use tuples to represent the different
dimensions, even if the number of dimensions vary.
is there any particular reason why you are using these nested
dictionaries?
Jul 18 '05 #10
>> String concatenation is not that lame, but I'd use tuples:
MY_DICT[(KEY_X, KEY_Y, KEY_Z)] = FOO
Tuples save on string operations.

What a nice way to simplify this common task. That's great. Thanks for
the advice.
Rich

Jul 18 '05 #11
On Mon, 26 Jan 2004 20:11:39 -0500, Rich Krauter wrote:
What a nice way to simplify this common task. That's great. Thanks for
the advice.

[HTML garbage repeating the same content]


What a hideous way to complicate this simple medium. That sucks.
Thanks for turning it off in future.

--
\ "My roommate got a pet elephant. Then it got lost. It's in the |
`\ apartment somewhere." -- Steven Wright |
_o__) |
Ben Finney <http://bignose.squidly.org/>
Jul 18 '05 #12
Oh crap. Sorry about the html emails. I've been meaning to turn that
off. Thanks for reminding me.
Rich

Jul 18 '05 #13
On Mon, 26 Jan 2004 21:32:28 -0500, Rich Krauter wrote:
Oh crap. Sorry about the html emails. I've been meaning to turn that
off. Thanks for reminding me.


Much better! Thanks for being considerate.

--
\ "When I turned two I was really anxious, because I'd doubled my |
`\ age in a year. I thought, if this keeps up, by the time I'm six |
_o__) I'll be ninety." -- Steven Wright |
Ben Finney <http://bignose.squidly.org/>
Jul 18 '05 #14

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Narendra C. Tulpule | last post by:
Hi, if you know the Python internals, here is a newbie question for you. If I have a list with 100 elements, each element being a long string, is it more efficient to maintain it as a dictionary...
6
by: Andy Baker | last post by:
Hi there, I'm learning Python at the moment and trying to grok the thinking behind it's scoping and nesting rules. I was googling for nested functions and found this Guido quote:...
2
by: David Pratt | last post by:
Hi. I like working with lists of dictionaries since order is preserved in a list when I want order and the dictionaries make it explicit what I have got inside them. I find this combination very...
2
by: techiepundit | last post by:
I'm parsing some data of the form: OuterName1 InnerName1=5,InnerName2=7,InnerName3=34; OuterName2 InnerNameX=43,InnerNameY=67,InnerName3=21; OuterName3 .... and so on.... These are fake...
8
by: Brian L. Troutwine | last post by:
I've got a problem that I can't seem to get my head around and hoped somebody might help me out a bit: I've got a dictionary, A, that is arbitarily large and may contains ints, None and more...
13
by: gonzlobo | last post by:
Greetings, and happyNewYear to all. I picked up Python a few weeks ago, and have been able to parse large files and process data pretty easily, but I believe my code isn't too efficient. I'm...
1
by: Matthew Schibler | last post by:
I'm a newbie to Python, with some experience using perl (where I used nested arrays and hashes extensively). I am building a script in python for a MUD I play, and I want to use the shelve module...
9
by: Brandon | last post by:
Hi all, I am not altogether experienced in Python, but I haven't been able to find a good example of the syntax that I'm looking for in any tutorial that I've seen. Hope somebody can point me...
1
by: Edwin.Madari | last post by:
by the way, iterating over bar will throw KeyError if that key does not exist in foo. to see that in action, simply set another key in bar after copy.deepcopy stmt in this example.. bar = 0 and...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.