471,873 Members | 1,854 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,873 software developers and data experts.

python shutting down sloooooooowly/tuning dictionaries

Is there a way to speed up killing python from within a python program?
Sometimes shutting down takes more than 10 times as much time as the actual
running of the program.

The programs are fairly simple (searching/organizing large boardgame
databases) but use a lot of memory (1-6GB). The memory is mostly used
for simple structures like trees or relations. Typically there will be
a few large dictionaries and many small dictionaries/sets/arrays/lists.

Since the entire database is too large I typically run the same
program 10-100 times on smaller parts. The annoying bit is that the
time for finishing one instance of a program before starting the next
instance can take a lot of time. I have started using a shell script to
check whether certain files have been written and then kill the
program from the os to speed up everything. But this solution is way
too ugly. There should be a better way, but I don't seem to be able to
find one.

A second question. Is it possible to control the resizing of
dictionaries? It seems that resizing always doubles the size, but for
dictionaries using more than half of the remaining memory than this
is a problem. I would rather have a slightly too full dictionaries in
main memory than have a sparse dictionary partially swapped out.

I would like to be able to restrict the maximum growth of each resize
operation to around 100MB or so.


- Till
Jul 18 '05 #1
0 1292

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

13 posts views Thread by Tim Howarth | last post: by
68 posts views Thread by Lad | last post: by
9 posts views Thread by corey.coughlin | last post: by
5 posts views Thread by Larry Bates | last post: by
reply views Thread by YellowAndGreen | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.