By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
445,885 Members | 1,474 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 445,885 IT Pros & Developers. It's quick & easy.

Child process hangs at pickle.dump??

P: 15

due to a memory leak (bug, I guess) in pyraf(or rather in IRAF) I have to fork an iterative process that goes through hundreds of image frames and does unspeakable things to them. In the child process, this returns a dictionary of image information for each image and is supposed to send it to the parent through pickle.dump. However, the child process hangs at pickle.dump and nothing happens. No error messages. The dictionary that is supposed to be dumped in the pipe is rather large, with possibly thousands of entires (about 2000-4000 is normal).

Is there some size limit on what you can dump in the pipe and in that case how do I increase it? I know nearly nothing about pipes and forks, so I'd be happy if you help.

Here's the code. Though you can't run it without rest of the functions maybe you'll see something horribly wrong with my forking?? The line "Finished dumping" never gets printed, it just hangs at "Dumping". I first thought the child is somehow unaware of the parent's import of pickle, so that's why I import it again, but that doesn't make a difference. Oh, and I do a os.execv("/bin/true",["true"]) at the end of the child because when IRAF closes in the child it apparently kills the child prematurely so then the code crashes cause there's no child. But just ignore IRAF/pyraf, just please tell me if there's some reason dump would hang in the child the way this is written.

thanks in advance.

Expand|Select|Wrap|Line Numbers
  1. imagedict={}
  3. for im in infodict:
  4.     Receive,Send =os.pipe()
  5.     pid = os.fork()
  6.     if pid !=0:  
  7.         # parent
  8.         os.close(Send)
  9.         Receive = os.fdopen(Receive) 
  10.         os.waitpid(pid, 0) 
  11.         imagedict[im]=pickle.load(Receive)          else:
  12.         # child
  13.         os.close(Receive)
  14.         Send=os.fdopen(Send,'w') 
  15.         import pickle
  16.         tmpdict=veryLargeProcedure(im)
  17.         print "Dumping"
  18.         pickle.dump(tmpdict,Send)
  19.         print "finished dumping"
  20.         Send.close()
  21.         os.execv("/bin/true",["true"])
Jan 16 '08 #1
Share this question for a faster answer!
Share on Google+

Post your reply

Sign in to post your reply or Sign up for a free account.