eval() of empty string hangs

[Chris Connett]

I have an interesting problem with eval().

---Background Info---
The program I'm working on launches a separate process with a popen to
do some highly specialized processing of input, then this process
leaves resultant data structured in files, in python syntax, at a
known absolute path, which is then read in, eval()'ed, and processed
further. It works fine when there are no errors in processing by this
external program, but when there are errors, it leaves an empty file.
When I .read() this file, I get an empty string, as expected.

---The interesting part---
When I eval the empty string in this context, instead of a
SyntaxError, like I expect and am prepared to handle, the program
hangs. If I run the example in a standard interactive shell, it
raises a SyntaxError, like I expect. Only here does it hang.

---More info---
This is Python 2.3.4, built from source, on Red Hat Enterprise Server
8. The code in question is started in a thread from a Zope external
method. Remember it works fine when there is an actual data structure
representation in the string, only when there should be a syntax error
does it hang (I've check by sticking in some other invalid evals, they
hang too), so it doesn't seem to be a Zope security issue (it's an
external method anyhow, so there shouldn't be any restrictions). It
is started in a new thread, so there is the possibility of the
exception silently crashing the thread, but I have a try-except to
catch everything around the offending code, and log some message, but
that message never comes either.

---Code---
def parseProofResults(self):
"""Parse the declarationOutput and referenceOutput files for PVS
entities and references.
"""

try:
self.logger.debug( 'before first parse' )
self.declarations = file(
self.temp_path + '/declarationOutput' )
self.logger.debug( 'opened file' )
self.declarations = self.declarations.read()
self.logger.debug( 'read:\n' + repr(self.declarations) )
self.logger.debug( self.declarations == '' )
self.declarations = eval( self.declarations )
self.logger.debug( 'before second parse' )
self.references = eval( file(
self.temp_path + '/referenceOutput' ).read() )
self.logger.debug( 'parsed' )
except SyntaxError, e:
raise ProofError, \
'Error parsing prover output, caused by unproven TCCs.'
-Where it's called from
# Stage 2
try:
candidate.parseProofResults()
except ProofError, e:
candidate.logger.log( submission.FAILURE, e )
return
except:
candidate.logger.error( 'Some error.' )
return

candidate.logger.log( submission.STATUS, 'Typechecking results
parsed.' )

---Log file excerpt---
At 2004-07-28 08:53:06,265, DEBUG for submission 344
before first parse
At 2004-07-28 08:53:06,266, DEBUG for submission 344
opened file
At 2004-07-28 08:53:06,266, DEBUG for submission 344
read:
''
At 2004-07-28 08:53:06,266, DEBUG for submission 344
True

---Summary---
I should be able to work around this, just check a special case for
the empty string, since that seems to be the only actual case that
arises, but this problem is a real puzzle to me, and I wanted to throw
it out there to see what others think.

[Peter Hansen]

Chris Connett wrote:

[...]

When I eval the empty string in this context, instead of a
SyntaxError, like I expect and am prepared to handle, the program
hangs. If I run the example in a standard interactive shell, it
raises a SyntaxError, like I expect. Only here does it hang. [...] I should be able to work around this, just check a special case for
the empty string, since that seems to be the only actual case that
arises, but this problem is a real puzzle to me, and I wanted to throw
it out there to see what others think.

As an alternative, and not requiring that you special case
anything, does it work if you simply append '\n' to the
string to be eval'ed in all cases?

-Peter

[Peter Otten]

Chris Connett wrote:

self.logger.debug( 'before second parse' )
self.references = eval( file(
self.temp_path + '/referenceOutput' ).read() )

You could modify the above to

self.logger.debug( 'before read' )
data = file(...).read()
self.logger.debug( 'before second parse' )
self.references = eval(data)

to verify that eval() is indeed the culprit.

Peter

[Peter Otten]

Sorry, I overlooked the earlier eval().

Peter

[Chris Connett]

Peter Hansen wrote:

As an alternative, and not requiring that you special case
anything, does it work if you simply append '\n' to the
string to be eval'ed in all cases?

I did try a number of ways of massaging the input string, though far
from exhaustive; I tried adding the single newline - it didn't help.

I did however localize the problem more tightly. It does not seem to be
eval that's hanging, but raise-ing is not working. When I tried my
original plan of special casing '', I added a check for equality, then
manually raised a SyntaxError. This exhibited the same behavior. I
then tried unconditionally raising SyntaxError, same result. Is it
possible that the exception is propagating up the wrong stack? This is
a separate thread, could it be on the main thread somehow? The creating
thread has ended by the time this exception is due to be raised. There
are numerous try-excepts to catch this, all should log at least some
message, but the log shows nothing.

[Peter Hansen]

Chris Connett wrote:

Is it
possible that the exception is propagating up the wrong stack? This is
a separate thread, could it be on the main thread somehow? The creating
thread has ended by the time this exception is due to be raised.

This doesn't make sense to me. What do you mean by 'creating thread'?
If it's what I think you mean, then either it's not really the
creating thread, or it hasn't really ended yet... after all, how
could a thread that has ended cause an exception to occur?

[Chris Connett]

Peter Hansen wrote:

Chris Connett wrote:

Is it possible that the exception is propagating up the wrong stack?
This is a separate thread, could it be on the main thread somehow?
The creating thread has ended by the time this exception is due to be
raised.

This doesn't make sense to me. What do you mean by 'creating thread'?
If it's what I think you mean, then either it's not really the
creating thread, or it hasn't really ended yet... after all, how
could a thread that has ended cause an exception to occur?

I didn't mean to confuse, just throwing some ideas out, sorry.

Here's the big picture:
A function is called, which creates a Submission object, and then calls
a sequence of methods on it. Since this is a normal everyday function,
the user must wait for it to finish. So after some basic integrity
checks, it creates a new thread to do the major processing, which could
take a significant amount of time. Once that thread is created and
started, the function the user called returns, and the user goes on
their merry way. The thread from which the user originally called the
first function should end shortly after that, but as always with
threading, no guarantees.

....

Well, I've found the problem. My own oversight, a NameError in the top
level except block, I forgot to qualify the name of the Exception being
caught. I should have used pychecker, I have no excuse not to have,
it's installed. Sorry to have wasted the group's time, but I have
learned from this greatly. Nothing like complete embarrassment in front
of one's peers to make a lesson stick. ;)