compile() and comments

[Ed Leafe]

I've noticed an odd behavior with compile() and code that does not
contain a trailing newline: if the last line is a comment inside of
any block, a syntax error is thrown, but if the last line is a non-
comment Python statement, there is no error. Here's an example (using
2.5.1 on OS X)

>>txt = """

.... def foo():
.... print 'bar' """

>>compcode = compile(t.strip (), "", "exec")
compcode

<code object <moduleat 0x79608, file "", line 2>

>>txt += " # Comment on last line"
compcode = compile(txt.str ip(), "", "exec")

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "", line 4
# Comment on last line
^
SyntaxError: invalid syntax

>>compcode = compile(txt.str ip() + "\n", "", "exec")
compcode

<code object <moduleat 0x79a40, file "", line 2>

Obviously the easy workaround is to add a newline and all is well, so
this isn't a show-stopper, but is this a bug?

-- Ed Leafe

[Fuzzyman]

Hello Ed,

It is certainly an odd restriction, but the docs for compile [1] do
explicitly state that the input must be newline terminated.

When compiling multi-line statements, two caveats apply: line
endings must be represented by a single newline character ('\n'), and
the input must be terminated by at least one newline character.

I always throw an extra newline onto any input to compile. I think the
behaviour maybe for detecting incomplete statements when you pass in
the obscure 'don't imply dedent' flag for interactive interpreter
loops.

[1] http://www.python.org/doc/2.5.2/lib/built-in-funcs.html

Michael Foord

On Oct 13, 1:06 pm, Ed Leafe <[e-mail] wrote:

I've noticed an odd behavior with compile() and code that does not
contain a trailing newline: if the last line is a comment inside of
any block, a syntax error is thrown, but if the last line is a non-
comment Python statement, there is no error. Here's an example (using
2.5.1 on OS X)

>>txt = """

... def foo():
... print 'bar' """

>>compcode = compile(t.strip (), "", "exec")
>>compcode

<code object <moduleat 0x79608, file "", line 2>

>>txt += " # Comment on last line"
>>compcode = compile(txt.str ip(), "", "exec")

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "", line 4
# Comment on last line
^
SyntaxError: invalid syntax

>>compcode = compile(txt.str ip() + "\n", "", "exec")
>>compcode

<code object <moduleat 0x79a40, file "", line 2>

Obviously the easy workaround is to add a newline and all is well, so
this isn't a show-stopper, but is this a bug?

-- Ed Leafe

--
http://www.ironpythoninaction.com/

[Ed Leafe]

On Oct 13, 2008, at 8:35 AM, Fuzzyman wrote:

It is certainly an odd restriction, but the docs for compile [1] do
explicitly state that the input must be newline terminated.

Understood; what I found odd was that if the last non-newline-
terminated statement was *not* a comment, no error was thrown.
-- Ed Leafe

[Terry Reedy]

Ed Leafe wrote:

On Oct 13, 2008, at 8:35 AM, Fuzzyman wrote:

>It is certainly an odd restriction, but the docs for compile [1] do
explicitly state that the input must be newline terminated.

Understood; what I found odd was that if the last
non-newline-terminated statement was *not* a comment, no error was thrown.

It kind of lulls one to sleep, then WHAM.

As near as I can tell, 2.5.2 and 3.0c1 require the comment to be on a
separate line to raise an exception.

>>print (compile("def f():\n pass #haha",'','exec '))

<code object <moduleat 00AADAD0, file "", line 1>

>>print (compile("def f():\n pass\n#haha",'' ,'exec')) # or
print (compile("def f():\n pass\n #haha",'','exec '))

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "", line 3
#haha

http://bugs.python.org/issue1479099
'compile' built-in function failures when missing EOL

Ideally, doc should match behavior. Consistent rejection would be
better. Other implementations might do this.

I also included this issue in doc issue
As near as I can tell, for 2.5.2 and 3.0c1, the limitation on compile
only applies when the last line only contains a comment.

>>print (compile("def f():\n pass #haha",'','exec '))

<code object <moduleat 00AADAD0, file "", line 1>

>>print (compile("def f():\n pass\n#haha",'' ,'exec')) # or
print (compile("def f():\n pass\n #haha",'','exec '))

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "", line 3
#haha

I would prefer more consistent behavior. I have opened a separate doc
issue that includes the documentation of this issue.
http://bugs.python.org/issue4118

Terry Jan Reedy

[Ed Leafe]

On Oct 13, 2008, at 7:20 PM, Terry Reedy wrote:

I would prefer more consistent behavior. I have opened a separate
doc issue that includes the documentation of this issue.
http://bugs.python.org/issue4118

Again, it was not a show-stopper by any means; more of a curiosity.
Thanks for verifying the inconsistency.

-- Ed Leafe

[Steve Holden]

Ed Leafe wrote:

On Oct 13, 2008, at 8:35 AM, Fuzzyman wrote:

>It is certainly an odd restriction, but the docs for compile [1] do
explicitly state that the input must be newline terminated.

Understood; what I found odd was that if the last
non-newline-terminated statement was *not* a comment, no error was thrown.

What if the last line is indented?

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC http://www.holdenweb.com/

[Terry Reedy]

Steve Holden wrote:

Ed Leafe wrote:

>On Oct 13, 2008, at 8:35 AM, Fuzzyman wrote:

>>It is certainly an odd restriction, but the docs for compile [1] do
explicitly state that the input must be newline terminated.

Understood; what I found odd was that if the last
non-newline-terminated statement was *not* a comment, no error was thrown.

What if the last line is indented?

Unterminated last-line comments raise an exception indented or not. I
did not test 'regular' non-indented lines.