module file length limitations on windows?

[Lonnie Princehouse]

I've run into some eccentric behavior... It appears that one of my
modules is being cut off at exactly 2^14 characters when I try to
import it. Has anyone else encountered this? I can't find any mention
of such a bug, and stranger yet, other modules that exceed 16384
characters seem to work just fine.

In particular, suppose that my module foo.py contains the following as
its last line:

thing = "goodbye world"

Now, suppose that the length of the file is 16383 characters. It works
just fine:

Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.

import foo

But if I make the string longer, it explodes:

thing = "goodbye world spam spam spam spam spam"

Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information. import foo

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "foo.py", line 583
thing = "goodbye world sp
^
SyntaxError: EOL while scanning single-quoted string
What in the world is going on here?!

This happens with Python 2.4 and 2.3.4 on win2k (under vmware), but it
does _not_ happen with 2.3.4 on Linux. Very strange! Could vmware be
the problem?

I have also tried replacing my unix newlines with DOS \r\n with the
exact same result.

I don't want to spend much time on this, since the workaround of
splitting the code into smaller files works just fine, but wow.. weird.

[Steve Holden]

Lonnie Princehouse wrote:

I've run into some eccentric behavior... It appears that one of my
modules is being cut off at exactly 2^14 characters when I try to
import it. Has anyone else encountered this? I can't find any mention
of such a bug, and stranger yet, other modules that exceed 16384
characters seem to work just fine.

In particular, suppose that my module foo.py contains the following as
its last line:

thing = "goodbye world"

Now, suppose that the length of the file is 16383 characters. It works
just fine:

Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.

import foo
But if I make the string longer, it explodes:

thing = "goodbye world spam spam spam spam spam"

Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
import foo
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "foo.py", line 583
thing = "goodbye world sp
^
SyntaxError: EOL while scanning single-quoted string
What in the world is going on here?!

This happens with Python 2.4 and 2.3.4 on win2k (under vmware), but it
does _not_ happen with 2.3.4 on Linux. Very strange! Could vmware be
the problem?

I have also tried replacing my unix newlines with DOS \r\n with the
exact same result.

I don't want to spend much time on this, since the workaround of
splitting the code into smaller files works just fine, but wow.. weird.

As a further data point, it doesn't appear to happen under Cygwin with
2.4 either:

sholden@dellboy ~
$ wc module.py
8177 2 8191 module.py

sholden@dellboy ~
$ python
Python 2.4 (#1, Dec 4 2004, 20:10:33)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.

import module hello
sholden@dellboy ~
$ vi module.py

sholden@dellboy ~
$ wc module.py
8177 5 8206 module.py

sholden@dellboy ~
$ python
Python 2.4 (#1, Dec 4 2004, 20:10:33)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright", "credits" or "license" for more information. import module

hello spam spam spam

Is it possible you've somehow inserted non-printing characters? Seems
bizarrely improbable to me that 2^14 would be a significant boundary to
the interpreter.

regards
Steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/
Holden Web LLC [phone] [phone]

[Lonnie Princehouse]

No non-printing characters.

However, I just tried copying the file (from a windows cmd prompt), and
the copy was cut off at the same point the interpreter is getting to.
When I edit the file with vim, though, the whole thing comes through.
I think this is a pretty strong indication that this monkey-business is
happening either in Windows or in vmware, and has nothing to do with
Python.

Confirmed. I rebooted the (virtual) machine and the problem is gone.
Weird.

[Stephen Waterbury]

[If there is a separate list for elementtree, please someone
clue me ... I didn't see one.]

Fredrik or other xml / elementtree gurus:

I see from the source that ElementTree.write() writes

<?xml version="1.0"? encoding=[whatever]>

at the beginning of the xml output if an encoding
other than utf-8 or us-ascii is selected. Shouldn't
it also write that if utf-8 or us-ascii is being
used? Or at least the

<?xml version="1.0"?>

?

Thanks,
Steve

[Fredrik Lundh]

Stephen Waterbury wrote:

[If there is a separate list for elementtree, please someone
clue me ... I didn't see one.]
the xml-sig is preferred, I think.
Fredrik or other xml / elementtree gurus:

I see from the source that ElementTree.write() writes

<?xml version="1.0"? encoding=[whatever]>

at the beginning of the xml output if an encoding
other than utf-8 or us-ascii is selected. Shouldn't
it also write that if utf-8 or us-ascii is being
used? Or at least the

<?xml version="1.0"?>

this is mostly for historical reasons; early versions only supported UTF-8 output,
and never generated encoding directives, so people ended up writing:

print >>myfile, "<document>"
for elem in list_of_elements:
ElementTree(elem).write(myfile)
print >>myfile, "</document>"

version 1.3 will probably include an option to always write the header.

(note that the XML header isn't needed for UTF-8 and ASCII; if it's not there,
a proper XML parser will treat the stream as UTF-8, and ASCII is a compatible
subset of UTF-8).

</F>