python24.dll and encodings ?

[Bugs]

I believe I read in a relatively recent thread that the reason
python24.dll is so large compared to previous releases is that all the
language encodings are linked into the library?

Are there any plans for future releases to split the encodings out so
that, for example, if someone wanted to make a smaller executable using
py2exe without all the language encodings, they could do so?

I suppose one could always compile their own version of the python24.dll
but for those of us that are compiler-challanged, splitting out the
encodings would be nice.

Is that even feasible?
Thanks!

[Martin v. Löwis]

Bugs wrote:

I believe I read in a relatively recent thread that the reason
python24.dll is so large compared to previous releases is that all the
language encodings are linked into the library?
Not only that (but also). In addition, it also contains modules that
were previously implemented as separate .pyd files (_csv, _sre,
_symtable, _winreg, datetime, mmap, parser).
Are there any plans for future releases to split the encodings out so
that, for example, if someone wanted to make a smaller executable using
py2exe without all the language encodings, they could do so?
I previously said that I would do such a thing if somebody provided a
specification other than "split out the encodings". I.e. write a PEP
that specifies how to determine whether an extension module should be
included into pythonxy.dll, and when it should not; this specification
should allow to reason about modules that haven't yet been contributed.

The PEP should be accompanied with an implementation, i.e. a set of
patches to PCbuild.

Then, the patch should be discussed, and find support in the community.
I suppose one could always compile their own version of the python24.dll
but for those of us that are compiler-challanged, splitting out the
encodings would be nice.

Is that even feasible?

Technically, yes. However, somebody needs to take the lead.

Regards,
Martin

[Bugs]

Martin v. Löwis wrote:

Not only that (but also). In addition, it also contains modules that
were previously implemented as separate .pyd files (_csv, _sre,
_symtable, _winreg, datetime, mmap, parser).
[snip]
I previously said that I would do such a thing if somebody provided a
specification other than "split out the encodings". I.e. write a PEP
that specifies how to determine whether an extension module should be
included into pythonxy.dll, and when it should not; this specification
should allow to reason about modules that haven't yet been contributed.

Thanks Martin! As the python24.dll now contains modules and codecs that
weren't linked into the DLL before 2.4 and as we now need a PEP to have
them split out again, I assume there was a PEP to have them included for
2.4? Can you point me to that PEP so I can evaluate writing a new PEP
to reverse it?
Thanks!

[Martin v. Löwis]

Bugs wrote:

Thanks Martin! As the python24.dll now contains modules and codecs that
weren't linked into the DLL before 2.4 and as we now need a PEP to have
them split out again, I assume there was a PEP to have them included for
2.4? Can you point me to that PEP so I can evaluate writing a new PEP
to reverse it?

There was no PEP. The current policy is "the maintainer of the Windows
port does whatever he considers the best thing". The previous maintainer
(Tim Peters) said this about the state of affairs in 2003:

http://mail.python.org/pipermail/pyt...er/038639.html

I offered a policy in:

http://mail.python.org/pipermail/pyt...er/038636.html

Thomas Heller proposed an explicit list in

http://mail.python.org/pipermail/pyt...er/038869.html

When I became maintainer of the Windows port, I made this policy:
everything should be included in pythonxy.dll, unless it requires
a library not included with the Python distribution.

I can't find a message where I communicated this policy at that time,
but I commented on the rationale several times since, e.g. in

http://groups.google.com/group/comp....5c8f597a86e743

Regards,
Martin

[Thomas Heller]

"Martin v. Löwis" <ma****@v.loewi s.de> writes:

Bugs wrote:

Thanks Martin! As the python24.dll now contains modules and codecs that
weren't linked into the DLL before 2.4 and as we now need a PEP to have
them split out again, I assume there was a PEP to have them included for
2.4? Can you point me to that PEP so I can evaluate writing a new PEP
to reverse it?

There was no PEP. The current policy is "the maintainer of the Windows
port does whatever he considers the best thing". The previous maintainer
(Tim Peters) said this about the state of affairs in 2003:

http://mail.python.org/pipermail/pyt...er/038639.html

I offered a policy in:

http://mail.python.org/pipermail/pyt...er/038636.html

Thomas Heller proposed an explicit list in

http://mail.python.org/pipermail/pyt...er/038869.html

When I became maintainer of the Windows port, I made this policy:
everything should be included in pythonxy.dll, unless it requires
a library not included with the Python distribution.

I can't find a message where I communicated this policy at that time,
but I commented on the rationale several times since, e.g. in

http://groups.google.com/group/comp....5c8f597a86e743

A PEP discussing the rationales would *really* be great.

Thomas

[Bugs]

Thomas Heller wrote:
[snip]

A PEP discussing the rationales would *really* be great.

My whole impetus for this thread was to minimize the size of Python
executables created with py2exe. Right now they tend to be rather
large, even for very small applications, as they include a bunch of
unutilized Python functionality and/or encodings.

It would be nice to see a Python environment structure that is more
desktop-application-friendly for applications that distribute the Python
environment WITH the application (i.e. using py2exe). Ideally, it would
be nice to be able to distribute as minimal amount of the Python runtime
environment as possible.

New PEP: "Small is beautiful!" ;-)

[Martin v. Löwis]

Bugs wrote:

New PEP: "Small is beautiful!" ;-)

A PEP would have to be more elaborate than that;
it best be accompanied with an implementation as well.
Somebody has to do the work.

Regards,
Martin

[Bugs]

Martin v. Löwis wrote:

Bugs wrote:

New PEP: "Small is beautiful!" ;-)

A PEP would have to be more elaborate than that;
it best be accompanied with an implementation as well.
Somebody has to do the work.

Regards,
Martin

That was my obviously ineffective attempt at humor, hence the ;-)

[Thomas Heller]

Bugs <do**@spam.me > writes:

Thomas Heller wrote:
[snip]

A PEP discussing the rationales would *really* be great.
My whole impetus for this thread was to minimize the size of Python
executables created with py2exe. Right now they tend to be rather
large, even for very small applications, as they include a bunch of
unutilized Python functionality and/or encodings.

That was what I guessed.
It would be nice to see a Python environment structure that is more
desktop-application-friendly for applications that distribute the
Python environment WITH the application (i.e. using py2exe). Ideally,
it would be nice to be able to distribute as minimal amount of the
Python runtime environment as possible.

Before py2exe had the single-file option, the most common 'complaint'
was the large number of files it generated. Now, the 'complaint' is
about the size of the executable. (Historically, the change in
Python2.4 was done by Martin on my request, that was before the
single-file option in py2exe. Plus, Python2.4 contains much more
encodings than 2.3 had.)

The PEP would probably have to find a balance between the number of
files and the size of a 'typical' py2exe built application. Personally,
I don't care too much on either the file number and the files sizes
since I use inno setup for distributing the applications.

Technically, as far as py2exe is concerned, it would be nice if the zlib
module was a builtin module because it would be easier to bootstrap the
executable - for obvious reasons the zlib module cannot be loaded from a
(compressed) zipfile. Unfortunately, this collides with Martin's
current policy to exclude extensions that need third party source code
from the python dll.

To summarize: Before someone requests a change to the builtin modules
list, a PEP would be needed to find the right balance.

Thomas

[Martin v. Löwis]

Thomas Heller wrote:

Technically, as far as py2exe is concerned, it would be nice if the zlib
module was a builtin module because it would be easier to bootstrap the
executable - for obvious reasons the zlib module cannot be loaded from a
(compressed) zipfile. Unfortunately, this collides with Martin's
current policy to exclude extensions that need third party source code
from the python dll.

That could change, of course. I personally would have no problems
requiring that zlib is available on the build machine - I always
have it available.

I would expect that others using the subversion trunk on Windows
would object, though. Perhaps it might be possible to create
some sort of compile time condition, ideally automatically
detected.

In any case, a PEP would have to answer these issues.

Regards,
Martin