py3k concerns. An example

[Aaron Watters]

Why is the migration to py3k a concern?
For example I have libraries which use string%dictionary
substitution where the dictionary is actually an object
which emulates a dictionary. The __getitem__ for
the object can be very expensive and is only called when
needed by the string substitution.

In py3k string%dictionary is going away. Why?
I have no idea.

The replacement is a string.format(...) method
which supports dictionary calling.
string.format(**dictionary)
But dictionary
calling doesn't support dictionary emulation.
So in the example below the substitution works
but the call fails.

=== code

class fdict(dict):
def __getitem__(self, item):
return "got("+item+")"

def fn(**d):
print d["boogie"]

if __name__=="__main__":
fd = fdict()
print "attempting string substitution with fake dictionary"
print
print "hello there %(boogie)s" % fd # <-- works
print
print "now attempting function call with fake dictionary"
print
fn(**fd) # <-- fails

=== output

% python2.6 dtest.py
attempting string substitution with fake dictionary

hello there got(boogie)

now attempting function call with fake dictionary

Traceback (most recent call last):
File "dtest.py", line 17, in <module>
fn(**fd)
File "dtest.py", line 7, in fn
print d["boogie"]
KeyError: 'boogie'

==== end of output

Consequently there is no simple way to translate
my code, I think. I suspect you will find this kind of subtle
issue in many places. Or worse, you won't find it
until after your program has been installed
in production.

It's a damn shame because
if string%dict was just left in it wouldn't be an issue.

Also, if making f(**d) support dict emulation
has any negative performance implications
then I don't want it please.

sigh. -- Aaron Watters

===
http://www.xfeedme.com/nucular/pydis...EXT=crack+open

[J. Cliff Dyer]

On Fri, 2008-04-18 at 08:58 -0700, Aaron Watters wrote:

Why is the migration to py3k a concern?
For example I have libraries which use string%dictionary
substitution where the dictionary is actually an object
which emulates a dictionary. The __getitem__ for
the object can be very expensive and is only called when
needed by the string substitution.

In py3k string%dictionary is going away. Why?
I have no idea.

The replacement is a string.format(...) method
which supports dictionary calling.
string.format(**dictionary)
But dictionary
calling doesn't support dictionary emulation.
So in the example below the substitution works
but the call fails.

=== code

class fdict(dict):
def __getitem__(self, item):
return "got("+item+")"

def fn(**d):
print d["boogie"]

if __name__=="__main__":
fd = fdict()
print "attempting string substitution with fake dictionary"
print
print "hello there %(boogie)s" % fd # <-- works
print
print "now attempting function call with fake dictionary"
print
fn(**fd) # <-- fails

=== output

% python2.6 dtest.py
attempting string substitution with fake dictionary

hello there got(boogie)

now attempting function call with fake dictionary

Traceback (most recent call last):
File "dtest.py", line 17, in <module>
fn(**fd)
File "dtest.py", line 7, in fn
print d["boogie"]
KeyError: 'boogie'

==== end of output

Consequently there is no simple way to translate
my code, I think. I suspect you will find this kind of subtle
issue in many places. Or worse, you won't find it
until after your program has been installed
in production.

It's a damn shame because
if string%dict was just left in it wouldn't be an issue.

Also, if making f(**d) support dict emulation
has any negative performance implications
then I don't want it please.

sigh. -- Aaron Watters

===
http://www.xfeedme.com/nucular/pydis...EXT=crack+open

I was with you on this issue right up until that last paragraph. You
want it, but only if its free. That's ridiculous. Every thing a
computer does requires processor cycles.

Do you really mean to tell me that string interpolation has been a major
bottleneck for you? Now I think you're just whining because you like to
hear yourself whine. Try coming up with a real standard for evaluation.
How much of a performance hit will actually cause you trouble? 1% extra
on string interpolation? 10%? 50%? 200%?

You do provide a link to a website called xfeedme.com. And I just fed
you. IHBT. HAND. :-/

--
Oook,
J. Cliff Dyer
Carolina Digital Library and Archives
UNC Chapel Hill

[Aaron Watters]

Try coming up with a real standard for evaluation.
How much of a performance hit will actually cause you trouble? 1% extra
on string interpolation? 10%? 50%? 200%?

You misread my comment. I don't want function calls to support
dictionary emulation if there is a speed penalty because this is
such a special case and function calls are so important.
I don't really know, but I think "fixing" the above issue for
string.format(...) might involve changing the representation of
every python stack frame... not worth it in this case if there
is any penalty (afaik there isn't).

An alternative is to provide an alternate interface to string.format
so
that you could pass in an object which might emulate a dictionary,
like string.formatDict(D) -- or you could even adopt a shortcut
notation like string % D -- hey there's an idea!
-- Aaron Watters

===
http://www.xfeedme.com/nucular/pydis...gnore+warnings

[Kay Schluehr]

On 18 Apr., 20:07, Aaron Watters <[e-mail]:

I don't really know, but I think "fixing" the above issue for
string.format(...) might involve changing the representation of
every python stack frame... not worth it in this case if there
is any penalty (afaik there isn't).

In Python 2.6 the format function simply calls PyDict_GetItem on the
dictionary object and raises a KeyError if it fails. A dict emulation
would require one more lookup if the fast lookup has no result. It
would be a speed penalty in the general case if this lookup would be
implemented in PyDict_GetItem but handling a special case in the
format implementation might not look that bad.

[Matimus]

On Apr 18, 8:58 am, Aaron Watters <[e-mail]:

Why is the migration to py3k a concern?
For example I have libraries which use string%dictionary
substitution where the dictionary is actually an object
which emulates a dictionary. The __getitem__ for
the object can be very expensive and is only called when
needed by the string substitution.

In py3k string%dictionary is going away. Why?
I have no idea.

The replacement is a string.format(...) method
which supports dictionary calling.
string.format(**dictionary)
But dictionary
calling doesn't support dictionary emulation.
So in the example below the substitution works
but the call fails.

=== code

class fdict(dict):
def __getitem__(self, item):
return "got("+item+")"

def fn(**d):
print d["boogie"]

if __name__=="__main__":
fd = fdict()
print "attempting string substitution with fake dictionary"
print
print "hello there %(boogie)s" % fd # <-- works
print
print "now attempting function call with fake dictionary"
print
fn(**fd) # <-- fails

=== output

% python2.6 dtest.py
attempting string substitution with fake dictionary

hello there got(boogie)

now attempting function call with fake dictionary

Traceback (most recent call last):
File "dtest.py", line 17, in <module>
fn(**fd)
File "dtest.py", line 7, in fn
print d["boogie"]
KeyError: 'boogie'

==== end of output

Consequently there is no simple way to translate
my code, I think. I suspect you will find this kind of subtle
issue in many places. Or worse, you won't find it
until after your program has been installed
in production.

It's a damn shame because
if string%dict was just left in it wouldn't be an issue.

Also, if making f(**d) support dict emulation
has any negative performance implications
then I don't want it please.

sigh. -- Aaron Watters

===http://www.xfeedme.com/nucular/pydistro.py/go?FREETEXT=crack+open

The reason it doesn't work is that you are unpacking the dictionary
with **, and you have done nothing to define any keys or define a
length. I would describe the way you are using dict as a hack, and not
part of any standard feature. You make a good point that it breaks
your code, but at the same time the format facility gives you the
ability to do something similar but in a standard way. You simply
define a class with a __format__ method and pass that in instead of
your dict.
class MyClass:
def __format__(self, spec):
return "got({0})".format(spec)

c = MyClass()
print ("hey everybody {0:some words}".format(c))
print ("lets try this {0:more words} {0:even more words}".format(c))

should produce:

hey everybody got(some words)
lets try this got(more words) got(even more words)

My point is that instead of exploiting the string formatting of
dictionaries feature to create a custom format specifier, you actually
get to define your own format specifiers, something which is much more
powerful.

And actually, you can do this too... which is even simpler and allows
you to use your a portion of your existing solution:

class fdict(dict):
def __getitem__(self, item):
return "got("+item+")"

fd = fdict()
print ("hello there {0[boogie]} hello there {0[george])".format(fd))

Which should result in:

hello there got(boogie) hello there got(george)

* Keep in mind that I have not tested any of this code, there may be
bugs. I don't have Py3k or 2.6 installed locally.

I think this is a good trade-off.

Adding to that... don't worry about py3k. Nobody is forcing you to
switch. In fact, you are encouraged not to until you are comfortable.
Py3k won't _break_ your code. You wrote the code for Python 2.x use it
in 2.x. Python 2.x probably has a good 5-10 years remaining.

Matt

[Kay Schluehr]

On 18 Apr., 23:09, Matimus <[e-mail]:

The reason it doesn't work is that you are unpacking the dictionary
with **, and you have done nothing to define any keys or define a
length.

This is a non-issue. The class derives from dict; it has all the
desired attributes. It is also not a problem in particular because
these properties are not requested by format ( at least not in the
code I have examined which was admittedly just a critical section that
caused the exception ).

Adding to that... don't worry about py3k. Nobody is forcing you to
switch. In fact, you are encouraged not to until you are comfortable.
Py3k won't _break_ your code. You wrote the code for Python 2.x use it
in 2.x. Python 2.x probably has a good 5-10 years remaining.

These advices start to get annoying.

Software hardly ever exists in isolation for the sake of the beauty of
the algorithm but is supplementary to a large framework/engine/
library. So if e.g. Django switches to 3 everyone who works with it
has to switch sooner or later as well or lose track otherwise, no
matter how long Python 1.5.2 or Python 2.5.2 or whatever version will
be maintained. If Pythons code base becomes fragmented it will be
harmful and affect almost everyones work.

[Ivan Illarionov]

On Fri, 18 Apr 2008 16:19:38 -0700, Kay Schluehr wrote:

On 18 Apr., 23:09, Matimus <[e-mail]:

>The reason it doesn't work is that you are unpacking the dictionary
with **, and you have done nothing to define any keys or define a
length.

This is a non-issue. The class derives from dict; it has all the desired
attributes. It is also not a problem in particular because these
properties are not requested by format ( at least not in the code I have
examined which was admittedly just a critical section that caused the
exception ).

>Adding to that... don't worry about py3k. Nobody is forcing you to
switch. In fact, you are encouraged not to until you are comfortable.
Py3k won't _break_ your code. You wrote the code for Python 2.x use it
in 2.x. Python 2.x probably has a good 5-10 years remaining.

These advices start to get annoying.

Software hardly ever exists in isolation for the sake of the beauty of
the algorithm but is supplementary to a large framework/engine/ library.
So if e.g. Django switches to 3 everyone who works with it has to switch
sooner or later as well or lose track otherwise, no matter how long
Python 1.5.2 or Python 2.5.2 or whatever version will be maintained. If
Pythons code base becomes fragmented it will be harmful and affect
almost everyones work.

This Py3k-Django-related FUD starts to get annoying. AFAIK Django is
going to support everything from 2.3 to 3.0 from single codebase. 2to3
tool will be called from setup.py and code will have few 'if
sys.version_info < (3, 0)/else' tricks. Proof of concept already exists:
http://wiki.python.org/moin/PortingDjangoTo3k

I work with Django as well as *on* Django and 3rd party Django addons and
don't see any reason to worry.

Your particular problem may be solved by using a real dictionary with
real keys which will probably have positive performance impact on your
code and make it more clean.

Even if Python code base will become fragmented why is it harmfull? This
is a way our life work - you rise and get better or die. It is essential
part of progress and evolution. IMHO, we'll only get better Python and
better Python libraries.

--
Ivan

[Jason Scheirer]

On Apr 18, 4:19 pm, Kay Schluehr <[e-mail]:

On 18 Apr., 23:09, Matimus <[e-mail]:

The reason it doesn't work is that you are unpacking the dictionary
with **, and you have done nothing to define any keys or define a
length.

This is a non-issue. The class derives from dict; it has all the
desired attributes. It is also not a problem in particular because
these properties are not requested by format ( at least not in the
code I have examined which was admittedly just a critical section that
caused the exception ).

Adding to that... don't worry about py3k. Nobody is forcing you to
switch. In fact, you are encouraged not to until you are comfortable.
Py3k won't _break_ your code. You wrote the code for Python 2.x use it
in 2.x. Python 2.x probably has a good 5-10 years remaining.

These advices start to get annoying.

Software hardly ever exists in isolation for the sake of the beauty of
the algorithm but is supplementary to a large framework/engine/
library. So if e.g. Django switches to 3 everyone who works with it
has to switch sooner or later as well or lose track otherwise, no
matter how long Python 1.5.2 or Python 2.5.2 or whatever version will
be maintained. If Pythons code base becomes fragmented it will be
harmful and affect almost everyones work.

This has happened before, though -- I remember the pain of moving to
2.0 from the 1.5 branch way back when, and it wasn't getting my 1.5
code to work in 2.0, it was being jealous of all the cool features of
2.0 that I had to wait to get. I was working in production with 1.5 in
2000 and all the libraries available for Python gradually stopped
supporting 1.5 to pick up interesting 2.0 features that actually made
them easier to work with, and new libraries all began to assume you
were using a 2.0+ Python version because that's what was current. By
2003-2004 everyone I knew had switched over to 2.0, but by then I had
had nearly 5 years to port my legacy 1.5 code to 2.0, take advantage
of the 2.0 version's features, and do plenty of testing of my 1.5
codebase in 2.0 before I switched my production systems over. Not to
mention the fact that plenty of warning was offered BEFORE 2.0 was
released and 1.5 was not abruptly ended, but gradually phased out
until only the teeniest far ends of the long tail were using it. The
2.6->3.0 process is going to be even less of a pain than the 1.5->2.0
conversion, which was not hard at all going forward into it. You may
not want to switch, but by the time you decide to it will be pretty
easy to move on -- the extreme opposite reality being your application
will be so frozen that both your Python version and your codebase will
be fossils, left to hum on completely unchanged on some forgotten
server like so much other legacy code.

[Carl Banks]

On Apr 18, 11:58 am, Aaron Watters <[e-mail]:

Why is the migration to py3k a concern?
For example I have libraries which use string%dictionary
substitution where the dictionary is actually an object
which emulates a dictionary. The __getitem__ for
the object can be very expensive and is only called when
needed by the string substitution.

In py3k string%dictionary is going away. Why?
I have no idea.

The replacement is a string.format(...) method
which supports dictionary calling.
string.format(**dictionary)
But dictionary
calling doesn't support dictionary emulation.
So in the example below the substitution works
but the call fails.

=== code

class fdict(dict):
def __getitem__(self, item):
return "got("+item+")"

def fn(**d):
print d["boogie"]

if __name__=="__main__":
fd = fdict()
print "attempting string substitution with fake dictionary"
print
print "hello there %(boogie)s" % fd # <-- works
print
print "now attempting function call with fake dictionary"
print
fn(**fd) # <-- fails

=== output

% python2.6 dtest.py
attempting string substitution with fake dictionary

hello there got(boogie)

now attempting function call with fake dictionary

Traceback (most recent call last):
File "dtest.py", line 17, in <module>
fn(**fd)
File "dtest.py", line 7, in fn
print d["boogie"]
KeyError: 'boogie'

==== end of output

Consequently there is no simple way to translate
my code, I think. I suspect you will find this kind of subtle
issue in many places. Or worse, you won't find it
until after your program has been installed
in production.

It's a damn shame because
if string%dict was just left in it wouldn't be an issue.

Also, if making f(**d) support dict emulation
has any negative performance implications
then I don't want it please.

sigh. -- Aaron Watters

If you don't like Python 3, DON'T USE IT.

It's been stated repeatedly that 2.x and 3.x are going to be supported
in parallel for years.

Refusing to use 3, thus casting your brain-share vote against it, is
far more likely to have an effect than you coming here and making
everyone's life miserable with your pitiful whining.
Carl Banks

[Gabriel Genellina]

En Fri, 18 Apr 2008 12:58:56 -0300, Aaron Watters <aa***********@gmail.comescribió:

Why is the migration to py3k a concern?
For example I have libraries which use string%dictionary
substitution where the dictionary is actually an object
which emulates a dictionary. The __getitem__ for
the object can be very expensive and is only called when
needed by the string substitution.

In py3k string%dictionary is going away. Why?
I have no idea.

But not soon. It's not listed in PEP 3100 and according to this message
http://mail.python.org/pipermail/pyt...il/013094.html
%s formatting will not disappear until Python 3.3
You have plenty of time to evaluate alternatives. Your code may become obsolete even before 3.3 is shipped.

--
Gabriel Genellina

[Paul McGuire]

On Apr 19, 4:42*am, Carl Banks <[e-mail]:

>
If you don't like Python 3, DON'T USE IT.

I've read this position a number of times in this and related threads,
and it overlooks one constituency of Python developers - those who
develop and support modules for use by other Python users. As the
supporter of pyparsing, I really can't just "not use" Py3 - ignoring
Py3 means shutting out/holding back those of my users who do want to
use it, and pretty much consigning my module to eventual dustbin
status. Ideally, I can implement some form of cross-compatible code
so that I need maintain only a single code base, and I have managed to
do so on a number of fronts (with the help of Robert A. Clark):
- add support for both __bool__ and __nonzero__ (__nonzero__ calls
__bool__, so that upgrading to Py3 actually saves a function call)
- convert isinstance(x,basestring) to isinstance(x,__BASESTRING__) and
dynamically set __BASESTRING__ to basestring or str
- similar treatment for sys.maxint/maxsize -__MAX_INT__

I dodged a bullet when 3.0a3 added back in support for the 2.x form of
"except" for exception handling. 3.0a2 only supported "except varname
as ExceptionType:" and there was no way I could do this in a multi-
version compatible way.

My remaining hassle is print as function vs. print as statement. I
provide a number of default diagnostic methods, and I have not fully
gotten all to print nice - converting "print x" to "print (x)" is
simple enough, and "print (x,y)" replaces "print x,y" well enough when
running under Py3, but the same code in Py2.x now prints a tuple
instead of a nice string like before. I will probably punt on the
whole issue in the next release and just use sys.write.stdout/stderr
throughout, and " ".join() the args (another function call!) before
calling.

I wasn't aware of the coming deprecation of '%' string interpolation,
but at least it is deferred until 3.3, which does give me a few years
I should think before I absolutely must address it. This is really
not so much an issue for me as it is for my "customers." Pyparsing
returns its parsed tokens using a class that is dict-like in behavior,
but without extending dict (duck-typing at its finest!). I really
like that my users can parse an expression and access any named fields
directly and neatly in an interpolated string using "%(field_name)s".
If this is removed, pyparsing will continue to work as-is, but I feel
like a nice ease-of-access mode will have been lost to those who use
it.

Overall, I think I'm getting off pretty easy, but then pyparsing is a
small module with very limited use of the standard lib. I can
imagine that maintainers of larger libraries are having some serious
fits trying to support both versions with a single code base. And as
much as we all love Python-the-language, language features alone do
not help a language and its community of users to grow and
proliferate. I think most would agree that it is the cornucopia of
libraries that really make Python an environment for developing
production applications.

-- Paul

[Stefan Behnel]

Gabriel Genellina wrote:

You have plenty of time to evaluate alternatives. Your code may become obsolete even before 3.3 is shipped.

Sure, and don't forget to save two bytes when storing the year. ;)

Stefan

[Ross Ridge]

Carl Banks <[e-mail]:

If you don't like Python 3, DON'T USE IT.

That's the plan.

Paul McGuire <pt***@austin.rr.comwrote:

>I've read this position a number of times in this and related threads,
and it overlooks one constituency of Python developers - those who
develop and support modules for use by other Python users. As the
supporter of pyparsing, I really can't just "not use" Py3 - ignoring
Py3 means shutting out/holding back those of my users who do want to
use it, and pretty much consigning my module to eventual dustbin
status.

Eh. You can ingore it until your users start asking for it.

>Ideally, I can implement some form of cross-compatible code
so that I need maintain only a single code base, and I have managed to
do so on a number of fronts (with the help of Robert A. Clark):
- add support for both __bool__ and __nonzero__ (__nonzero__ calls
__bool__, so that upgrading to Py3 actually saves a function call)

Doing sometthing like the following would save the function call in
both cases:

class C(object):
def __bool__(self):
return False

__nonzero__ = __bool__

>- convert isinstance(x,basestring) to isinstance(x,__BASESTRING__) and
dynamically set __BASESTRING__ to basestring or str
- similar treatment for sys.maxint/maxsize -__MAX_INT__

I don't thiink using double underscores here is appropriate. It suggests
it's part of the language. Since "basestring" is no longer part of the
language, you could do:

if "basestring" not in globals():
basestring = str

>Overall, I think I'm getting off pretty easy, but then pyparsing is a
small module with very limited use of the standard lib.

Has the standard library changed that much? I thought was it mainly the
deletion of old seldom used modules that happens in new releases anyways.

>[...] And as much as we all love Python-the-language, language features
alone do not help a language and its community of users to grow
and proliferate. I think most would agree that it is the cornucopia
of libraries that really make Python an environment for developing
production applications.

Definately.

Ross Ridge

--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] rr****@csclub.uwaterloo.ca
-()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
db //

[Gabriel Genellina]

En Mon, 21 Apr 2008 16:42:41 -0300, Ross Ridge
<rr****@caffeine.csclub.uwaterloo.caescribió:

>Ideally, I can implement some form of cross-compatible code
so that I need maintain only a single code base, and I have managed to
do so on a number of fronts (with the help of Robert A. Clark):

Perhaps you can manage to keep your code compatible with all versions, but
AFAIK the reccomended strategy is to write code compatible with Python 2.6
and use the 2to3 tool to generate the 3.0 source. And *not* edit the 3.0
code unless one wants to maintain two branches.

>Overall, I think I'm getting off pretty easy, but then pyparsing is a
small module with very limited use of the standard lib.

Has the standard library changed that much? I thought was it mainly the
deletion of old seldom used modules that happens in new releases anyways.

*and* renaming of old module names that don't follow PEP8, and merging
others into packages for better structure.
That's another point where using the 2to3 tool is necesary -it takes care
of such changes- unless one wants to maintain two branches.

--
Gabriel Genellina

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

In py3k string%dictionary is going away.

Why do you say that? It's not going away in Python 3.0.

Regards,
Martin

[Hrvoje Niksic]

"Martin v. Löwis" <ma****@v.loewis.dewrites:

>In py3k string%dictionary is going away.

Why do you say that? It's not going away in Python 3.0.

I also got the impression that it was going away. PEP 3101's abstract
says:

This PEP proposes a new system for built-in string formatting
operations, intended as a replacement [sic] for the existing '%'
string formatting operator.

Under "Backward compatibility" it says that both systems can coexist
"until it comes time to deprecate the older system".

[Stefan Behnel]

Gabriel Genellina wrote:

>Has the standard library changed that much? I thought was it mainly the
deletion of old seldom used modules that happens in new releases anyways.

*and* renaming of old module names that don't follow PEP8, and merging
others into packages for better structure.
That's another point where using the 2to3 tool is necesary -it takes
care of such changes- unless one wants to maintain two branches.

conditional imports were never a problem in Python. Once things are in their
final place, you can add a try-except and import from two different places -
unless (I assume) you want to use 2to3, which might even break this approach.

Stefan

[=?ISO-8859-2?Q?=22Martin_v=2E_L=F6wis=22?=]

>>In py3k string%dictionary is going away.

>Why do you say that? It's not going away in Python 3.0.

I also got the impression that it was going away. PEP 3101's abstract
says:

This PEP proposes a new system for built-in string formatting
operations, intended as a replacement [sic] for the existing '%'
string formatting operator.

Under "Backward compatibility" it says that both systems can coexist
"until it comes time to deprecate the older system".

The PEP may say that it's going away, but it doesn't say that it
goes away in 3.0 - and indeed, it won't.

At some point in the future, somebody will likely propose that the PEP
will be executed. At that time, huge flame wars will start. I expect
that they settle in changing the PEP to explain that the old mechanism
gets removed in Python 4, to be release in 2018 :-)

Regards,
Martin

[Hrvoje Niksic]

"Martin v. Löwis" <ma****@v.loewis.dewrites:

>>>In py3k string%dictionary is going away.
Why do you say that? It's not going away in Python 3.0.

I also got the impression that it was going away. PEP 3101's abstract
says:

This PEP proposes a new system for built-in string formatting
operations, intended as a replacement [sic] for the existing '%'
string formatting operator.

Under "Backward compatibility" it says that both systems can coexist
"until it comes time to deprecate the older system".

The PEP may say that it's going away, but it doesn't say that it
goes away in 3.0 - and indeed, it won't.

Thanks for clarifying it. It is certainly unclear from the wording of
the PEP, given that 3.0 is regarded as the Python version allowed to
break backward compatibility.

At some point in the future, somebody will likely propose that the
PEP will be executed. At that time, huge flame wars will start. I
expect that they settle in changing the PEP to explain that the old
mechanism gets removed in Python 4, to be release in 2018 :-)

Indeed.

[Paul McGuire]

On Apr 21, 9:01*pm, "Gabriel Genellina" <[e-mail]>
wrote:

>
Perhaps you can manage to keep your code compatible with all versions, but*
AFAIK the reccomended strategy is to write code compatible with Python 2.6*
and use the 2to3 tool to generate the 3.0 source. And *not* edit the 3.0 *
code unless one wants to maintain two branches.

Gabriel -

(Thanks for chiming in on this sub-thread, I really enjoy reading your
posts.)

My point is that the recommended strategy MAY work for those who write
end point applications (I consider maintaining 2 branches to be in the
"not working" category), but it does NOT WORK for people who maintain
modules for other people to use, because those people may be on a
range of Python versions that extend beyond 2.6-3.0. So if I upgrade
my module to 2.6, those running on earlier versions can no longer use
it. At some point in the future, I'll probably be able to say "no
more support for pre-2.6", but it is a bit early to start saying that
now.

Likewise, I don't want to say "no support for 3.0" - people DO want to
try 3.0 out, and I WANT them to want and be able to use my module too.

Given the recommended strategy, and ruling out dual codebase, whom do
I tell that they can't use the next version of my module?

Again, to me, this is a non-issue because I've been able to create a
cross-version compatible single codebase for pyparsing. But it was a
bit dicey there for a while, and I think other module developers/
maintainers may not be so lucky.

So, I feel that the recommended strategy was devised with a narrow
group of developers in mind, and leaves module developers/maintainers,
who wish to target as broad a set of users as possible, faced with
choosing one of these strategies:
- create (if possible) single cross-version compatible code
- forego support of 3.0 users
- discontinue pre-2.6 support for future versions of their module
- maintain dual codebase

-- Paul

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Again, to me, this is a non-issue because I've been able to create a

cross-version compatible single codebase for pyparsing. But it was a
bit dicey there for a while, and I think other module developers/
maintainers may not be so lucky.

I'm more optimistic. I tried it for Django, and while the port is not
complete, it goes really well and supports "basic" operations (i.e.
the Django tutorial).

Based on your experience, and other reports, I think there is a fair
chance that you can support a wide range of versions (2.x and 3.x)
from a single code base for most projects.

- create (if possible) single cross-version compatible code
- forego support of 3.0 users
- discontinue pre-2.6 support for future versions of their module
- maintain dual codebase

One needs to consider the drawbacks in each case; for the single
codebase approach, the drawback probably is that readability
suffers, and the need for testing increases (but it does always
if you support multiple targets). It also requires expertise to
create such cross-version code in the first place, but that's
"just" a learning issue (i.e. you have to keep the rules in mind
that you want to follow).

Regards,
Martin

[Aaron Watters]

On Apr 24, 10:10*am, Paul McGuire <[e-mail]

end point applications (I consider maintaining 2 branches to be in the
"not working" category), but it does NOT WORK for people who maintain
modules for other people to use, because those people may be on a
range of Python versions that extend beyond 2.6-3.0. *So if I upgrade
my module to 2.6, those running on earlier versions can no longer use
it. *At some point in the future, I'll probably be able to say "no
more support for pre-2.6", but it is a bit early to start saying that
now.

In my view using a conversion tool on an ongoing basis is
not an option. It just adds a dependancy. What happens when
the conversion tool is upgraded in a non-backwards-compatible
way? Or do we have assurance that it won't be ;)?

Will changes to the converter
mean that the users of my
converted libraries have to start
using my tools in a different way? Even if it breaks some
users in a very subtle way it's not acceptible.
I have no interest in adding additional dependancies,
with an additional degree of freedom to break.

This is all made worse because it's binary -- with
upgrades to C or C# you usually had the option of cross
linking between old style components and new style
components (and at least with C these could usually
be made to work) and you could port the older stuff
with care. With py 3.0 and python 2.6 *everything*
either works with the interpreter or none of it does.
(Don't ask my users to install two interpreters:
they'll just give up and use something else.)

So if I want to support both I have to do everything
twice in the expected case and in the best case test
everything twice, at the same time, if I want to
support both versions and keep features in sync.
This is of course assuming that all the supporting
libraries do the same thing. If they don't and
one of the libraries doesn't support 2.* and another
doesn't support 3.*... I guess I'm just screwed.

I still think it's a shame, and I think it's
different in kind from python 1.x->2.x. 2.x
broke very little as I recall. Most 1.x code
just worked in 2.x and most of the rest required
very minor change. I think this is not the case
for 2.x->3.x.

-- Aaron (Scummy) Watters, hoping to shut up now.

ps: I didn't notice that % was vanishing later (3.3).
that's a bit better (whew ;) ).

pps: I have to note that it would be nice if the
ad-hominem (sp?) invective would drop out of
these threads -- it doesn't add a lot, I think.

===

http://www.xfeedme.com/nucular/pydis...bage+left+over

[Terry Reedy]

"Aaron Watters" <aa***********@gmail.comwrote in message
news:83**********************************@u36g2000 prf.googlegroups.com...

The reason that successive versions of 2.x broke so little is that starting
at about 2.2, all breakages (starting with int division change) were put
off until until 3.0 instead of being implemented as decided upon (with
warning, deprecation, and then removal). Now the debt comes due.

The new policy of mass breakage was a result of complaint about the old
policy of gradual breakage. Of course, the new policy will get complaints
both from those who preferred the old policy and those who want Python
frozen with nothing ever removed for improvements. No change, gradual
change, and jump change all have problems.

In a year, we will have a better idea of which was better.

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

In my view using a conversion tool on an ongoing basis is

not an option. It just adds a dependancy. What happens when
the conversion tool is upgraded in a non-backwards-compatible
way? Or do we have assurance that it won't be ;)?

The latter: if you include a copy of the converter with your
package, it won't change unless you change it. If you rely
on the copy of the converter that ships with Python 3.0, you
have the assurance that it won't change in a
non-backwards-compatible manner in all of 3.0.y.

Of course, the copies include in later 3.x releases may change, but
you'll have to test for the later 3.x releases, anyway, as
they may show incompatible changes themselves.

Will changes to the converter
mean that the users of my
converted libraries have to start
using my tools in a different way?

No. If the 2to3 converter is changed to support additional
source patterns to convert, it either won't affect your
code (if you don't use the pattern), or it may convert some
of your code that it didn't previously - then that conversion
may or may not be correct. So as a consequence, your library
may stop working; then you'll have to adjust either your
library, or go back to an older version of the converter.

In no case, however, will the users of the library need to
adjust their code to changing 2to3 output. 2to3 won't change
the library API.

I have no interest in adding additional dependancies,
with an additional degree of freedom to break.

Then remove the freedom by fixing a specific 2to3 version
(e.g. by including it).

So if I want to support both I have to do everything
twice in the expected case and in the best case test
everything twice, at the same time, if I want to
support both versions and keep features in sync.

This I don't understand. Why do you think you have to
do everything twice?

I still think it's a shame

[...]

pps: I have to note that it would be nice if the
ad-hominem (sp?) invective would drop out of
these threads -- it doesn't add a lot, I think.

shame
1 a. a painful emotion caused by consciousness of guilt,
shortcoming, or impropriety
2 a condition of humiliating disgrace or disrepute

So who do you think should feel guilt? Or should
be disgraced or disreputed?

Regards,
Martin

[Gabriel Genellina]

En Thu, 24 Apr 2008 11:10:55 -0300, Paul McGuire <pt***@austin.rr.com>
escribió:

On Apr 21, 9:01*pm, "Gabriel Genellina" <[e-mail]>
wrote:

>>
Perhaps you can manage to keep your code compatible with all versions,
but *
AFAIK the reccomended strategy is to write code compatible with Python
2.6 *
and use the 2to3 tool to generate the 3.0 source. And *not* edit the
3.0 *
code unless one wants to maintain two branches.

Gabriel -

(Thanks for chiming in on this sub-thread, I really enjoy reading your
posts.)

(And I enjoy using your parser! Not that I have to parse text so often,
but when it comes, the "pythonicity" of pyparsing is a great thing!)

My point is that the recommended strategy MAY work for those who write
end point applications (I consider maintaining 2 branches to be in the
"not working" category), but it does NOT WORK for people who maintain
modules for other people to use, because those people may be on a
range of Python versions that extend beyond 2.6-3.0. So if I upgrade
my module to 2.6, those running on earlier versions can no longer use
it. At some point in the future, I'll probably be able to say "no
more support for pre-2.6", but it is a bit early to start saying that
now.

Likewise, I don't want to say "no support for 3.0" - people DO want to
try 3.0 out, and I WANT them to want and be able to use my module too.

Given the recommended strategy, and ruling out dual codebase, whom do
I tell that they can't use the next version of my module?

Ok, code that is "2.6 compatible" doesn't mean that it only runs on 2.6...
I'm *trying* to write code that is "2to3 friendly" but anyway compatible
with older Python versions, and it should not require 2.6 to run. That
means not using "with" as a variable name, for example. (And on the other
side, also refrain from using some new 2.6 features like class decorators
and binary literals)
I hope the final version of the 2to3 tool will be a little more robust -
or at least, that one will always be able to write code in a way that it
can handle (and I *dont* want to maintain a 2.x and 3.x branches of any
code either)
Based on my limited experience I'd say that this approach *could* work,
that is, write the code base for 2.x (x >= 3, in my case) and
automatically convert to 3.0. That last stage may fail, but -I hope!- not
so often in the future. As always, YMMV...
Also note that I *don't* write code for other developers (thanks God!),
just final users (with Python 2.3/4/5)

Again, to me, this is a non-issue because I've been able to create a
cross-version compatible single codebase for pyparsing. But it was a
bit dicey there for a while, and I think other module developers/
maintainers may not be so lucky.

That's a bit tricky at least. How did you manage to avoid problems with
(string/unicode) and (bytes/string) in 3.0?

So, I feel that the recommended strategy was devised with a narrow
group of developers in mind, and leaves module developers/maintainers,
who wish to target as broad a set of users as possible, faced with
choosing one of these strategies:
- create (if possible) single cross-version compatible code
- forego support of 3.0 users
- discontinue pre-2.6 support for future versions of their module
- maintain dual codebase

I hope the first alternative will be actually viable, perhaps with help
from tools like 2to3...

--
Gabriel Genellina

[Donn Cave]

In article <48***********************@news.freenet.de>,
"Martin v. Löwis" <ma****@v.loewis.dewrote:

I still think it's a shame

[...]

pps: I have to note that it would be nice if the
ad-hominem (sp?) invective would drop out of
these threads -- it doesn't add a lot, I think.

shame
1 a. a painful emotion caused by consciousness of guilt,
shortcoming, or impropriety
2 a condition of humiliating disgrace or disrepute

- [in sing.] a regrettable or unfortunate situation or action: `it is a
shame that they are not better known'

If English isn't your 1st language, you deserve a lot of credit for your
mastery of it, but you need a better dictionary.

Donn Cave, do**@u.washington.edu

[Aaron Watters]

shame
1 a. a painful emotion caused by consciousness of guilt,
* * *shortcoming, or impropriety
2 a condition of humiliating disgrace or disrepute

Sigh. This is stupid (in the usual usage), but
I must reply because I can't control myself. I meant
usage 5:

"something regrettable, unfortunate, or outrageous:
it's a shame that he wasn't told."
-- http://www.yourdictionary.com/shame

I think outrageous is appropriate here because
I think it's outrageous to change the basic
usage for things like dictionary.keys() when
it would be so easy to leave the old definition
and add a new method like dictionary.keySet().
This would save me personally a great deal of
painful tedium, I suspect (especially considering
that I've implemented a lot of "dictionary-like"
objects -- so I'll have to change the way their
"keys" method works -- or something -- I haven't
figured it out yet...).

I know that the designers of Python are motivated
by a desire to attain a Platonic ideal of
aesthetic perfection primarily with a weaker desire
to make lives easy for people writing libraries
and tools somewhere further down the list,
but from my perspective it's a shame^H^H^H^H^H
regretable and unfortunate that the aesthetics
so often trumps other considerations.

In C# and java, for example, this sort of issue
has never been a problem
in my experience: stuff I wrote many versions ago
still works just fine with no changes (but please
note that I don't write gui stuff, which is less
stable -- I'm speaking of algorithmic and system
libraries).
-- Aaron Watters

===
btw: usage (5) for "shame" in the python source:
http://www.xfeedme.com/nucular/pydis...FREETEXT=shame

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

This would save me personally a great deal of

painful tedium, I suspect (especially considering
that I've implemented a lot of "dictionary-like"
objects -- so I'll have to change the way their
"keys" method works -- or something -- I haven't
figured it out yet...).

[...]

In C# and java, for example, this sort of issue
has never been a problem
in my experience: stuff I wrote many versions ago
still works just fine with no changes (but please
note that I don't write gui stuff, which is less
stable -- I'm speaking of algorithmic and system
libraries).

I don't see the connection. Why do you think your
..keys() implementation breaks just because dict.keys
has a different semantics now?

An existing application of an existing dict-like
object will continue to work just fine in Python 3,
right?

I can't find the change to dictionaries outrageous.

Regards,
Martin

[Aaron Watters]

On Apr 27, 9:18*pm, "Martin v. Löwis" <[e-mail]:

An existing application of an existing dict-like
object will continue to work just fine in Python 3,
right?

Unless I mix my psuedodicts with standard dicts
in the same list, for example, or pass them to
functions intended to accept any dict-like object,
including the especially important case of standard
dicts.

Who knows? Maybe I'm wrong about this being a much of
problem. 20+ years experience warns me strongly
to be very afraid, however. It would be great if I
didn't have to think about it.

Can anyone recommend a good book on Ruby :)?
-- Aaron Watters

===
http://www.xfeedme.com/nucular/pydis...ssary+breakage

[Ben Finney]

Aaron Watters <aa***********@gmail.comwrites:

I think it's outrageous to change the basic usage for things like
dictionary.keys() when it would be so easy to leave the old
definition and add a new method like dictionary.keySet().

Except that name would be outrageously non-conformant with PEP 8.

--
\ "Yesterday I parked my car in a tow-away zone. When I came back |
`\ the entire area was missing." -- Steven Wright |
_o__) |
Ben Finney

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Unless I mix my psuedodicts with standard dicts

in the same list, for example, or pass them to
functions intended to accept any dict-like object,
including the especially important case of standard
dicts.

Who knows? Maybe I'm wrong about this being a much of
problem.

I think so. I expect that in the typical application, you either
won't notice the difference in behavior at all, or you get an
exception the first time, in which case you can easily adjust
the code to support both cases. This expectation is supported
by experience both from adjusting the Python standard library
itself, and from converting Django to Python 3.

In my experience (from the same applications), by far the most
significant change in Python 3 is the change to the meaning of
string literals, and the disposal of the 2.x str type.

Regards,
Martin

[Paul McGuire]

On Apr 24, 9:21*am, "Martin v. Löwis" <[e-mail]:

Again, to me, this is a non-issue because I've been able to create a
cross-version compatible single codebase for pyparsing. *But it was a
bit dicey there for a while, and I think other module developers/
maintainers may not be so lucky.

I'm more optimistic. I tried it for Django, and while the port is not
complete, it goes really well and supports "basic" operations (i.e.
the Django tutorial).

Based on your experience, and other reports, I think there is a fair
chance that you can support a wide range of versions (2.x and 3.x)
from a single code base for most projects.

- create (if possible) single cross-version compatible code
- forego support of 3.0 users
- discontinue pre-2.6 support for future versions of their module
- maintain dual codebase

One needs to consider the drawbacks in each case; for the single
codebase approach, the drawback probably is that readability
suffers, and the need for testing increases (but it does always
if you support multiple targets). It also requires expertise to
create such cross-version code in the first place, but that's
"just" a learning issue (i.e. you have to keep the rules in mind
that you want to follow).

Regards,
Martin

Martin --

Well, my concerns about 3.0 continue to be founded. Up until now I
have been just reading "What's New" docs, and relying on input from
others as to what changes I will have to make. Yesterday I downloaded
and installed 3.0a5, and it is clear that there is *no* way to have a
single code base for any but the most trivial module.

The biggest sticking point is the incompatibility of "except" syntax.
Apparently the promise of syntax compatibility in the 3.0a4 docs was
an overstatement. Syntax can't be conditionalized out with "if
version >= 3:" type code, so I must have separate Python 2.x and
Python 3.x codebases, as will anyone whose modules include exception
handlers.

I also tripped over the conversion from "<iterator>.next()" to
"next(<iterator>)". Fortunately, this only occurs in my test cases,
so my runtime module will not be affected by extra conditional
testing, just to pick the right code form for the particular Python
version that is running.

Conversion of print from statement to function is definitely more
painful than I thought it would be. Again, I am fortunate that this
largely affects only my test code, with only minimal impact on my
runtime module.

I continue to struggle with some of the refactoring done in references
to func.func_code, im_self, etc. changing to __code__, __self__, and
so on. There is a bit of code in Pyparsing that normalizes user
callbacks to a standard call using 3 arguments, and I have to
compensate for functions that have self args, or are callables. I see
that there is some movement to "get rid of bound methods," but I don't
fully understand how this will be done, nor how I will adjust my code
to compensate.

In fact, I will probably set this effort aside for now, so as to avoid
converting/conditionalizing code to alpha versions of Python, and
having to change them again for later alphas or the final release.

By the way Martin, your work on 2to3 was especially helpful in this
effort - thank you!

In summary, I am adjusting my plans for future support of Python
versions in pyparsing. As of a pyparsing release some time after
August, I will only support Python 3.0 versions with any new features
or bugfixes, and the pyparsing version that is compatible with Python
2.x will be frozen.

-- Paul

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Well, my concerns about 3.0 continue to be founded. Up until now I

have been just reading "What's New" docs, and relying on input from
others as to what changes I will have to make. Yesterday I downloaded
and installed 3.0a5, and it is clear that there is *no* way to have a
single code base for any but the most trivial module.

I think there is a way, if you can use 2to3.

The biggest sticking point is the incompatibility of "except" syntax.
Apparently the promise of syntax compatibility in the 3.0a4 docs was
an overstatement. Syntax can't be conditionalized out with "if
version >= 3:" type code, so I must have separate Python 2.x and
Python 3.x codebases, as will anyone whose modules include exception
handlers.

No, you don't need to maintain two code bases. Just maintain the 2.x
one, and generate the 3.x code every time you need it updated.

2to3 deals with the change to the except clauses just fine.

I also tripped over the conversion from "<iterator>.next()" to
"next(<iterator>)".

2to3 should also fix that one. Alternatively, you can add your own
next implementation to 2.x.

Conversion of print from statement to function is definitely more
painful than I thought it would be. Again, I am fortunate that this
largely affects only my test code, with only minimal impact on my
runtime module.

Again, 2to3 fixes these just fine.

I continue to struggle with some of the refactoring done in references
to func.func_code, im_self, etc. changing to __code__, __self__, and
so on.

There are also 2to3 fixers for these, although they might be incomplete.

By the way Martin, your work on 2to3 was especially helpful in this
effort - thank you!

Thanks - but this is really Collin Winter's work (with various
contributions of others, of course).

In summary, I am adjusting my plans for future support of Python
versions in pyparsing. As of a pyparsing release some time after
August, I will only support Python 3.0 versions with any new features
or bugfixes, and the pyparsing version that is compatible with Python
2.x will be frozen.

I don't want to interfere with your project plans, but I don't think
there is a factual need for such a process. Instead, you could continue
to write 2.x code, and produce 3.x code either at release or at install
time.

Regards,
Martin

[Hrvoje Niksic]

Paul McGuire <pt***@austin.rr.comwrites:

I see that there is some movement to "get rid of bound methods," but
I don't fully understand how this will be done, nor how I will
adjust my code to compensate.

Isn't it supposed to get rid of *un*bound methods? That is,
Class.method will simply return the function object used to define
'method', rather than a specialized "unbound method" object.
object.method will continue to return bound method objects, which are
required for method calls to work.