RELEASED Python 2.4, alpha 1

[Anthony Baxter]

On behalf of the Python development team and the Python community, I'm
happy to announce the first alpha of Python 2.4.

Python 2.4a1 is an alpha release. We'd greatly appreciate it if you
could download it, kick the tires and let us know of any problems you
find, but it is not suitable for production usage.

http://www.python.org/2.4/

In this release we have a number of new modules, a number of existing
modules that have been reimplemented in C for speed, a large number of
improvements and additions to existing modules and an even larger list
of bugs squished. See either the highlights, the What's New in Python
2.4, or the detailed NEWS file -- all available from the Python 2.4
webpage.

There will be at least one more alpha release in a couple of weeks to
pick up a few new features that didn't make it into the first alpha,
before we release 2.4 betas and then the final release.

Please log any problems you have with this release in the SourceForge
bug tracker (noting that you're using 2.4a1):

http://sourceforge.net/bugs/?group_id=5470

Enjoy the new release,
Anthony

Anthony Baxter
an*****@python.org
Python Release Manager
(on behalf of the entire python-dev team)

[Iwan van der Kleyn]

Anthony Baxter wrote:

On behalf of the Python development team and the Python community, I'm
happy to announce the first alpha of Python 2.4.

Congratulations on a terrific job, as always. Just to be curious:
clearly, function decorators didn't make it in this release. Will they
be incorporated in the beta?

Regards,

Iwan

[Michele Simionato]

Anthony Baxter <an*****@python.org> wrote in message news:<ma*************************************@pyth on.org>...

On behalf of the Python development team and the Python community, I'm
happy to announce the first alpha of Python 2.4.
<snip>

Uhm ... I see generator expressions have late bindings, just as list comprehensions:

f1,f2,f3=tuple(lambda : i for i in [1,2,3])
f1() 3 f2() 3 f3()

3

I was more in the camp of early bindings; I would like to know if late
bindings are final or subject to changes and it there a pronouncement
from Guido.
Michele Simionato

[Christopher T King]

On 9 Jul 2004, Michele Simionato wrote:

Uhm ... I see generator expressions have late bindings, just as list
comprehensions:

f1,f2,f3=tuple(lambda : i for i in [1,2,3])
f1() 3 f2() 3 f3() 3

I think this is a property of lambdas (or functions in general), rather
than comprehensions:

i=1
f1=lambda: i
i=2
f2=lambda: i
i=3
f3=lambda: i
f1() 3 f2() 3 f3()

3

[Irmen de Jong]

Anthony Baxter wrote:

On behalf of the Python development team and the Python community, I'm
happy to announce the first alpha of Python 2.4.

Great stuff.

One thing ; the format test isn't working:
(Mandrake 10, gcc 3.3.2)

$ make test

....
test_format
test test_format produced unexpected output:
************************************************** ********************
*** line 2 of actual output doesn't appear in expected output after line 1:
+ u'%f' % (1.0,) == u'1,000000' != '1.000000'
************************************************** ********************
test_fpformat
....
test_zlib
249 tests OK.
1 test failed:
test_format
30 tests skipped:
test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
.....
I'm wondering where the u'1,000000' comes from... because when I type
on the interactive prompt:
Python 2.4a1 (#1, Jul 9 2004, 15:42:46)
[GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

u'%f' % (1.0,) u'1.000000'

--Irmen.

[Tim Peters]

[Michele Simionato]

Uhm ... I see generator expressions have late bindings, just as list
comprehensions:

f1,f2,f3=tuple(lambda : i for i in [1,2,3])
f1() 3 f2() 3 f3()

3

I was more in the camp of early bindings; I would like to know if late
bindings are final or subject to changes and it there a pronouncement
from Guido.

Guido Pronounced: the expression in the leftmost "for" clause is
evaluated immediately, but all the rest is delayed. So in your
example, only "[1, 2, 3]" is evaluated at the time the genexp is
created. If you had tried to iterate instead over, say, range(1/0),
the ZeroDivisionError would have been raised immediately, which is the
real point of evaluating that one piece "early".

Don't ask me to justify the rest <wink>.

Guido doesn't really care about examples like yours. He thinks
genexps will overwhelmingly be consumed "on the same line" they're
created, and that people doing fancy-pants stuff like you're doing
there probably shouldn't (but could find more-or-less obvious
workarounds if they had to, given that they're obsessed enough to try
such fancy-pants stuff to begin with).

The world won't end either way (IMO), and (also IMO) Python has pushed
delayed code blocks in a scoped language without explicit scope
declarations about as far as it can without becoming plainly
incomprehensible.

[Michele Simionato]

Christopher T King <sq******@WPI.EDU> wrote in message news:<Pi*************************************@ccc3 .wpi.edu>...

I think this is a property of lambdas (or functions in general), rather
than comprehensions:

i=1
f1=lambda: i
i=2
f2=lambda: i
i=3
f3=lambda: i
f1() 3 f2() 3 f3()

3

No, it is due to the fact that looping does not create a new lexical
scope. This was discussed in depth in the past. Google in this
newgroup.
Scheme does it right:

; scheme is the same as in Python here
msi> (define i 1)
msi> (define (f1) i)
msi> (set! i 2)
msi> (define (f2) i)
msi> (set! i 3)
msi> (define (f3) i)
msi> (f1)
3
msi> (f2)
3
msi> (f3)
3
; Scheme is different in looping constructs
msi> (define-values (f1 f2 f3) (apply values (map (lambda(i) (lambda()
i)) '(1 2 3))))
msi> (f1)
1
msi> (f2)
2
msi> (f3)
3

After long discussions in previous threads, I do understand well why
it is so;
I also understand why in regular "for" loops Python behavior has to be
the one
it is, since "for" does not create a new lexical scope. In my opinion,
however, it would have made more sense to have generator-expressions
with
early bindings.
But, anyway, Tim Peters is right that this issue does not make a
big difference for casual programmers and sophysticated programmers
know the workarounds.

Michele Simionato

[Michele Simionato]

Tim Peters <ti********@gmail.com> wrote in message news:<ma*************************************@pyth on.org>...

Guido Pronounced: the expression in the leftmost "for" clause is
evaluated immediately, but all the rest is delayed. So in your
example, only "[1, 2, 3]" is evaluated at the time the genexp is
created. If you had tried to iterate instead over, say, range(1/0),
the ZeroDivisionError would have been raised immediately, which is the
real point of evaluating that one piece "early".

Don't ask me to justify the rest <wink>.

Guido doesn't really care about examples like yours. He thinks
genexps will overwhelmingly be consumed "on the same line" they're
created, and that people doing fancy-pants stuff like you're doing
there probably shouldn't (but could find more-or-less obvious
workarounds if they had to, given that they're obsessed enough to try
such fancy-pants stuff to begin with).

The world won't end either way (IMO), and (also IMO) Python has pushed
delayed code blocks in a scoped language without explicit scope
declarations about as far as it can without becoming plainly
incomprehensible.

It happened to me more than once to think that Guido made something
wrong and to change my mind months later. So this maybe one of those
occasions. The problems is mostly for people coming from functional
languages. Incidentally, I got caught by this a couple of year ago
and this was the reason for my first post on the newsgroup (I was
playing with Tkinter at the time and lambda's are useful as callback
functions). At the time I had no experience with functional languages,
but still I had a functional mindset due to my strong mathematical
background and experience with Mathematica/Maple.

The simplest workaround is Pythonic in the sense that it is explicit

f1,f2,f3=tuple(lambda i=i: i for i in [1,2,3])

as one explicitly rebinds "i" at each iteration but still I cannot
find it other than hackish, since there is no point here in creating
a function with default arguments other than fixing the
binding-in-iteration
issue. What I would need is a better way to create function objects
than
lambda's; for instance I would like the ability to subclass the
function type and customize it to my needs (but I have already talked
about this in the
past
http://groups.google.it/groups?hl=it...gle.com&rnum=1
so I want repeat myself).
If I had that, the binding-in-iteration issue would be minor for me
and I
would not protest anymore. Actually I would probably think that it is
good
to have a broken binding-in-iteration behavior, so people are
encouraged
not to use lambda's and to generate their functions in other ways.

Michele Simionato

[Denis S. Otkidach]

On Sat, 9 Jul 2004, Michele Simionato wrote:

MS> The simplest workaround is Pythonic in the sense that it is
MS> explicit
MS>
MS> f1,f2,f3=tuple(lambda i=i: i for i in [1,2,3])
MS>
MS> as one explicitly rebinds "i" at each iteration but still I
MS> cannot
MS> find it other than hackish, since there is no point here in
MS> creating
MS> a function with default arguments other than fixing the
MS> binding-in-iteration

You can bind explicitly without default argument hack:

f1, f2, f3 = [(lambda i: lambda: i)(i) for i in [1, 2, 3]]
f1() 1 f2() 2 f3()

3

I think the same will apply to generator expression too.

--
Denis S. Otkidach
http://www.python.ru/ [ru]

[Bengt Richter]

On 9 Jul 2004 11:38:00 -0700, mi***************@gmail.com (Michele Simionato) wrote:

Anthony Baxter <an*****@python.org> wrote in message news:<ma*************************************@pyth on.org>...

On behalf of the Python development team and the Python community, I'm
happy to announce the first alpha of Python 2.4.
<snip>
Uhm ... I see generator expressions have late bindings, just as list comprehensions:

f1,f2,f3=tuple(lambda : i for i in [1,2,3])
f1()3 f2()3 f3()3

I guess I don't know what you mean by "late binding" -- i.e., I don't see
a semantic difference between (I don't have the generator expression version yet):

f1,f2,f3=[lambda : i for i in [1,2,3]]
f1(),f2(),f3() (3, 3, 3)

and
lamb = lambda : i
f1,f2,f3=[lamb for i in [1,2,3]]
f1(),f2(),f3() (3, 3, 3)

ISTM it is a matter of late lookup, more than late binding. I.e.,
the lambda expression specifies lookup of a name "i" when it is executed:
import dis
dis.dis(lambda : i) 1 0 LOAD_GLOBAL 0 (i)
3 RETURN_VALUE

The list comprehension didn't generate a different lookup: f1,f2,f3=[lambda : i for i in [1,2,3]]
dis.dis(f1) 1 0 LOAD_GLOBAL 0 (i)
3 RETURN_VALUE

BTW, if list comprehension variables bound in a loop-private scope instead of
the enclosing scope, the lookup of i would fail unless otherwise set ;-)
Will generator expressions bind in the eclosing scope too? Have the consequences been discussed?

Anyway, as I think you know, to get your desired end result, you have to create lambdas
that will do their lookups referring to different i's -- which you can do with closures, e.g.,
f1,f2,f3=[(lambda i: lambda : i)(i) for i in [1,2,3]]
f1(),f2(),f3() (1, 2, 3) dis.dis(f1) 1 0 LOAD_DEREF 0 (i)
3 RETURN_VALUE

Or here's another of several other possible ways:
f1,f2,f3=[(lambda i:i).__get__(i,int) for i in [1,2,3]]
f1(),f2(),f3() (1, 2, 3) dis.dis(f1) 1 0 LOAD_FAST 0 (i)
3 RETURN_VALUE

BTW, those are bound methods ... f1,f2,f3

(<bound method int.<lambda> of 1>, <bound method int.<lambda> of 2>, <bound method int.<lambda> of 3>)

I was more in the camp of early bindings; I would like to know if late
bindings are final or subject to changes and it there a pronouncement
from Guido.
Michele Simionato

Regards,
Bengt Richter

[Michele Simionato]

bo**@oz.net (Bengt Richter) wrote in message

I guess I don't know what you mean by "late binding" -- i.e., I don't see
a semantic difference between (I don't have the generator expression version yet):

>>> f1,f2,f3=[lambda : i for i in [1,2,3]]
>>> f1(),f2(),f3() (3, 3, 3)

and
>>> lamb = lambda : i
>>> f1,f2,f3=[lamb for i in [1,2,3]]
>>> f1(),f2(),f3() (3, 3, 3)

ISTM it is a matter of late lookup, more than late binding. I.e.,
the lambda expression specifies lookup of a name "i" when it is executed:
Yes, I was unsure of the right term to use, I meant late lookup (so
I used the term "early binding" improperly, but you understood what
I asked anyway ;).

>>> import dis
>>> dis.dis(lambda : i) 1 0 LOAD_GLOBAL 0 (i)
3 RETURN_VALUE

The list comprehension didn't generate a different lookup: >>> f1,f2,f3=[lambda : i for i in [1,2,3]]
>>> dis.dis(f1) 1 0 LOAD_GLOBAL 0 (i)
3 RETURN_VALUE

BTW, if list comprehension variables bound in a loop-private scope instead of
the enclosing scope, the lookup of i would fail unless otherwise set ;-)
Will generator expressions bind in the eclosing scope too? Have the consequences been discussed?

Python 2.4a1 (#1, Jul 10 2004, 02:19:27)
[GCC 3.3.1 (Mandrake Linux 9.2 3.3.1-2mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

ls=[i for i in [1,2,3]]
i 3 it=(j for j in [1,2,3])
j

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'j' is not defined

I like this.
Anyway, as I think you know, to get your desired end result, you have to create lambdas
that will do their lookups referring to different i's -- which you can do with closures, e.g.,
>>> f1,f2,f3=[(lambda i: lambda : i)(i) for i in [1,2,3]]
>>> f1(),f2(),f3() (1, 2, 3) >>> dis.dis(f1) 1 0 LOAD_DEREF 0 (i)
3 RETURN_VALUE

Or here's another of several other possible ways:
>>> f1,f2,f3=[(lambda i:i).__get__(i,int) for i in [1,2,3]]
>>> f1(),f2(),f3() (1, 2, 3) >>> dis.dis(f1) 1 0 LOAD_FAST 0 (i)
3 RETURN_VALUE

BTW, those are bound methods ... >>> f1,f2,f3

(<bound method int.<lambda> of 1>, <bound method int.<lambda> of 2>, <bound method int.<lambda> of 3>)

Yes Bengt, but would you seriously use such "solutions"? The default argument
trick is ugly but far better than those horrors! ;) What I do in reality
is to I create a custom function factory (or a factory of callable
objects) and I pass it to the list comprehension with "i" as a parameter.
My complaint is that it is more verbose than I would like for short
functions where I would consider a "lambda".

Michele Simionato
Michele

[Anthony Baxter]

> Uhm ... I see generator expressions have late bindings, just as list comprehensions:

I think you'll find if you check the PEP that this is the decision of the BDFL.

The feeling was that early binding was too different and would lead to
more confusion than late binding. While late binding can give suprising
results if you're not aware of it, it is at least consistent with other language
features (such as lambdas and list comps)

Anthony

[Anthony Baxter]

On Fri, 09 Jul 2004 11:22:27 +0200, Iwan van der Kleyn <no**@none.net> wrote:

Congratulations on a terrific job, as always. Just to be curious:
clearly, function decorators didn't make it in this release. Will they
be incorporated in the beta?

They are planned for the next alpha. There will be at least one more
alpha release before the beta - possibly two! The more folks that download
the first alpha and let us know of problems _now_, the sooner a 2.4 final
will be done.

[Michael Hudson]

Irmen de Jong <[e-mail]> writes:

Anthony Baxter wrote:

On behalf of the Python development team and the Python community, I'm
happy to announce the first alpha of Python 2.4.

Great stuff.

One thing ; the format test isn't working:
(Mandrake 10, gcc 3.3.2)

$ make test

...
test_format
test test_format produced unexpected output:
************************************************** ********************
*** line 2 of actual output doesn't appear in expected output after line 1:
+ u'%f' % (1.0,) == u'1,000000' != '1.000000'
************************************************** ********************

[snippety]

This has the foul, rotten stench of something involving locales. What
is the default locale on your system? There's probably some test that
runs before test_format on your system that is messing things up.
Binary chop?

Cheers,
mwh

--
MARVIN: Oh dear, I think you'll find reality's on the blink again.
-- The Hitch-Hikers Guide to the Galaxy, Episode 12

[Irmen de Jong]

Michael Hudson wrote:

Irmen de Jong <[e-mail]> writes:

Anthony Baxter wrote:

On behalf of the Python development team and the Python community, I'm
happy to announce the first alpha of Python 2.4.
Great stuff.

One thing ; the format test isn't working:
(Mandrake 10, gcc 3.3.2)

$ make test

...
test_format
test test_format produced unexpected output:
************************************************ **********************
*** line 2 of actual output doesn't appear in expected output after line 1:
+ u'%f' % (1.0,) == u'1,000000' != '1.000000'
************************************************ **********************

[snippety]

This has the foul, rotten stench of something involving locales. What
is the default locale on your system? There's probably some test that
runs before test_format on your system that is messing things up.

My default locale is NL_nl, which does indeed contain ',' for the decimal symbol.
And I have to agree with your feeling that an earlier test messes something up,
because when I run the test_format test by itself, it runs fine without error.

However, when I switch locales to en_US, the test still fails! (same error).

Binary chop?

Sorry, what do you mean by that?

--Irmen

[Martin v. Löwis]

Irmen de Jong wrote:

I'm wondering where the u'1,000000' comes from... because when I type
on the interactive prompt:
Python 2.4a1 (#1, Jul 9 2004, 15:42:46)
[GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> u'%f' % (1.0,) u'1.000000' >>>

Does the test also fail if you run it as "python Lib/test/regrtest.py
test_format"? If not, does it fail if you run it as
python Lib/test/regrtest.py <all test cases that are executed before
test_format> test_format? If yes, please try to eliminate all prior
test cases that don't contribute to the failure, and report the
minimum sequence of test cases needed to make it fail.

Regards,
Martin

[Martin v. Löwis]

Irmen de Jong wrote:

I'm wondering where the u'1,000000' comes from... because when I type
on the interactive prompt:
Python 2.4a1 (#1, Jul 9 2004, 15:42:46)
[GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> u'%f' % (1.0,) u'1.000000' >>>

Does the test also fail if you run it as "python Lib/test/regrtest.py
test_format"? If not, does it fail if you run it as
python Lib/test/regrtest.py <all test cases that are executed before
test_format> test_format? If yes, please try to eliminate all prior
test cases that don't contribute to the failure, and report the
minimum sequence of test cases needed to make it fail.

Regards,
Martin

[Tim Peters]

[Irmen de Jong]
,,,

As Michael Hudson pointed out it's probably a test that
messes with the locale settings and forgets to set them back,
or some such.

In case you don't already know it, regrtest.py's little-used -f
argument lets you specify a file, containing the names of the tests
you want to run. It ignore lines in the file starting with '#'. So
it's a convenient way to do a binary-search kind of reduction when
looking for a minimal *set* of failing tests: edit the file, run,
stick '#' in front of half the remaining lines if it still fails then,
etc.

[Martin v. Löwis]

Irmen de Jong wrote:

I'll try when I have some time for that...
As Michael Hudson pointed out it's probably a test that
messes with the locale settings and forgets to set them back,
or some such.

Rather "some such". If this was a systematic error, it would
fail for many more people.

Regards,
Martin

[Irmen de Jong]

Tim Peters wrote:

[Irmen de Jong]
,,,

As Michael Hudson pointed out it's probably a test that
messes with the locale settings and forgets to set them back,
or some such.

In case you don't already know it, regrtest.py's little-used -f
argument lets you specify a file, containing the names of the tests
you want to run.

That was great, I quickly found out that test__locale is the cause.
Not only test_format but also test_unicode fail on my Mandrake 10 box.
My default locale is nl_NL. Behold:

[irmen@atlantis Python-2.4a1]$ cat testcases.in
test__locale
test_format
test_unicode

[irmen@atlantis Python-2.4a1]$ ./python Lib/test/regrtest.py -f testcases.in
test__locale
test_format
test test_format produced unexpected output:
************************************************** ********************
*** line 2 of actual output doesn't appear in expected output after line 1:
+ u'%f' % (1.0,) == u'1,000000' != '1.000000'
************************************************** ********************
test_unicode
test test_unicode failed -- Traceback (most recent call last):
File "/home/irmen/BUILD/Python-2.4a1/Lib/test/test_unicode.py", line 358, in
test_formatt ing
string_tests.MixinStrUnicodeUserStringTest.test_fo rmatting(self)
File "/home/irmen/BUILD/Python-2.4a1/Lib/test/string_tests.py", line 615, in
test_formatt ing
self.checkequal('0042.00', '%07.2f', '__mod__', 42)
File "/home/irmen/BUILD/Python-2.4a1/Lib/test/string_tests.py", line 56, in checkequal
realresult
AssertionError: u'0042.00' != u'0042,00'

1 test OK.
2 tests failed:
test_format test_unicode
[irmen@atlantis Python-2.4a1]$
--Irmen

[Martin v. Löwis]

Irmen de Jong wrote:

That was great, I quickly found out that test__locale is the cause.
Not only test_format but also test_unicode fail on my Mandrake 10 box.
My default locale is nl_NL.

Ok. Can you please run test__locale in verbose mode to find out what
the last locale is that it uses? Let's call it X.

Can you please then run interactively

locale.setlocale(locale.LC_NUMERIC, X)
print '%07.2f' % 42

It appears from the test results that this gives '0042,00' for
you; please confirm that it does.

It is bad that test__locale does not restore the locale, but perhaps
we take out that test entirely. However, the behaviour of % is not
supposed to change, so we need to investigate why it changes on your
machine.

Regards,
Martin

[Irmen de Jong]

Irmen de Jong wrote:

That was great, I quickly found out that test__locale is the cause.
Not only test_format but also test_unicode fail on my Mandrake 10 box.
My default locale is nl_NL. Behold:

[...]

I just saw that there are already 2 bug reports on this on
the bug tracker, to which I have added my observations:
bug #992078
bug #992081
--Irmen.