int() 24 times slower then long() in Python 2.3

[Willem]

When I run the follwing code using Python 2.3:

from time import clock
t1 = clock ()
for i in range (10000): a = int ('bbbbaaaa', 16)
t2 = clock ()
for i in range (10000): a = long ('bbbbaaaa', 16)
t3 = clock ()
print (t2-t1) / (t3-t2)

it prints out: 23.9673206147

That seems to mean that the int-conversion is 24 times slower than the
long conversion.
The reason is that in Python 2.3 the int-conversion generates
warning messages that you *never* see but that consume a *lot* of CPU.

So it may happen that old code performing well under Python 2.2 suddenly
slows down a considerable amount under Python 2.3 without any percievable
change in the output ...

Willem Vree

BTW.
The message that you don't see is: OverflowWarning : string/unicode conversion
and you can make it appear by starting the progam with

Python -Wall script.py

or including the following lines in the source text:
import warnings
warnings.resetw arnings()

I suppose that Python is getting over-engineered by too professional
programmers. The first signs of decay?

[Peter Otten]

Willem wrote:

When I run the follwing code using Python 2.3:

from time import clock
t1 = clock ()
for i in range (10000): a = int ('bbbbaaaa', 16)
t2 = clock ()
for i in range (10000): a = long ('bbbbaaaa', 16)
t3 = clock ()
print (t2-t1) / (t3-t2)

it prints out: 23.9673206147

That seems to mean that the int-conversion is 24 times slower than the
long conversion.
The reason is that in Python 2.3 the int-conversion generates
warning messages that you *never* see but that consume a *lot* of CPU. I suppose that Python is getting over-engineered by too professional
programmers. The first signs of decay?

No. I think warning.py needs some _more_ engineering - throw in a dictionary
to speed up repetetive calls or so. In the meantime:

import warnings
def dummy(*args, **kw):
pass
warnings.warn = dummy

Now watch it speed up :)

Peter

[James Henderson]

On Tuesday 13 July 2004 3:09 pm, Willem wrote:

When I run the follwing code using Python 2.3:

from time import clock
t1 = clock ()
for i in range (10000): a = int ('bbbbaaaa', 16)
t2 = clock ()
for i in range (10000): a = long ('bbbbaaaa', 16)
t3 = clock ()
print (t2-t1) / (t3-t2)

it prints out: 23.9673206147

That seems to mean that the int-conversion is 24 times slower than the
long conversion.
The reason is that in Python 2.3 the int-conversion generates
warning messages that you *never* see but that consume a *lot* of CPU.

So it may happen that old code performing well under Python 2.2 suddenly
slows down a considerable amount under Python 2.3 without any percievable
change in the output ...

I don't think this code did work perfectly well under Python2.2 (see below),
or am I missing something?

Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.

for i in range (10000): a = int ('bbbbaaaa', 16) ....
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: int() literal too large: bbbbaaaa

--
James Henderson, Logical Progression Ltd
http://www.logicalprogression.net
http://mailmanager.sourceforge.net

[Istvan Albert]

Willem wrote:

So it may happen that old code performing well under Python 2.2 suddenly
slows down a considerable amount under Python 2.3 without any percievable
change in the output ...

IMO the speed at which a bunch of invalid conversions are
executed means nothing at all. Could you come up with an example
that show the same symptoms in a meaningful context?

i.

[paolo veronelli]

On 13 Jul 2004 14:34:25 -0700, Paul Rubin <"http://phr.cx"@NOSPAM. invalid>
wrote:

"Nick Smallbone" <ni**@nick8325. freeserve.co.uk > writes:

>
> IMO the speed at which a bunch of invalid conversions are
> executed means nothing at all. Could you come up with an example
> that show the same symptoms in a meaningful context?

What do you mean? bbbbaaaa is a hex number.

>>> int('bbbbaaaa', 16)

3149638314L

It's not an int. It has to attempt to convert to int, trap the error
and recover from it, and then convert to a long.

I don't undersand the meaning of int()
If I want an int() (two bytes?) I want two bytes.It should truncate.If the
result is the same with long() a part from the warning
I think int() is unmeaningfull.T ransparency is away from warnings.
This python is saying I used the wrong function and there at least two
cases:
I have wrong results.
He is doing my businness.

IMHO I want wrong results.
--
.....lotta dura per la verdura

[James Henderson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday 14 July 2004 8:32 am, paolo veronelli wrote:

On 13 Jul 2004 14:34:25 -0700, Paul Rubin <"http://phr.cx"@NOSPAM. invalid>

wrote:

"Nick Smallbone" <ni**@nick8325. freeserve.co.uk > writes:

> IMO the speed at which a bunch of invalid conversions are
> executed means nothing at all. Could you come up with an example
> that show the same symptoms in a meaningful context?

What do you mean? bbbbaaaa is a hex number.

>>> int('bbbbaaaa', 16)

3149638314L

It's not an int. It has to attempt to convert to int, trap the error
and recover from it, and then convert to a long.

I don't undersand the meaning of int()
If I want an int() (two bytes?) I want two bytes.It should truncate.If the
result is the same with long() a part from the warning
I think int() is unmeaningfull.T ransparency is away from warnings.
This python is saying I used the wrong function and there at least two
cases:
I have wrong results.
He is doing my businness.

IMHO I want wrong results.
--
....lotta dura per la verdura

I think you should look at PEP 237 "Unifying Long Integers and Integers" - it
may even reassure you. As Peter Hansen has already hinted in reply to
another of your messages, Python is in the process of unifying longs and
ints.

It used to be that calling int() on a big number gave an exception - perhaps a
more Pythonic version of your requested giving the wrong result - and now
that it doesn't I suppose there is a sense in which the current distinction
between ints and longs is "unmeaningf ul" from the user's point of view (it's
not meaningless under the hood of course). According to the PEP the decision
to keep two types was that it:

is far easier to implement, is backwards
compatible at the C API level, and in addition can be implemented
partially as a transitional measure.

Perhaps you thing that ints and longs should not be unified but I won't start
arguing till you come out and say it. :)

James
- --
James Henderson, Logical Progression Ltd
http://www.logicalprogression.net
http://mailmanager.sourceforge.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFA9QYUd1y XQ13iWmgRAmmSAK DWOH+vv9RAvPape mqXDSTax+P1uQCg zbbc
AHRxEquLPpWE32t asFiJOsw=
=taJY
-----END PGP SIGNATURE-----

[paolo veronelli]

On Wed, 14 Jul 2004 11:08:16 +0100, James Henderson
<ja***@logicalp rogression.net> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday 14 July 2004 8:32 am, paolo veronelli wrote:

On 13 Jul 2004 14:34:25 -0700, Paul Rubin
<"http://phr.cx"@NOSPAM. invalid>

wrote:

> "Nick Smallbone" <ni**@nick8325. freeserve.co.uk > writes:
>> > IMO the speed at which a bunch of invalid conversions are
>> > executed means nothing at all. Could you come up with an example
>> > that show the same symptoms in a meaningful context?
>>
>> What do you mean? bbbbaaaa is a hex number.
>>
>> >>> int('bbbbaaaa', 16)
>>
>> 3149638314L
>
> It's not an int. It has to attempt to convert to int, trap the error
> and recover from it, and then convert to a long.
I don't undersand the meaning of int()
If I want an int() (two bytes?) I want two bytes.It should truncate.If
the
result is the same with long() a part from the warning
I think int() is unmeaningfull.T ransparency is away from warnings.
This python is saying I used the wrong function and there at least two
cases:
I have wrong results.
He is doing my businness.

IMHO I want wrong results.

I think you should look at PEP 237 "Unifying Long Integers and Integers"
- it
may even reassure you. As Peter Hansen has already hinted in reply to
another of your messages, Python is in the process of unifying longs and
ints.

It used to be that calling int() on a big number gave an exception -
perhaps a
more Pythonic version of your requested giving the wrong result - and now
that it doesn't I suppose there is a sense in which the current
distinction
between ints and longs is "unmeaningf ul" from the user's point of view
(it's
not meaningless under the hood of course). According to the PEP the
decision
to keep two types was that it:

is far easier to implement, is backwards
compatible at the C API level, and in addition can be implemented
partially as a transitional measure.

Perhaps you thing that ints and longs should not be unified but I won't
start
arguing till you come out and say it. :)

James

I don't want to change any way the base of PEP 237,but this kind of
performance
flaw is a problem and it's possible to be the first of a long list.
It's good to know, always ,why we pay a price.Better an Error then a ghost
warning,which
is itself the cause of the flaw.
Python is fast ,this is why I use it.Anyone has a different goal in using
it.

Regards Paolino


--
.....lotta dura per la verdura

[James Henderson]

On Wednesday 14 July 2004 12:28 pm, paolo veronelli wrote:

On Wed, 14 Jul 2004 11:08:16 +0100, James Henderson wrote:

I think you should look at PEP 237 "Unifying Long Integers and Integers"
- it
may even reassure you. As Peter Hansen has already hinted in reply to
another of your messages, Python is in the process of unifying longs and
ints.

It used to be that calling int() on a big number gave an exception -
perhaps a
more Pythonic version of your requested giving the wrong result - and now
that it doesn't I suppose there is a sense in which the current
distinction
between ints and longs is "unmeaningf ul" from the user's point of view
(it's
not meaningless under the hood of course). According to the PEP the
decision
to keep two types was that it:

is far easier to implement, is backwards
compatible at the C API level, and in addition can be implemented
partially as a transitional measure.

Perhaps you thing that ints and longs should not be unified but I won't
start
arguing till you come out and say it. :)

James

I don't want to change any way the base of PEP 237,but this kind of
performance
flaw is a problem and it's possible to be the first of a long list.
It's good to know, always ,why we pay a price.Better an Error then a ghost
warning,which
is itself the cause of the flaw.
Python is fast ,this is why I use it.Anyone has a different goal in using
it.

Regards Paolino

I actually agree it's a shame that even ignored warnings have such a
performance impact but at least these warnings are scheduled to be removed
sometime in Python 2.4 (they're still there in 2.4a1 though). Apparently
they were added for "those concerned about extreme backwards compatibility".

James
--
James Henderson, Logical Progression Ltd
http://www.logicalprogression.net
http://mailmanager.sourceforge.net

[Willem]

James Henderson <ja***@logicalp rogression.net> wrote in message news:<ma******* *************** *************** @python.org>...

On Wednesday 14 July 2004 12:28 pm, paolo veronelli wrote:

On Wed, 14 Jul 2004 11:08:16 +0100, James Henderson wrote:

I think you should look at PEP 237 "Unifying Long Integers and Integers"
- it
may even reassure you. As Peter Hansen has already hinted in reply to
another of your messages, Python is in the process of unifying longs and
ints.

It used to be that calling int() on a big number gave an exception -
perhaps a
more Pythonic version of your requested giving the wrong result - and now
that it doesn't I suppose there is a sense in which the current
distinction
between ints and longs is "unmeaningf ul" from the user's point of view
(it's
not meaningless under the hood of course). According to the PEP the
decision
to keep two types was that it:

is far easier to implement, is backwards
compatible at the C API level, and in addition can be implemented
partially as a transitional measure.

Perhaps you thing that ints and longs should not be unified but I won't
start
arguing till you come out and say it. :)

James

I don't want to change any way the base of PEP 237,but this kind of
performance
flaw is a problem and it's possible to be the first of a long list.
It's good to know, always ,why we pay a price.Better an Error then a ghost
warning,which
is itself the cause of the flaw.
Python is fast ,this is why I use it.Anyone has a different goal in using
it.

Regards Paolino

I actually agree it's a shame that even ignored warnings have such a
performance impact but at least these warnings are scheduled to be removed
sometime in Python 2.4 (they're still there in 2.4a1 though). Apparently
they were added for "those concerned about extreme backwards compatibility".

James

That was indeed the reason of my original posting: As a casual
programmer I was disturbed that upgrading to Python 2.3 actually
slowed down my program significantly and that I had to spend so much
time to find the reason: somebody has decided to hide these particular
overflow warnings ...
Why warning and then suppressing it by default?
I understand the problem will disappear in the next version.
I apologize for my remark on "over-engineering". I posted the message
when I was still angry, having spent so much time in finding the "bug"

Willem

[James Henderson]

On Thursday 15 July 2004 9:58 am, Willem wrote:

That was indeed the reason of my original posting: As a casual
programmer I was disturbed that upgrading to Python 2.3 actually
slowed down my program significantly
As I said before, I don't think the particular code you posted would have
worked at all before 2.3.
and that I had to spend so much
time to find the reason: somebody has decided to hide these particular
overflow warnings ...
Why warning and then suppressing it by default?

The warnings are there because some people may want to know about when the
type conversion is happening, since it's a new feature.

They're suppressed by default because most people won't want to know. :)

J.
--
James Henderson, Logical Progression Ltd
http://www.logicalprogression.net
http://mailmanager.sourceforge.net