471,344 Members | 1,556 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,344 software developers and data experts.

How to avoid overflow errors

I thought that overflow errors would be a thing of the past now that
Python automatically converts ints to longs as needed. Unfortunately,
that is not the case.
>>class MyInt(int):
.... pass
....
>>MyInt(sys.maxint)
2147483647
>>MyInt(sys.maxint+1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: long int too large to convert to int
How do I subclass int and/or long so that my class also auto-converts
only when needed?

--
Steven.
Sep 15 '07 #1
8 4525
On 9/14/07, Steven D'Aprano <st***@remove-this-cybersource.com.auwrote:
I thought that overflow errors would be a thing of the past now that
Python automatically converts ints to longs as needed. Unfortunately,
that is not the case.
>class MyInt(int):
... pass
...
>MyInt(sys.maxint)
2147483647
>MyInt(sys.maxint+1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: long int too large to convert to int

Not totally unrelated, but in Py3k, as it seems, overflows are really
things of the past:
Python 3.0a1 (py3k:58061, Sep 9 2007, 13:18:37)
[GCC 4.1.3 20070831 (prerelease) (Ubuntu 4.1.2-16ubuntu1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>class MyInt(int):
.... pass
....
>>import sys
MyInt(sys.maxint)
2147483647
>>MyInt(sys.maxint+1)
2147483648
Thanks to your mail, only now I see how important this change
(int/liong unification) really is!

How do I subclass int and/or long so that my class also auto-converts
only when needed?
What about just subclassing long - is this not an option?

Steven.
--
http://mail.python.org/mailman/listinfo/python-list

--
http://www.advogato.org/person/eopadoan/
Bookmarks: http://del.icio.us/edcrypt
Sep 15 '07 #2
"Eduardo O. Padoan" <ed************@gmail.comwrites:
Not totally unrelated, but in Py3k, as it seems, overflows are really
things of the past:
Python 3.0a1 (py3k:58061, Sep 9 2007, 13:18:37)
[GCC 4.1.3 20070831 (prerelease) (Ubuntu 4.1.2-16ubuntu1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>class MyInt(int):
... pass
...
>import sys
MyInt(sys.maxint)
2147483647
>MyInt(sys.maxint+1)
2147483648
I'd be interested in knowing what happens in 3.0a1 with

a = itertools.count(sys.maxint)
print a.next()
print a.next()
Sep 15 '07 #3
Steven D'Aprano wrote:
I thought that overflow errors would be a thing of the past now that
Python automatically converts ints to longs as needed. Unfortunately,
that is not the case.
>>>class MyInt(int):
... pass
...
>>>MyInt(sys.maxint)
2147483647
>>>MyInt(sys.maxint+1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: long int too large to convert to int
How do I subclass int and/or long so that my class also auto-converts
only when needed?
Use __new__.

pyimport sys
py>
pyclass MyLong(long):
.... pass
....
pyclass MyInt(int):
.... def __new__(cls, *args, **kwargs):
.... try:
.... return int.__new__(cls, *args, **kwargs)
.... except OverflowError:
.... return MyLong(*args, **kwargs)
....
pyMyInt(sys.maxint**2)
4611686014132420609L
pytype(_)
<class '__main__.MyLong'>
James
Sep 15 '07 #4
On 14 Sep 2007 18:08:00 -0700, Paul Rubin
<"http://phr.cx"@nospam.invalidwrote:
"Eduardo O. Padoan" <ed************@gmail.comwrites:
Not totally unrelated, but in Py3k, as it seems, overflows are really
things of the past:
Python 3.0a1 (py3k:58061, Sep 9 2007, 13:18:37)
[GCC 4.1.3 20070831 (prerelease) (Ubuntu 4.1.2-16ubuntu1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>class MyInt(int):
... pass
...
>>import sys
>>MyInt(sys.maxint)
2147483647
>>MyInt(sys.maxint+1)
2147483648

I'd be interested in knowing what happens in 3.0a1 with

a = itertools.count(sys.maxint)
print a.next()
print a.next()
>>print(next(a))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: cannot count beyond PY_SSIZE_T_MAX

Hum, you've got me there. it is the same as in 2.x. Maybe the message
should be less crypt, at least - nothing that googling for
PY_SSIZE_T_MAX cant help.

--
http://www.advogato.org/person/eopadoan/
Bookmarks: http://del.icio.us/edcrypt
Sep 15 '07 #5
On Fri, 14 Sep 2007 22:59:13 -0300, Eduardo O. Padoan wrote:
On 14 Sep 2007 18:08:00 -0700, Paul Rubin
<"http://phr.cx"@nospam.invalidwrote:
>"Eduardo O. Padoan" <ed************@gmail.comwrites:
Not totally unrelated, but in Py3k, as it seems, overflows are really
things of the past:
Python 3.0a1 (py3k:58061, Sep 9 2007, 13:18:37) [GCC 4.1.3 20070831
(prerelease) (Ubuntu 4.1.2-16ubuntu1)] on linux2 Type "help",
"copyright", "credits" or "license" for more information.
class MyInt(int):
... pass
...
import sys
MyInt(sys.maxint)
2147483647
MyInt(sys.maxint+1)
2147483648

I'd be interested in knowing what happens in 3.0a1 with

a = itertools.count(sys.maxint)
print a.next()
print a.next()

>>>print(next(a))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: cannot count beyond PY_SSIZE_T_MAX

Hum, you've got me there. it is the same as in 2.x. Maybe the message
should be less crypt, at least - nothing that googling for
PY_SSIZE_T_MAX cant help.
This should demonstrate that OverflowError will not disappear entirely
even in Python 3.

Even if we were to get rid of all OverflowErrors resulting from integer
operations in Python, there are going to be some C extensions--including
some in the Python standard library--that will store 32-bit integers.
Modules such as array and struct will still raise it, and probably other
modules were a long integer sometimes doesn't make sense (socket, for
instance). itertools.count() should work for arbitrary integers, though.
Carl Banks
Sep 15 '07 #6
On Fri, 14 Sep 2007 21:43:38 -0300, Eduardo O. Padoan wrote:
>How do I subclass int and/or long so that my class also auto-converts
only when needed?

What about just subclassing long - is this not an option?
Of course it's an option. As it turned out, that was the easiest way for
me to proceed. I was a little concerned that the overhead of using longs
might have been excessive, but for values close to zero (and by close I
mean < millions) there's no significant time difference between
arithmetic with ints and longints.
--
Steven.
Sep 15 '07 #7
On Fri, 14 Sep 2007 18:19:45 -0700, James Stroud wrote:
>How do I subclass int and/or long so that my class also auto-converts
only when needed?

Use __new__.
The disadvantage of that is that your example code requires me to
duplicate my methods in the long version and the int version. It's easy
enough to work around that (class factory function) but it just seems all
rather untidy...
--
Steven.
Sep 15 '07 #8
On 9/15/07, Carl Banks <pa************@gmail.comwrote:
On Fri, 14 Sep 2007 22:59:13 -0300, Eduardo O. Padoan wrote:
On 14 Sep 2007 18:08:00 -0700, Paul Rubin
<"http://phr.cx"@nospam.invalidwrote:
"Eduardo O. Padoan" <ed************@gmail.comwrites:
Not totally unrelated, but in Py3k, as it seems, overflows are really
things of the past:
Python 3.0a1 (py3k:58061, Sep 9 2007, 13:18:37) [GCC 4.1.3 20070831
(prerelease) (Ubuntu 4.1.2-16ubuntu1)] on linux2 Type "help",
"copyright", "credits" or "license" for more information.
>>class MyInt(int):
... pass
...
>>import sys
>>MyInt(sys.maxint)
2147483647
>>MyInt(sys.maxint+1)
2147483648

I'd be interested in knowing what happens in 3.0a1 with

a = itertools.count(sys.maxint)
print a.next()
print a.next()
>>print(next(a))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: cannot count beyond PY_SSIZE_T_MAX

Hum, you've got me there. it is the same as in 2.x. Maybe the message
should be less crypt, at least - nothing that googling for
PY_SSIZE_T_MAX cant help.

This should demonstrate that OverflowError will not disappear entirely
even in Python 3.
No one is denying that by now :)
Even if we were to get rid of all OverflowErrors resulting from integer
operations in Python, there are going to be some C extensions--including
some in the Python standard library--that will store 32-bit integers.
Modules such as array and struct will still raise it, and probably other
modules were a long integer sometimes doesn't make sense (socket, for
instance). itertools.count() should work for arbitrary integers, though.
Agreed. I was lookind at itertoolsmodule.c, and even for my very
limited C knowlegment, it does not seem to be too hard.
Anyway: http://bugs.python.org/issue1165

Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list

--
http://www.advogato.org/person/eopadoan/
Bookmarks: http://del.icio.us/edcrypt
Sep 15 '07 #9

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

2 posts views Thread by Mark | last post: by
44 posts views Thread by JKop | last post: by
5 posts views Thread by Tom Szabo | last post: by
1 post views Thread by Chua Wen Ching | last post: by
8 posts views Thread by starffly | last post: by
7 posts views Thread by amit.atray | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.