pickle and infinity

[Bart Ogryczak]

Hello,
I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error). What could I do about it? Example code:

>>x = 1e310 #actually it would be a result of calculations
type(x)

<type 'float'>

>>x

1.#INF

>>import pickle
pickle.loads(pickle.dumps(x))

[...]
ValueError: invalid literal for float(): 1.#INF

[Bart Ogryczak]

To make things more interesting -- Solaris version:

>>x = 1e310
x

Infinity

>>import pickle
pickle.dumps(x)

'FInfinity\n.'

>>pickle.loads(_)

Infinity

>>pickle.dumps(x,1)

[...]
SystemError: frexp() result out of range

[Grant Edwards]

On 2006-11-29, Bart Ogryczak <B.********@gmail.comwrote:

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return
overflow/underflow error). What could I do about it?

Here's what I did. I'm sure it'll fall down on some systems,
but it works on Linux and Windows.

__________________________________________________ ____________________
# unpickle can't handle nan/inf/ind floats, so we need to
# handle that ourselves

def myload_float(self):
s = self.readline()[:-1]
try:
f = float(s)
except ValueError:
s = s.upper()
if s in ["1.#INF", "INF"]:
f = 1e300*1e300
elif s in ["-1.#INF", "-INF"]:
f = -1e300*1e300
elif s in ["NAN","1.#QNAN","-1.#QNAN","QNAN","1.#IND","IND","-1.#IND"]:
f = (1e300*1e300)/(1e300*1e300)
else:
raise ValueError, "Don't know what to do with "+`s`
self.append(f)

# unpickle routine that overrides the float load method

def unpickle(f):
unpickler = pickle.Unpickler(f)
unpickler.dispatch[pickle.FLOAT] = myload_float
return unpickler.load()
__________________________________________________ ____________________

--
Grant Edwards grante Yow! I represent a
at sardine!!
visi.com

[Fredrik Lundh]

Bart Ogryczak wrote:

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error).

Python 2.X relies on the C library to serialize floats, and, as you've
noticed, some C libraries can produce values that they themselves cannot
read.

</F>

[Bart Ogryczak]

Fredrik Lundh wrote:

Bart Ogryczak wrote:

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error).

Python 2.X relies on the C library to serialize floats, and, as you've
noticed, some C libraries can produce values that they themselves cannot
read.

Actually I´d be very happy, if Python would throw FloatPointExeption,
rather then allow infinite values. Haven´t got idea how to do it.
fpectl would do the trick, but it´s not present nither on Solaris, nor
on Windows platform. And I´d need it on both.

[Grant Edwards]

On 2006-11-29, Bart Ogryczak <B.********@gmail.comwrote:

>
Fredrik Lundh wrote:

>Bart Ogryczak wrote:

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error).

Python 2.X relies on the C library to serialize floats, and, as you've
noticed, some C libraries can produce values that they themselves cannot
read.

Actually I´d be very happy, if Python would throw FloatPointExeption,
rather then allow infinite values.

I'd be very unhappy. I use both NaNs and infinities in many
programs, so it needs to be selectable should that feature be
added.

Haven´t got idea how to do it. fpectl would do the trick, but
it´s not present nither on Solaris, nor on Windows platform.
And I´d need it on both.

In my experience anything than needs to be portable and
involves NaN, inf, or denormals is rather painful.

--
Grant Edwards grante Yow! I wish I was a
at sex-starved manicurist
visi.com found dead in the Bronx!!

[Bart Ogryczak]

Grant Edwards wrote:

On 2006-11-29, Bart Ogryczak <B.********@gmail.comwrote:

Fredrik Lundh wrote:

Bart Ogryczak wrote:

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error).

Python 2.X relies on the C library to serialize floats, and, as you've
noticed, some C libraries can produce values that they themselves cannot
read.

Actually I´d be very happy, if Python would throw FloatPointExeption,
rather then allow infinite values.

I'd be very unhappy. I use both NaNs and infinities in many
programs, so it needs to be selectable should that feature be
added.

Like I´ve said. I´d be satisfied with something like fpectl, which
gives you option to either throw FloatPointExeption or go on with NaNs
and infs. NaNs and infinities are pain in the a** if you´re
integrating with C libraries or you´re using pickle, pyro, whatever.
The problem is, that the error pops out in the middle of the code that
really has nothing to do with the actual erroneous calculation. Its
impossible to trace back where the hell the NaN or infinite value comes
from.

Haven´t got idea how to do it. fpectl would do the trick, but
it´s not present nither on Solaris, nor on Windows platform.
And I´d need it on both.

In my experience anything than needs to be portable and
involves NaN, inf, or denormals is rather painful.

That´s not very comforting :-/