473,322 Members | 1,425 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Numerics, NaNs, IEEE 754 and C99


The numerical robustness of Python is very poor - this is not its fault,
but that of IEEE 754 and (even more) C99. In particular, erroneous
numerical operations often create apparently valid numbers, and the
NaN state can be lost without an exception being raised. For example,
try int(float("nan")).

Don't even ASK about complex, unless you know FAR more about numerical
programming than 99.99% of programmers :-(

Now, I should like to improve this, but there are two problems. The
first is political, and is whether it would be acceptable in Python to
restore the semantics that were standard up until about 1980 in the
numerical programming area. I.e. one where anything that is numerically
undefined or at a singularity which can deliver more than one value is
an error state (e.g. raises an an exception or returns a NaN). This
is heresy in the C99 and Java camps, and is none too acceptable in the
IEEE 754R one.

My question here is would such an attempt be opposed tooth and nail
in the Python context, the way it was in C99?

The second is technical. I can trivially provide options to select
between a restricted range of behaviours, but the question is how.
Adding a method to an built-in class doesn't look easy, from my
investigations of floatobject.c, and it is very doubtful that is the
best way, anyway - one of the great problems with "object orientation"
is how it handles issues that occur at class conversions. A run-time
option or reading an environment variable has considerable merit from
a sanity point of view, but calling a global function is also possible.

Any ideas?
Regards,
Nick Maclaren.
Jun 14 '06 #1
33 2386
On 2006-06-14, Nick Maclaren <nm**@cus.cam.ac.uk> wrote:

The numerical robustness of Python is very poor - this is not its fault,
but that of IEEE 754 and (even more) C99. In particular, erroneous
numerical operations often create apparently valid numbers, and the
NaN state can be lost without an exception being raised. For example,
try int(float("nan")).

Don't even ASK about complex, unless you know FAR more about numerical
programming than 99.99% of programmers :-(

Now, I should like to improve this, but there are two problems. The
first is political, and is whether it would be acceptable in Python to
restore the semantics that were standard up until about 1980 in the
numerical programming area. I.e. one where anything that is numerically
undefined or at a singularity which can deliver more than one value is
an error state (e.g. raises an an exception or returns a NaN).


That's fine as long as the behavior is selectable. I almost
always want a quiet NaN.

While you're at it, the pickle modules need to be fixed so they
support NaN and Inf. ;)

--
Grant Edwards grante Yow! Yow!
at
visi.com
Jun 14 '06 #2

In article <12*************@corp.supernews.com>,
Grant Edwards <gr****@visi.com> writes:
|> >
|> > Now, I should like to improve this, but there are two problems. The
|> > first is political, and is whether it would be acceptable in Python to
|> > restore the semantics that were standard up until about 1980 in the
|> > numerical programming area. I.e. one where anything that is numerically
|> > undefined or at a singularity which can deliver more than one value is
|> > an error state (e.g. raises an an exception or returns a NaN).
|>
|> That's fine as long as the behavior is selectable. I almost
|> always want a quiet NaN.

That is one of the two modes that I regard as respectable. However,
because integer arithmetic doesn't have a NaN value (which could be
fixed, in Python), anything that returns an integer has to raise an
exception. On that matter, division by zero and several other currently
trapped numeric errors could be modified to return NaN for people like
you (if the option were selected, of course).

I will take a look at adding NaN to integers, but that is a much
hairier hack - and it STILL doesn't deal with comparisons (which can
be done only in a purely functional programming language).

|> While you're at it, the pickle modules need to be fixed so they
|> support NaN and Inf. ;)

Yup. On my list :-)
Regards,
Nick Maclaren.
Jun 14 '06 #3
On 2006-06-14, Nick Maclaren <nm**@cus.cam.ac.uk> wrote:
|>> Now, I should like to improve this, but there are two problems. The
|>> first is political, and is whether it would be acceptable in Python to
|>> restore the semantics that were standard up until about 1980 in the
|>> numerical programming area. I.e. one where anything that is numerically
|>> undefined or at a singularity which can deliver more than one value is
|>> an error state (e.g. raises an an exception or returns a NaN).
|>
|> That's fine as long as the behavior is selectable. I almost
|> always want a quiet NaN.

That is one of the two modes that I regard as respectable. However,
because integer arithmetic doesn't have a NaN value (which could be
fixed, in Python), anything that returns an integer has to raise an
exception. On that matter, division by zero and several other currently
trapped numeric errors could be modified to return NaN for people like
you (if the option were selected, of course).
The division by zero trap is really annoying. In my world the
right thing to do is to return Inf.
|> While you're at it, the pickle modules need to be fixed so they
|> support NaN and Inf. ;)

Yup. On my list :-)


--
Grant Edwards grante Yow! Were these parsnips
at CORRECTLY MARINATED in
visi.com TACO SAUCE?
Jun 14 '06 #4
Grant Edwards a écrit :
The division by zero trap is really annoying. In my world the
right thing to do is to return Inf.


Your world is flawed then, this is a big mistake. NaN is the only
aceptable return value for a division by zero.
Jun 14 '06 #5
Grant Edwards wrote:
While you're at it, the pickle modules need to be fixed so they
support NaN and Inf. ;)

The NaN problem is portability -- NaN values are not standard, and
pretending they are won't help. There are many possible NaNs, several
of which have desirable behaviors, and different processors (and
Floating Point settings) choose different bit representations for
those NaNs. There are at least: Inf, -Inf, NaN, Ind (Indeterminant).

Being able to pickle some of these will produce values that "don't
behave right" on a different machine. Up until now, I think you can
send a pickle of a data structure and unpickle it on a different
processor to get equivalent data.

--Scott David Daniels
sc***********@acm.org
Jun 14 '06 #6
On 2006-06-14, Christophe <ch*************@free.fr> wrote:
Grant Edwards a écrit :
The division by zero trap is really annoying. In my world the
right thing to do is to return Inf.


Your world is flawed then, this is a big mistake. NaN is the
only aceptable return value for a division by zero.


You're probably right if you're talking about math, but I'm not
doing math. I'm doing engineering. In all of the situations
I've ever encountered, Inf was a much better choice.

Aside from Python, every FP library or processor I've ever used
returned Inf for divide by zero (which is the behavior required
by IEEE 754).

I need my Python programs to work the same way as everything
else.
http://standards.ieee.org/reading/ie.../754-1985.html

In IEEE Std 754-1985, subclause 7.2- Division by Zero, it says:

"If the divisor is zero and the dividend is a finite nonzero
number, then the division by zero shall be signaled. The
result, when no trap occurs, shall be a correctly signed
(infinity symbol)(6.3)."

While this is apparently the convention decided on by the
committee, it is mathematically incorrect and it seems as if
it should have been designated as Not-a-Number, since
division by zero is mathematically undefined and implies that
0*infinity=1, which is patently absurd.

Why was this convention chosen instead of NaN, since it leads
to a further degradation of our children's math abilities,
given that the IEEE floating-point standard would be considered
to be authoritative on this subject, yet produces an erroneous
results.

Interpretation for IEEE Std 754-1985

When a non-zero number is divided by a zero number, that is a
divide by zero. It is interpreted as an attempt to
take a limit of the ratio of two numbers as the denominator
becomes too small to be represented in the number system
while the numerator remains representable. Such a limit is best
represented by an infinity of the appropriate sign.

When zero is divided by zero, no such extrapolation can be
made. If it is caused by an attempt to take the limit of
the ratio of two numbers when both become two small to be
represented, then the limit cannot be determined. If it
is caused by some mistake in the programming, then no limit
exists. Thus, this case is thought to be invalid and a NaN
of appropriate sign is returned. (The sign is the only bit of
information that can be determined.)

While counter examples to the mathematical interpretation of
both of these results can be constructed they tend to be either
the result of extreme scaling or an attempt to evaluate a
non-analytic function. The former can be resolved by
rescaling. But, as the latter involve functions that cannot
(formally) be evaluated on a computer (without extreme effort
anyway) in the region of their non-analyticity, usually no good
solution exists.

--
Grant Edwards grante Yow! I represent a
at sardine!!
visi.com
Jun 14 '06 #7
On 2006-06-14, Scott David Daniels <sc***********@acm.org> wrote:
Grant Edwards wrote:
While you're at it, the pickle modules need to be fixed so they
support NaN and Inf. ;)
The NaN problem is portability -- NaN values are not standard,
My copy of IEEE 754 defines them quite precisely. :)
and pretending they are won't help. There are many possible
NaNs, several of which have desirable behaviors, and different
processors (and Floating Point settings) choose different bit
representations for those NaNs. There are at least: Inf,
-Inf, NaN, Ind (Indeterminant).
I don't think +Inf and -Inf aren't NaNs (in IEEE 754
terminology). I think Pickle ought to handle them as well.
Being able to pickle some of these will produce values that
"don't behave right" on a different machine.
The values "don't behave right" now. Having them "behave
right" on 99.9% of the hosts in the world would be a vast
improvement.
Up until now, I think you can send a pickle of a data
structure and unpickle it on a different processor to get
equivalent data.


No, you can't. Nan, Inf, and Ind floating point values don't
work.

--
Grant Edwards grante Yow! Yow! STYROFOAM...
at
visi.com
Jun 14 '06 #8
Grant Edwards a écrit :
On 2006-06-14, Christophe <ch*************@free.fr> wrote:
Grant Edwards a écrit :
The division by zero trap is really annoying. In my world the
right thing to do is to return Inf.


Your world is flawed then, this is a big mistake. NaN is the
only aceptable return value for a division by zero.

You're probably right if you're talking about math, but I'm not
doing math. I'm doing engineering. In all of the situations
I've ever encountered, Inf was a much better choice.


You should have been more precise then : "In my ideal world, when
dividing a non zero value by a zero value, the result should be +Inf or
-Inf according the the sign rules"

On that point, you should also note that +0 and -0 are sometimes
considered two different floating point numbers in Python :)
Jun 14 '06 #9
Christophe wrote:
Grant Edwards a écrit :
The division by zero trap is really annoying. In my world the
right thing to do is to return Inf.


Your world is flawed then, this is a big mistake. NaN is the only
aceptable return value for a division by zero.

Sorry, but this is not true.

The IEEE standard specifies (plus or minus) infinity as the result of
division by zero. This makes sense since such is the limit of division
by a quantity that goes to zero. The IEEE standard then goes on to
define reasonable results for arithmetic between infinities and real
values. The production of, and arithmetic on, infinities is a choice
that any application may want allow or not.

Gary Herron
Jun 14 '06 #10

In article <ma***************************************@python. org>,
Gary Herron <gh*****@islandtraining.com> writes:
|> Christophe wrote:
|> > Grant Edwards a =E9crit :
|> >
|> >> The division by zero trap is really annoying. In my world the
|> >> right thing to do is to return Inf.
|> >
|> > Your world is flawed then, this is a big mistake. NaN is the only=20
|> > aceptable return value for a division by zero.
|> >
|> Sorry, but this is not true.
|>
|> The IEEE standard specifies (plus or minus) infinity as the result of
|> division by zero. This makes sense since such is the limit of division
|> by a quantity that goes to zero. The IEEE standard then goes on to
|> define reasonable results for arithmetic between infinities and real
|> values. The production of, and arithmetic on, infinities is a choice
|> that any application may want allow or not.

That is true, and it is numerical nonsense. Christophe is right. Despite
Kahan's eminence as a numerical analyst, he is no software engineer. And
I am amazed at ANY engineer that can believe that inverting zero should
give infinity - while there ARE circumstances where it is correct, it is
NEVER a safe thing to do by default.

The reason is that it is correct only when you know for certain that the
sign of zero is meaningful. IEEE 754 gets this wrong by conflating true
zero, sign-unknown zero and positive zero. Inter alia, it means that
you get situations like:

A = 0.0; B = -A; C = B+0.0; A == B == C; 1/A != 1/B != 1/C;
Regards,
Nick Maclaren.
Jun 14 '06 #11

In article <44********@nntp0.pdx.net>,
Scott David Daniels <sc***********@acm.org> writes:
|> Grant Edwards wrote:
|>
|> > While you're at it, the pickle modules need to be fixed so they
|> > support NaN and Inf. ;)
|>
|> The NaN problem is portability -- NaN values are not standard, and
|> pretending they are won't help. There are many possible NaNs, several
|> of which have desirable behaviors, and different processors (and
|> Floating Point settings) choose different bit representations for
|> those NaNs. There are at least: Inf, -Inf, NaN, Ind (Indeterminant).

The main meaning of NaN is Indeterminate (i.e. it could be anything).
If you mean 'Missing', that is what was and is done in statistical
packages, and it follows very different rules. There are several other
meanings of NaN, but let's not go there, here. Though I have document
I could post if people are interested.

|> Being able to pickle some of these will produce values that "don't
|> behave right" on a different machine. Up until now, I think you can
|> send a pickle of a data structure and unpickle it on a different
|> processor to get equivalent data.

No, it could be done right. The unpickling would need to detect those
values and raise an exception. You have to allow for it even on the
same 'systems' because one Python might have been compiled with hard
underflow and one with soft. Your really DON'T want to import denorms
into programs that don't expect them.
Regards,
Nick Maclaren.
Jun 14 '06 #12
On 2006-06-14, Christophe <ch*************@free.fr> wrote:
The division by zero trap is really annoying. In my world the
right thing to do is to return Inf.

Your world is flawed then, this is a big mistake. NaN is the
only aceptable return value for a division by zero.
You're probably right if you're talking about math, but I'm not
doing math. I'm doing engineering. In all of the situations
I've ever encountered, Inf was a much better choice.


You should have been more precise then : "In my ideal world, when
dividing a non zero value by a zero value, the result should be +Inf or
-Inf according the the sign rules"


True. I've been dealing with IEEE 754 so long that I assume
things like that go without saying.
On that point, you should also note that +0 and -0 are sometimes
considered two different floating point numbers in Python :)


Different but equal.

[Don't tell the Supreme Court.]

--
Grant Edwards grante Yow! I just had a NOSE
at JOB!!
visi.com
Jun 14 '06 #13
Jeez, 12 posts in this IEEE 754 thread, and still
no message from uncle timmy ? ;)

Please, we need enlightenment here and *now* :)

platform-dependent accident'ly yours,

SB

Jun 14 '06 #14

In article <ma***************************************@python. org>,
Gary Herron <gh*****@islandtraining.com> writes:
|>
|> The IEEE standard specifies (plus or minus) infinity as the result of
|> division by zero. This makes sense since such is the limit of division
|> by a quantity that goes to zero. The IEEE standard then goes on to
|> define reasonable results for arithmetic between infinities and real
|> values. The production of, and arithmetic on, infinities is a choice
|> that any application may want allow or not.

The mistake you have made (and it IS a mistake) is in assuming that the
denominator approaches zero from the direction indicated by its sign.
There are many reasons why it is likely to not be, but let's give only
two:

It may be a true zero - i.e. a count that is genuinely zero, or
the result of subtracting a number from itself.

It may be a negative zero that has had its sign flipped by an
aritfact of the code. For example:

lim(x->0 from above) 0.001*b/(a-1.001*a)

I fully agree that infinity arithmetic is fairly well-defined for
most operations, but it most definitely is not in this case. It should
be reserved for when the operations have overflowed.
Regards,
Nick Maclaren.
Jun 14 '06 #15
On 14 Jun 2006 10:33:21 -0700, Sébastien Boisgérault
<Se*******************@gmail.com> wrote:
Jeez, 12 posts in this IEEE 754 thread, and still
no message from uncle timmy ? ;)


Somebody reboot the timbot, please. Seems to have hung.

--
Cheers,
Simon B,
si***@brunningonline.net,
http://www.brunningonline.net/simon/blog/
Jun 14 '06 #16
On 2006-06-14, Sébastien Boisgérault <Se*******************@gmail.com> wrote:
Jeez, 12 posts in this IEEE 754 thread, and still no message
from uncle timmy ? ;)

Please, we need enlightenment here and *now* :)


What we need is fewer people like me who do nothing but
complain about it...

--
Grant Edwards grante Yow! I am covered with
at pure vegetable oil and I am
visi.com writing a best seller!
Jun 14 '06 #17
On 2006-06-14, Nick Maclaren <nm**@cus.cam.ac.uk> wrote:

In article <ma***************************************@python. org>,
Gary Herron <gh*****@islandtraining.com> writes:
|>
|> The IEEE standard specifies (plus or minus) infinity as the result of
|> division by zero. This makes sense since such is the limit of division
|> by a quantity that goes to zero. The IEEE standard then goes on to
|> define reasonable results for arithmetic between infinities and real
|> values. The production of, and arithmetic on, infinities is a choice
|> that any application may want allow or not.

The mistake you have made (and it IS a mistake) is in assuming
that the denominator approaches zero from the direction
indicated by its sign.
I assume the "you" in that sentence refers to the IEEE FP
standards group. I just try to follow the standard, but I have
found that the behavior required by the IEEE standard is
generally what works best for my applications.
There are many reasons why it is likely to not be, but let's give only
two:

It may be a true zero - i.e. a count that is genuinely zero, or
the result of subtracting a number from itself.
I do real-world engineering stuff with measured physical
quatities. There generally is no such thing as "true zero".
I fully agree that infinity arithmetic is fairly well-defined for
most operations, but it most definitely is not in this case. It should
be reserved for when the operations have overflowed.


All I can say is that 1/0 => Inf sure seems to work well for
me.

--
Grant Edwards grante Yow! World War Three can
at be averted by adherence
visi.com to a strictly enforced
dress code!
Jun 14 '06 #18

In article <12*************@corp.supernews.com>,
Grant Edwards <gr****@visi.com> writes:
|>
|> I assume the "you" in that sentence refers to the IEEE FP
|> standards group. I just try to follow the standard, but I have
|> found that the behavior required by the IEEE standard is
|> generally what works best for my applications.

Well, it could be, but actually it was a reference to the sentence "This
makes sense since such is the limit of division by a quantity that goes
to zero."

|> I do real-world engineering stuff with measured physical
|> quatities. There generally is no such thing as "true zero".

It is extremely unusual for even such programs to use ONLY continuous
interval scale quantities, but they might dominate your usage, I agree.
Such application areas are very rare, but do exist. For example, I can
tell that you don't use statistics in your work, and therefore do not
handle events (including the analysis of failure rates).

|> > I fully agree that infinity arithmetic is fairly well-defined for
|> > most operations, but it most definitely is not in this case. It should
|> > be reserved for when the operations have overflowed.
|>
|> All I can say is that 1/0 => Inf sure seems to work well for me.

Now, can you explain why 1/0 => -Inf wouldn't work as well? I.e. why
are ALL of your zeroes, INCLUDING those that arise from subtractions,
are known to be positive?

If you can, then you have a case (and an EXTREMELY unusual application
domain. If you can't, then I am afraid that your calculations are
unreliable, at best.

The point here is that +infinity is the correct answer when the zero is
known to be a positive infinitesimal, just as -infinity is when it is
known to be a negative one. NaN is the only numerically respectable
result if the sign is not known, or it might be a true zero.
Regards,
Nick Maclaren.
Jun 15 '06 #19
Nick Maclaren a écrit :
Now, can you explain why 1/0 => -Inf wouldn't work as well? I.e. why
are ALL of your zeroes, INCLUDING those that arise from subtractions,
are known to be positive?


I would say that the most common reason people assume 1/0 = Inf is
probably because they do not make use of negative numbers or they forgot
they exist at all.
Jun 15 '06 #20

In article <44***********************@news.free.fr>,
Christophe <ch*************@free.fr> writes:
|>
|> > Now, can you explain why 1/0 => -Inf wouldn't work as well? I.e. why
|> > are ALL of your zeroes, INCLUDING those that arise from subtractions,
|> > are known to be positive?
|>
|> I would say that the most common reason people assume 1/0 = Inf is
|> probably because they do not make use of negative numbers or they forgot
|> they exist at all.

Usually the latter :-(

Yes, if you are working with the non-negative real numbers (or even
non-negative integers), then the best result of 1/0 is +infinity.
Thus, if you are working with events, a count of zero is a true zero,
but its inverse can reasonably be said to be +infinity.

However, it is common for non-experts to assume that they are working
in the non-negative real domain, because numbers are conventionally
measured that way, but in fact that is not so. The use of elapsed
times is one such "gotcha". I have seen quite a few programs that
have assumed that elapsed times were always positive, and which have
blown up when applied to real problems, where the measurement of an
elapsed time may be negative.

Hence, the SAFE approach is to make the inverse of all zeros a NaN.
Regards,
Nick Maclaren.
Jun 15 '06 #21
On 2006-06-15, Nick Maclaren <nm**@cus.cam.ac.uk> wrote:

In article <12*************@corp.supernews.com>,
Grant Edwards <gr****@visi.com> writes:
|>
|> I assume the "you" in that sentence refers to the IEEE FP
|> standards group. I just try to follow the standard, but I have
|> found that the behavior required by the IEEE standard is
|> generally what works best for my applications.

Well, it could be, but actually it was a reference to the
sentence "This makes sense since such is the limit of division
by a quantity that goes to zero."
That sentence was written by the IEEE FP standards committee
explaining why they chose the behavior they did.
The point here is that +infinity is the correct answer when the zero is
known to be a positive infinitesimal, just as -infinity is when it is
known to be a negative one. NaN is the only numerically respectable
result if the sign is not known, or it might be a true zero.


OK, you're right. I still prefer that Python follow the
standard they everything else does. Part of what I use Python
for is to simulate devices that obey the IEEE 754 floating
point standards. I need Python to follow the standard.

--
Grant Edwards grante Yow! Yow! Now we can
at become alcoholics!
visi.com
Jun 15 '06 #22
On 2006-06-15, Nick Maclaren <nm**@cus.cam.ac.uk> wrote:
Hence, the SAFE approach is to make the inverse of all zeros a
NaN.


OK. You're right. I'm wrong about what my Python programs
should do.

In any case, that ship sailed.

Maybe you can argue convincingly that theoretically your
approach is better than IEEE 754. You're not going to get the
standard changed. You're not going to get all of the computers
on the planet re-worked. Making Python incompatible with IEEE
754 is a bad idea.

I might be able to come up with a convincing arguement that
driving on the right-hand side of the road is better than
driving on the left, but that doesn't make a good idea to do so
when in England, Japan, or Australia.

--
Grant Edwards grante Yow! This PIZZA symbolizes
at my COMPLETE EMOTIONAL
visi.com RECOVERY!!
Jun 15 '06 #23
Nick Maclaren wrote:
...
Now, I should like to improve this, but there are two problems. The
first is political, and is whether it would be acceptable in Python to
restore the semantics that were standard up until about 1980 in the
numerical programming area. I.e. one where anything that is numerically
undefined or at a singularity which can deliver more than one value is
an error state (e.g. raises an an exception or returns a NaN). This
is heresy in the C99 and Java camps, and is none too acceptable in the
IEEE 754R one.

...


As one of the few people on this group who will admit to programming
before 1980, let alone doing numerical programming then (I presume
writing code to process gigabytes of seismic data via Fast Fourier
equations meet your criteria), the only semantics that existed then
were for the program to *CRASH* hard when a singularity or undefined
state occurred. When using the various flavors of FORTRAN and vector
processors the undefined states could take a few microseconds to
manifest, but crash and burn they would, with some difficulty to
analyze the burning wreckage.

C is not a mathematically based language as FORTRAN is, so it is funny
to criticize it for being what it never was, however IEEE754 and NaN's
and the other standardizations put into place make back analyzing what
is wrong with your program and/or data far easier then what we had
before.
Curtis

Jun 15 '06 #24

In article <12*************@corp.supernews.com>,
Grant Edwards <gr****@visi.com> writes:
|> On 2006-06-15, Nick Maclaren <nm**@cus.cam.ac.uk> wrote:
|>
|> > Hence, the SAFE approach is to make the inverse of all zeros a
|> > NaN.
|>
|> OK. You're right. I'm wrong about what my Python programs
|> should do.
|>
|> In any case, that ship sailed.

And sank, spawning a zillion lifeboats heading in different directions :-)

|> Maybe you can argue convincingly that theoretically your
|> approach is better than IEEE 754. You're not going to get the
|> standard changed. You're not going to get all of the computers
|> on the planet re-worked. Making Python incompatible with IEEE
|> 754 is a bad idea.

No, that is wrong, on many counts.

Firstly, a FAR more common assumption is that integers wrap in twos'
complement - Python does not do that.

Secondly, it is NOT incompatible with IEEE 754, which is a pure hardware
standard. All it does is to trap the exception and take appropriate
action (as permitted by that standard).

Thirdly, virtually no hardware sticks strictly to IEEE 754, and no
language that I know of has EVER attempted to make it the strict basis
for its arithmetic model.

Fourthly, I am not proposing to change any hardware, and could even
provide a Python option to deliver mathematically incorrect results
when you want them.
Regards,
Nick Maclaren.
Jun 15 '06 #25

In article <11********************@u72g2000cwu.googlegroups.c om>,
"ra************@gmail.com" <ra************@gmail.com> writes:
|>
|> As one of the few people on this group who will admit to programming
|> before 1980, let alone doing numerical programming then (I presume
|> writing code to process gigabytes of seismic data via Fast Fourier
|> equations meet your criteria), the only semantics that existed then
|> were for the program to *CRASH* hard when a singularity or undefined
|> state occurred. When using the various flavors of FORTRAN and vector
|> processors the undefined states could take a few microseconds to
|> manifest, but crash and burn they would, with some difficulty to
|> analyze the burning wreckage.

Yes, it does, but I had used some dozens of systems, Fortrans and other
languages by then. I did not make myself entirely clear, which partly
accounts for your response, but I am afraid that you are mistaken.

What I was referring to was that the consensus was that it was a
program error to calculate a value at a singularity or otherwise when
the result was mathematically undefined.

Yes, the languages specified that to be undefined behaviour (and usually
still do), but it was clear that a good implementation was ALOLOWED to
trap and diagnose the failures. Many did and some still do - Python is
pretty good, but not in this area.

Examples of numerically near-bulletproof compilers included Egtran, XFAT
(if I recall after 35+ years), WATFIV, Delft Algol, SPITBOL, FLACC and
many others (and some in the 1980s, too). MOST of the better Fortran
run-time systems (even IBM's Mod II library) used to detect and flag
invalid arguments to library routines.

==>> Why should Python regard that, provably achievable, level of
robustness as positevely undesirable?

|> C is not a mathematically based language as FORTRAN is, so it is funny
|> to criticize it for being what it never was, however IEEE754 and NaN's
|> and the other standardizations put into place make back analyzing what
|> is wrong with your program and/or data far easier then what we had
|> before.

As someone who has done it more-or-less continually since the 1960s,
I am afraid that is not so. Yes, NaNs COULD do that, if the (fixable)
flaws in IEEE 754 were sorted out, but Java and C99 have picked up on the
flaws and used them as justification for ignoring 90% of the principles
of IEEE 754.

==>> But the flaws in those are not the point here. I am asking
whether people would positively OBJECT (as the did in C99) to
errors being detected, possibly as an overridable option.
Regards,
Nick Maclaren.
Jun 15 '06 #26
On 2006-06-15, Nick Maclaren <nm**@cus.cam.ac.uk> wrote:
|> Making Python incompatible with IEEE |> 754 is a bad idea.

No, that is wrong, on many counts.

Firstly, a FAR more common assumption is that integers wrap in twos'
complement - Python does not do that.
It used to, and I sure wish there was still a way to get that
behavior back. Now I have to do all sorts of bitwise ands that
I didn't use to.
Secondly, it is NOT incompatible with IEEE 754, which is a
pure hardware standard. All it does is to trap the exception
and take appropriate action (as permitted by that standard).
There are two allowed (selectable?) behaviors for 1/0: trap and
Inf.
Thirdly, virtually no hardware sticks strictly to IEEE 754,
and no language that I know of has EVER attempted to make it
the strict basis for its arithmetic model.
All the HW and libraries I've used returned Inf for 1/0.
Fourthly, I am not proposing to change any hardware,
IMO, having Python's FP results disagree with common HW FP
results would be a pretty bad thing.
and could even provide a Python option to deliver
mathematically incorrect results when you want them.


As long as I can get IEEE 754 results, that's cool. I'd prefer
if Python just let the HW do it's thing when possible.

--
Grant Edwards grante Yow! How do I get HOME?
at
visi.com
Jun 15 '06 #27

In article <12*************@corp.supernews.com>,
Grant Edwards <gr****@visi.com> writes:
|> >
|> > Firstly, a FAR more common assumption is that integers wrap in twos'
|> > complement - Python does not do that.
|>
|> It used to, and I sure wish there was still a way to get that
|> behavior back. Now I have to do all sorts of bitwise ands that
|> I didn't use to.

Given that it is perhaps THE most common cause of severe failure
in large scientific codes, I don't agree that it should be the default.
A 'twos complement' class would be quite reasonable.

|> > Secondly, it is NOT incompatible with IEEE 754, which is a
|> > pure hardware standard. All it does is to trap the exception
|> > and take appropriate action (as permitted by that standard).
|>
|> There are two allowed (selectable?) behaviors for 1/0: trap and
|> Inf.

Er, no. The second is REQUIRED to set an exception flag, which IEEE 754
assumes that the code will test and take appropriate action (which can
be anything, including aborting the program and replacing it by a NaN).
See http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf

|> > Thirdly, virtually no hardware sticks strictly to IEEE 754,
|> > and no language that I know of has EVER attempted to make it
|> > the strict basis for its arithmetic model.
|>
|> All the HW and libraries I've used returned Inf for 1/0.

Well, I have clearly used a much wider range than you have, then.
But it isn't just that aspect I was referring to. You will also find
that a high proportion of hardware traps into the kernel on overflow,
and it is software that returns the infinity.

|> > Fourthly, I am not proposing to change any hardware,
|>
|> IMO, having Python's FP results disagree with common HW FP
|> results would be a pretty bad thing.

No matter WHAT results are returned, they will do that! I was triggered
into this project by attempting to write NaN-safe code and finding that
the first few versions of Python I tried all did different things.
When I saw that it just mapped what C does, all became clear.

|> > and could even provide a Python option to deliver
|> > mathematically incorrect results when you want them.
|>
|> As long as I can get IEEE 754 results, that's cool. I'd prefer
|> if Python just let the HW do it's thing when possible.

Including crash and burn?

Seriously. I don't think that you realise just how many layers of
fixup there are on a typical "IEEE 754" system, and how many variations
there are even for a single platform.
Regards,
Nick Maclaren.
Jun 15 '06 #28
Grant Edwards wrote:
Firstly, a FAR more common assumption is that integers wrap in twos'
complement - Python does not do that.


It used to


for integers ? what version was that ?

</F>

Jun 15 '06 #29
On 2006-06-15, Nick Maclaren <nm**@cus.cam.ac.uk> wrote:
|>> Firstly, a FAR more common assumption is that integers wrap in twos'
|>> complement - Python does not do that.
|>
|> It used to, and I sure wish there was still a way to get that
|> behavior back. Now I have to do all sorts of bitwise ands that
|> I didn't use to.

Given that it is perhaps THE most common cause of severe
failure in large scientific codes, I don't agree that it
should be the default. A 'twos complement' class would be
quite reasonable.
Since Python went to non-fixed-length integers, a couple of
those classes have been written. I tried one of them and it
was rather clumsy. There was no conversion/coercion between
different widths or between fixed-width integers and "regular"
integers. That mean you had to call a constructor even if all
you wanted to do was add 2 to a value.
|> > Secondly, it is NOT incompatible with IEEE 754, which is a
|> > pure hardware standard. All it does is to trap the exception
|> > and take appropriate action (as permitted by that standard).
|>
|> There are two allowed (selectable?) behaviors for 1/0: trap and
|> Inf.

Er, no. The second is REQUIRED to set an exception flag,
But it's not required to generate a trap according to my
reading of the spec.
which IEEE 754 assumes that the code will test and take
appropriate action (which can be anything, including aborting
the program and replacing it by a NaN). See
http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf
IEEE Std 754-1985, subclause 7.2 - Division by Zero

"If the divisor is zero and the dividend is a finite nonzero
number, then the division by zero shall be signaled. The
result, when no trap occurs, shall be a correctly signed
(infinity symbol)(6.3)."

To me it looks like the spec specifically allows for a case
where "no trap occurrs" and the result is Inf.
|> As long as I can get IEEE 754 results, that's cool. I'd prefer
|> if Python just let the HW do it's thing when possible.

Including crash and burn?
No. :)
Seriously. I don't think that you realise just how many
layers of fixup there are on a typical "IEEE 754" system, and
how many variations there are even for a single platform.


Probably not.

--
Grant Edwards grante Yow! My face is new, my
at license is expired, and I'm
visi.com under a doctor's care!!!!
Jun 15 '06 #30
On 2006-06-15, Fredrik Lundh <fr*****@pythonware.com> wrote:
Grant Edwards wrote:
Firstly, a FAR more common assumption is that integers wrap in twos'
complement - Python does not do that.


It used to


for integers ? what version was that ?


Am I remebering incorrectly? Didn't the old fixed-width
integers operate modulo-wordsize? I distinctly remember having
to rewrite a bunch of checksum code when fixed-width integers
went away.

--
Grant Edwards grante Yow! My uncle Murray
at conquered Egypt in 53
visi.com B.C. And I can prove
it too!!
Jun 15 '06 #31
[Nick Maclaren]
Firstly, a FAR more common assumption is that integers wrap in twos'
complement - Python does not do that.
[Grant Edwards] It used to
[Fredrik Lundh]
for integers ? what version was that ?

[Grant] Am I remebering incorrectly?
Mostly but not entirely.
Didn't the old fixed-width integers operate modulo-wordsize?
Not in Python.
I distinctly remember having to rewrite a bunch of checksum code when
fixed-width integers went away.


Best guess is that you're working on a 32-bit box, and remember this
Python <= 2.2 behavior specific to the left shift operator:
1 << 31 -2147483648 1 << 32 0

On a 64-bit box (except Win64, which didn't exist at the time ;-)),
those returned 2**31 and 2**32 instead, while "1 << 64" wrapped to 0
(etc).

Python 2.3 starting warning about that:
1 << 31 __main__:1: FutureWarning: x<<y losing bits or changing sign will
return a long in Python 2.4 and up
-2147483648

and Python 2.4 started producing platform-independent results:
1 << 31 2147483648L 1 << 32

4294967296L

+ - * / on short ints always complained about overflow before int-long
unification, although IIRC very early Pythons missed the overflow
error in (on a 32-bit box) int(-(2L**31))/-1.
Jun 15 '06 #32

In article <12*************@corp.supernews.com>,
Grant Edwards <gr****@visi.com> writes:
|>
|> > which IEEE 754 assumes that the code will test and take
|> > appropriate action (which can be anything, including aborting
|> > the program and replacing it by a NaN). See
|> > http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf
|>
|> IEEE Std 754-1985, subclause 7.2 - Division by Zero
|>
|> "If the divisor is zero and the dividend is a finite nonzero
|> number, then the division by zero shall be signaled. The
|> result, when no trap occurs, shall be a correctly signed
|> (infinity symbol)(6.3)."
|>
|> To me it looks like the spec specifically allows for a case
|> where "no trap occurrs" and the result is Inf.

That is correct. And it is a mandatory case. But it does NOT say what
the software should then do with the exception.
Regards,
Nick Maclaren.
Jun 15 '06 #33
On 2006-06-15, Tim Peters <ti********@gmail.com> wrote:
[Grant]
Am I remebering incorrectly?
Mostly but not entirely.
Didn't the old fixed-width integers operate modulo-wordsize?


Not in Python.
I distinctly remember having to rewrite a bunch of checksum code when
fixed-width integers went away.


Best guess is that you're working on a 32-bit box, and remember this
Python <= 2.2 behavior specific to the left shift operator:
1 << 31 -2147483648 1 << 32

0


That's probably it.

I've also spent some time on/off fighting with 32-bit constants
that have the high-order bit set. You've got to jump through
hoops when you need to pass a value like 0xC0000000 to an
extension that demands a 32-bit value.
+ - * / on short ints always complained about overflow before
int-long unification,


I was definitely mis-remembering things. I had to have been
the left shift that caused the problems.

--
Grant Edwards grante Yow! I'm CONTROLLED by
at the CIA!! EVERYONE is
visi.com controlled by the CIA!!
Jun 15 '06 #34

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: A. Barnet | last post by:
Hi, all, I posted a similar message at comp.soft-sys.sas. I used code like the following via SAS PROC SQL pass-thru to create a table named user.tmp in Oracle: drop table user.tmp; create...
3
by: Manzanita | last post by:
Hi. I'm using boost::numerics::ublas to solve linear symmetric and hermitian systems (cholesky, qr, etc.). I'm aware that ublas only provide basic algebra operations (blas1, blas2 and blas3) so...
1
by: GooglePoster | last post by:
Hello, I am looking for a function/utility to read in an IEEE value and convert this to decimal, for testing purposes. Also, I need to convert decimal values to IEEE values to send out on a...
54
by: Andy | last post by:
Hi, I don't know if this is the correct group to post this, but when I multiply a huge floating point value by a really small (non-zero) floating point value, I get 0 (zero) for the result. This...
10
by: John | last post by:
Hi all... Either I'm doing really something really stupid - or maybe there is some bug somewhere (optimizing?). In a function I have: int x1, y1, x2, y2; float dR, dX, dY; dR =...
2
by: Benjamin Rutt | last post by:
Does anyone have C code laying around to do this? I have to read in some binary data files that contains some 4-byte IBM/370 floating point values. I would like a function to convert 4-byte...
8
by: Learner | last post by:
Hello, I have a situation that I need to be able to identify if there are any numerical values in a string variable that might have numerics including charecters. For instanse Dim strValue as...
13
by: revuesbio | last post by:
Hi Does anyone have the python version of the conversion from msbin to ieee? Thank u
5
by: ms292606 | last post by:
Hi, I am currently working on a program to decode data that is collected on a gps receiver and i've ran into a problem. Some of the data is encoded in IEEE 754. I wrote the following functions to...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.