Weird numerical behavior regarding zero

[matthias_k]

Hi,

I am currently implementing a solver for linear optimization problems
using the revised simplex method and I stumbled accross some strange
behavior regarding the treatment of the number 0.
I am not sure whether this is compiler or language related though.

The first problem is, that there are two representations of zero, namely
+0 and -0. Does IEEE for floating points allow this? Maybe I didn't
activate IEEE floating point numbers?
Anyway, this does have some very annoying consequences, since this
expression:
x / -0 yields -inf (x is positive).

Which brings me to my second question. Shouldn't division by zero make
my computer go up in flames or so? Because, on my system (Arch Linux 0.7
i686) with my compiler (g++ 3.4.3), dividing by zero is allowed.
In fact, it yields +inf when dividing by +0 and -inf when dividing by
-0, given that the nominator (is that the english term for "Zähler"?) is
positive of course.

Any comments/ideas?

Regards,
Matthias

[Andrew Koenig]

"matthias_k " <no****@digital raid.com> wrote in message
news:ct******** *****@news.t-online.com...

The first problem is, that there are two representations of zero, namely
+0 and -0. Does IEEE for floating points allow this? Maybe I didn't
activate IEEE floating point numbers?
IEEE floating point requires +0 and -0 as distinct representations .
Anyway, this does have some very annoying consequences, since this
expression:
x / -0 yields -inf (x is positive).
Yes, that is what IEEE floating point requires.
Which brings me to my second question. Shouldn't division by zero make my
computer go up in flames or so? Because, on my system (Arch Linux 0.7
i686) with my compiler (g++ 3.4.3), dividing by zero is allowed.

C++ says that what happens after division by zero is undefined.

IEEE says that the result is either a floating-point trap or an
appropriately signed infinity, depending on how you have set things up.

The behavior you have described so far is completely normal according to
IEEE.

[matthias_k]

Andrew Koenig wrote:

"matthias_k " <no****@digital raid.com> wrote in message
news:ct******** *****@news.t-online.com...

The first problem is, that there are two representations of zero, namely
+0 and -0. Does IEEE for floating points allow this? Maybe I didn't
activate IEEE floating point numbers?

IEEE floating point requires +0 and -0 as distinct representations .

Anyway, this does have some very annoying consequences, since this
expression:
x / -0 yields -inf (x is positive).

Yes, that is what IEEE floating point requires.

Which brings me to my second question. Shouldn't division by zero make my
computer go up in flames or so? Because, on my system (Arch Linux 0.7
i686) with my compiler (g++ 3.4.3), dividing by zero is allowed.

C++ says that what happens after division by zero is undefined.

IEEE says that the result is either a floating-point trap or an
appropriately signed infinity, depending on how you have set things up.

The behavior you have described so far is completely normal according to
IEEE.

Thanks Andrew. That is terrible news. :)
Does that mean I have to multiply each -0 with -1 to get a sane
representation of zero?

[Dave Moore]

"matthias_k " <no****@digital raid.com> wrote in message
news:ct******** *****@news.t-online.com...

Hi,

I am currently implementing a solver for linear optimization problems
using the revised simplex method and I stumbled accross some strange
behavior regarding the treatment of the number 0.
I am not sure whether this is compiler or language related though.

The first problem is, that there are two representations of zero, namely
+0 and -0.

Depending on how you are obtaining your "signed zero" values, many different
things could be happening. You should use the constants defined in
std::numeric_li mits against your values to figure out what is happening.
For example, if the problem values results from subtraction operations, you
have to check them against the machine accuracy (e.g.
std::numeric_li mits<float>::ep silon()) to make sure they haven't lost all
significance. I guess this is what happened in your case, and the sign bit
is just staying set.
HTH,

Dave Moore

[matthias_k]

Dave Moore wrote:

"matthias_k " <no****@digital raid.com> wrote in message
news:ct******** *****@news.t-online.com...

Hi,

I am currently implementing a solver for linear optimization problems
using the revised simplex method and I stumbled accross some strange
behavior regarding the treatment of the number 0.
I am not sure whether this is compiler or language related though.

The first problem is, that there are two representations of zero, namely
+0 and -0.

Depending on how you are obtaining your "signed zero" values, many different
things could be happening. You should use the constants defined in
std::numeric_li mits against your values to figure out what is happening.
For example, if the problem values results from subtraction operations, you
have to check them against the machine accuracy (e.g.
std::numeric_li mits<float>::ep silon()) to make sure they haven't lost all
significance. I guess this is what happened in your case, and the sign bit
is just staying set.
HTH,

Dave Moore

I am doing lots of subtraction, multiplication and division.
What do you mean by check against the machine accuracy? Does -0 imply
that it's not 0 but something really close to 0? I can't check that for
each arithmetical operation, that's too expensive.

[Pete Becker]

matthias_k wrote:

Does that mean I have to multiply each -0 with -1 to get a sane
representation of zero?

No, it means that you have to rethink what you mean by zero. <g>

The problem is that for negative values of x, 1/x is also negative. As x
approaches zero from below, 1/x becomes a larger and larger negative
value. The limit of 1/x as x approaches zero from below is negative
infinity. Similarly, for positive valus of y, 1/y is also positive. As y
approaches zero from above, 1/y becomes a larger and larger positive
value. The limit of 1/y as y approaches zero from above is positive
infinity. If you don't distinguish between negative and positive zeroes
you end up with the limits of both of those expressions being the same.
While it's possible to do math that way, it's far more intuitive to have
negative and positive infinity as two distinct values, and that in turn
means that you need two distinct zeroes, one positive and one negative.

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

[Jerry Coffin]

matthias_k wrote:

[ ... ]

The first problem is, that there are two representations of zero, namely +0 and -0. Does IEEE for floating points allow this? Maybe I didn't
activate IEEE floating point numbers?
The C and C++ standards allow this. The IEEE floating point standards
_require_ it, if memory serves at all.
Anyway, this does have some very annoying consequences, since this
expression:
x / -0 yields -inf (x is positive).

Which brings me to my second question. Shouldn't division by zero make my computer go up in flames or so?
According to C it gives undefined behavior -- which can include
returning a value it considers reasonable. C99 added a number of
functions for controlling how floating point is done. AFAIK, they
haven't officially been adopted into C++ yet, but your compiler may
include them anyway -- if you have an fenv.h, chances are you can use
its functions from C++ as well as from C. I could be wrong, but at
first glance I'd expect FE_OVERFLOW to apply to division by 0.
Because, on my system (Arch Linux 0.7
i686) with my compiler (g++ 3.4.3), dividing by zero is allowed.
In fact, it yields +inf when dividing by +0 and -inf when dividing by -0, given that the nominator (is that the english term for "Zähler"?) is positive of course.

"Numerator" , is the word you seem to be looking for. In any case, as I
said above, the C++ standard simply makes division by 0 undefined
behavior, so anthing is possible.

The usual way to deal with negative zeros is not by multiplication, but
by addition -- a negative zero plus a positive zero gives a positive
zero, so to remove negative zeros, you add 0.0 to anything that might
have resulted in a negative zero.

--
Later,
Jerry.

The universe is a figment of its own imagination.

[Karl Heinz Buchegger]

matthias_k wrote:

Depending on how you are obtaining your "signed zero" values, many different
things could be happening. You should use the constants defined in
std::numeric_li mits against your values to figure out what is happening.
For example, if the problem values results from subtraction operations, you
have to check them against the machine accuracy (e.g.
std::numeric_li mits<float>::ep silon()) to make sure they haven't lost all
significance. I guess this is what happened in your case, and the sign bit
is just staying set.
HTH,

Dave Moore

I am doing lots of subtraction, multiplication and division.
What do you mean by check against the machine accuracy? Does -0 imply
that it's not 0 but something really close to 0? I can't check that for
each arithmetical operation, that's too expensive.

Taking your comments into account I would say that lots
of surprises are waiting for you.

First of all, you really shouldn't divide by 0. No matter if
IEEE allows it or not. Simply don't do it. If you are bound
to divide by 0, then something strange has happend and your
program has to react on that.

What Dave means. How do you know that something is exactly
+0 or -0. In programmers real life you can't distinguish between
them, because you don't know if the sign is correct or not.
If eg. the result of a subtraction or addition happens to be
-0 then you never know if the sign is indeed '-' or if it
(mathematically ) should be '+'. This is because of small round
of errors which happen all the time in floating point arithmetic.
So the best you can do is:
do the arithmetic and before you you perform the division,
check if you don't divide by a number that is small enough
to be assumed to be 0. If you figure out that you are going
to do that, consider it to be an error and react to it.
Do *not* depend on the result to be +inf or -inf, those
results will be wrong on most of the cases anyway.

In a computer 2 * 0.1 does *not* equal 0.2, but something that
is very close to 0.2 but not exactly 0.2
You need to take that into account with every floating point
operation you make.

And no: checking for division by 0.0 is *never* too expensive
no matter how many operations you have to perform. There is only
one exception: If you can proove that your division will never
be executed with something that is even close to 0.0. In all
other cases: Do the check, but do it correct:

if( fabs( some_number ) < some_epsilon ) {
// some_number is close to 0.0
// treat it as 0.0 and react to it
}
else
op = 5.0 / some_number;

As for the value of some_epsilon. It mostly depends on the
history of some_number, how many 'floating point debris'
has accumulated in it. This is dependent on
a) the number of operations
b) the range of the numbers participating in those operations.

Using a value like std::numeric_li mits<float>::ep silon() isn't going
to help.

And by the way: Never use float, until you know what you
do, have the knowledge to fight that beast, are willing to fight
that beast and have a very, very, very, very good reason to use
it. In all other cases (that is 99.99% of all cases) use double.

--
Karl Heinz Buchegger
kb******@gascad .at

[Pete Becker]

Karl Heinz Buchegger wrote:

And no: checking for division by 0.0 is *never* too expensive
no matter how many operations you have to perform.

The reason for having both forms of zero, infinities, and NaNs is so
that you don't have to check before every operation. Once an error
occurs it propogates through the subsequent calculations, so you only
have to check at the end. The underlying assumption is that errors are
rare, so not checking for them makes complicated computations much faster.

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

[White Wolf]

Pete Becker wrote:

matthias_k wrote:

Does that mean I have to multiply each -0 with -1 to get a sane
representation of zero?

No, it means that you have to rethink what you mean by zero. <g>

[SNIP]

Just as an (off-topic) sidenote to that: (probably) that definition of
floating point zero has kept a lot of people wondering for quite some time
in Hungary. A public digital thermometer was installed at the capital, and
it was sometimes showing -0 Celsius (Centigrade) and sometimes +0. People
came up with loads of explanations as to why that thermometer made a
difference between -0 and +0 degrees, why was it important. I guess that
Forest Gump was closest to the solution when he said: for no particular
reason. :-)

--
WW aka Attila
:::
My software never has bugs. It just develops random features.