Rounding Problem

[Jason]

I am having a rounding problem in a value i am trying to display in VB.NET.

If I have the following code:
Dim x As Single = 2726.795
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.79

If I have the following code:
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.80

I need to use the first example as I read the value of x in as a string from
an xml file then convert it to a single in order to round and format it to 2
decimal places for a report display.

Any ideas why the first example is not rounding to 2726.80?

[Larry Lard]

[the short answer is at the end]

Jason wrote:

I am having a rounding problem in a value i am trying to display in VB.NET.

If I have the following code:
Dim x As Single = 2726.795
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.79

If I have the following code:
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.80
The two lines of the second example are the same as the second two
lines of the first example. I suspect you have a typo somewhere.

Incidentally, I see you are running without the benefit of Option
Strict (Math.Round returns a Double which cannot be implicitly
converted to a Single). I suggest turning it on - the additional code
sometimes required is a small price to pay for the increased
compile-time checking.
My guess is that you meant the first example to be:

Dim x As Single = 2726.795
Dim y As Single = Math.Round(x, 2)
Dim s As String = String.Format("{0:0.00}", y)

which *does* produce 2726.795.

Assuming that is the right correction:
Any ideas why the first example is not rounding to 2726.80?
The first parameter to the particular overload of Math.Round being used
is a Double. That means that when a Single is supplied (as in the
corrected first example), it must be converted to a Double. This is OK
to be done implicitly, since it is a widening conversion. Let's have a
look at the actual exact values involved:

Dim x As Single = 2726.795
Dim xAsDouble As Double = x

Console.WriteLine(x.ToString("G9"))
Console.WriteLine(xAsDouble.ToString("R"))

The output is

2726.79492
2726.794921875

So we see that trying to put 2726.795 into a Single actually puts in a
value slightly *less* than 2726.795, which is why it gets rounded to
....79. With a double, on the other hand (as in the second example):

Dim x As Double = 2726.795

Console.WriteLine(x.ToString("G17"))

2726.7950000000001

So trying to put 2726.295 into a Double actually puts in a value
slightly *more* than 2726.295, which is why it gets rounded to ...80.
That's the why. The next question is the what-to-do.

I'd say the first recommendation is this: don't use floating-point
types to deal with exact values. Actually it's more than a
recommendation, it's a golden rule. It's customary for me to give my
usual example at this point:

Debug.Print(0.1+0.1+0.1-0.3) 'guess output before running!
The next recommendation would be, don't use Single at all unless you
have a very good reason. Single only has about *seven* decimal digits
of precision, which we run into pretty quickly in the real world. Any
memory gains you think you might be getting by using Singles over
Doubles are almost certainly illusory. Any performance gains you think
you are getting are _wrong_: "Double is the most efficient of the
fractional data types, because the processors on current platforms
perform floating-point operations in double precision", to quote the
docs. In short, if you are going to use floating point (with the above
caveat that you shouldn't if... you shouldn't), use Double not Single.

And now for the short answer:
I need to use the first example as I read the value of x in as a string from
an xml file then convert it to a single in order to round and format it to 2
decimal places for a report display.

Use Decimal as the intermediate data type to avoid loss of precision.

--
Larry Lard
Replies to group please

[Kerry Moorman]

Larry,

Using Decimal cuts right to the chase:

Dim myDec As Decimal = 2726.795

MsgBox(Math.Round(myDec, 2))

This code produces 2726.80, which the OP does not want.

Of course, the reason that Math.Round produces this value is very clearly
laid out in the docs on Math.Round (IEEE Standard 754, section 4 or banker's
rounding).

The real question is probably "How to do rounding that rounds 2726.795 to
2726.79.

Maybe using INT or FIX or FLOOR, etc is the required approach.

Kerry Moorman

"Larry Lard" wrote:

Use Decimal as the intermediate data type to avoid loss of precision.

[Jim Wooley]

> Larry,

Using Decimal cuts right to the chase:

Dim myDec As Decimal = 2726.795

MsgBox(Math.Round(myDec, 2))

This code produces 2726.80, which the OP does not want.

Of course, the reason that Math.Round produces this value is very
clearly laid out in the docs on Math.Round (IEEE Standard 754, section
4 or banker's rounding).

The real question is probably "How to do rounding that rounds 2726.795
to 2726.79.

Maybe using INT or FIX or FLOOR, etc is the required approach.

Round includes an additional overloaded version which allows you to specify
the rounding method you want: banker's round or away from 0 rounding. Unfortunately
the framework isn't necessarily consistant in rounding approaches. Math.Round
uses bankers rounding by default, but Format and ToString(c2) use away from
0 rounding. See my write-up at http://devauthority.com/blogs/jwoole...03/24/806.aspx
for more.

Jim Wooley
http://devauthority.com/blogs/jwooley/default.aspx

[Kerry Moorman]

Jim

That information, about MidpointRounding.AwayFromZero, is good to know and
may go a ways to help clear up the confusion that has always surrounded the
Math.Round method. I had not seen that before you pointed it out.

Unfortunately, its a .Net 2.0-only feature.

Kerry Moorman
"Jim Wooley" wrote:

Round includes an additional overloaded version which allows you to specify
the rounding method you want: banker's round or away from 0 rounding. Unfortunately
the framework isn't necessarily consistant in rounding approaches. Math.Round
uses bankers rounding by default, but Format and ToString(c2) use away from
0 rounding. See my write-up at http://devauthority.com/blogs/jwoole...03/24/806.aspx
for more.

Jim Wooley
http://devauthority.com/blogs/jwooley/default.aspx

[Jason]

Cheers for the reply.

I will switch from using Single to Double as that has solved my problem. The
reason I have been using Single is that was the type I get back from my
DataSet schema when connecting to the Access database I am working with. I
will explicitly convert to Double in code for rounding functions.

"Larry Lard" wrote:

[the short answer is at the end]

Jason wrote:

I am having a rounding problem in a value i am trying to display in VB.NET.

If I have the following code:
Dim x As Single = 2726.795
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.79

If I have the following code:
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.80

The two lines of the second example are the same as the second two
lines of the first example. I suspect you have a typo somewhere.

Incidentally, I see you are running without the benefit of Option
Strict (Math.Round returns a Double which cannot be implicitly
converted to a Single). I suggest turning it on - the additional code
sometimes required is a small price to pay for the increased
compile-time checking.
My guess is that you meant the first example to be:

Dim x As Single = 2726.795
Dim y As Single = Math.Round(x, 2)
Dim s As String = String.Format("{0:0.00}", y)

which *does* produce 2726.795.

Assuming that is the right correction:

Any ideas why the first example is not rounding to 2726.80?

The first parameter to the particular overload of Math.Round being used
is a Double. That means that when a Single is supplied (as in the
corrected first example), it must be converted to a Double. This is OK
to be done implicitly, since it is a widening conversion. Let's have a
look at the actual exact values involved:

Dim x As Single = 2726.795
Dim xAsDouble As Double = x

Console.WriteLine(x.ToString("G9"))
Console.WriteLine(xAsDouble.ToString("R"))

The output is

2726.79492
2726.794921875

So we see that trying to put 2726.795 into a Single actually puts in a
value slightly *less* than 2726.795, which is why it gets rounded to
....79. With a double, on the other hand (as in the second example):

Dim x As Double = 2726.795

Console.WriteLine(x.ToString("G17"))

2726.7950000000001

So trying to put 2726.295 into a Double actually puts in a value
slightly *more* than 2726.295, which is why it gets rounded to ...80.
That's the why. The next question is the what-to-do.

I'd say the first recommendation is this: don't use floating-point
types to deal with exact values. Actually it's more than a
recommendation, it's a golden rule. It's customary for me to give my
usual example at this point:

Debug.Print(0.1+0.1+0.1-0.3) 'guess output before running!
The next recommendation would be, don't use Single at all unless you
have a very good reason. Single only has about *seven* decimal digits
of precision, which we run into pretty quickly in the real world. Any
memory gains you think you might be getting by using Singles over
Doubles are almost certainly illusory. Any performance gains you think
you are getting are _wrong_: "Double is the most efficient of the
fractional data types, because the processors on current platforms
perform floating-point operations in double precision", to quote the
docs. In short, if you are going to use floating point (with the above
caveat that you shouldn't if... you shouldn't), use Double not Single.

And now for the short answer:

I need to use the first example as I read the value of x in as a string from
an xml file then convert it to a single in order to round and format it to 2
decimal places for a report display.

Use Decimal as the intermediate data type to avoid loss of precision.

--
Larry Lard
Replies to group please