P: n/a

I am having a rounding problem in a value i am trying to display in VB.NET.
If I have the following code:
Dim x As Single = 2726.795
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.79
If I have the following code:
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.80
I need to use the first example as I read the value of x in as a string from
an xml file then convert it to a single in order to round and format it to 2
decimal places for a report display.
Any ideas why the first example is not rounding to 2726.80?  
Share this Question
P: n/a

[the short answer is at the end]
Jason wrote: I am having a rounding problem in a value i am trying to display in VB.NET.
If I have the following code: Dim x As Single = 2726.795 Dim y As Single = Math.Round(2726.795, 2) Dim s as String = String.Format("{0:0.00}", y) The value of s is 2726.79
If I have the following code: Dim y As Single = Math.Round(2726.795, 2) Dim s as String = String.Format("{0:0.00}", y) The value of s is 2726.80
The two lines of the second example are the same as the second two
lines of the first example. I suspect you have a typo somewhere.
Incidentally, I see you are running without the benefit of Option
Strict (Math.Round returns a Double which cannot be implicitly
converted to a Single). I suggest turning it on  the additional code
sometimes required is a small price to pay for the increased
compiletime checking.
My guess is that you meant the first example to be:
Dim x As Single = 2726.795
Dim y As Single = Math.Round(x, 2)
Dim s As String = String.Format("{0:0.00}", y)
which *does* produce 2726.795.
Assuming that is the right correction:
Any ideas why the first example is not rounding to 2726.80?
The first parameter to the particular overload of Math.Round being used
is a Double. That means that when a Single is supplied (as in the
corrected first example), it must be converted to a Double. This is OK
to be done implicitly, since it is a widening conversion. Let's have a
look at the actual exact values involved:
Dim x As Single = 2726.795
Dim xAsDouble As Double = x
Console.WriteLine(x.ToString("G9"))
Console.WriteLine(xAsDouble.ToString("R"))
The output is
2726.79492
2726.794921875
So we see that trying to put 2726.795 into a Single actually puts in a
value slightly *less* than 2726.795, which is why it gets rounded to
....79. With a double, on the other hand (as in the second example):
Dim x As Double = 2726.795
Console.WriteLine(x.ToString("G17"))
2726.7950000000001
So trying to put 2726.295 into a Double actually puts in a value
slightly *more* than 2726.295, which is why it gets rounded to ...80.
That's the why. The next question is the whattodo.
I'd say the first recommendation is this: don't use floatingpoint
types to deal with exact values. Actually it's more than a
recommendation, it's a golden rule. It's customary for me to give my
usual example at this point:
Debug.Print(0.1+0.1+0.10.3) 'guess output before running!
The next recommendation would be, don't use Single at all unless you
have a very good reason. Single only has about *seven* decimal digits
of precision, which we run into pretty quickly in the real world. Any
memory gains you think you might be getting by using Singles over
Doubles are almost certainly illusory. Any performance gains you think
you are getting are _wrong_: "Double is the most efficient of the
fractional data types, because the processors on current platforms
perform floatingpoint operations in double precision", to quote the
docs. In short, if you are going to use floating point (with the above
caveat that you shouldn't if... you shouldn't), use Double not Single.
And now for the short answer:
I need to use the first example as I read the value of x in as a string from an xml file then convert it to a single in order to round and format it to 2 decimal places for a report display.
Use Decimal as the intermediate data type to avoid loss of precision.

Larry Lard
Replies to group please  
P: n/a

Larry,
Using Decimal cuts right to the chase:
Dim myDec As Decimal = 2726.795
MsgBox(Math.Round(myDec, 2))
This code produces 2726.80, which the OP does not want.
Of course, the reason that Math.Round produces this value is very clearly
laid out in the docs on Math.Round (IEEE Standard 754, section 4 or banker's
rounding).
The real question is probably "How to do rounding that rounds 2726.795 to
2726.79.
Maybe using INT or FIX or FLOOR, etc is the required approach.
Kerry Moorman
"Larry Lard" wrote: Use Decimal as the intermediate data type to avoid loss of precision.  
P: n/a

> Larry, Using Decimal cuts right to the chase:
Dim myDec As Decimal = 2726.795
MsgBox(Math.Round(myDec, 2))
This code produces 2726.80, which the OP does not want.
Of course, the reason that Math.Round produces this value is very clearly laid out in the docs on Math.Round (IEEE Standard 754, section 4 or banker's rounding).
The real question is probably "How to do rounding that rounds 2726.795 to 2726.79.
Maybe using INT or FIX or FLOOR, etc is the required approach.
Round includes an additional overloaded version which allows you to specify
the rounding method you want: banker's round or away from 0 rounding. Unfortunately
the framework isn't necessarily consistant in rounding approaches. Math.Round
uses bankers rounding by default, but Format and ToString(c2) use away from
0 rounding. See my writeup at http://devauthority.com/blogs/jwoole...03/24/806.aspx
for more.
Jim Wooley http://devauthority.com/blogs/jwooley/default.aspx  
P: n/a

Jim
That information, about MidpointRounding.AwayFromZero, is good to know and
may go a ways to help clear up the confusion that has always surrounded the
Math.Round method. I had not seen that before you pointed it out.
Unfortunately, its a .Net 2.0only feature.
Kerry Moorman
"Jim Wooley" wrote: Round includes an additional overloaded version which allows you to specify the rounding method you want: banker's round or away from 0 rounding. Unfortunately the framework isn't necessarily consistant in rounding approaches. Math.Round uses bankers rounding by default, but Format and ToString(c2) use away from 0 rounding. See my writeup at http://devauthority.com/blogs/jwoole...03/24/806.aspx for more.
Jim Wooley http://devauthority.com/blogs/jwooley/default.aspx  
P: n/a

Cheers for the reply.
I will switch from using Single to Double as that has solved my problem. The
reason I have been using Single is that was the type I get back from my
DataSet schema when connecting to the Access database I am working with. I
will explicitly convert to Double in code for rounding functions.
"Larry Lard" wrote: [the short answer is at the end]
Jason wrote: I am having a rounding problem in a value i am trying to display in VB.NET.
If I have the following code: Dim x As Single = 2726.795 Dim y As Single = Math.Round(2726.795, 2) Dim s as String = String.Format("{0:0.00}", y) The value of s is 2726.79
If I have the following code: Dim y As Single = Math.Round(2726.795, 2) Dim s as String = String.Format("{0:0.00}", y) The value of s is 2726.80
The two lines of the second example are the same as the second two lines of the first example. I suspect you have a typo somewhere.
Incidentally, I see you are running without the benefit of Option Strict (Math.Round returns a Double which cannot be implicitly converted to a Single). I suggest turning it on  the additional code sometimes required is a small price to pay for the increased compiletime checking.
My guess is that you meant the first example to be:
Dim x As Single = 2726.795 Dim y As Single = Math.Round(x, 2) Dim s As String = String.Format("{0:0.00}", y)
which *does* produce 2726.795.
Assuming that is the right correction:
Any ideas why the first example is not rounding to 2726.80?
The first parameter to the particular overload of Math.Round being used is a Double. That means that when a Single is supplied (as in the corrected first example), it must be converted to a Double. This is OK to be done implicitly, since it is a widening conversion. Let's have a look at the actual exact values involved:
Dim x As Single = 2726.795 Dim xAsDouble As Double = x
Console.WriteLine(x.ToString("G9")) Console.WriteLine(xAsDouble.ToString("R"))
The output is
2726.79492 2726.794921875
So we see that trying to put 2726.795 into a Single actually puts in a value slightly *less* than 2726.795, which is why it gets rounded to ....79. With a double, on the other hand (as in the second example):
Dim x As Double = 2726.795
Console.WriteLine(x.ToString("G17"))
2726.7950000000001
So trying to put 2726.295 into a Double actually puts in a value slightly *more* than 2726.295, which is why it gets rounded to ...80.
That's the why. The next question is the whattodo.
I'd say the first recommendation is this: don't use floatingpoint types to deal with exact values. Actually it's more than a recommendation, it's a golden rule. It's customary for me to give my usual example at this point:
Debug.Print(0.1+0.1+0.10.3) 'guess output before running!
The next recommendation would be, don't use Single at all unless you have a very good reason. Single only has about *seven* decimal digits of precision, which we run into pretty quickly in the real world. Any memory gains you think you might be getting by using Singles over Doubles are almost certainly illusory. Any performance gains you think you are getting are _wrong_: "Double is the most efficient of the fractional data types, because the processors on current platforms perform floatingpoint operations in double precision", to quote the docs. In short, if you are going to use floating point (with the above caveat that you shouldn't if... you shouldn't), use Double not Single.
And now for the short answer:
I need to use the first example as I read the value of x in as a string from an xml file then convert it to a single in order to round and format it to 2 decimal places for a report display.
Use Decimal as the intermediate data type to avoid loss of precision.
 Larry Lard Replies to group please
  This discussion thread is closed Replies have been disabled for this discussion.   Question stats  viewed: 1943
 replies: 5
 date asked: May 30 '06
