472,958 Members | 2,444 Online

# Rounding Problem

I am having a rounding problem in a value i am trying to display in VB.NET.

If I have the following code:
Dim x As Single = 2726.795
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.79

If I have the following code:
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.80

I need to use the first example as I read the value of x in as a string from
an xml file then convert it to a single in order to round and format it to 2
decimal places for a report display.

Any ideas why the first example is not rounding to 2726.80?

May 30 '06 #1
5 2216 [the short answer is at the end]

Jason wrote:
I am having a rounding problem in a value i am trying to display in VB.NET.

If I have the following code:
Dim x As Single = 2726.795
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.79

If I have the following code:
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.80
The two lines of the second example are the same as the second two
lines of the first example. I suspect you have a typo somewhere.

Incidentally, I see you are running without the benefit of Option
Strict (Math.Round returns a Double which cannot be implicitly
converted to a Single). I suggest turning it on - the additional code
sometimes required is a small price to pay for the increased
compile-time checking.
My guess is that you meant the first example to be:

Dim x As Single = 2726.795
Dim y As Single = Math.Round(x, 2)
Dim s As String = String.Format("{0:0.00}", y)

which *does* produce 2726.795.

Assuming that is the right correction:
Any ideas why the first example is not rounding to 2726.80?
The first parameter to the particular overload of Math.Round being used
is a Double. That means that when a Single is supplied (as in the
corrected first example), it must be converted to a Double. This is OK
to be done implicitly, since it is a widening conversion. Let's have a
look at the actual exact values involved:

Dim x As Single = 2726.795
Dim xAsDouble As Double = x

Console.WriteLine(x.ToString("G9"))
Console.WriteLine(xAsDouble.ToString("R"))

The output is

2726.79492
2726.794921875

So we see that trying to put 2726.795 into a Single actually puts in a
value slightly *less* than 2726.795, which is why it gets rounded to
....79. With a double, on the other hand (as in the second example):

Dim x As Double = 2726.795

Console.WriteLine(x.ToString("G17"))

2726.7950000000001

So trying to put 2726.295 into a Double actually puts in a value
slightly *more* than 2726.295, which is why it gets rounded to ...80.
That's the why. The next question is the what-to-do.

I'd say the first recommendation is this: don't use floating-point
types to deal with exact values. Actually it's more than a
recommendation, it's a golden rule. It's customary for me to give my
usual example at this point:

Debug.Print(0.1+0.1+0.1-0.3) 'guess output before running!
The next recommendation would be, don't use Single at all unless you
have a very good reason. Single only has about *seven* decimal digits
of precision, which we run into pretty quickly in the real world. Any
memory gains you think you might be getting by using Singles over
Doubles are almost certainly illusory. Any performance gains you think
you are getting are _wrong_: "Double is the most efficient of the
fractional data types, because the processors on current platforms
perform floating-point operations in double precision", to quote the
docs. In short, if you are going to use floating point (with the above
caveat that you shouldn't if... you shouldn't), use Double not Single.

And now for the short answer:
I need to use the first example as I read the value of x in as a string from
an xml file then convert it to a single in order to round and format it to 2
decimal places for a report display.

Use Decimal as the intermediate data type to avoid loss of precision.

--
Larry Lard

May 30 '06 #2
Larry,

Using Decimal cuts right to the chase:

Dim myDec As Decimal = 2726.795

MsgBox(Math.Round(myDec, 2))

This code produces 2726.80, which the OP does not want.

Of course, the reason that Math.Round produces this value is very clearly
laid out in the docs on Math.Round (IEEE Standard 754, section 4 or banker's
rounding).

The real question is probably "How to do rounding that rounds 2726.795 to
2726.79.

Maybe using INT or FIX or FLOOR, etc is the required approach.

Kerry Moorman

"Larry Lard" wrote:

Use Decimal as the intermediate data type to avoid loss of precision.

May 30 '06 #3
> Larry,

Using Decimal cuts right to the chase:

Dim myDec As Decimal = 2726.795

MsgBox(Math.Round(myDec, 2))

This code produces 2726.80, which the OP does not want.

Of course, the reason that Math.Round produces this value is very
clearly laid out in the docs on Math.Round (IEEE Standard 754, section
4 or banker's rounding).

The real question is probably "How to do rounding that rounds 2726.795
to 2726.79.

Maybe using INT or FIX or FLOOR, etc is the required approach.

the rounding method you want: banker's round or away from 0 rounding. Unfortunately
the framework isn't necessarily consistant in rounding approaches. Math.Round
uses bankers rounding by default, but Format and ToString(c2) use away from
0 rounding. See my write-up at http://devauthority.com/blogs/jwoole...03/24/806.aspx
for more.

Jim Wooley
http://devauthority.com/blogs/jwooley/default.aspx
May 30 '06 #4
Jim

That information, about MidpointRounding.AwayFromZero, is good to know and
may go a ways to help clear up the confusion that has always surrounded the
Math.Round method. I had not seen that before you pointed it out.

Unfortunately, its a .Net 2.0-only feature.

Kerry Moorman
"Jim Wooley" wrote:

the rounding method you want: banker's round or away from 0 rounding. Unfortunately
the framework isn't necessarily consistant in rounding approaches. Math.Round
uses bankers rounding by default, but Format and ToString(c2) use away from
0 rounding. See my write-up at http://devauthority.com/blogs/jwoole...03/24/806.aspx
for more.

Jim Wooley
http://devauthority.com/blogs/jwooley/default.aspx

May 30 '06 #5

I will switch from using Single to Double as that has solved my problem. The
reason I have been using Single is that was the type I get back from my
DataSet schema when connecting to the Access database I am working with. I
will explicitly convert to Double in code for rounding functions.

"Larry Lard" wrote:
[the short answer is at the end]

Jason wrote:
I am having a rounding problem in a value i am trying to display in VB.NET.

If I have the following code:
Dim x As Single = 2726.795
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.79

If I have the following code:
Dim y As Single = Math.Round(2726.795, 2)
Dim s as String = String.Format("{0:0.00}", y)
The value of s is 2726.80

The two lines of the second example are the same as the second two
lines of the first example. I suspect you have a typo somewhere.

Incidentally, I see you are running without the benefit of Option
Strict (Math.Round returns a Double which cannot be implicitly
converted to a Single). I suggest turning it on - the additional code
sometimes required is a small price to pay for the increased
compile-time checking.
My guess is that you meant the first example to be:

Dim x As Single = 2726.795
Dim y As Single = Math.Round(x, 2)
Dim s As String = String.Format("{0:0.00}", y)

which *does* produce 2726.795.

Assuming that is the right correction:
Any ideas why the first example is not rounding to 2726.80?

The first parameter to the particular overload of Math.Round being used
is a Double. That means that when a Single is supplied (as in the
corrected first example), it must be converted to a Double. This is OK
to be done implicitly, since it is a widening conversion. Let's have a
look at the actual exact values involved:

Dim x As Single = 2726.795
Dim xAsDouble As Double = x

Console.WriteLine(x.ToString("G9"))
Console.WriteLine(xAsDouble.ToString("R"))

The output is

2726.79492
2726.794921875

So we see that trying to put 2726.795 into a Single actually puts in a
value slightly *less* than 2726.795, which is why it gets rounded to
....79. With a double, on the other hand (as in the second example):

Dim x As Double = 2726.795

Console.WriteLine(x.ToString("G17"))

2726.7950000000001

So trying to put 2726.295 into a Double actually puts in a value
slightly *more* than 2726.295, which is why it gets rounded to ...80.
That's the why. The next question is the what-to-do.

I'd say the first recommendation is this: don't use floating-point
types to deal with exact values. Actually it's more than a
recommendation, it's a golden rule. It's customary for me to give my
usual example at this point:

Debug.Print(0.1+0.1+0.1-0.3) 'guess output before running!
The next recommendation would be, don't use Single at all unless you
have a very good reason. Single only has about *seven* decimal digits
of precision, which we run into pretty quickly in the real world. Any
memory gains you think you might be getting by using Singles over
Doubles are almost certainly illusory. Any performance gains you think
you are getting are _wrong_: "Double is the most efficient of the
fractional data types, because the processors on current platforms
perform floating-point operations in double precision", to quote the
docs. In short, if you are going to use floating point (with the above
caveat that you shouldn't if... you shouldn't), use Double not Single.

And now for the short answer:
I need to use the first example as I read the value of x in as a string from
an xml file then convert it to a single in order to round and format it to 2
decimal places for a report display.

Use Decimal as the intermediate data type to avoid loss of precision.

--
Larry Lard
 3 by: Caesar Augustus | last post by: Hello Tom, Long time listener, first time caller... I have been working with vb code to automate a salary increase process that not only rounds to the nearest penny but also rounds to the... 0 by: Rick Gabel | last post by: Trying to use the decimal datatype with SQL Server2000 and C#.Net and having a rounding problem. I am using SqlCommand/SqlParameter process to retrieve and insert a record into the table. One... 11 by: cj | last post by: Lets assume all calculations are done with decimal data types so things are as precise as possible. When it comes to the final rounding to cut a check to pay dividends for example in VB rounding... 4 by: Altman | last post by: I am having a rounding problem all over the place and am struggling to fix it. My problem is that I have an unit price and a quantity. When I multiply them together they come out to 3.705. I... 29 by: Marco | last post by: Hello, I have : float f = 36.09999999; When I do : char cf; sprintf(cf,"%0.03lf", f); I get : 36.100 18 by: jdrott1 | last post by: i'm trying to round my currency string to end in 9. it's for a pricing application. this is the function i'm using to get the item in currency: FormatCurrency(BoxCost, , , , TriState.True) if... 206 by: md | last post by: Hi Does any body know, how to round a double value with a specific number of digits after the decimal points? A function like this: RoundMyDouble (double &value, short numberOfPrecisions) ... 20 by: jacob navia | last post by: Hi "How can I round a number to x decimal places" ? This question keeps appearing. I would propose the following solution #include #include 30 by: bdsatish | last post by: The built-in function round( ) will always "round up", that is 1.5 is rounded to 2.0 and 2.5 is rounded to 3.0. If I want to round to the nearest even, that is my_round(1.5) = 2 # As... 2 by: isladogs | last post by: The next Access Europe meeting will be on Wednesday 4 Oct 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central... 0 by: Aliciasmith | last post by: In an age dominated by smartphones, having a mobile app for your business is no longer an option; it's a necessity. Whether you're a startup or an established enterprise, finding the right mobile app... 2 by: giovanniandrean | last post by: The energy model is structured as follows and uses excel sheets to give input data: 1-Utility.py contains all the functions needed to calculate the variables and other minor things (mentions... 4 by: NeoPa | last post by: Hello everyone. I find myself stuck trying to find the VBA way to get Access to create a PDF of the currently-selected (and open) object (Form or Report). I know it can be done by selecting :... 3 by: NeoPa | last post by: Introduction For this article I'll be using a very simple database which has Form (clsForm) & Report (clsReport) classes that simply handle making the calling Form invisible until the Form, or all... 1 by: Teri B | last post by: Hi, I have created a sub-form Roles. In my course form the user selects the roles assigned to the course. 0ne-to-many. One course many roles. Then I created a report based on the Course form and... 0 by: isladogs | last post by: The next Access Europe meeting will be on Wednesday 1 Nov 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM) Please note that the UK and Europe revert to winter time on... 3 by: nia12 | last post by: Hi there, I am very new to Access so apologies if any of this is obvious/not clear. I am creating a data collection tool for health care employees to complete. It consists of a number of... 0 by: isladogs | last post by: The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...