Bug! Format statement introduces a round off

[RD]

The statements

The value passed to Amount by the function call is 111111111111111.98 and is
a double

then we 4execute the following code snippet

Dim New_amt as string
New_amt = Format(amount, "000000000000000.00")

The new_amt value returned is

"111111111111112.00"

Which is wrong.

Does any one know why and how to work around thisÉ

Thanks

Bob

[Tom Dacon]

It's not a bug. The value you're passing into Format has more significant
digits than the double data type can handle, so it's not unreasonable to see
roundoff with values like that. For double-precision variables you get
around fifteen or sixteen significant digits, depending on the value. If you
need more significant digits, use the Decimal type, which has (I think)
twenty-seven significant digits.

Tom Dacon
Dacon Software Consulting

"RD" <no****@nospam.net> wrote in message
news:Ow**************@TK2MSFTNGP10.phx.gbl...

The statements

The value passed to Amount by the function call is 111111111111111.98 and
is
a double

then we 4execute the following code snippet

Dim New_amt as string
New_amt = Format(amount, "000000000000000.00")

The new_amt value returned is

"111111111111112.00"

Which is wrong.

Does any one know why and how to work around thisÉ

Thanks

Bob

[Herfried K. Wagner [MVP]]

"RD" <no****@nospam.net> schrieb:

The value passed to Amount by the function call is
111111111111111.98 and is a double

then we 4execute the following code snippet

Dim New_amt as string
New_amt = Format(amount, "000000000000000.00")

The new_amt value returned is

"111111111111112.00"

That's not a bug. The floating point number literal 111....1.98 cannot be
represented in a 'Double'.

<URL:http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html>
<URL:http://www.math.grin.edu/~stone/courses/fundamentals/IEEE-reals.html>
<URL:http://support.microsoft.com/?scid=kb;[LN];42980>

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>

[RD]

Can you tell me what I can use to correctly represent it, that would be very
helpfull

Thank you for your help.

Bob
"Herfried K. Wagner [MVP]" <hi***************@gmx.at> wrote in message
news:OQ**************@TK2MSFTNGP11.phx.gbl...

"RD" <no****@nospam.net> schrieb:

The value passed to Amount by the function call is
111111111111111.98 and is a double

then we 4execute the following code snippet

Dim New_amt as string
New_amt = Format(amount, "000000000000000.00")

The new_amt value returned is

"111111111111112.00"
That's not a bug. The floating point number literal 111....1.98 cannot be
represented in a 'Double'.

<URL:http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html> <URL:http://www.math.grin.edu/~stone/courses/fundamentals/IEEE-reals.html>
<URL:http://support.microsoft.com/?scid=kb;[LN];42980>

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>

[Jay B. Harlow [MVP - Outlook]]

RD,
As Tom suggested, a System.Decimal value.

Something like:

Dim amount As Decimal = 111111111111111.98D
Dim New_amt As String
New_amt = amount.ToString("000000000000000.00")

FWIW: The D on 111111111111111.98D indicates that it is a Decimal literal,
where as 111111111111111.98 is a Double literal and 111111111111111.98F is a
Single literal.

Hope this helps
Jay

"RD" <no****@nospam.net> wrote in message
news:ey**************@TK2MSFTNGP11.phx.gbl...

Can you tell me what I can use to correctly represent it, that would be
very
helpfull

Thank you for your help.

Bob
"Herfried K. Wagner [MVP]" <hi***************@gmx.at> wrote in message
news:OQ**************@TK2MSFTNGP11.phx.gbl...

"RD" <no****@nospam.net> schrieb:

> The value passed to Amount by the function call is
> 111111111111111.98 and is a double
>
> then we 4execute the following code snippet
>
> Dim New_amt as string
> New_amt = Format(amount, "000000000000000.00")
>
> The new_amt value returned is
>
> "111111111111112.00"

That's not a bug. The floating point number literal 111....1.98 cannot
be
represented in a 'Double'.

<URL:http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html>

<URL:http://www.math.grin.edu/~stone/courses/fundamentals/IEEE-reals.html>
<URL:http://support.microsoft.com/?scid=kb;[LN];42980>

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>