471,627 Members | 2,034 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,627 software developers and data experts.

Decimal Precision - short code, strange problem

Hi

Could anyone help me to understand why in the following code sum1 is
different from sum2?
Basically it's the same calculation done. Keep in mind the x is 28
digits so it's within the range of decimal.

<codesnip>
decimal x = (-0.0084682975822291150192357277M);
decimal sum1 , sum2;

sum2 = (3500.91M * x) + (x * 3500M + 822m);
sum1 = (3500.91M * x) + (x * 3500M) + 822m;

Console.WriteLine (sum1);
Console.WriteLine (sum2);
</codesnip>

Thank you,
Saar Carmi

Feb 6 '06 #1
2 1532
<sa*******@gmail.com> wrote:
Could anyone help me to understand why in the following code sum1 is
different from sum2?
Basically it's the same calculation done. Keep in mind the x is 28
digits so it's within the range of decimal.

<codesnip>
decimal x = (-0.0084682975822291150192357277M);
decimal sum1 , sum2;

sum2 = (3500.91M * x) + (x * 3500M + 822m);
sum1 = (3500.91M * x) + (x * 3500M) + 822m;

Console.WriteLine (sum1);
Console.WriteLine (sum2);
</codesnip>


I believe I understand the reason for the difference. Suppose you have
a type which is accurate to 3 significant digits. Now consider the
difference between:

100 + 1.49 + 0.03
and
100 + (1.49 + 0.03)

The first would be evaluated as:

(100 + 1.49) + 0.03
=> 101 + 0.03
=> 101

The second is:

100 + (1.49 + 0.03)
=> 100 + 1.52
=> 102

Do you follow why the order is important?

I haven't checked whether or not that's what's happening in your case,
but I suspect it is. You could print out each part of the sum and do it
manually to verify it.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Feb 6 '06 #2
Seems you're right.
Thank you.

Feb 7 '06 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

21 posts views Thread by Batista, Facundo | last post: by
687 posts views Thread by cody | last post: by
8 posts views Thread by nick | last post: by
10 posts views Thread by Paul Sullivan | last post: by
3 posts views Thread by Boot2TheHead | last post: by
23 posts views Thread by neha_chhatre | last post: by
1 post views Thread by XIAOLAOHU | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.