468,532 Members | 1,708 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 468,532 developers. It's quick & easy.

Pricision of float

Hi,

Just wonder why

PRINT CAST(0.0573542567654 AS float)

will give the rounded reult

0.0573543

rather than the original number?

"float" should be 'big' enough to hold numbers that have even more
decimal places. How come it round up at the 7 decimal place?

since I need to do some calculations with accumulated values. The
rounded figure will cause significant error after a number of opertions.

Are there any way to work around it?

thanks


*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!
Jul 20 '05 #1
2 8413
oj
Float is never exact. You will need decimal or numeric instead.

--
-oj
http://www.rac4sql.net
"Benny" <an*******@devdex.com> wrote in message
news:3f*********************@news.frii.net...
Hi,

Just wonder why

PRINT CAST(0.0573542567654 AS float)

will give the rounded reult

0.0573543

rather than the original number?

"float" should be 'big' enough to hold numbers that have even more
decimal places. How come it round up at the 7 decimal place?

since I need to do some calculations with accumulated values. The
rounded figure will cause significant error after a number of opertions.

Are there any way to work around it?

thanks


*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!

Jul 20 '05 #2
Benny (an*******@devdex.com) writes:
Just wonder why

PRINT CAST(0.0573542567654 AS float)

will give the rounded reult

0.0573543

rather than the original number?

"float" should be 'big' enough to hold numbers that have even more
decimal places. How come it round up at the 7 decimal place?

since I need to do some calculations with accumulated values. The
rounded figure will cause significant error after a number of opertions.

Are there any way to work around it?


PRINT does not work on floats, it works with string data, so there is
an implicit conversion to varchar, and that conversion gives you
seven decimals by default.

Here is one option:

PRINT convert(varchar(30), CAST(0.0573542567654 AS float), 2)

You can also use the str() function to format floats as string. Look
up this function in Books Online.
--
Erland Sommarskog, SQL Server MVP, so****@algonet.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techinf...2000/books.asp
Jul 20 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

5 posts views Thread by Pat | last post: by
16 posts views Thread by Gerald Lafreniere | last post: by
6 posts views Thread by Dave win | last post: by
9 posts views Thread by Sisyphus | last post: by
11 posts views Thread by Marc Pelletier | last post: by
20 posts views Thread by ehabaziz2001 | last post: by
8 posts views Thread by vjnr83 | last post: by
13 posts views Thread by Shirsoft | last post: by
reply views Thread by NPC403 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.