Pricision of float

[Benny]

Hi,

Just wonder why

PRINT CAST(0.0573542567654 AS float)

will give the rounded reult

0.0573543

rather than the original number?

"float" should be 'big' enough to hold numbers that have even more
decimal places. How come it round up at the 7 decimal place?

since I need to do some calculations with accumulated values. The
rounded figure will cause significant error after a number of opertions.

Are there any way to work around it?

thanks


*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!

[oj]

Float is never exact. You will need decimal or numeric instead.

--
-oj
http://www.rac4sql.net
"Benny" <an*******@devdex.com> wrote in message
news:3f*********************@news.frii.net...

Hi,

Just wonder why

PRINT CAST(0.0573542567654 AS float)

will give the rounded reult

0.0573543

rather than the original number?

"float" should be 'big' enough to hold numbers that have even more
decimal places. How come it round up at the 7 decimal place?

since I need to do some calculations with accumulated values. The
rounded figure will cause significant error after a number of opertions.

Are there any way to work around it?

thanks


*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!

[Erland Sommarskog]

Benny (an*******@devdex.com) writes:

Just wonder why

PRINT CAST(0.0573542567654 AS float)

will give the rounded reult

0.0573543

rather than the original number?

"float" should be 'big' enough to hold numbers that have even more
decimal places. How come it round up at the 7 decimal place?

since I need to do some calculations with accumulated values. The
rounded figure will cause significant error after a number of opertions.

Are there any way to work around it?

PRINT does not work on floats, it works with string data, so there is
an implicit conversion to varchar, and that conversion gives you
seven decimals by default.

Here is one option:

PRINT convert(varchar(30), CAST(0.0573542567654 AS float), 2)

You can also use the str() function to format floats as string. Look
up this function in Books Online.
--
Erland Sommarskog, SQL Server MVP, so****@algonet.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techinf...2000/books.asp