On Thu, 13 Nov 2008 07:02:12 -0800 (PST), Tim wrote:

>Folks,

Can anyone thow some clarifying light on the following?

Hi Tim,

In addition to Erland's reply, here are a few othe rpoints to keep in

mind:

1) What you see on your screen is how the front end chooses to render

the numbers. Not all clients use the same number of digits when

displaying a floating point number. One way to get a reliable look at

the real precision is to cast the number to e.g. DECIMAL(38,30) (unless

the number is too big or too small to fit is the range of fixed point

numbers.

2) Precision of a floating point number is measured in "significant

figures", i.e. it's calculated from the first non-zero number. So the

numbers 123000000, 123, 1.23, and 0.0000123 all have a precision of

three significant figures.

3) Floating point numbers are actually stored in some binary format.

Some numbers that can be represented as an exact number in decimal can

not be in binary (just as 1/3 can not be represented exactly in

decimal). The actual precision is based on the internals of the actual

storage; the precision quoted in Books Online is an approximation (and I

believe it's a "safe" approximation, i.e. you will get at least that

precision for any number - but don't take my word for this, I'm not 100%

sure).

Does this answer all your questions?

--

Hugo Kornelis, SQL Server MVP

My SQL Server blog:

http://sqlblog.com/blogs/hugo_kornelis