By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,341 Members | 1,460 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,341 IT Pros & Developers. It's quick & easy.

Significant digits in double with Format strings?

P: n/a

MSDN documentation for format strings seems ambiguous. It's says
things like "significant digits or number of decimal places" all over
the place. Well those two things are different.

How do I show a specific number of significant digits to the right of
the decimal place with a format string? NOT number of decimal places,
significant digits.

For example;

if I have the number .001
the format string "{0:N2}" shows 0.00 yet if you were showing two
significant digits shouldn't that number display as 0.001?

another example is a number like 0.00123 showing 2 significant digits
would show 0.0012

It seems the number 2 in the format string is the number of decimal
places not the number of significant figures.

How can I use a format string to do this?

In my application the user can set the number of significant digits they
want to see.

thanks
mike
Mar 13 '07 #1
Share this Question
Share on Google+
7 Replies


P: n/a
Michael Howes <mh****@xfortebio.comwrote:
MSDN documentation for format strings seems ambiguous. It's says
things like "significant digits or number of decimal places" all over
the place. Well those two things are different.
Indeed.
How do I show a specific number of significant digits to the right of
the decimal place with a format string? NOT number of decimal places,
significant digits.

For example;

if I have the number .001
the format string "{0:N2}" shows 0.00 yet if you were showing two
significant digits shouldn't that number display as 0.001?
Yes - but MSDN for "standard numeric format strings" says that "N"
specifies the number of decimal places, not significant digits. "G"
specifies the number of significant places, and works fine.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Mar 13 '07 #2

P: n/a
> For example;

if I have the number .001
the format string "{0:N2}" shows 0.00 yet if you were showing two
significant digits shouldn't that number display as 0.001?

Yes - but MSDN for "standard numeric format strings" says that "N"
specifies the number of decimal places, not significant digits. "G"
specifies the number of significant places, and works fine.
Ah...thanks!

the MSDN page I saw didn't say much about G. It just said "General",
so I sort of ignored that.
http://msdn2.microsoft.com/en-us/library/s8s7t687.aspx

phew, thanks...I didn't really want to write it and was more surprised
that I didn't think it was built it.

mike
Mar 13 '07 #3

P: n/a
>>
For example;

if I have the number .001
the format string "{0:N2}" shows 0.00 yet if you were showing two
significant digits shouldn't that number display as 0.001?

Yes - but MSDN for "standard numeric format strings" says that "N"
specifies the number of decimal places, not significant digits. "G"
specifies the number of significant places, and works fine.
One more thing. I noticed G doesn't pad 0s to the right

for example if I have the number .01 and use the format string {0:G2}
shouldn't the resulting string be 0.010?

Also how do I not have the format string add a 0 to the left of the
decimal? Most of the numbers I'm working with are less than 1

thanks
mike

Mar 13 '07 #4

P: n/a
Michael Howes <mh****@xfortebio.comwrote:
if I have the number .001
the format string "{0:N2}" shows 0.00 yet if you were showing two
significant digits shouldn't that number display as 0.001?
Yes - but MSDN for "standard numeric format strings" says that "N"
specifies the number of decimal places, not significant digits. "G"
specifies the number of significant places, and works fine.

One more thing. I noticed G doesn't pad 0s to the right

for example if I have the number .01 and use the format string {0:G2}
shouldn't the resulting string be 0.010?
No, not as far as I'm aware. For instance, 12 to 5 significant figures
is just 12, not 12.000. (IIRC!)
Also how do I not have the format string add a 0 to the left of the
decimal? Most of the numbers I'm working with are less than 1
Personally I think that 0.1 is a lot clearer than .1 - it's easy to
miss the .

I suspect you'd have to trim the 0 off yourself.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Mar 14 '07 #5

P: n/a

"Jon Skeet [C# MVP]" <sk***@pobox.comwrote in message
news:MP************************@msnews.microsoft.c om...
Michael Howes <mh****@xfortebio.comwrote:
>if I have the number .001
the format string "{0:N2}" shows 0.00 yet if you were showing two
significant digits shouldn't that number display as 0.001?

Yes - but MSDN for "standard numeric format strings" says that "N"
specifies the number of decimal places, not significant digits. "G"
specifies the number of significant places, and works fine.

One more thing. I noticed G doesn't pad 0s to the right

for example if I have the number .01 and use the format string {0:G2}
shouldn't the resulting string be 0.010?

No, not as far as I'm aware. For instance, 12 to 5 significant figures
is just 12, not 12.000. (IIRC!)
Definitely not. 12.000 indicates accuracy of within .005, if not better,
while 12 could be off by 1 or more.
>
> Also how do I not have the format string add a 0 to the left of the
decimal? Most of the numbers I'm working with are less than 1

Personally I think that 0.1 is a lot clearer than .1 - it's easy to
miss the .

I suspect you'd have to trim the 0 off yourself.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

Mar 14 '07 #6

P: n/a
On Mar 14, 4:59 pm, "Ben Voigt" <r...@nospam.nospamwrote:
for example if I have the number .01 and use the format string {0:G2}
shouldn't the resulting string be 0.010?
No, not as far as I'm aware. For instance, 12 to 5 significant figures
is just 12, not 12.000. (IIRC!)

Definitely not. 12.000 indicates accuracy of within .005, if not better,
while 12 could be off by 1 or more.
Yup, you're right. Looks like a bug then.

Jon

Mar 14 '07 #7

P: n/a
>>> for example if I have the number .01 and use the format string {0:G2}
>>>shouldn't the resulting string be 0.010?
No, not as far as I'm aware. For instance, 12 to 5 significant figures
is just 12, not 12.000. (IIRC!)
Definitely not. 12.000 indicates accuracy of within .005, if not better,
while 12 could be off by 1 or more.

Yup, you're right. Looks like a bug then.
anyone have ideas for ways around this problem?
I was trying to think of a way to maybe format twice?
I have a lot of numbers I try and show quickly so am not that into the
idea of writing my own loops to count decimals and sig figs

thanks
mike
Mar 15 '07 #8

This discussion thread is closed

Replies have been disabled for this discussion.