By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
434,805 Members | 1,427 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 434,805 IT Pros & Developers. It's quick & easy.

Decimal/Single/...: which Numeric-type to use?

P: n/a
Hi,

I'm having some troubles with my numeric-types in my VB.NET 2005
application, together with a SQL Server 2000.

- I first used Single in my application, and Decimal in my database. But a
Single with value 4.475 was converted to a Decimal with value
4.4749999999999996D. So after inserting and selecting it from the database I
got another value than the original!

- So I used also the Decimal in my application, but now all my numbers are
formatted with all the decimals: when my value is "1", it shows "1.0000". A
single value roudns it nicely to 1, but the decimal value shows the whole
decimal part, even when it isn't needed.
What I need is: the users should only see the decimal part when needed, AND
my values have to be the same (no 4.475 to 4.474999999999 changements).

This seems such a simple and basic need, but I just keep having troubles
with it :-(

What should I do/use? Another numeric datatype in my application? Another
datatype in my Database??

Any help/hints/links/information would be really appreciated!

Thansk a lot in advance,

Pieter
Mar 22 '06 #1
Share this Question
Share on Google+
11 Replies


P: n/a
guy
er, it depends
are the calculations that you are performing financial style calcs where
precision is everything? if so use Decimal. or are they scientific calcs
where you have a wide range of numbers that need ot be accurate to a a given
nummber of digits? (eg calculation of stresses in an engineering project) in
this case use Double. I wouldnt use Single unless... it is a leagacy app
being converted or you have REALLY big arrays and are tight on memory

dont worry about the formatting, that is trivial comapred with getting the
data right, you can sort it out later.

hth

guy

"Pieter" wrote:
Hi,

I'm having some troubles with my numeric-types in my VB.NET 2005
application, together with a SQL Server 2000.

- I first used Single in my application, and Decimal in my database. But a
Single with value 4.475 was converted to a Decimal with value
4.4749999999999996D. So after inserting and selecting it from the database I
got another value than the original!

- So I used also the Decimal in my application, but now all my numbers are
formatted with all the decimals: when my value is "1", it shows "1.0000". A
single value roudns it nicely to 1, but the decimal value shows the whole
decimal part, even when it isn't needed.
What I need is: the users should only see the decimal part when needed, AND
my values have to be the same (no 4.475 to 4.474999999999 changements).

This seems such a simple and basic need, but I just keep having troubles
with it :-(

What should I do/use? Another numeric datatype in my application? Another
datatype in my Database??

Any help/hints/links/information would be really appreciated!

Thansk a lot in advance,

Pieter

Mar 22 '06 #2

P: n/a
Precision is indeed everything. Most of the time it are prices, weights,
heights etc. So it has to be exact.

And I do worry about the formatting, because when I show the decimal value
in a Textbox, it shows all the zeros after the decimal, and I don't want it
to do it. Especially in some cases where it contains most of the time whole
numbers (1, 2, 3, ...) and rarely 1.001 etc. So I don't want the user to be
seeing the whole time 1.0000, 2.0000, 3.0000, etc.

Basicly: I need exact precision, but without the zeros at the end.

How should I do that? there isn't a Format-property on the textbox...

"guy" <gu*@discussions.microsoft.com> wrote in message
news:52**********************************@microsof t.com...
er, it depends
are the calculations that you are performing financial style calcs where
precision is everything? if so use Decimal. or are they scientific calcs
where you have a wide range of numbers that need ot be accurate to a a
given
nummber of digits? (eg calculation of stresses in an engineering project)
in
this case use Double. I wouldnt use Single unless... it is a leagacy app
being converted or you have REALLY big arrays and are tight on memory

dont worry about the formatting, that is trivial comapred with getting the
data right, you can sort it out later.

hth

guy

Mar 22 '06 #3

P: n/a
guy
look at the overloads of Decimal.ToString

this one may help...
(it is the online version of Help)

http://msdn.microsoft.com/library/de...ringtopic3.asp

hth

guy

"Pieter" wrote:
Precision is indeed everything. Most of the time it are prices, weights,
heights etc. So it has to be exact.

And I do worry about the formatting, because when I show the decimal value
in a Textbox, it shows all the zeros after the decimal, and I don't want it
to do it. Especially in some cases where it contains most of the time whole
numbers (1, 2, 3, ...) and rarely 1.001 etc. So I don't want the user to be
seeing the whole time 1.0000, 2.0000, 3.0000, etc.

Basicly: I need exact precision, but without the zeros at the end.

How should I do that? there isn't a Format-property on the textbox...

"guy" <gu*@discussions.microsoft.com> wrote in message
news:52**********************************@microsof t.com...
er, it depends
are the calculations that you are performing financial style calcs where
precision is everything? if so use Decimal. or are they scientific calcs
where you have a wide range of numbers that need ot be accurate to a a
given
nummber of digits? (eg calculation of stresses in an engineering project)
in
this case use Double. I wouldnt use Single unless... it is a leagacy app
being converted or you have REALLY big arrays and are tight on memory

dont worry about the formatting, that is trivial comapred with getting the
data right, you can sort it out later.

hth

guy


Mar 22 '06 #4

P: n/a
Pieter,

I have the idea that you do not use precise but rounded decimal values.

Be aware that it uses Banking roundings as default. In version 2005 you can
as well use other roundings (as standard is used in the Benelux by
instance). I could not find the information on MSDN, so have a look for
yourself.

Cor

"Pieter" <pi**********@hotmail.com> schreef in bericht
news:Ot**************@TK2MSFTNGP10.phx.gbl...
Precision is indeed everything. Most of the time it are prices, weights,
heights etc. So it has to be exact.

And I do worry about the formatting, because when I show the decimal value
in a Textbox, it shows all the zeros after the decimal, and I don't want
it to do it. Especially in some cases where it contains most of the time
whole numbers (1, 2, 3, ...) and rarely 1.001 etc. So I don't want the
user to be seeing the whole time 1.0000, 2.0000, 3.0000, etc.

Basicly: I need exact precision, but without the zeros at the end.

How should I do that? there isn't a Format-property on the textbox...

"guy" <gu*@discussions.microsoft.com> wrote in message
news:52**********************************@microsof t.com...
er, it depends
are the calculations that you are performing financial style calcs where
precision is everything? if so use Decimal. or are they scientific calcs
where you have a wide range of numbers that need ot be accurate to a a
given
nummber of digits? (eg calculation of stresses in an engineering project)
in
this case use Double. I wouldnt use Single unless... it is a leagacy app
being converted or you have REALLY big arrays and are tight on memory

dont worry about the formatting, that is trivial comapred with getting
the
data right, you can sort it out later.

hth

guy


Mar 22 '06 #5

P: n/a
Thanks, but this means that I have to change manually every numeric property
of my objects to a string? And that I can't anymore DataBind my controls
directly to the propertys of my objects? It seems a little bit a werid
solution for such a problem.

Why does a Single threats the trailing zeros right (cuts them off) but
doesn't have ap recision, and does a decimal the opposite?
Isn't there somehow a datatype that combines the two? I can't be the only
one with this problem????

"guy" <gu*@discussions.microsoft.com> wrote in message
news:16**********************************@microsof t.com...
look at the overloads of Decimal.ToString

this one may help...
(it is the online version of Help)

http://msdn.microsoft.com/library/de...ringtopic3.asp

hth

guy

Mar 22 '06 #6

P: n/a
I did some test:
I used single in my application, and Float, Real, Decimal and Numeric in my
database.
The combination Single-float/Real works great, and give mle the exact
results that I wanted. Should I use this? Or is there some problem that I'm
not seeing?

It doesn't round the precision, and it cuts off the trailing zeroes...

"Pieter" <pi**********@hotmail.com> wrote in message
news:uu**************@TK2MSFTNGP11.phx.gbl...
Hi,

I'm having some troubles with my numeric-types in my VB.NET 2005
application, together with a SQL Server 2000.

- I first used Single in my application, and Decimal in my database. But a
Single with value 4.475 was converted to a Decimal with value
4.4749999999999996D. So after inserting and selecting it from the database
I got another value than the original!

- So I used also the Decimal in my application, but now all my numbers are
formatted with all the decimals: when my value is "1", it shows "1.0000".
A single value roudns it nicely to 1, but the decimal value shows the
whole decimal part, even when it isn't needed.
What I need is: the users should only see the decimal part when needed,
AND my values have to be the same (no 4.475 to 4.474999999999
changements).

This seems such a simple and basic need, but I just keep having troubles
with it :-(

What should I do/use? Another numeric datatype in my application? Another
datatype in my Database??

Any help/hints/links/information would be really appreciated!

Thansk a lot in advance,

Pieter

Mar 22 '06 #7

P: n/a
guy
another thing to consider...
in one project i have developed, i built a set of structuers - Distance,
Area, volume, Weight etc. which wrapped a double (you could equally well use
a decimal) with appropriate overloads and formatting. It sounds a lot of work
but more than paid for itself in functionality and ensuring correct variable
usage.

"Pieter" wrote:
Thanks, but this means that I have to change manually every numeric property
of my objects to a string? And that I can't anymore DataBind my controls
directly to the propertys of my objects? It seems a little bit a werid
solution for such a problem.

Why does a Single threats the trailing zeros right (cuts them off) but
doesn't have ap recision, and does a decimal the opposite?
Isn't there somehow a datatype that combines the two? I can't be the only
one with this problem????

"guy" <gu*@discussions.microsoft.com> wrote in message
news:16**********************************@microsof t.com...
look at the overloads of Decimal.ToString

this one may help...
(it is the online version of Help)

http://msdn.microsoft.com/library/de...ringtopic3.asp

hth

guy


Mar 22 '06 #8

P: n/a
Ok, thanks for the help on this issue.
So if I am right I should define my propertys like this:

Public Property FraisEmballageOriginal() As Decimal

Get

Return m_decFraisEmballageOriginal.ToString("0.##")

End Get
"guy" <gu*@discussions.microsoft.com> wrote in message
news:FA**********************************@microsof t.com...
another thing to consider...
in one project i have developed, i built a set of structuers - Distance,
Area, volume, Weight etc. which wrapped a double (you could equally well
use
a decimal) with appropriate overloads and formatting. It sounds a lot of
work
but more than paid for itself in functionality and ensuring correct
variable
usage.

Mar 22 '06 #9

P: n/a


Pieter wrote:
Hi,

I'm having some troubles with my numeric-types in my VB.NET 2005
application, together with a SQL Server 2000.

- I first used Single in my application, and Decimal in my database.
Use the same type to process and store floating numbers, otherwise you
will multiply loss of precision.

Single numbers are "float", and uses base 2, that is it can represent
multiples of halves without precision loss (upto the precision limits).
Decimal is base10 and can represent decimal-numbers without loss (upto
the precision limit).
Single with value 4.475 was converted to a Decimal with value
4.4749999999999996D. So after inserting and selecting it from the database I
got another value than the original!
Actually what happened is that Signgle (float) cannot represent 4.475
without loss, it becomes a bit-pattern which represents
4.4749999999999996. So the conversion to decimal does not loose
precision, the use of float to represent base10 numbers does.

If you format 4.475 as: (4.475).ToString("##.###") you *will* get 4.475,
since the presentation will do rounding.
- So I used also the Decimal in my application, but now all my numbers are
formatted with all the decimals: when my value is "1", it shows "1.0000". A
single value roudns it nicely to 1, but the decimal value shows the whole
decimal part, even when it isn't needed.
You should look into how the numbers are presented to the user. Have a
look at ToString(format) and NumberFormatInfo.
What I need is: the users should only see the decimal part when needed, AND
my values have to be the same (no 4.475 to 4.474999999999 changements).
That can be done with Decimal, which can represent base10 without loss
of precision. beware you you may stil inadvertently generate numbers
which does *not* have representation without loss in base 10, for
example by division.
This seems such a simple and basic need, but I just keep having troubles
with it :-(
floating-point precision numbers are not "reals" as in math, they do not
have infinite precision, that's the problem here.
What should I do/use? Another numeric datatype in my application? Another
datatype in my Database??


Decimal all-round if your *really* need the precision (not just as
4.475), otherwise just use the same in app and databse. And learn to
format the numbers presented to the users, have a look at ToString with
formatting arguments and the string-conversions in string.Format.

Whatever you do, dont start using strings to represent numbers in your
calculations and interface, that won't help you.
--
Helge Jensen
mailto:he**********@slog.dk
sip:he**********@slog.dk
-=> Sebastian cover-music: http://ungdomshus.nu <=-
Mar 22 '06 #10

P: n/a
Hello, Pieter,

It is important to realize that you cannot represent all decimal
fractions exactly with a floating point variable. (It is a little like
trying to represent a number like 2/3 exactly as a decimal number with a
fixed number of decimal places. You can write 0.66667, or you can write
0.66666666667. But neither is precisely correct. There is always some
"round-off error".

For measurements such as weights and heights this is not normally a
problem. The "accuracy" of the measurement is usually less than the
precision of the variable used to store it. (Note there is an important
distinction between accuracy and precision.) Floating point is
typically the best choice for these values, as it provides the
possibility for a very wide range of values.

However, currency values are typically considered to be exact, and so
the "round-off error" that is normal with Single and Double precision
floating point values is a problem. You should use Decimal for these
numbers. Usually with currency values, you want to always display the
same number of places following the decimal, even if they are zero.
I.e. If an amount is Euros, Dollars, Pounds, Kronar, etc. I think it
would be unusual to display 12.22 in one case and 12.2 in another
(instead of 12.20).

Anyway, I wonder if your observation that:
The combination Single-float/Real works great, and give mle the exact
results that I wanted.
is a result of simply having chosen values that can be represented
exactly in floating point, and that if you experimented further you
might find that you again see values like "4.4749999999999996D" instead
of the "4.475" that you hope to see.

Cheers,
Randy

Pieter wrote:
I did some test:
I used single in my application, and Float, Real, Decimal and Numeric in my
database.
The combination Single-float/Real works great, and give mle the exact
results that I wanted. Should I use this? Or is there some problem that I'm
not seeing?

It doesn't round the precision, and it cuts off the trailing zeroes...

"Pieter" <pi**********@hotmail.com> wrote in message
news:uu**************@TK2MSFTNGP11.phx.gbl...
Hi,

I'm having some troubles with my numeric-types in my VB.NET 2005
application, together with a SQL Server 2000.

- I first used Single in my application, and Decimal in my database. But a
Single with value 4.475 was converted to a Decimal with value
4.4749999999999996D. So after inserting and selecting it from the database
I got another value than the original!

- So I used also the Decimal in my application, but now all my numbers are
formatted with all the decimals: when my value is "1", it shows "1.0000".
A single value roudns it nicely to 1, but the decimal value shows the
whole decimal part, even when it isn't needed.
What I need is: the users should only see the decimal part when needed,
AND my values have to be the same (no 4.475 to 4.474999999999
changements).

This seems such a simple and basic need, but I just keep having troubles
with it :-(

What should I do/use? Another numeric datatype in my application? Another
datatype in my Database??

Any help/hints/links/information would be really appreciated!

Thansk a lot in advance,

Pieter


Mar 22 '06 #11

P: n/a
guy
Pieter,
the key is to separate what you display to the operator and the actual value
in your classes.
use the data type of your choice in the class, but build a formatting method
to present it to the operator that does not interfere with applicationss
internals
"Pieter" wrote:
Ok, thanks for the help on this issue.
So if I am right I should define my propertys like this:

Public Property FraisEmballageOriginal() As Decimal

Get

Return m_decFraisEmballageOriginal.ToString("0.##")
*** this wont work, the data type is decimal so you cant return a string *** End Get
"guy" <gu*@discussions.microsoft.com> wrote in message
news:FA**********************************@microsof t.com...
another thing to consider...
in one project i have developed, i built a set of structuers - Distance,
Area, volume, Weight etc. which wrapped a double (you could equally well
use
a decimal) with appropriate overloads and formatting. It sounds a lot of
work
but more than paid for itself in functionality and ensuring correct
variable
usage.


Mar 22 '06 #12

This discussion thread is closed

Replies have been disabled for this discussion.