I have the following field in a query: - Quantity: Sum([qrytransactions].[Quantity]*[StockMultiplier])
Where [Quantity] is a decimal, such as 100 or 100.23 etc. and [StockMultiplier] is either -1 or +1.
In the source tables, both fields are defined as type double.
The Sum query works fine, unless the sum should be equal to zero. In these cases, I get some very obscure rounding errors. For example, given the following data: - Quantity, StockMultiplier
-
100, 1
-
0.118, 1
-
0.112, 1
-
100.23, -1
I get the following result in the query: -1.4210854715202 E-14 (rather than 0)
Can anyone help with this?
I tried writing a function to "force" the result I wanted. If I express the return value as Currency I see the query result as required (but if it is double the problem persists). This seems far from ideal though. - Expr1: Sum(CalculateQuantity([qrytransactions].[Quantity],[StockMultiplier]))
- Public Function CalculateQuantity(qty As Double, multiple As Double) As Currency
-
-
If multiple = -1 Then
-
CalculateQuantity = -qty
-
Else
-
CalculateQuantity = qty
-
End If
-
-
End Function
6 14600
You've touched on a large & complex subject and there are very probably long explanations of the causes. One is that currency variables have four decimal place precision, as I recall .
A possible fix, if you know how many decimal places you want in your result, is to multiply your numbers by that number, and take the integer of the results to a series of long integer variables, add them add them all up and divide the sum by the number of decimal places.
If the mutiplier is always an integer, it would be better to define it as integer.
You could modify this general approach to fit your needs more precisely. Post again if this is not clear.
Hi. Just to add to YouMike's reply, it should be borne in mind that ALL floating-point arithmetic is approximate to some degree - small rounding errors of the order of 10^-14 occur with Excel just as with Access, for example. This is because it is simpy not possible to represent most values so exactly that there will never be rounding errors - and for some values (1/3 being one of many) there are no exact representations in decimal form in any event.
If you are comparing floating point values to 0 you will always run into rounding problems. The norm in these circumstances is to compare the value to a near-zero value below the smallest value you would count as zero instead. Using currency values as an example, you can compare the absolute value of the result to 0.0001, for instance, instead of to 0 - this is well below the lowest currency value you would normally encounter, yet well above the 'noise threshold' of around 10^-11 which rounding errors typically involve.
-Stewart
PS many years ago software for banking etc used binary-coded decimal (BCD)representa tions of each digit in a number to be able to represent that number exactly, which floating-point numbers cannot do. However, calculations themselves can still result in inexact values even with BCD used (divisions, for instance, taking the case of 1 / 3 again). I don't know if BCD is still used these days.
Thanks for your answers, both of you. It seems that the easiest solution is to compare to a nearly non-zero number. Knowing that 10^-11 is the threshold is very useful.
NeoPa 32,564
Recognized Expert Moderator MVP
The Data Type Currency and the Field Size Decimal are provided for exactly that reason. I believe they actually use a form of BCD but the help description is not greatly helpful there I'm afraid.
What is special about BCD is that the underlying data is stored and worked out using decimal rather than converting to binary and then back for display.
This means that rounding errors (from binary to decimal) are avoided.
While some fractions will always be held inaccurately in decimal, decimal numbers themselves (of which currency values are some) will always be handled accurately.
In short, this is what you, and everybody else, SHOULD be using for any values where arithmetic rounding errors are an issue.
I'm a hypocrite of course, as I still use the defaults ones in most cases ;)
ADezii 8,834
Recognized Expert Expert
Thanks for your answers, both of you. It seems that the easiest solution is to compare to a nearly non-zero number. Knowing that 10^-11 is the threshold is very useful.
I think NeoPa has a very valid point in Post #5 in that you can use the Decimal Data Type to avoid the problem of rounding errors. Remember that Decimal is not a Data Type within itself, but a SubType of Variant. You cannot simply declare a Variable as Type Decimal, no such animal exists. You must create a Variant whose SubType is Decimal using the CDec() Function.
An important point regarding Currency fields is that they have an underlying precision of of four decimal places. This can bring its own rounding problems if you need to multiply a currency value with a decimal quantity to get a total value.
If you are working in a situation where VAT or sales tax need to be calculated as a percentage of value, this is another area where care is needed. Here in South Africa, we have 14% VAT and I have used rounding via Int(CalcValue*1 00 + 0.5)/100 as a predictable means of calculation.
Sign in to post your reply or Sign up for a free account.
Similar topics |
by: Batista, Facundo |
last post by:
Here I send it.
Suggestions and all kinds of recomendations are more than welcomed.
If it all goes ok, it'll be a PEP when I finish writing/modifying the code.
Thank you.
.. Facundo
|
by: John Bentley |
last post by:
John Bentley:
INTRO
The phrase "decimal number" within a programming context is ambiguous. It could
refer to the decimal datatype or the related but separate concept of a generic
decimal number. "Decimal Number" sometimes serves to distinguish Base 10
numbers, eg "15", from Base 2 numbers, Eg "1111". At other times "Decimal
Number" serves to...
|
by: Peter Blatt |
last post by:
Does 5 represent the total numer of digits (including the fractional portion) or only the number of places
BEFORE the decimal point? Moreover does the number include the decimal point?
Are there differences between the databases servers ?
Peter
|
by: Tariq |
last post by:
I've a SQL view performing a number of very precise calculations ie.
495/(1.112 - (.00043499*((Col1 + Col2 + Col3))) + ( .00000055 * ((Col1
+ Col2 + Col3)) * ((Col1 + Col2 + Col3))) - (.00028826 * Col4))) -
450)/100
Obviously this sometimes causes a precision issues because the float
and real datatypes don't hold precise values... My...
|
by: cody |
last post by:
no this is no trollposting and please don't get it wrong but iam very
curious why people still use C instead of other languages especially C++.
i heard people say C++ is slower than C but i can't believe that. in pieces
of the application where speed really matters you can still use "normal"
functions or even static methods which is...
| |
by: nick |
last post by:
printf("%lf",3.25);
the result is 3.25000
i want the answer correct to 3 decimal places
What should i do?
thanks!
|
by: Paul Sullivan |
last post by:
decimal d;
d = 1.1M
OR
d= (decimal) 1.1
Discussioon
|
by: Pieter |
last post by:
Hi,
I'm having some troubles with my numeric-types in my VB.NET 2005
application, together with a SQL Server 2000.
- I first used Single in my application, and Decimal in my database. But a
Single with value 4.475 was converted to a Decimal with value
4.4749999999999996D. So after inserting and selecting it from the database I
got...
|
by: Boot2TheHead |
last post by:
This one cost me a solid half hour yesterday. I'm wondering why on
earth the default precision for a decimal type is 18,0. Maybe I'm
mistaken. A decimal datatype sort of implies that you'd want something
after the decimal!
Question is, can I set this database-wide? Like all new decimal
datatypes have a precision of 12,6 or something like...
|
by: neha_chhatre |
last post by:
which is the best format specifier(data type) if i have to work with
decimal number.
also please tell me the syntax for truncating a decimal number
please reply as soon as possible
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language...
| |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it. ...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| |
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...
| |