Very strange thing in VB6

[Allcomp]

Hello,

I have seen something really strange in VB6

If I do a Int ( (5 * 1.2)) , I receive the value 5, but I should receive 6?
Is this a bug or something really "normal".
I can see that if I do
? int ((5 * 1.2 + 0.0000000000000003))
I receive 6. If I add something smaller, I have 5 as a result
What is strange is that If I do a ? 5*1.2, I receive 6 so (5 * 1.2)
should do 6
If I do Int ( (5 * 12 / 10)), the result is OK (6)
What can I do to have it correct?
Is there an option in VB6?

Thank you
Marc Allard
Allcomp

[Chris]

On Wed, 27 Apr 2005 10:35:31 +0200 Allcomp <ma**@nospam.allcomp.be> wrote:

*From:* Allcomp <ma**@nospam.allcomp.be>
*Date:* Wed, 27 Apr 2005 10:35:31 +0200

Hello,

I have seen something really strange in VB6

If I do a Int ( (5 * 1.2)) , I receive the value 5, but I should receive 6?
Is this a bug or something really "normal".
I can see that if I do
? int ((5 * 1.2 + 0.0000000000000003))
I receive 6. If I add something smaller, I have 5 as a result
What is strange is that If I do a ? 5*1.2, I receive 6 so (5 * 1.2)
should do 6
If I do Int ( (5 * 12 / 10)), the result is OK (6)
What can I do to have it correct?
Is there an option in VB6?

Thank you
Marc Allard
Allcomp

Isn't this just typical of the inaccuracies often produced when using binary to
manipulate decimal numbers?

I suspect the actual result is probably something like 5.999999999 (etc) and the
INT of that will be 5. Using Cint(5 * 1.2) will give the correct answer of 6 as
Cint round fractions whereas INT does not.

Chris.

[Rick Rothstein]

> > I have seen something really strange in VB6

If I do a Int ( (5 * 1.2)) , I receive the value 5, but I should receive 6? Is this a bug or something really "normal".
I can see that if I do
? int ((5 * 1.2 + 0.0000000000000003))
I receive 6. If I add something smaller, I have 5 as a result
What is strange is that If I do a ? 5*1.2, I receive 6 so (5 * 1.2)
should do 6
If I do Int ( (5 * 12 / 10)), the result is OK (6)
What can I do to have it correct?
Is there an option in VB6?

Thank you
Marc Allard
Allcomp

Isn't this just typical of the inaccuracies often produced
when using binary to manipulate decimal numbers?

Yes. See http://support.microsoft.com/default...kb;en-us;42980
for more details.

I suspect the actual result is probably something
like 5.999999999 (etc) and the INT of that will be 5.
If you execute this statement: Print 5 * 1.2 - 6
You will get -2.22044604925031E-16 as a response proving that 5*1.2 is
not exactly 6 when the considerations of the above article are kept in
mind.

Using Cint(5 * 1.2) will give the correct answer of 6 as
Cint round fractions whereas INT does not.

The only problem with that suggestion is the CInt function uses Banker's
Rounding, so you may still get unexpected results (for a different
reason than above)...

MsgBox CInt(3 * 1.5)

displays 4, not the expected value of 5.
Rick - MVP

[Rick Rothstein]

> I have seen something really strange in VB6

If I do a Int ( (5 * 1.2)) , I receive the value 5, but I should receive 6? Is this a bug or something really "normal".
I can see that if I do
? int ((5 * 1.2 + 0.0000000000000003))
I receive 6. If I add something smaller, I have 5 as a result
What is strange is that If I do a ? 5*1.2, I receive 6 so (5 * 1.2)
should do 6
If I do Int ( (5 * 12 / 10)), the result is OK (6)
What can I do to have it correct?
Is there an option in VB6?

See my response to Chris' posting for some explanation of why you are
seeing what you wrote about. The solution is to use the Format function
as this avoids the round-off errors that come from trying to cram a
non-exact binary value into a binary representation. And, the Format
function also uses normal rounding instead of Banker's Rounding. So,
replace your statement with this and it will work fine.

Print Format(5 * 1.2, "0")

Rick - MVP

[Raoul Watson]

"Allcomp" <ma**@nospam.allcomp.be> wrote in message
news:42***********************@news.skynet.be...

Hello,

I have seen something really strange in VB6

If I do a Int ( (5 * 1.2)) , I receive the value 5, but I should receive 6? Is this a bug or something really "normal".
I can see that if I do
? int ((5 * 1.2 + 0.0000000000000003))
I receive 6. If I add something smaller, I have 5 as a result
What is strange is that If I do a ? 5*1.2, I receive 6 so (5 * 1.2)
should do 6
If I do Int ( (5 * 12 / 10)), the result is OK (6)
What can I do to have it correct?
Is there an option in VB6?

Thank you
Marc Allard
Allcomp

I suggest you write your own rounding routine.

VB's ailment in its math function is caused because there are errors in the
underlying code. For example, if you do

? round (68.505,2)

you get 68.5, which is incorrect; for if you round 68.505 to two decimal
points you should get 68.51.

This is caused because the underlying code that computes:

? int(68.505 * 100 + .5)/100

gives 68.5!

A more serious error can be seen here:

? int(68.505 * 1000 + 5)
68509 !!! (should have been 68510)

So the bigger the multiplication factor of a fraction, the larger the error
(the above error is a full ONE).

Some of you may argue that's just because certain numbers don't have the
exact binary representation. Yes, this is a known fact. However, those who
have worked long enough with BASIC dating back to QuickBasic will be able to
jump in and swear that QB does not have the math issues that VB has.

[Rick Rothstein]

> I suggest you write your own rounding routine.

Not necessary... see below.

VB's ailment in its math function is caused because there are errors in the underlying code. For example, if you do

? round (68.505,2)

you get 68.5, which is incorrect; for if you round 68.505 to two decimal points you should get 68.51.

This is caused because the underlying code that computes:

? int(68.505 * 100 + .5)/100

gives 68.5!
Uh! That is not correct. The Round function does not return what it does
because of the "underlying code" as you are assuming... the Round
function acts exactly as it was designed to... it uses something known
as Banker's Rounding. Now, personally, I think Microsoft's decision to
use this rounding method exclusively was a poor decision on their part,
but make no mistake, this decision was made on purpose. Here are a
couple of links from Microsoft that talk about it...

http://support.microsoft.com/default...b;en-us;196652
http://support.microsoft.com/default...b;en-us;194983

A more serious error can be seen here:

? int(68.505 * 1000 + 5)
68509 !!! (should have been 68510)

So the bigger the multiplication factor of a fraction, the larger the error (the above error is a full ONE).

Some of you may argue that's just because certain numbers don't have the exact binary representation. Yes, this is a known fact.
http://support.microsoft.com/default...kb;en-us;42980
http://support.microsoft.com/default...kb;en-us;35826
http://support.microsoft.com/default...kb;en-us;61436

However, those who have worked long enough with BASIC dating
back to QuickBasic will be able to jump in and swear that QB
does not have the math issues that VB has.

That depended on the version....

http://support.microsoft.com/default...kb;en-us;11936
Rick - MVP

[Rick Rothstein]

> > I suggest you write your own rounding routine.

Not necessary... see below.
Whoops! I forgot to cover this in my last posting. The Format function
avoids all of the problems shown...
For example, if you do

? round (68.505,2)

you get 68.5, which is incorrect; for if you round 68.505
to two decimal points you should get 68.51.
? Format(68.505, "0.00") ==> 68.51
A more serious error can be seen here:

? int(68.505 * 1000 + 5)
68509 !!! (should have been 68510)

? Format(68.505 * 1000 + 5, "0") ==> 68510

Rick - MVP

[Raoul Watson]

"Rick Rothstein" <ri************@NOSPAMcomcast.net> wrote in message
news:79********************@comcast.com...

I suggest you write your own rounding routine.

Not necessary... see below.

VB's ailment in its math function is caused because there are errors

in the

underlying code. For example, if you do

? round (68.505,2)

you get 68.5, which is incorrect; for if you round 68.505 to two

decimal

points you should get 68.51.

This is caused because the underlying code that computes:

? int(68.505 * 100 + .5)/100

gives 68.5!

Uh! That is not correct. The Round function does not return what it does
because of the "underlying code" as you are assuming... the Round
function acts exactly as it was designed to... it uses something known
as Banker's Rounding. Now, personally, I think Microsoft's decision to
use this rounding method exclusively was a poor decision on their part,
but make no mistake, this decision was made on purpose. Here are a
couple of links from Microsoft that talk about it...

http://support.microsoft.com/default...b;en-us;196652
http://support.microsoft.com/default...b;en-us;194983

A more serious error can be seen here:

? int(68.505 * 1000 + 5)
68509 !!! (should have been 68510)

So the bigger the multiplication factor of a fraction, the larger the

error

(the above error is a full ONE).

Some of you may argue that's just because certain numbers don't have

the

exact binary representation. Yes, this is a known fact.

http://support.microsoft.com/default...kb;en-us;42980
http://support.microsoft.com/default...kb;en-us;35826
http://support.microsoft.com/default...kb;en-us;61436

However, those who have worked long enough with BASIC dating
back to QuickBasic will be able to jump in and swear that QB
does not have the math issues that VB has.

That depended on the version....

http://support.microsoft.com/default...kb;en-us;11936
Rick - MVP

You are giving a whole bunch of known excuses from Microsoft.

If you have worked with other math packages such as SANE from Apple, you
would agree that the programmer should never have to worry about
inaccuracies due to internal data representation. When he does a
calculation, it will come out EXACTLY the way it suppose to.

[J French]

On Thu, 28 Apr 2005 08:39:18 GMT, "Raoul Watson"
<Wa*****@IntelligenCIA.com> wrote:

<snip>

You are giving a whole bunch of known excuses from Microsoft.

If you have worked with other math packages such as SANE from Apple, you
would agree that the programmer should never have to worry about
inaccuracies due to internal data representation. When he does a
calculation, it will come out EXACTLY the way it suppose to.

I am afraid I don't fully agree with that
- the problem is down to the Institute of Electrical & Electronic
Engineers

MS did not develop the IEEE format, if anything they were slow at
adopting it.

The problem is that IEEE is simply 'faulty'
- some numbers can only be stored as approximations

MS to give them their credit, realized the problem of IEEE and their
own original format and used BCD in their Business Basic (1981-2)
which was targeted at replacing CBasic ( DR's CB86 )

They also brought in the Currency variable.

I do, however think that their use of banker's rounding anywhere is
pure insanity.

[Raoul Watson]

"J French" <er*****@nowhere.uk> wrote in message
news:42****************@news.btclick.com...

I am afraid I don't fully agree with that
- the problem is down to the Institute of Electrical & Electronic
Engineers

The problem is that IEEE is simply 'faulty'
- some numbers can only be stored as approximations

Agree with you a 100% on that buddy. MS implemention is my issue..

Ever since Kahan brought his graduate students, the 754 floating point
standard became a faulty committee . Only one representative from the
computing field was there. When the standard was ratified, exactly two
companies produced floating point arithmetic chips that attached to
microprocessor CPU chips. Intel sold the 8087 and National Semiconductor
sold a chip for the NS32 processors.

The supposedly target audience had ignored it for over 20 years. It has
never been used by Cray, makers of the premier scientific computers. It has
never been used by IBM mainframes, which at the time did a great deal of
scientific computing. It was never used by CDC. It was never used by DEC. No
RISC microprocessor has ever implemented IEEE 754 floating point. Since the
RISC processors were always by far the leaders in floating point
performance, the fact that they for twenty years never implemented the
standard is telling. (They generally use the IEEE format, and there is
usually a software-fixup technique that allows them to claim complete
conformance!) No one uses them in that mode, however. It's way too slow.

In the early 80's, when Apple developped SANE (Standard Apple Numeric
Environment) it supports all the requirements of the IEEE standard and yet
went beyond it to "shield" the users from the deficiencies by developing
libraries for high quality engineering, scientific, financial and accounting
applications.

MS could have done the same thing or stick to the BCD and improve on it.

To have a sleuth of non-working functions and come back to the end user by
explaining that's because of the IEEE standard and how numbers are
manipukated internally is a pisspoor excuse.

[J French]

On Fri, 29 Apr 2005 11:42:33 GMT, "Raoul Watson"
<Wa*****@IntelligenCIA.com> wrote:

"J French" <er*****@nowhere.uk> wrote in message
news:42****************@news.btclick.com...

I am afraid I don't fully agree with that
- the problem is down to the Institute of Electrical & Electronic
Engineers

The problem is that IEEE is simply 'faulty'
- some numbers can only be stored as approximations

Agree with you a 100% on that buddy. MS implemention is my issue..

Ever since Kahan brought his graduate students, the 754 floating point
standard became a faulty committee . Only one representative from the
computing field was there. When the standard was ratified, exactly two
companies produced floating point arithmetic chips that attached to
microprocessor CPU chips. Intel sold the 8087 and National Semiconductor
sold a chip for the NS32 processors.

The supposedly target audience had ignored it for over 20 years. It has
never been used by Cray, makers of the premier scientific computers. It has
never been used by IBM mainframes, which at the time did a great deal of
scientific computing. It was never used by CDC. It was never used by DEC. No
RISC microprocessor has ever implemented IEEE 754 floating point. Since the
RISC processors were always by far the leaders in floating point
performance, the fact that they for twenty years never implemented the
standard is telling. (They generally use the IEEE format, and there is
usually a software-fixup technique that allows them to claim complete
conformance!) No one uses them in that mode, however. It's way too slow.

In the early 80's, when Apple developped SANE (Standard Apple Numeric
Environment) it supports all the requirements of the IEEE standard and yet
went beyond it to "shield" the users from the deficiencies by developing
libraries for high quality engineering, scientific, financial and accounting
applications.

MS could have done the same thing or stick to the BCD and improve on it.

To have a sleuth of non-working functions and come back to the end user by
explaining that's because of the IEEE standard and how numbers are
manipukated internally is a pisspoor excuse.

Thanks for the info
I guess that SANE holds numbers internally in its own format, and only
allows the problems to occur when they hit the disk

I must confess that in the late 1980's we managed to improve on the MS
IEEE conversion routines that were in BASIC 7.1

The VB5 conversion routines are slightly less accurate than those in
B7.1 - which gave me a shock when porting code.

However, I still reckon that the IEEE standard is faulty
- the crazy thing is that BCD appears to be built into the
'co-processor'

Personally I would have added a new data type
IEEE would be a 'sort of optional extra' for Import/Export

Following stupid standards - is ... well stupid <G>

[DoppelHerz]

Hi,

The fucntion you use (Int) does exactly what it is supposed to do. The
VB helpfile sais:

Int, Fix Functions
Returns the integer portion of a number.
Syntax
Int(number)

Int performs the "return integer" stuff on all bits and bobs of the
calculation not just the result, so int reads your calculation as
(5 * 1)

The way around it is to pass values, not calculations to the Int
function. You should have written

lngResult = (5 * 1.2)
Int(lngResult)

Regards DP

Allcomp wrote:

Hello,

I have seen something really strange in VB6

If I do a Int ( (5 * 1.2)) , I receive the value 5, but I should receive 6?
Is this a bug or something really "normal".
I can see that if I do
? int ((5 * 1.2 + 0.0000000000000003))
I receive 6. If I add something smaller, I have 5 as a result
What is strange is that If I do a ? 5*1.2, I receive 6 so (5 * 1.2)
should do 6
If I do Int ( (5 * 12 / 10)), the result is OK (6)
What can I do to have it correct?
Is there an option in VB6?

Thank you
Marc Allard
Allcomp

[Rick Rothstein]

> Int performs the "return integer" stuff on all bits and

bobs of the calculation not just the result, so int reads
your calculation as (5 * 1)
NO! That is not how the Int function works! If it did work that way,
then

MsgBox Int(1.2 * 3.4)

would display 3; but that is NOT what is displayed... the value 4 is
displayed because the values are NOT made into integers before the
multiplication. The problem is due to the reasons discussed in the
several articles I cited links for in my previous postings to this
thread. You should go back and read them in order to correct your
mistaken idea of how the Int function works.

The way around it is to pass values, not calculations
to the Int function. You should have written

lngResult = (5 * 1.2)
Int(lngResult)

Since you named your variable lngResult, one has to assume you Dim'med
it as Long. If that is correct, then the Int function call is not needed
because the act of assigning a floating point number to a variable that
has been declared as type Long automatically forces the number to an
integer (non-floating point) number. So the Int function actually does
nothing in the above calculation. However, one needs to be aware of a
potential problem when assigning floating point values to a type Long
variable... VB uses Banker's Rounding on the floating point number to
make it into an integer value prior to the assignment. Unexpected
results can follow if your calculation produces a floating point value
that end in exactly ".5" because Banker's Rounding rounds these numbers
to the nearest EVEN preceding digit. Consider this code snippet...

Dim lngResult As Long
lngResult = 1 * 1.5
Debug.Print lngResult
lngResult = 2 * 1.5
Debug.Print lngResult
lngResult = 3 * 1.5
Debug.Print lngResult
lngResult = 4 * 1.5
Debug.Print lngResult
lngResult = 5 * 1.5
Debug.Print lngResult
lngResult = 6 * 1.5
Debug.Print lngResult
lngResult = 7 * 1.5
Debug.Print lngResult
lngResult = 8 * 1.5
Debug.Print lngResult
lngResult = 9 * 1.5
Debug.Print lngResult

It prints out the following values...

2
3
4
6
8
9
10
12
14

Notice that the numbers 5, 7, 11, 13 do not appear.

Okay, that takes care of the case where you Dim'med lngResult as Long.
If, on the other hand, you didn't declare it as anything (a very bad
practice in my opinion), then it is a Variant. In that case, the
calculation forces the variable to a data sub-type of Double (during the
assignment) Doubles have "quirks" of their own. Double are 16 digit
values, but only expose 15 digits of them. The 16th digit provides a
guard digit to help maintain accuracy during chained calculations. In
essence, VB uses this 16 decimal place to absorb the errors inherent in
floating point values not being able to be stored exactly using the IEEE
standard. Now I'm not 100% sure if the 16th place is lost during the
assignment, or if it is retained but ignored every time a value is
retrieved from a variable of type or sub-type Double, but the net effect
is the calculation 5*1.2 becomes the value of 6, exactly, when stored in
a Double type variable. You can see this like so...

Dim lngResult As Double
lngResult = 5 * 1.2
Debug.Print lngResult - 6
Debug.Print 5 * 1.2 - 6

The first debug print out displays 0 indicated that the value stored in
the Double type variable is (or is being treated as if it is) exactly 6
(to 15 digits of accuracy). The second debug statement, on the other
hand, displays

-2.22044604925031E-16

indicating that during chained calculations, the 16th decimal place is
still being used. Floating point numbers can be a real pain in the ass
to work with.
Rick - MVP

Rick - MVP

[Steve Gerrard]

"Rick Rothstein" <ri************@NOSPAMcomcast.net> wrote in message
news:Ma********************@comcast.com...

Now I'm not 100% sure if the 16th place is lost during the
assignment, or if it is retained but ignored every time a value is
retrieved from a variable of type or sub-type Double, but the net effect
is the calculation 5*1.2 becomes the value of 6, exactly, when stored in
a Double type variable. You can see this like so...

I think it is the case that the 16th place is lost in the assignment, but only
if the result can't retain that precision. If you do

dim dblResult As Double ' I refuse to use lng here :)

dblResult = 5 * 1.2 - 6
Debug.Print dblResult

dblResult will still have that check bit quantity, i.e.
e -2.22044604925031E-16.

If you do this loop, you can then see that the variable loses the ability to
retain the "dribble" when it gets up to 3, presumably because the exponent has
to become larger.

Private Sub Command1_Click()
Dim dblResult As Double
Dim i As Long

dblResult = 5 * 1.2 - 6
Debug.Print dblResult

For i = 1 To 4
dblResult = dblResult + i
dblResult = dblResult - i
Debug.Print i, dblResult
Next i

End Sub

'result:
-2.22044604925031E-16
1 -2.22044604925031E-16
2 -2.22044604925031E-16
3 0
4 0
Floating point numbers can be a real pain in the ass
to work with.

Oh yes indeed :)

[Allcomp]

Thank you for your answer and sorry for the late answer (I was not in
the office).

In fact, I use double because I take some values from a dxf (double) and
by luck (bad luck), the result of calculation was 1.2
I have already opened a lot of DXF (mode than 100 differents) and it is
the first time I have exactly 1.2 to multiply by 5 (the two values
depent on the DXF size, coordonates...)

Thank you
Marc Allard
Allcomp
Steve Gerrard wrote:

"Rick Rothstein" <ri************@NOSPAMcomcast.net> wrote in message
news:Ma********************@comcast.com...

Now I'm not 100% sure if the 16th place is lost during the
assignment, or if it is retained but ignored every time a value is
retrieved from a variable of type or sub-type Double, but the net effect
is the calculation 5*1.2 becomes the value of 6, exactly, when stored in
a Double type variable. You can see this like so...

I think it is the case that the 16th place is lost in the assignment, but only
if the result can't retain that precision. If you do

dim dblResult As Double ' I refuse to use lng here :)

dblResult = 5 * 1.2 - 6
Debug.Print dblResult

dblResult will still have that check bit quantity, i.e.
e -2.22044604925031E-16.

If you do this loop, you can then see that the variable loses the ability to
retain the "dribble" when it gets up to 3, presumably because the exponent has
to become larger.

Private Sub Command1_Click()
Dim dblResult As Double
Dim i As Long

dblResult = 5 * 1.2 - 6
Debug.Print dblResult

For i = 1 To 4
dblResult = dblResult + i
dblResult = dblResult - i
Debug.Print i, dblResult
Next i

End Sub

'result:
-2.22044604925031E-16
1 -2.22044604925031E-16
2 -2.22044604925031E-16
3 0
4 0

Floating point numbers can be a real pain in the ass
to work with.

Oh yes indeed :)