Help | Site Map
Connecting Tech Pros Worldwide
 
 
LinkBack Thread Tools
  #1  
Old July 19th, 2005, 02:20 PM
Ted
Guest
 
Posts: n/a
Default Help with Fix function

I've written a little function to remove everything after the 2nd decimal
place for prices which is as follows: -

ReturnConvertedCurrency = (fix(iSterling * session("ExchangeRate") *
100) / 100)

However, it sometimes returns incorrect values. i.e. Why does the
following: -

response.write(FormatNumber((fix(2.30 * 1 * 100) / 100) , 2))

or

response.write(FormatNumber((int(2.30 * 1 * 100) / 100) , 2))

return 2.29 ?

Any help would be greatly appreciated as I've been ripping my hair out for a
few hours trying to solve this one.

Ted.


  #2  
Old July 19th, 2005, 02:20 PM
Aaron [SQL Server MVP]
Guest
 
Posts: n/a
Default Re: Help with Fix function

I didn't know this existed in Fix as well.
http://www.aspfaq.com/2477

Don't know why you need fix or int here though. These work fine for me:

response.write(FormatNumber((2.30 * 100)/100.0, 2))
response.write(FormatPercent((2.30)/100.0, 2))

--
http://www.aspfaq.com/
(Reverse address to reply.)




"Ted" <spam@spam.com> wrote in message
news:OgNpgkAcEHA.2972@TK2MSFTNGP12.phx.gbl...[color=blue]
> I've written a little function to remove everything after the 2nd decimal
> place for prices which is as follows: -
>
> ReturnConvertedCurrency = (fix(iSterling * session("ExchangeRate")[/color]
*[color=blue]
> 100) / 100)
>
> However, it sometimes returns incorrect values. i.e. Why does the
> following: -
>
> response.write(FormatNumber((fix(2.30 * 1 * 100) / 100) , 2))
>
> or
>
> response.write(FormatNumber((int(2.30 * 1 * 100) / 100) , 2))
>
> return 2.29 ?
>
> Any help would be greatly appreciated as I've been ripping my hair out for[/color]
a[color=blue]
> few hours trying to solve this one.
>
> Ted.
>
>[/color]


  #3  
Old July 19th, 2005, 02:21 PM
Ted
Guest
 
Posts: n/a
Default Re: Help with Fix function

> Don't know why you need fix or int here though. These work fine for me:[color=blue]
>
> response.write(FormatNumber((2.30 * 100)/100.0, 2))
> response.write(FormatPercent((2.30)/100.0, 2))[/color]

The reason I am using fix() is so: -

response.write(FormatNumber((fix(2.309 * 1 * 100) / 100) , 2))

will result in 2.30

Maybe I'm goint to have to re-think my appraoch to this problem.

As it's an eCommerce site I'm putting together, I need the maths to add up
precisely on the basket screen etc. I have got some functions that will
inevitably return pesky floats (adding 17.5% tax, currency conversions etc).

What I was doing was chopping off everthing after the second decimal place
in all of my calculations using the function I mentioned. This, until now,
had worked OK and all the totals had added up. Until I got the dreaded 2.30
resulting in 2.29.

Any ideas how I should tackle this problem?

Thanks for the reply,

Ted.
[color=blue][color=green]
> > I've written a little function to remove everything after the 2nd[/color][/color]
decimal[color=blue][color=green]
> > place for prices which is as follows: -
> >
> > ReturnConvertedCurrency = (fix(iSterling *[/color][/color]
session("ExchangeRate")[color=blue]
> *[color=green]
> > 100) / 100)
> >
> > However, it sometimes returns incorrect values. i.e. Why does the
> > following: -
> >
> > response.write(FormatNumber((fix(2.30 * 1 * 100) / 100) , 2))
> >
> > or
> >
> > response.write(FormatNumber((int(2.30 * 1 * 100) / 100) , 2))
> >
> > return 2.29 ?
> >
> > Any help would be greatly appreciated as I've been ripping my hair out[/color][/color]
for[color=blue]
> a[color=green]
> > few hours trying to solve this one.
> >
> > Ted.
> >
> >[/color]
>
>[/color]


  #4  
Old July 19th, 2005, 02:21 PM
Aaron [SQL Server MVP]
Guest
 
Posts: n/a
Default Re: Help with Fix function

Is it really correct to just truncate digits instead of rounding? I would
think that, especially for an e-commerce site, you would be more interested
in being correct than using fix().

--
http://www.aspfaq.com/
(Reverse address to reply.)




"Ted" <spam@spam.com> wrote in message
news:uU9jcpBcEHA.3904@TK2MSFTNGP12.phx.gbl...[color=blue][color=green]
> > Don't know why you need fix or int here though. These work fine for me:
> >
> > response.write(FormatNumber((2.30 * 100)/100.0, 2))
> > response.write(FormatPercent((2.30)/100.0, 2))[/color]
>
> The reason I am using fix() is so: -
>
> response.write(FormatNumber((fix(2.309 * 1 * 100) / 100) , 2))
>
> will result in 2.30
>
> Maybe I'm goint to have to re-think my appraoch to this problem.
>
> As it's an eCommerce site I'm putting together, I need the maths to add up
> precisely on the basket screen etc. I have got some functions that will
> inevitably return pesky floats (adding 17.5% tax, currency conversions[/color]
etc).[color=blue]
>
> What I was doing was chopping off everthing after the second decimal place
> in all of my calculations using the function I mentioned. This, until now,
> had worked OK and all the totals had added up. Until I got the dreaded[/color]
2.30[color=blue]
> resulting in 2.29.
>
> Any ideas how I should tackle this problem?
>
> Thanks for the reply,
>
> Ted.
>[color=green][color=darkred]
> > > I've written a little function to remove everything after the 2nd[/color][/color]
> decimal[color=green][color=darkred]
> > > place for prices which is as follows: -
> > >
> > > ReturnConvertedCurrency = (fix(iSterling *[/color][/color]
> session("ExchangeRate")[color=green]
> > *[color=darkred]
> > > 100) / 100)
> > >
> > > However, it sometimes returns incorrect values. i.e. Why does the
> > > following: -
> > >
> > > response.write(FormatNumber((fix(2.30 * 1 * 100) / 100) , 2))
> > >
> > > or
> > >
> > > response.write(FormatNumber((int(2.30 * 1 * 100) / 100) , 2))
> > >
> > > return 2.29 ?
> > >
> > > Any help would be greatly appreciated as I've been ripping my hair out[/color][/color]
> for[color=green]
> > a[color=darkred]
> > > few hours trying to solve this one.
> > >
> > > Ted.
> > >
> > >[/color]
> >
> >[/color]
>
>[/color]


  #5  
Old July 19th, 2005, 02:21 PM
Ted
Guest
 
Posts: n/a
Default Re: Help with Fix function

If I use currency formats then there may be descrepancies when totaling the
basket (amongst other things)

i.e. after adding VAT or coverting a currency item values might end up
like: -

£3.0090
£5.1070

which appear to the user as: -

£3.01
£5.11

but totals as: -

£8.13

Problems also occur when bankers rounding is introduced. I just want
everything to be predictable. Any suggestions for the best way to achieve
this?

Regards,

Ted.

[color=blue]
> Is it really correct to just truncate digits instead of rounding? I would
> think that, especially for an e-commerce site, you would be more[/color]
interested[color=blue]
> in being correct than using fix().
>
> --
> http://www.aspfaq.com/
> (Reverse address to reply.)
>
>
>
>
> "Ted" <spam@spam.com> wrote in message
> news:uU9jcpBcEHA.3904@TK2MSFTNGP12.phx.gbl...[color=green][color=darkred]
> > > Don't know why you need fix or int here though. These work fine for[/color][/color][/color]
me:[color=blue][color=green][color=darkred]
> > >
> > > response.write(FormatNumber((2.30 * 100)/100.0, 2))
> > > response.write(FormatPercent((2.30)/100.0, 2))[/color]
> >
> > The reason I am using fix() is so: -
> >
> > response.write(FormatNumber((fix(2.309 * 1 * 100) / 100) , 2))
> >
> > will result in 2.30
> >
> > Maybe I'm goint to have to re-think my appraoch to this problem.
> >
> > As it's an eCommerce site I'm putting together, I need the maths to add[/color][/color]
up[color=blue][color=green]
> > precisely on the basket screen etc. I have got some functions that will
> > inevitably return pesky floats (adding 17.5% tax, currency conversions[/color]
> etc).[color=green]
> >
> > What I was doing was chopping off everthing after the second decimal[/color][/color]
place[color=blue][color=green]
> > in all of my calculations using the function I mentioned. This, until[/color][/color]
now,[color=blue][color=green]
> > had worked OK and all the totals had added up. Until I got the dreaded[/color]
> 2.30[color=green]
> > resulting in 2.29.
> >
> > Any ideas how I should tackle this problem?
> >
> > Thanks for the reply,
> >
> > Ted.
> >[color=darkred]
> > > > I've written a little function to remove everything after the 2nd[/color]
> > decimal[color=darkred]
> > > > place for prices which is as follows: -
> > > >
> > > > ReturnConvertedCurrency = (fix(iSterling *[/color]
> > session("ExchangeRate")[color=darkred]
> > > *
> > > > 100) / 100)
> > > >
> > > > However, it sometimes returns incorrect values. i.e. Why does the
> > > > following: -
> > > >
> > > > response.write(FormatNumber((fix(2.30 * 1 * 100) / 100) ,[/color][/color][/color]
2))[color=blue][color=green][color=darkred]
> > > >
> > > > or
> > > >
> > > > response.write(FormatNumber((int(2.30 * 1 * 100) / 100) ,[/color][/color][/color]
2))[color=blue][color=green][color=darkred]
> > > >
> > > > return 2.29 ?
> > > >
> > > > Any help would be greatly appreciated as I've been ripping my hair[/color][/color][/color]
out[color=blue][color=green]
> > for[color=darkred]
> > > a
> > > > few hours trying to solve this one.
> > > >
> > > > Ted.
> > > >
> > > >
> > >
> > >[/color]
> >
> >[/color]
>
>[/color]


  #6  
Old July 19th, 2005, 02:21 PM
Aaron [SQL Server MVP]
Guest
 
Posts: n/a
Default Re: Help with Fix function

> i.e. after adding VAT or coverting a currency item values might end up[color=blue]
> like: -
>
> £3.0090
> £5.1070
>
> which appear to the user as: -
>
> £3.01
> £5.11
>
> but totals as: -
>
> £8.13
>
> Problems also occur when bankers rounding is introduced. I just want
> everything to be predictable. Any suggestions for the best way to achieve
> this?[/color]

Yes. Round each product to the nearest penny, instead of adding products
together at fractions of pennies (introducing the possibility of rounding
down by mistake). In the above example, you should be adding 3.01 and 5.11
and getting 8.12. Why would you carry forward 3.0090? If they only bought
one product, would you charge them 3.0090? Of course not.

A better "fix" function isn't going to fix your problem, if you'll pardon
the pun.


  #7  
Old July 19th, 2005, 02:21 PM
[MSFT]
Guest
 
Posts: n/a
Default Re: Help with Fix function

Hi Ted,

I don't understand why you said total is 0.13.

0.0090+0.1070=0.1160
0.01+0.11=0.12

It cannot be 0.13.

Luke

  #8  
Old July 19th, 2005, 02:22 PM
Nathan Sokalski
Guest
 
Posts: n/a
Default Re: Help with Fix function

I definitely understand that certain operations used on an ecommerce site
will give pesky floats, as you mentioned:

As it's an eCommerce site I'm putting together, I need the maths to add up
precisely on the basket screen etc. I have got some functions that will
inevitably return pesky floats (adding 17.5% tax, currency conversions etc).

But rather than change your formatting and rounding techniques, I would
suggest adding the total in whatever currency the prices were originally
given in, and then multiplying that total by the exchange rate and tax
percentages. That way, you will not need to worry about adding fractions of
a cent (or whatever currency is being used); you can simply round or
concatenate the decimal places one time for the total rather doing it for
every item. Example (my conversion & tax factors may be wrong here):

Cost Conv Tax
£15.99 * 1.76 * 17.5% = 33.06732 = 33.06
£10.50 * 1.76 * 17.5% = 21.714 = 21.71
£ 8.27 * 1.76 * 17.5% = 17.10236 = 17.10
71.88368 = 71.88 (This is of by
..01)

OR

Cost Conv Tax
£15.99
£10.50
£ 8.27
£34.76 * 1.76 * 17.5% = 71.88368 = 71.88 (This does not show any unexpected
decimals)

If only some items are taxed, you may need to add up the taxable and
non-taxable items separately, but otherwise the same. You may want to
display the operations differently on the site, and I am not experienced in
economics or finance, but I usually see tax and conversions done at the end,
which I believe makes it simpler for both the user and in the code. Just a
suggestion.
--
Nathan Sokalski
njsokalski@hotmail.com
www.nathansokalski.com

"Ted" <spam@spam.com> wrote in message
news:uU9jcpBcEHA.3904@TK2MSFTNGP12.phx.gbl...[color=blue][color=green]
> > Don't know why you need fix or int here though. These work fine for me:
> >
> > response.write(FormatNumber((2.30 * 100)/100.0, 2))
> > response.write(FormatPercent((2.30)/100.0, 2))[/color]
>
> The reason I am using fix() is so: -
>
> response.write(FormatNumber((fix(2.309 * 1 * 100) / 100) , 2))
>
> will result in 2.30
>
> Maybe I'm goint to have to re-think my appraoch to this problem.
>
> As it's an eCommerce site I'm putting together, I need the maths to add up
> precisely on the basket screen etc. I have got some functions that will
> inevitably return pesky floats (adding 17.5% tax, currency conversions[/color]
etc).[color=blue]
>
> What I was doing was chopping off everthing after the second decimal place
> in all of my calculations using the function I mentioned. This, until now,
> had worked OK and all the totals had added up. Until I got the dreaded[/color]
2.30[color=blue]
> resulting in 2.29.
>
> Any ideas how I should tackle this problem?
>
> Thanks for the reply,
>
> Ted.
>[color=green][color=darkred]
> > > I've written a little function to remove everything after the 2nd[/color][/color]
> decimal[color=green][color=darkred]
> > > place for prices which is as follows: -
> > >
> > > ReturnConvertedCurrency = (fix(iSterling *[/color][/color]
> session("ExchangeRate")[color=green]
> > *[color=darkred]
> > > 100) / 100)
> > >
> > > However, it sometimes returns incorrect values. i.e. Why does the
> > > following: -
> > >
> > > response.write(FormatNumber((fix(2.30 * 1 * 100) / 100) , 2))
> > >
> > > or
> > >
> > > response.write(FormatNumber((int(2.30 * 1 * 100) / 100) , 2))
> > >
> > > return 2.29 ?
> > >
> > > Any help would be greatly appreciated as I've been ripping my hair out[/color][/color]
> for[color=green]
> > a[color=darkred]
> > > few hours trying to solve this one.
> > >
> > > Ted.
> > >
> > >[/color]
> >
> >[/color]
>
>[/color]


 

Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are Off
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

What is Bytes?

We are a network of experts and professionals in IT and software development that help one another with answers to tough questions and share insights. Get the best answers to your questions from over network members.
Post your question now . . .
It's fast and it's free

Popular Articles