By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
459,253 Members | 1,693 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 459,253 IT Pros & Developers. It's quick & easy.

insufficient incrementation

Dormilich
Expert Mod 5K+
P: 8,639
Hi,

Iíve got a mathematical script, which is currently running fine*, although it bothers me that JS does not always increment correctly. sometimes it happens that the incremented value is slightly bigger/smaller (around the 12th decimal place) than it should be (Iím usually incrementing by 0.1). because of that it affects the accuracy of the results.

incrementation:
- Fn.search() line 239 & 248
value assignment:
- Fn() line 83-85

full code

can anyone shed some light on this matter?

* - even if you canít read German, after some trials you should know what each button/field is for.
Aug 21 '09 #1

✓ answered by gits

that is due to javascript's number representation - taken from MDC here:

Numbers in JavaScript are "double-precision 64-bit format IEEE 754 values", according to the spec. This has some interesting consequences. There's no such thing as an integer in JavaScript, so you have to be a little careful with your arithmetic if you're used to maths in C or Java. Watch out for stuff like:

Expand|Select|Wrap|Line Numbers
  1. 0.1 + 0.2 = 0.30000000000000004
so basicly there are some things to do when you do math with decimal values in JavaScript ... you could round the values or implement some function that fix that issue, in case you need to.

kind regards

Share this Question
Share on Google+
6 Replies


gits
Expert Mod 5K+
P: 5,390
that is due to javascript's number representation - taken from MDC here:

Numbers in JavaScript are "double-precision 64-bit format IEEE 754 values", according to the spec. This has some interesting consequences. There's no such thing as an integer in JavaScript, so you have to be a little careful with your arithmetic if you're used to maths in C or Java. Watch out for stuff like:

Expand|Select|Wrap|Line Numbers
  1. 0.1 + 0.2 = 0.30000000000000004
so basicly there are some things to do when you do math with decimal values in JavaScript ... you could round the values or implement some function that fix that issue, in case you need to.

kind regards
Aug 21 '09 #2

Dormilich
Expert Mod 5K+
P: 8,639
besides rounding, do you have any idea how to do this?
Aug 21 '09 #3

gits
Expert Mod 5K+
P: 5,390
not really ... i don't use JavaScript math with that precision ... mostly in business cases where simple rounding rules will do the job. but i guess there are some theoretical concepts that could handle it ... may be you could start further research from here?

kind regards
Aug 21 '09 #4

Dormilich
Expert Mod 5K+
P: 8,639
actually, I hardly need more than a 3 digits precision.* (I use it for solving dissociation equations when comparing experiment data with theoretical values)


* - besides that, itís a fun/self-training project

[EDIT]
do you see anything, where I should do improvements?
Aug 21 '09 #5

gits
Expert Mod 5K+
P: 5,390
:) are you asking for a code review? ... besides of the mixing of different ways to build/declare objects/methods i don't see anything that i would change. personally i always prefer the 'classical' way to build JavaScript objects and would always use the prototype for their methods, but as i said ... that is my personal view ... i think that methods shouldn't alloc memory everytime an instance of the obj is created while member-variables that store 'plain-data' need to do so ...

kind regards
Aug 21 '09 #6

Dormilich
Expert Mod 5K+
P: 8,639
@gits
thatís already quite helpful.
Aug 21 '09 #7

Post your reply

Sign in to post your reply or Sign up for a free account.