473,326 Members | 2,805 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

calculation problem

Hello All,
I have a calculation problem, the system changes it's accuracy during the
program,
attached is a sample with the bug, you can compile it and run it in debug
mode, then look at the changed result of ans1 & ans2.
So the question now is how do I change the accuracy?
Thanks Maor.


using System;

using System.Drawing;

using System.Windows.Forms;

using Microsoft.DirectX;

using Microsoft.DirectX.Direct3D;

using Direct3D=Microsoft.DirectX.Direct3D;

namespace test

{

public class test

{

/// <summary>

/// The main entry point for the application.

/// </summary>

static void Main()

{

PresentParameters presentParams = new PresentParameters();

presentParams.Windowed=true; // We don't want to run fullscreen

presentParams.SwapEffect = SwapEffect.Discard; // Discard the frames

presentParams.EnableAutoDepthStencil = true; // Turn on a Depth stencil

presentParams.AutoDepthStencilFormat = DepthFormat.D16; // And the stencil
format

double a = 10.0;

double b = 220.0;

double ans1 = a/b;

Device device = new Device(0, DeviceType.Hardware, new Panel(),
CreateFlags.SoftwareVertexProcessing, presentParams); //Create a device
double ans2 = a/b;

decimal a4 = 10;

decimal b4 = 220;

decimal ans4 = a4/b4;

double ans5 = (double)ans1;

float ans6 = (float)ans1;

//ans6++;

//ans6--;

double ans7 = (double)ans6;

double ans9 = (double)ans4;

double ans8 = decimal.ToDouble(ans4);

decimal.

decimal ans3 = decimal.Divide((decimal)a,(decimal)b);

int r =2; //for break point

}

}

}
Nov 15 '05 #1
1 1170
You may want to describe what the actual problem is (so that people dont
have to install SDK's and build the software to understand what you are
getting at.)

Andy Mortimer [MS]
Please do not send email directly to this alias. This alias is for
newsgroup purposes only

This posting is provided "AS IS" with no warranties, and confers no rights.
OR if you wish to include a script sample in your post please add "Use of
included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm"

Nov 15 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

8
by: Aspersion | last post by:
I'm building an ASP page that has a lot of text and graphics. There is a calculation facility on the page. The user enters several numbers in a form and presses a button to see the calculated...
10
by: Michael G | last post by:
double A = floor((2447297.0 - 122.1)/365.25); i get differing results. It varies between 6699 and 6700. The correct answer is 6699. The 6700 result really throws the entire group of calculations...
0
by: anaxamandr | last post by:
Hi. I have a long loop in ASP that performs a rather lengthy calculation. I would love for my users to be able to stop that calculation, if they so choose, mid way through the process. I attempted...
0
by: gavo | last post by:
Hi. using A2K; i have a form containing a continous subform. The question is, how can i call a calculation for one of the fields in the continous subform from the main form. At the moment i...
4
by: Michiel Alsters | last post by:
Hello everybody, I hope anybody can help me. I'll try to give a brief overview of my problem. I have running a program that performs a heavy calculation. To give the user feedback what the...
7
by: th-ko | last post by:
Hi, I would like to know whether a certain coding pattern exist to handle the following problem. I am using a const methode to get and value a from the class. In case the value was not calculated...
10
by: 60325 | last post by:
This is the page where I collect the data in drop-down boxes with values of 1-10 and send it to a submitted page to do calculations. Example: Employee1 TeamScore(1-10) Employee2 ...
5
by: The alMIGHTY N | last post by:
Hi all, Let's say I have a simple math formula: sum (x * y / 1000) / (sum z / 1000) I have to do this across 50 items, each with an x, y and z value, when the page first loads AND when a...
3
by: mattmao | last post by:
Okay, I was asked by a friend about the result of this limit: http://bbs.newwise.com/attdata/forumid_14/20070922_fe7f77c81050413a20fbDWYOGm7zeRj3.jpg Not n->zero but n-> + infinite I really...
1
by: csolomon | last post by:
I have an unbound text box which I use to get a calculation, called DM_SampleWt. This text box's control source is: =GetSize(,Nz(,0),.!,.!.(2)) I have written a function that will get the required...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.