473,385 Members | 1,606 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Problems with Very Large Numbers

My app is trying to multiply and add some very large numbers. Here is the line that I am am having problems with:

Seconds = CDbl(PartB) * CDbl(16777216)

Seconds is a double and PartB is a short. Partb is 15 before this line. After the multiplication, the value held in seconds is 549420. This obviously isn't correct. I know that the problem is the conversion between types and how much space vb.net uses for its evaluation. However, I'm not sure how to fix it. Any ideas? Thanks for the help.
Nov 20 '05 #1
1 1603
Hi,

I can't reproduce your error.
Dim PartB As Short = 15
Dim seconds As Double = CDbl(PartB) * CDbl(16777216)
Trace.WriteLine(seconds.ToString)

Trace.WriteLine(15 * 16777216)
Both methods return a value of 251658240.

Ken
--------------

"Jonathan" <Jo******@discussions.microsoft.com> wrote in message
news:36**********************************@microsof t.com:
My app is trying to multiply and add some very large numbers. Here is the
line that I am am having problems with:

Seconds = CDbl(PartB) * CDbl(16777216)

Seconds is a double and PartB is a short. Partb is 15 before this line.
After the multiplication, the value held in seconds is 549420. This
obviously isn't correct. I know that the problem is the conversion between
types and how much space vb.net uses for its evaluation. However, I'm not
sure how to fix it. Any ideas? Thanks for the help.


--
Outgoing mail is certified Virus Free.
Checked by AVG Anti-Virus (http://www.grisoft.com).
Version: 7.0.230 / Virus Database: 263.3.6 - Release Date: 6/25/2004
Nov 20 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Alex Vinokur | last post by:
Dann Corbit has implemented function int ilog2 (unsigned long) at http://groups.google.com/groups?selm=lkPa4.2165%24I41.1498%40client Is exist a similar C++-function for very large numbers,...
5
by: Todd Steury | last post by:
Greetings Python'ers: I'm just an amature who occasionally uses Python for complex mathematical models. The current model I'm working with occasionally generates really large numbers that are...
5
by: Scott | last post by:
I have a customer that had developed an Access97 application to track their business information. The application grew significantly and they used the Upsizing Wizard to move the tables to SQL...
18
by: Zero | last post by:
Hi, I am calculating an integer to the pwer of a large integer, e.g. 2^5000. It turns out that the result I always get is zero. I am sure that the result is too large to store in such type...
6
by: Ruy Castelli | last post by:
Hello, I'm learning to code in C# and I created a DataGrid component, which I couldn't get it to work properly either using "Next >>" and "<< Previous" buttons or using actual page numbers. ...
9
by: MDoyle | last post by:
Here is the easiest question you’ll get all day from a complete newbie. I installed Visual Studio on my laptop last night and have been tryin to write my very first C# programs this morning. I’m...
0
by: zephyrus360 | last post by:
This is about a technique to find the mod of a very large integer with a normal small integer. I recently encountered this problem when I needed to compute the modulus of a very large number with...
19
by: Ant | last post by:
Hi all, I have a question on PyParsing. I am trying to create a parser for a hierarchical todo list format, but have hit a stumbling block. I have parsers for the header of the list (title and...
25
by: stararic | last post by:
hi, I'm writing a program a kind of calculator that should work with very big numbers (thousand of digits). So, numbers are rappresented by a struct that contains a pointer to integer, e 2 int for...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.