473,748 Members | 7,142 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Integer Division Broken?

Ben
Hi, I have an interesting example from my debugger. I have 2 variables:
sourcewidthnet and targetwidthnet. Notice the results in the debugger.
I'm going to be forced to use the int function of the decimal.toint32
since the \ operator doesn't appear to work, or I greatly misunderstand
the documentation on the \ operator.

decimal.Remaind er(sourcewidthn et, targetwidthnet) 0.375D
sourcewidthnet 96D
targetwidthnet 2.125D
SourceWidthNet / TargetWidthNet
45.176470588235 294117647058824 D(SourceWidthNe t \ TargetWidthNet)
48 Long
decimal.ToInt32 ((SourceWidthNe t / TargetWidthNet) ) 45 Integer

I can't justify the: 96 \ 2.125 = 48 answer at all!

Thanks for your input.

Nov 21 '05 #1
2 1645
It is correct. 96 \ 2 = 48. When you do an integer division, the divisor
is the integer part, which in this case is 2.

"Ben" <be***********@ bankscorporatio n.com> wrote in message
news:11******** **************@ z14g2000cwz.goo glegroups.com.. .
Hi, I have an interesting example from my debugger. I have 2 variables:
sourcewidthnet and targetwidthnet. Notice the results in the debugger.
I'm going to be forced to use the int function of the decimal.toint32
since the \ operator doesn't appear to work, or I greatly misunderstand
the documentation on the \ operator.

decimal.Remaind er(sourcewidthn et, targetwidthnet) 0.375D
sourcewidthnet 96D
targetwidthnet 2.125D
SourceWidthNet / TargetWidthNet
45.176470588235 294117647058824 D(SourceWidthNe t \ TargetWidthNet)
48 Long
decimal.ToInt32 ((SourceWidthNe t / TargetWidthNet) ) 45 Integer

I can't justify the: 96 \ 2.125 = 48 answer at all!

Thanks for your input.

Nov 21 '05 #2
Ben
Ok, I finally see the documentation in the remarks section:

Remarks
Before division is performed, any floating-point numeric expressions
are coerced to Long if Option Strict is Off. If Option Strict is On, a
compiler error results

Silly me, thanks.

Still to get what I'm looking for the int() and decimal.tointxx
routines will work, thanks again.

Nov 21 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

12
2130
by: Tim Rowe | last post by:
If I do from __future__ import division then eval(1/2) gives me 0.5 as expected. But if I do print input("enter a sum: ") and enter 1/2 as the sum I get 0 as if I hadn't done the import. I thought input was supposed to give the same behaviour as an eval on raw input -- why the difference here?
24
19384
by: Teis Draiby | last post by:
In .NET, can I be sure that the result of a division between two integers always is truncated rather that rounded to nearest? Example: 99 / 50 = 1 regards, Teis
3
5564
by: Mark McConnell | last post by:
Regarding deserializing XML into a custom object... I've been able to deserialize an XML doc into my custom object and everything works great. The problem I am encountering is when one of the XML elements contains a NULL and it maps to a variable of type Integer in my custom object, the deserializer doesn't like this. For example, my custom object is declared like: Public Class Member
20
9166
by: GS | last post by:
The stdint.h header definition mentions five integer categories, 1) exact width, eg., int32_t 2) at least as wide as, eg., int_least32_t 3) as fast as possible but at least as wide as, eg., int_fast32_t 4) integer capable of holding a pointer, intptr_t 5) widest integer in the implementation, intmax_t Is there a valid motivation for having both int_least and int_fast?
3
6123
by: Janice | last post by:
I got this question from my textbook and I cannot understand the theory. When a signed positive integer X divided by pow(2,k), the result is shifting k bits to right and putting w-k bits of 0 from the most significant bits. However, when the X is a negative number divided by pow(2,k), the shifting and sign bit extension method doesnt give the correct answer? What kind of bias should we make on the X before the division? Thanx
10
3191
by: Mike S | last post by:
Does anyone know the logic behind why in VB.NET the result of a floating-point division ('/') is -rounded- on being converted to an integer type, such as with statements like Dim x As Integer = 2/3 'after assignment, x is 1, whereas a sane person would say it should be 0 Does Microsoft have a reason for this design decision? I understand that this type of rounding can reduce the overall error in long computation chains by reducing the...
8
6562
by: Candace | last post by:
I am using the following code to pick off each digit of a number, from right to left. The number I am working with is 84357. So for the first iteration it should return the number 7 and for the second iteration it should return the number 5, and so on. But for some reason on the first iteration returns the expected results. Each subsequent iteration returns the number plus 1. In order words, when I run the program I am getting: 7, 6, 4, and...
5
1605
by: bdsatish | last post by:
How does (a/b) work when both 'a' and 'b' are pure integers ? 4 -5 Why is it -5 ? I expect it to be -4 ? Because, in C/C++, 9/2 is 4 and so negative of it, (-9/2) is -4. What should I do to get C-like behavior ?
14
5167
by: Default User | last post by:
Hi, If I have three 64 bit integers and I want to do this operation on them: x*y/z Lets say that what we are multiplying by (y) is offset by what we are dividing by (z) so that the final answer will fit in a 64-bit integer. Let me simplify it by using unsigned chars (8 bits):
0
8991
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9541
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9370
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9321
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8242
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6796
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
1
3312
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2782
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2215
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.