469,085 Members | 1,017 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,085 developers. It's quick & easy.

PRINT 1/(1+26)=0?

Why does this T-SQL round incorrectly?

PRINT 1/(1+26)

when it should be 0.037037037?

Thanks,
Michael

Nov 20 '06 #1
5 1402
I should have been more specific:
DECLARE @A int
DECLARE @B int
DECLARE @C float

SET @A = 1
SET @B = 26
SET @C = @A / (@A + @B)
PRINT @C

mpa...@htxml.com wrote:
Why does this T-SQL round incorrectly?

PRINT 1/(1+26)

when it should be 0.037037037?

Thanks,
Michael
Nov 20 '06 #2
Ok, I just figured it out:

DECLARE @A int
DECLARE @B int
DECLARE @C float

SET @A = 1
SET @B = 26
SET @C = CAST(@A AS float) / (CAST(@A AS float) + CAST(@B AS float))
PRINT @C

This is unlike any other programming language that I have used before
-- so strange.

mp****@htxml.com wrote:
Why does this T-SQL round incorrectly?

PRINT 1/(1+26)

when it should be 0.037037037?

Thanks,
Michael
Nov 20 '06 #3
If any one of the operators had a decimal point then the results would
have had a decimal point. Performaing pure integer arithmatic, as in
the original example, returns an integer result.

Roy Harvey
Beacon Falls, CT

On 20 Nov 2006 12:57:04 -0800, mp****@htxml.com wrote:
>Ok, I just figured it out:

DECLARE @A int
DECLARE @B int
DECLARE @C float

SET @A = 1
SET @B = 26
SET @C = CAST(@A AS float) / (CAST(@A AS float) + CAST(@B AS float))
PRINT @C

This is unlike any other programming language that I have used before
-- so strange.

mp****@htxml.com wrote:
>Why does this T-SQL round incorrectly?

PRINT 1/(1+26)

when it should be 0.037037037?

Thanks,
Michael
Nov 20 '06 #4
So is PRINT CAST(1 AS float)/(CAST(1 AS float)+CAST(26 AS float)) the
best way to get the correct answer for PRINT 1/(1+26)?

- Michael

Roy Harvey wrote:
If any one of the operators had a decimal point then the results would
have had a decimal point. Performaing pure integer arithmatic, as in
the original example, returns an integer result.

Roy Harvey
Beacon Falls, CT

On 20 Nov 2006 12:57:04 -0800, mp****@htxml.com wrote:
Ok, I just figured it out:

DECLARE @A int
DECLARE @B int
DECLARE @C float

SET @A = 1
SET @B = 26
SET @C = CAST(@A AS float) / (CAST(@A AS float) + CAST(@B AS float))
PRINT @C

This is unlike any other programming language that I have used before
-- so strange.

mp****@htxml.com wrote:
Why does this T-SQL round incorrectly?

PRINT 1/(1+26)

when it should be 0.037037037?

Thanks,
Michael
Nov 20 '06 #5
On 20 Nov 2006 14:53:34 -0800, mp****@htxml.com wrote:
>So is PRINT CAST(1 AS float)/(CAST(1 AS float)+CAST(26 AS float)) the
best way to get the correct answer for PRINT 1/(1+26)?

- Michael
If a value is a constant, simply adding a decimal point to it is
sufficient, and only one of the operators needs the decimal point:

PRINT 1.0/(1+26)

Alternately, if none of the values are constants, converting any one
of them is sufficient. However, I would not choose to convert to
FLOAT, but to an appropriately sized DECIMAL.

PRINT CONVERT(DECIMAL(15,6),1)/(1+26)

Roy Harvey
Beacon Falls, CT
Nov 20 '06 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

12 posts views Thread by neutrino | last post: by
10 posts views Thread by Bob Bedford | last post: by
1 post views Thread by Steff | last post: by
4 posts views Thread by shortbackandsided.removeme | last post: by
4 posts views Thread by benjamin.cordes | last post: by
9 posts views Thread by Stef Mientki | last post: by
12 posts views Thread by jt2 | last post: by
10 posts views Thread by Prisoner at War | last post: by
reply views Thread by zhoujie | last post: by
reply views Thread by kglaser89 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.