473,383 Members | 1,866 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,383 software developers and data experts.

how truncate scientific notation from Real?

mo
Hi
We've got some numbers stored as Reals which are returning values in
scientific notation that we need rounded down to 3 digits to the right
of the decimal.
ie 8.7499999E-2 needs to return 8.75.
Round, cast, convert, formatnumber in the dts package all fail.
Help!
Thanks Moe

Jul 23 '05 #1
2 5168
mo (mo@idcomm.com) writes:
We've got some numbers stored as Reals which are returning values in
scientific notation that we need rounded down to 3 digits to the right
of the decimal.
ie 8.7499999E-2 needs to return 8.75.
Round, cast, convert, formatnumber in the dts package all fail.


It's seems a little funny that 0.0875 would be rounded to 8.75. Maybe
you could clarify?

Anyway, a real is an approximate number. Most numbers cannot represented
exactly as real. How a real number is displayed, depends on the routine
that converts it to text. For instance when I run

select convert(real, 8.7499999E-2)

in Query Analyzer, I see just that. But if I run the same statement in
SQL Server Mgmt Studio, the replacement for Query Analyzer (and
Enterprise Manager) in SQL 2005, I get back 0,875.

If you want make sure that the client only presents three digits, you
could to this:

select convert(decimal(10, 2), 8.7499999E-2 * 100)

but it's probably better to do this in the client.

--
Erland Sommarskog, SQL Server MVP, es****@sommarskog.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techinf...2000/books.asp
Jul 23 '05 #2
mo
Thank you so much! The convert with decimal worked great!
Moe

Jul 23 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Skeleton Man | last post by:
Hi, I have a script that generates numbers beyond the range of it, and so it's throwing them out as scientfic notation (e.g. 5E+10 instead of 50,000,000,000). The only way I've found of...
2
by: Woodster | last post by:
I am using the std::setprecision function to format variables of type double in a string however I am unsure how to stop this appearing in scientific notation. For example std::stringstream...
1
by: masoud bayan | last post by:
I have some values in type of double such 0.00009 , 0.0000007, when I want to show them in a text box and convert them to string they are changed to scientific notation while I want to show them as...
1
by: Nick | last post by:
Well, the project I am working on has now come to a screeching halt! I have been developing a program that heavily utilizes ADO.NET record sets. To generate reports, I convert the recordset to XML,...
0
by: Greg | last post by:
I am working on an application that requires working with numbers in scientific notation. I am using SqlServer as the database and I have created strongly typed data adapters and datasets. The...
7
by: Dustan | last post by:
How can I get a number into scientific notation? I have a preference for the format '1 E 50' (as an example), but if it's well known, it works.
9
by: Joe Attardi | last post by:
Hi all, Math is not my strongest area so forgive me if I use some of the wrong terminology. It seems that scientific notation is immune to rounding errors. For example: (4.98 * 100) + 5.51 ...
2
by: rSmoke | last post by:
I have a DataSet that contains a table with about 6 columns of high accuracy decimal values. When I try to write out the DataSet using the WriteXML() function the XML is written fine, but the...
2
by: Greg | last post by:
I am working on an application that requires working with numbers in scientific notation. I am using SqlServer as the database and I have created strongly typed data adapters and datasets. The...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.