473,595 Members | 2,646 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Selecting reals into doubles


I have attached some SQL which produces what to me, at least, is
rather unexpected results. Selecting real columns into double
precision columns loses some precision. Is this expected or documented
anywhere?

Thanks,
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Nov 23 '05 #1
2 1355
On Wed, Mar 03, 2004 at 11:19:15 +0000,
Will Newton <wi**@gbdirect. co.uk> wrote:

I have attached some SQL which produces what to me, at least, is
rather unexpected results. Selecting real columns into double
precision columns loses some precision. Is this expected or documented
anywhere?
You left out the output. But probably what you are seeing are the effects
of increased precision not decreased precision. Neither of the two
numbers you entered is exactly representable as floating point numbers.
When being printed as single precision numbers you got the same thing
back as you entered because within the number of digits used to display
single precision numbers those are going to be the closest to what is
stored. This isn't going to be the case for double precision numbers
in general.

If you really want exact decimal fractions, you want to use the numeric type
to store them.

Thanks, DROP TABLE precision_test;
DROP TABLE precision_test2 ;

CREATE TABLE precision_test
(
foo real
);

INSERT INTO precision_test
SELECT 20.20
UNION SELECT 1969.22;

CREATE TABLE precision_test2
(
foo double precision
);

INSERT INTO precision_test2 (foo) SELECT foo from precision_test;
SELECT * FROM precision_test;
SELECT * FROM precision_test2 ;
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postg resql.org so that your
message can get through to the mailing list cleanly

Nov 23 '05 #2
Will Newton <wi**@gbdirect. co.uk> writes:
I have attached some SQL which produces what to me, at least, is
rather unexpected results. Selecting real columns into double
precision columns loses some precision. Is this expected or documented
anywhere?


You shouldn't be surprised; this is a fundamental behavior of floating
point arithmetic anywhere.

There isn't any "loss of precision" per se --- the value represented in
the float8 column is the same as what was in the float4 column. The
difference is that the float8 output routine is programmed to print
about 15 digits of precision whereas the float4 routine prints no more
than 6. So you get to see the fact that the stored value wasn't really
20.2 but only something close to it.

If you find this surprising maybe you should be using type "numeric".

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 23 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
332
by: Daniel Lidström | last post by:
Hi, I'm currently using this method to extract doubles from a string: System::String* sp = S" "; System::String* tokens = s->Trim()->Split(sp->ToCharArray()); m_Northing = System::Double::Parse(tokens, nfi); m_Easting = System::Double::Parse(tokens, nfi); m_Elevation = System::Double::Parse(tokens, nfi);
43
2174
by: J.K. Becker | last post by:
Hi there, I am trying to multiply doubles with floats (actually I tried every possible combination by now) and it never works (well, it does something but it is always wrong). I have no idea what it is and where to look for help, maybe some of you know? double=float*double; (or every possible combination of it). An example: 0.3 * 0.7 would result in 1.7 (with lots more digits). Anyone any idea?
42
1569
by: Dooglo | last post by:
I'm new VB and programming all together, but I'm getting he hang of it. My question is; I'm writting a program to figure square feet and yards when the user inputs "Length in feet and inch ( seperate boxes), Width in feet and inches (seperate boxes), and thickness in feet and inches (all so in seperate boxes). I haven't quit figured out the split function yet to use a single text box for each. Anyways, when I enter data in each box;...
5
4488
by: Tales Normando | last post by:
The title says it all. Anyone?
12
6842
by: John Smith | last post by:
This code for the comparison of fp types is taken from the C FAQ. Any problems using it in a macro? /* compare 2 doubles for equality */ #define DBL_ISEQUAL(a,b) (fabs((a)-(b))<=(DBL_EPSILON)*fabs((a))) Do the same issues involved in comparing 2 fp types for equality apply to comparing a float to zero? E.g. is if(x == 0.0) considered harmful?
27
2535
by: brad | last post by:
Does anyone else feel that unittesting is too much work? Not in general, just the official unittest module for small to medium sized projects? It seems easier to write some quick methods that are used when needed rather than building a program with integrated unittesting. I see the value of it (the official unittest that is)... especially when there's a lot of source code. But this... if len(x) != y: sys.exit('...')
5
2795
by: =?Utf-8?B?TWFuanJlZSBHYXJn?= | last post by:
Hi, I developed a custom Matrix class derived from custom VC++ doubles Array class derived from a RealArray which is defined as: typedef CArray<double,doubleRealArray; Now I need to convert this custom Matrix class to standard doubles Matrix (VC++) as I am passing it to some function that understands only the standard class.
2
1839
by: utab | last post by:
Dear all, On a formatted printing operation that I am trying to accomplish for doubles, I have a question. I would like to print a double, d, and its square, d*d, but I would like to make the argument to be computed upon the length of the numbers(d and d*d) so that it lines up neatly in the columns. For this operation, I need the number of digits in a double, however, due to floating point representation, the double values are not...
48
2504
by: Bill Cunningham | last post by:
Is this valid C syntax ? double=double/int; I seem to be having trouble here. Bill
0
7954
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
7882
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8260
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8379
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8250
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
5837
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
3908
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1490
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
1222
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.