Dear python experts,
I am new to python and this site, so I apologize if this is off topic (i.e. is it a SciPy question?). I will try to demonstrate my problem below:
--------------------------------------------------------
#!/usr/local/bin/python
from scipy import *
from scipy.stats import *
a=norm(loc=0,scale=1)
a_data = a.rvs(10)
problem = zeros(10)
print problem
h_x1_x2 = -sum(problem * log2(a_data))
print h_x1_x2
#NaN
----------------------------------------------------------
I need a way of handling NaNs for example R has the 'na.omit' option. Does anybody know if this exists?
Many thanks
Andy 22 4392
On 2006-05-03, Andy McDonagh <an**********@earthlink.net> wrote: Dear python experts,
I am new to python and this site, so I apologize if this is off topic (i.e. is it a SciPy question?). I will try to demonstrate my problem below: -------------------------------------------------------- #!/usr/local/bin/python
from scipy import * from scipy.stats import * a=norm(loc=0,scale=1) a_data = a.rvs(10)
problem = zeros(10) print problem
h_x1_x2 = -sum(problem * log2(a_data))
print h_x1_x2 #NaN ----------------------------------------------------------
I need a way of handling NaNs
NaNs are handled.
Apparently they aren't handled the way you want them to be?
for example R has the 'na.omit' option. Does anybody know if this exists?
It would help if you explain how you want NaNs handled, but I
don't recall that tehre are any "options" for handling NaNs
other than the default way in scipy.
--
Grant Edwards grante Yow! It's NO USE... I've
at gone to "CLUB MED"!!
visi.com <snip> NaNs are handled.
Throwing an exception would be nice in regular Python (non-scipy).
This works to catch NaN on OSX and Linux:
# assuming x is a number
if x+1==x or x!=x:
#x is NaN
But is expensive as a precautionary measure.
Assert can be used for testing, if production code can be run with -0
or -OO.
On 2006-05-05, Ivan Vinogradov <vi*****@mcmaster.ca> wrote: <snip> NaNs are handled. Throwing an exception would be nice in regular Python (non-scipy).
That would break most of my Python programs (at least most of
the ones in which I do floating point). My main problem with
NaNs (and Infs) is that there isn't a string represention that
is consistent across platforms.
This works to catch NaN on OSX and Linux:
# assuming x is a number if x+1==x or x!=x: #x is NaN
But is expensive as a precautionary measure. Assert can be used for testing, if production code can be run with -0 or -OO.
There are those of us that need NaNs in production code, so it
would have to be something that could be configured. I find
that in my programs the places where I need to do something
"exceptional" with a NaN are very limited. The vast majority
of the time, I need them to propagate quietly.
--
Grant Edwards grante Yow! Thousands of days of
at civilians... have produced
visi.com a... feeling for the
aesthetic modules --
> <snip> There are those of us that need NaNs in production code, so it would have to be something that could be configured. I find that in my programs the places where I need to do something "exceptional" with a NaN are very limited. The vast majority of the time, I need them to propagate quietly.
Our programming expectations may differ, but an option to catch NaNs as
an exception is a great idea.
Regards, Ivan.
Ivan Vinogradov wrote: <snip> There are those of us that need NaNs in production code, so it would have to be something that could be configured. I find that in my programs the places where I need to do something "exceptional" with a NaN are very limited. The vast majority of the time, I need them to propagate quietly.
Our programming expectations may differ, but an option to catch NaNs as an exception is a great idea.
numpy lets the programmer control how NaNs are handled in numpy code. Producing
a NaN can be ignored, create a warning, raise an exception or call a function.
It's not well documented at the moment, but the functions are seterr(),
seterrcall(), seterrobj(), geterr(), geterrcall(), and geterrobj().
Pure Python has a similar, but somewhat less flexible method, on UNIX platforms. http://docs.python.org/dev/lib/module-fpectl.html
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
On 2006-05-05, Ivan Vinogradov <vi*****@mcmaster.ca> wrote: <snip> There are those of us that need NaNs in production code, so it would have to be something that could be configured. I find that in my programs the places where I need to do something "exceptional" with a NaN are very limited. The vast majority of the time, I need them to propagate quietly.
Our programming expectations may differ, but an option to catch NaNs as an exception is a great idea.
Unless it's done in hardware, it would be hopelessly slow.
There some platforms where it's possible for an application to
enable and handle FP interrupts (all of the exampls for that
seem to be in Fortran). I don't know if the common platforms
(IA32/MS-Windows, IA32/Linux) even have that ability.
--
Grant Edwards grante Yow! I'm into SOFTWARE!
at
visi.com
On 2006-05-05, Robert Kern <ro*********@gmail.com> wrote: Our programming expectations may differ, but an option to catch NaNs as an exception is a great idea.
[...]
Pure Python has a similar, but somewhat less flexible method, on UNIX platforms.
http://docs.python.org/dev/lib/module-fpectl.html
For which "Unix" platforms? It's not there under Linux:
Python 2.4.2 (#1, Feb 14 2006, 07:55:13)
[GCC 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8)] on linux2
Type "help", "copyright", "credits" or "license" for more information. import fpectl
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ImportError: No module named fpectl
--
Grant Edwards grante Yow! Do I hear th'
at SPINNING of various
visi.com WHIRRING, ROUND, and WARM
WHIRLOMATICS?!
Grant Edwards wrote: On 2006-05-05, Robert Kern <ro*********@gmail.com> wrote: Pure Python has a similar, but somewhat less flexible method, on UNIX platforms.
http://docs.python.org/dev/lib/module-fpectl.html
For which "Unix" platforms? It's not there under Linux:
Python 2.4.2 (#1, Feb 14 2006, 07:55:13) [GCC 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import fpectl
Traceback (most recent call last): File "<stdin>", line 1, in ? ImportError: No module named fpectl
You might have to enable it during the Python build process.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
On 5-May-06, at 6:45 PM, Grant Edwards wrote: On 2006-05-05, Robert Kern <ro*********@gmail.com> wrote:
Our programming expectations may differ, but an option to catch NaNs as an exception is a great idea.
[...]
Pure Python has a similar, but somewhat less flexible method, on UNIX platforms.
http://docs.python.org/dev/lib/module-fpectl.html
For which "Unix" platforms? It's not there under Linux:
<snip>
It doesn't seem to be here under OSX either (universal Python install).
Since numpy seems to be working on a variety of platforms/hardware,
how hard would it be to extract this functionality from it to add to
Python proper?
Cheers, Ivan.
Ivan Vinogradov wrote: It doesn't seem to be here under OSX either (universal Python install).
It's not enabled by default. In the source distribution, it is
Modules/fpectlmodule.c .
Since numpy seems to be working on a variety of platforms/hardware, how hard would it be to extract this functionality from it to add to Python proper?
Harder than just enabling fpectl.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
Ivan Vinogradov wrote: <snip> NaNs are handled.
Throwing an exception would be nice in regular Python (non-scipy).
This works to catch NaN on OSX and Linux:
# assuming x is a number if x+1==x or x!=x: #x is NaN
x != x works, but: x = 1e100 x + 1 == x
True
Robert Kern <ro*********@gmail.com> writes: Ivan Vinogradov wrote:
It doesn't seem to be here under OSX either (universal Python install).
It's not enabled by default. In the source distribution, it is Modules/fpectlmodule.c .
Since numpy seems to be working on a variety of platforms/hardware, how hard would it be to extract this functionality from it to add to Python proper?
Harder than just enabling fpectl.
Last thing I heard fpectl was considered to be completely broken -- it's
likely not disabled by default for no reason. A short google turned up this:
Comment By: Michael Hudson (mwh)
Date: 2006-02-20 12:59
Message:
Logged In: YES
user_id=6656
Waaa, the correct thing to is to remove the --with-fpectl from configure!
I've
been meaning to post to python-dev for a while now proposing the ripping out
of this code. It's just not useful any more. At any rate, I am actively
opposed to
advertising its existence more widely.
And in pep 356 (python 2.5 release schedule) we find:
Possible features for 2.5
[...]
- Remove the fpectl module?
So "just enabling fpectl" doesn't look like a viable long term solution.
'as
Em Sex, 2006-05-05 Ã*s 16:37 -0400, Ivan Vinogradov escreveu: This works to catch NaN on OSX and Linux:
# assuming x is a number if x+1==x or x!=x: #x is NaN
This works everywhere:
nan = float('nan')
..
..
..
if x == nan:
# x is not a number
--
Felipe.
Felipe Almeida Lessa <fe**********@gmail.com> writes: Em Sex, 2006-05-05 às 16:37 -0400, Ivan Vinogradov escreveu: This works to catch NaN on OSX and Linux:
# assuming x is a number if x+1==x or x!=x: #x is NaN
This works everywhere:
nan = float('nan')
. . .
if x == nan: # x is not a number
No it doesn't.
'as
"Felipe Almeida Lessa" <fe**********@gmail.com> wrote in message
news:11**********************@kenshin.CASA... This works everywhere:
nan = float('nan')
Not. nan = float('nan')
Traceback (most recent call last):
File "<pyshell#4>", line 1, in -toplevel-
nan = float('nan')
ValueError: invalid literal for float(): nan
Above is Windows, which requires something else.
tjr
Terry Reedy wrote: "Felipe Almeida Lessa" <fe**********@gmail.com> wrote in message news:11**********************@kenshin.CASA... This works everywhere:
nan = float('nan')
Not.
nan = float('nan') Traceback (most recent call last): File "<pyshell#4>", line 1, in -toplevel- nan = float('nan') ValueError: invalid literal for float(): nan
Above is Windows, which requires something else.
I think he meant: float("NaN")
nan
That's Python 2.4.1 on Mac OS X.
"Ryan Forsythe" <ry***@cs.uoregon.edu> wrote in message
news:44**************@cs.uoregon.edu... Terry Reedy wrote: "Felipe Almeida Lessa" <fe**********@gmail.com> wrote in message news:11**********************@kenshin.CASA... This works everywhere:
nan = float('nan')
Not.
> nan = float('nan')
Traceback (most recent call last): File "<pyshell#4>", line 1, in -toplevel- nan = float('nan') ValueError: invalid literal for float(): nan
Above is Windows, which requires something else.
I think he meant:
float("NaN") nan
That's Python 2.4.1 on Mac OS X. float("NaN")
Traceback (most recent call last):
File "<pyshell#5>", line 1, in -toplevel-
float("NaN")
ValueError: invalid literal for float(): NaN
As Tim Peters has said often enough, this sort of thing is specific to the
underlying C library and will remain so until someone cares enough to write
or fund a cross-platform solution.
Terry Jan Reedy
Felipe Almeida Lessa wrote: Em Sex, 2006-05-05 Ã*s 16:37 -0400, Ivan Vinogradov escreveu:
This works to catch NaN on OSX and Linux:
# assuming x is a number if x+1==x or x!=x: #x is NaN
This works everywhere:
nan = float('nan')
Have you tried it on Windows?
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
Alexander Schmolck wrote: Robert Kern <ro*********@gmail.com> writes:Ivan Vinogradov wrote:
Since numpy seems to be working on a variety of platforms/hardware, how hard would it be to extract this functionality from it to add to Python proper?
Harder than just enabling fpectl.
Last thing I heard fpectl was considered to be completely broken -- it's likely not disabled by default for no reason.
Fair enough. If you want to go through numpy's code to rip out its floating
point error handling, knock yourself out. It's not going to be trivial, though.
It's heavily embedded in the ufunc machinery.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
On 2006-05-06, Robert Kern <ro*********@gmail.com> wrote: Since numpy seems to be working on a variety of platforms/hardware, how hard would it be to extract this functionality from it to add to Python proper?
Harder than just enabling fpectl.
Last thing I heard fpectl was considered to be completely broken -- it's likely not disabled by default for no reason.
Fair enough. If you want to go through numpy's code to rip out its floating point error handling, knock yourself out. It's not going to be trivial, though. It's heavily embedded in the ufunc machinery.
Does numpy's NaN handling only work within numpy functions, or
does it enable HW FP signals and then catch them for "normal"
floating point operations that take place outside of numpy code?
--
Grant Edwards grante Yow! I'm in direct contact
at with many advanced fun
visi.com CONCEPTS.
On 2006-05-06, Terry Reedy <tj*****@udel.edu> wrote: That's Python 2.4.1 on Mac OS X.
float("NaN")
Traceback (most recent call last): File "<pyshell#5>", line 1, in -toplevel- float("NaN") ValueError: invalid literal for float(): NaN
As Tim Peters has said often enough, this sort of thing is specific to the underlying C library and will remain so until someone cares enough to write or fund a cross-platform solution.
Yea, that's a royal PITA. I had to add my own code into the
"pickle" classes to handle NaNs. Having a pickler that only
works for a subset of the legal IEEE FP values proved to be
sort of useless.
--
Grant Edwards grante Yow! Were these parsnips
at CORRECTLY MARINATED in
visi.com TACO SAUCE?
Grant Edwards wrote: On 2006-05-06, Robert Kern <ro*********@gmail.com> wrote:
>Since numpy seems to be working on a variety of platforms/hardware, >how hard would it be to extract this functionality from it to add to >Python proper?
Harder than just enabling fpectl.
Last thing I heard fpectl was considered to be completely broken -- it's likely not disabled by default for no reason.
Fair enough. If you want to go through numpy's code to rip out its floating point error handling, knock yourself out. It's not going to be trivial, though. It's heavily embedded in the ufunc machinery.
Does numpy's NaN handling only work within numpy functions, or does it enable HW FP signals and then catch them for "normal" floating point operations that take place outside of numpy code?
Just in numpy code, it seems.
In [1]: import numpy
i
In [2]: import math
In [3]: numpy.seterr(invalid='raise')
Out[3]: {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under':
'ignore'}
In [4]: numpy.log(-1.0)
---------------------------------------------------------------------------
exceptions.FloatingPointError Traceback (most recent call
last)
/Users/kern/<ipython console>
FloatingPointError: invalid encountered in log
In [5]: math.log(-1.0)
Out[5]: nan
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: WSeeger |
last post by:
When creating a new class, is it encouraged to always include error
handling routines within your LET and GET procedures? It's seems that
most text books never seem to include much about error...
|
by: Hans-Joachim Widmaier |
last post by:
Hi all.
Handling files is an extremely frequent task in programming, so most
programming languages have an abstraction of the basic files offered by
the underlying operating system. This is...
|
by: Master of C++ |
last post by:
Hi,
I am an absolute newbie to Exception Handling, and I am trying to
retrofit exception handling to a LOT of C++ code that I've written
earlier. I am just looking for a bare-bones, low-tech...
|
by: Anthony England |
last post by:
Everyone knows that global variables get re-set in an mdb when an un-handled
error is encountered, but it seems that this also happens when the variable
is defined as private at form-level.
So...
|
by: Stefan Johansson |
last post by:
Hi all
I'am moving from Visual Foxpro and have a question
regarding "best practice" error handling in vb .net.
In VFP I have always used a "central" error handling
object in order to have a...
|
by: Al Williams |
last post by:
Hi,
I have error handling in place throughout my application. I also start the
application wrapped in error handling code to catch any unexpected
exceptions (i.e. exceptions that occur where I...
|
by: Gustaf |
last post by:
I'm confused about structured error handling. The following piece of
code is a simplification of a class library I'm working on. It works,
and it does what I want, but I'm still not convinced that...
|
by: yogeshnelwadkar |
last post by:
Hello, i have a problem with replacing c++ exception handling with
structured exception handling. How to replace the " catch(...) " in
c++ exception handling with, __except , a structured...
|
by: Zytan |
last post by:
Ok something simple like int.Parse(string) can throw these exceptions:
ArgumentNullException, FormatException, OverflowException
I don't want my program to just crash on an exception, so I must...
|
by: Lysander |
last post by:
Thought I would give something back with a few articles. This article is a bit of code to add error handling. When I have time, I want to write articles on multilingual databases, and Access...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: aa123db |
last post by:
Variable and constants
Use var or let for variables and const fror constants.
Var foo ='bar';
Let foo ='bar';const baz ='bar';
Functions
function $name$ ($parameters$) {
}
...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
| |