473,388 Members | 953 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,388 software developers and data experts.

Questions about mathematical and statistical functionality in Python

Greetings Pythoners!

I hope you'll indulge an ignorant outsider. I work at a financial software
firm, and the tool I currently use for my research is R, a software
environment for statistical computing and graphics. R is designed with
matrix manipulation in mind, and it's very easy to do regression and time
series modeling, and to plot the results and test hypotheses. The kinds of
functionality we rely on the most are standard and robust versions of
regression and principal component / factor analysis, bayesian methods such
as Gibbs sampling and shrinkage, and optimization by linear, quadratic,
newtonian / nonlinear, and genetic programming; frequently used graphics
include QQ plots and histograms. In R, these procedures are all available
as functions (some of them are in auxiliary libraries that don't come with
the standard distribution, but are easily downloaded from a central
repository).

For a variety of reasons, the research group is considering adopting Python.
Naturally, I am curious about the mathematical, statistical, and graphical
functionality available in Python. Do any of you out there use Python in
financial research, or other intense mathematical/statistical computation?
Can you compare working in Python with working in a package like R or S-Plus
or Matlab, etc.? Which of the procedures I mentioned above are available in
Python? I appreciate any insight you can provide. Thanks!

-- TMK --
212-460-5430 home
917-656-5351 cell
Jun 14 '07 #1
4 3056
On Jun 14, 4:02 pm, "Talbot Katz" <topk...@msn.comwrote:
Greetings Pythoners!

I hope you'll indulge an ignorant outsider. I work at a financial software
firm, and the tool I currently use for my research is R, a software
environment for statistical computing and graphics. R is designed with
matrix manipulation in mind, and it's very easy to do regression and time
series modeling, and to plot the results and test hypotheses. The kinds of
functionality we rely on the most are standard and robust versions of
regression and principal component / factor analysis, bayesian methods such
as Gibbs sampling and shrinkage, and optimization by linear, quadratic,
newtonian / nonlinear, and genetic programming; frequently used graphics
include QQ plots and histograms. In R, these procedures are all available
as functions (some of them are in auxiliary libraries that don't come with
the standard distribution, but are easily downloaded from a central
repository).

For a variety of reasons, the research group is considering adopting Python.
Naturally, I am curious about the mathematical, statistical, and graphical
functionality available in Python. Do any of you out there use Python in
financial research, or other intense mathematical/statistical computation?
Can you compare working in Python with working in a package like R or S-Plus
or Matlab, etc.? Which of the procedures I mentioned above are available in
Python? I appreciate any insight you can provide. Thanks!

-- TMK --
212-460-5430 home
917-656-5351 cell
I'd look at following modules:

matplotlib - http://matplotlib.sourceforge.net/
numpy - http://numpy.scipy.org/

Finally, this website lists other resources: http://www.astro.cornell.edu/staff/loredo/statpy/

Mike

Jun 14 '07 #2
Talbot Katz wrote:
I hope you'll indulge an ignorant outsider. I work at a financial
software firm, and the tool I currently use for my research is R, a
software environment for statistical computing and graphics. R is
designed with matrix manipulation in mind, and it's very easy to do
regression and time series modeling, and to plot the results and test
hypotheses. The kinds of functionality we rely on the most are standard
and robust versions of regression and principal component / factor
analysis, bayesian methods such as Gibbs sampling and shrinkage, and
optimization by linear, quadratic, newtonian / nonlinear, and genetic
programming; frequently used graphics include QQ plots and histograms.
In R, these procedures are all available as functions (some of them are
in auxiliary libraries that don't come with the standard distribution,
but are easily downloaded from a central repository).
I use both R and Python for my work. I think R is probably better for
most of the stuff you are mentioning. I do any sort of heavy
lifting--database queries/tabulation/aggregation in Python and load the
resulting data frames into R for analysis and graphics.
--
Michael Hoffman
Jun 14 '07 #3
Michael Hoffman wrote:
Talbot Katz wrote:
>I hope you'll indulge an ignorant outsider. I work at a financial
software firm, and the tool I currently use for my research is R, a
software environment for statistical computing and graphics. R is
designed with matrix manipulation in mind, and it's very easy to do
regression and time series modeling, and to plot the results and test
hypotheses. The kinds of functionality we rely on the most are standard
and robust versions of regression and principal component / factor
analysis, bayesian methods such as Gibbs sampling and shrinkage, and
optimization by linear, quadratic, newtonian / nonlinear, and genetic
programming; frequently used graphics include QQ plots and histograms.
In R, these procedures are all available as functions (some of them are
in auxiliary libraries that don't come with the standard distribution,
but are easily downloaded from a central repository).

I use both R and Python for my work. I think R is probably better for
most of the stuff you are mentioning. I do any sort of heavy
lifting--database queries/tabulation/aggregation in Python and load the
resulting data frames into R for analysis and graphics.
I would second that. It is not either/or. Use Python, including Numpy
and matplotlib and packages from SciPy, for some things, and R for
others. And you can even embed R in Python using RPy - see
http://rpy.sourceforge.net/

We use the combination of Python, Numpy (actually, the older Numeric
Python package, but soon to be converted to Numpy), RPy and R in our
NetEpi Analysis project - exploratory epidemiological analysis of large
data sets - see http://sourceforge.net/projects/netepi - and it is a
good combination - Python for the Web interface, data manipulation and
data heavy-lifting, and for some of the more elementary statistics, and
R for more involved statistical analysis and graphics (with teh option
of using matplotlib or other Python-based graphics packages for some
tasks if we wish). The main thing to remember, though, is that indexing
is zero-based in Python and 1-based in R...

Tim C
Jun 14 '07 #4
On Thursday 14 June 2007 5:54 pm, Tim Churches wrote:
Michael Hoffman wrote:
Talbot Katz wrote:
I hope you'll indulge an ignorant outsider. I work at a financial
software firm, and the tool I currently use for my research is R, a
software environment for statistical computing and graphics. R is
designed with matrix manipulation in mind, and it's very easy to do
regression and time series modeling, and to plot the results and test
hypotheses. The kinds of functionality we rely on the most are standard
and robust versions of regression and principal component / factor
analysis, bayesian methods such as Gibbs sampling and shrinkage, and
optimization by linear, quadratic, newtonian / nonlinear, and genetic
programming; frequently used graphics include QQ plots and histograms.
In R, these procedures are all available as functions (some of them are
in auxiliary libraries that don't come with the standard distribution,
but are easily downloaded from a central repository).
I use both R and Python for my work. I think R is probably better for
most of the stuff you are mentioning. I do any sort of heavy
lifting--database queries/tabulation/aggregation in Python and load the
resulting data frames into R for analysis and graphics.

I would second that. It is not either/or. Use Python, including Numpy
and matplotlib and packages from SciPy, for some things, and R for
others. And you can even embed R in Python using RPy - see
http://rpy.sourceforge.net/

We use the combination of Python, Numpy (actually, the older Numeric
Python package, but soon to be converted to Numpy), RPy and R in our
NetEpi Analysis project - exploratory epidemiological analysis of large
data sets - see http://sourceforge.net/projects/netepi - and it is a
good combination - Python for the Web interface, data manipulation and
data heavy-lifting, and for some of the more elementary statistics, and
R for more involved statistical analysis and graphics (with teh option
of using matplotlib or other Python-based graphics packages for some
tasks if we wish). The main thing to remember, though, is that indexing
is zero-based in Python and 1-based in R...

Tim C
Thirded. I use R, Python, Matlab along with other languages (I hate pipeline
pilot) in my work and from what I've seen nothing can compare with R when it
comes to stats. I love R, from its brilliant CRAN system (PyPI needs serious
work to be considered in the same class as CPAN et al) to its delicious Emacs
integration.

I just wish there was a way to distribute R packages without requiring the
user to separately install R.

In a similar vein, I wish there was a reasonable Free Software equivalent to
Spotfire. The closest I've found (and they're nowhere near as good) are
Orange (http://www.ailab.si/orange) and WEKA
(http://www.cs.waikato.ac.nz/ml/weka/). Orange is written in Python, but its
tied to QT 2.x as the 3.x series was not available on Windows under the GPL.
Josh Gilbert
Jun 15 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: dont bother | last post by:
Hey, I was just wondering if python has any support modules for statistical techniques as Bayesian, Markovian Style Models etc. Any hints? Thanks Dont __________________________________
6
by: administrata | last post by:
Hi! I'm programming maths programs. And I got some questions about mathematical signs. 1. Inputing suqare like a * a, It's too long when I do time-consuming things. Can it be simplified? 2....
162
by: techievasant | last post by:
hello everyone, Iam vasant from India.. I have a test+interview on C /C++ in the coming month so plz help me by giving some resources of FAQS, interview questions, tracky questions, multiple...
81
by: Matt | last post by:
I have 2 questions: 1. strlen returns an unsigned (size_t) quantity. Why is an unsigned value more approprate than a signed value? Why is unsighned value less appropriate? 2. Would there...
3
by: Thomas Nelson | last post by:
Sorry if this is a FAQ, but I couldn't find a good summary through google. What kinds of statistical analysis tools exist in python? I really just need t-tests, chi-squared test, and other such...
5
by: jeremito | last post by:
I am extending python with C++ and need some help. I would like to convert a string to a mathematical function and then make this a C++ function. My C++ code would then refer to this function to...
6
by: Qun Cao | last post by:
Hi Everyone, I am a beginner on cross language development. My problem at hand is to build a python interface for a C++ application built on top of a 3D game engine. The purpose of this python...
8
by: Krypto | last post by:
Hi, I have used Python for a couple of projects last year and I found it extremely useful. I could write two middle size projects in 2-3 months (part time). Right now I am a bit rusty and trying...
2
by: mc | last post by:
I'm looking for a library which can do mathematical stuff like solving equations. Or calculation the nulls of a function and so on. Does anyone know one? Thanks in advance!
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.