473,287 Members | 1,868 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

Python advocacy in scientific computation

Someone asked me to write a brief essay regarding the value-add
proposition for Python in the Fortran community. Slightly modified to
remove a few climatology-related specifics, here it is.

I would welcome comments and corrections, and would be happy to
contribute some version of this to the Python website if it is of
interest.

===

The established use of Fortran in continuum models such as climate
models has some benefits, including very high performance and
flexibility in dealing with regular arrays, backward compatibility with
the existing code base, and the familiarity with the language among the
modeling community. Fortran 90 and later versions have taken many of
the lessons of object oriented programming and adapted them so that
logical separation of modules is supported, allowing for more effective
development of large systems. However, there are many purposes to which
Fortran is ill-suited which are increasingly part of the modeling
environment.

These include: source and version control and audit trails for runs,
build system management, test specification, deployment testing (across
multiple platforms), post-processing analysis, run-time and
asynchronous visualization, distributed control and ensemble
management. To achieve these goals, a combination of shell scripts,
specialized build tools, specialized applications written in several
object-oriented languages, and various web and network deployment
strategies have been deployed in an ad hoc manner. Not only has much
duplication of effort occurred, a great deal of struggling up the
learning curves of various technologies has been required as one need
or another has been addressed in various ad hoc ways.

A new need arises as the ambitions of physical modeling increase; this
is the rapid prototyping and testing of new model components. As the
number of possible configurations of a model increases, the expense and
difficulty of both unit testing and integration testing becomes more
demanding.

Fortunately, there is Python. Python is a very flexible language that
has captured the enthusiasm of commercial and scientific programmers
alike. The perception of Python programmers coming from almost any
other language is that they are suddenly dramatically several times
more productive than previously, in terms of functionality delivered
per unit of programmer time.

One slogan of the Python community is that the language "fits your
brain". Why this might be the case is an interesting question. There
are no startling computer science breakthroughs original to the
language, Rather, Python afficionados will claim that the language
combines the best features of such various languages as Lisp, Perl,
Java, and Matlab. Eschewing allegiance to a specific theory of how to
program, Python's design instead offers the best practices from many
other software cultures.

The synergies among these programming modes is in some ways harder to
explain than to experience. The Python novice may nevertheless observe
that a single language can take the place of shell scripts, makefiles,
desktop computation environments, compiled languages to build GUIs, and
scripting languages to build web interfaces. In addition, Python is
useful as a wrapper for Fortran modules, facilitating the
implementation of true test-driven design processes in Fortran models.

Another Python advocacy slogan is "batteries included". The point here
is that (in part because Python is dramatically easier to write than
other languages) there is a very broad range of very powerful standard
libraries that make many tasks which are difficult in other languages
astonishingly easy in Python. For instance, drawing upon the standard
libraries (no additional download required) a portable webserver
(runnable on both Microsoft and Unix-based platforms) can be
implemented in seven lines of code. (See
http://effbot.org/librarybook/simplehttpserver.htm ) Installation of
pure python packages is also very easy, and installation of mixed
language products with a Python component is generally not
significantly harder than a comparable product with no Python
component.

Among the Python components and Python bindings of special interest to
scientists are the elegant and powerful matplotlib plotting package,
which began by emulating and now surpasses the plotting features of
Matlab, SWIG, which allows for runtime interoperability with various
languages, f2py which specifically interoperates with Fortran, NetCDF
libraries (which cope with NetCDF files with dramatically less fuss
than the standard C or Fortran bindings), statistics packages including
bindings to the R language, linear algebra packages, various
platform-specific and portable GUI libraries, genetic algorithms,
optimization libraries, and bindings for high performance differential
equation solvers (notably, using the Argonne National Laboratory
package PetSC). An especially interesting Python trick for runtime
visualization in models that were not designed to support it, pioneered
by David Beazley's SWILL, embeds a web server in your model code.

See especially http://starship.python.net/~hinsen/ScientificPython/ and
http://scipy.org as good starting points to learn about scientific uses
of Python.

mt

Feb 14 '06 #1
53 4251
Nicely done. But now for a couple of small nits:

other language is that they are suddenly dramatically several times
more productive
'suddenly dramatically several times' seems a bit redundantly
repeditively excessive, don't you think?

Among the Python components and Python bindings of special interest to
scientists are the elegant and powerful matplotlib plotting package,
which began by emulating and now surpasses the plotting features of
Matlab, SWIG, which allows for runtime interoperability with various
languages, f2py which specifically interoperates with Fortran, NetCDF
libraries (which cope with NetCDF files with dramatically less fuss
than the standard C or Fortran bindings), statistics packages including
bindings to the R language, linear algebra packages, various
platform-specific and portable GUI libraries, genetic algorithms,
optimization libraries, and bindings for high performance differential
equation solvers (notably, using the Argonne National Laboratory
package PetSC).


As the length of the sentence built up, and the inumerable commas
passed by, my brain exploded. I'd suggest turning this into a bullet
list.

Feb 14 '06 #2
In article <11**********************@z14g2000cwz.googlegroups .com>,
Michael Tobis <mt****@gmail.com> wrote:
Feb 14 '06 #3
Michael Tobis wrote:
Someone asked me to write a brief essay regarding the value-add
proposition for Python in the Fortran community. Slightly modified to
remove a few climatology-related specifics, here it is.

Thank you - this was very good reading.
I would welcome comments and corrections, and would be happy to
contribute some version of this to the Python website if it is of
interest.


A slight broadening of the perspective could show another advantage:
Python is also used for data processing, at least in astronomy. Modeling
and processing the data in the same environment is very practical. Spend
more time on modeling and processing the critical data sections -
critical data section may depend on model parameters and sampling (which
is often incomplete and uneven). You also avoid wasting CPU cycles to
model things not in the data.

A theorist may be perfectly happy with Fortran, and an observer could do
his stuff with simple scripts. But if they need to work together, Python
is a very good option.
Feb 15 '06 #4
Michael Tobis wrote:
Someone asked me to write a brief essay regarding the value-add
proposition for Python in the Fortran community. Slightly modified to
remove a few climatology-related specifics, here it is.


Great text. Do you want to put it onto a Wiki page at wiki.python.org?

Georg
Feb 15 '06 #5
I have posted your essay in a thread "Python for Fortran programmers"
in comp.lang.fortran since it is written in part for a Fortran
audience, and since you are more likely to get critical (but hopefully
constructive) comments there.

Feb 15 '06 #6
Hi,

Like it - an area that doesn't come out strongly enough for me is
Python's ability to drop down to and integrate with low level
algorithms. This allows me to to optimise the key bits of design in
python very quickly and then if I still need more poke i can drop down
to low level programming languages. Optimise design, not code unless I
really need to.

To be fair the same is at least partly true for Java ( though supporting
JNI code scares me ) but my prototyping productivity isn't as high.

The distributed / HPC packages may also be worth noting - PyMPI and
PyGlobus.

p

Michael Tobis wrote:
Someone asked me to write a brief essay regarding the value-add
proposition for Python in the Fortran community. Slightly modified to
remove a few climatology-related specifics, here it is.

I would welcome comments and corrections, and would be happy to
contribute some version of this to the Python website if it is of
interest.

===

The established use of Fortran in continuum models such as climate
models has some benefits, including very high performance and
flexibility in dealing with regular arrays, backward compatibility with
the existing code base, and the familiarity with the language among the
modeling community. Fortran 90 and later versions have taken many of
the lessons of object oriented programming and adapted them so that
logical separation of modules is supported, allowing for more effective
development of large systems. However, there are many purposes to which
Fortran is ill-suited which are increasingly part of the modeling
environment.

These include: source and version control and audit trails for runs,
build system management, test specification, deployment testing (across
multiple platforms), post-processing analysis, run-time and
asynchronous visualization, distributed control and ensemble
management. To achieve these goals, a combination of shell scripts,
specialized build tools, specialized applications written in several
object-oriented languages, and various web and network deployment
strategies have been deployed in an ad hoc manner. Not only has much
duplication of effort occurred, a great deal of struggling up the
learning curves of various technologies has been required as one need
or another has been addressed in various ad hoc ways.

A new need arises as the ambitions of physical modeling increase; this
is the rapid prototyping and testing of new model components. As the
number of possible configurations of a model increases, the expense and
difficulty of both unit testing and integration testing becomes more
demanding.

Fortunately, there is Python. Python is a very flexible language that
has captured the enthusiasm of commercial and scientific programmers
alike. The perception of Python programmers coming from almost any
other language is that they are suddenly dramatically several times
more productive than previously, in terms of functionality delivered
per unit of programmer time.

One slogan of the Python community is that the language "fits your
brain". Why this might be the case is an interesting question. There
are no startling computer science breakthroughs original to the
language, Rather, Python afficionados will claim that the language
combines the best features of such various languages as Lisp, Perl,
Java, and Matlab. Eschewing allegiance to a specific theory of how to
program, Python's design instead offers the best practices from many
other software cultures.

The synergies among these programming modes is in some ways harder to
explain than to experience. The Python novice may nevertheless observe
that a single language can take the place of shell scripts, makefiles,
desktop computation environments, compiled languages to build GUIs, and
scripting languages to build web interfaces. In addition, Python is
useful as a wrapper for Fortran modules, facilitating the
implementation of true test-driven design processes in Fortran models.

Another Python advocacy slogan is "batteries included". The point here
is that (in part because Python is dramatically easier to write than
other languages) there is a very broad range of very powerful standard
libraries that make many tasks which are difficult in other languages
astonishingly easy in Python. For instance, drawing upon the standard
libraries (no additional download required) a portable webserver
(runnable on both Microsoft and Unix-based platforms) can be
implemented in seven lines of code. (See
http://effbot.org/librarybook/simplehttpserver.htm ) Installation of
pure python packages is also very easy, and installation of mixed
language products with a Python component is generally not
significantly harder than a comparable product with no Python
component.

Among the Python components and Python bindings of special interest to
scientists are the elegant and powerful matplotlib plotting package,
which began by emulating and now surpasses the plotting features of
Matlab, SWIG, which allows for runtime interoperability with various
languages, f2py which specifically interoperates with Fortran, NetCDF
libraries (which cope with NetCDF files with dramatically less fuss
than the standard C or Fortran bindings), statistics packages including
bindings to the R language, linear algebra packages, various
platform-specific and portable GUI libraries, genetic algorithms,
optimization libraries, and bindings for high performance differential
equation solvers (notably, using the Argonne National Laboratory
package PetSC). An especially interesting Python trick for runtime
visualization in models that were not designed to support it, pioneered
by David Beazley's SWILL, embeds a web server in your model code.

See especially http://starship.python.net/~hinsen/ScientificPython/ and
http://scipy.org as good starting points to learn about scientific uses
of Python.

mt

Feb 17 '06 #7
In article <45************@individual.net>,
Georg Brandl <g.*************@gmx.net> wrote:
Michael Tobis wrote:
Someone asked me to write a brief essay regarding the value-add
proposition for Python in the Fortran community. Slightly modified to
remove a few climatology-related specifics, here it is.


Great text. Do you want to put it onto a Wiki page at wiki.python.org?

Feb 28 '06 #8
Maybe I'd also emphasize the nice COM interface that allow your wrapped
Fortran to be made available in your Excel macros in a snap. It happens
that Fortran programmers/users tends to be poor Office users except for
Excel which they master at unbelievable level...
My own best low work/high user satisfaction ever is just this, wrap
LOWTRAN call to make is usable from Excel, a 1/2h work and a 100+ users
2 days later !

Feb 28 '06 #9
Michael Tobis skrev:

Being a scientist, I can tell you that your not getting it right. If
you speak computer science or business talk no scientist are going to
listen. Lets just see how you argue:
These include: source and version control and audit trails for runs,
build system management, test specification, deployment testing (across
multiple platforms), post-processing analysis, run-time and
asynchronous visualization, distributed control and ensemble
management.
At this point, no scientist will no longer understand what the heck you
are talking about. All have stopped reading and are busy doing
experiments in the laboratory instead. Perhaps it sound good to a CS
geek, but not to a busy researcher.

Typically a scientist need to:

1. do a lot of experiments

2. analyse the data from experiments

3. run a simulation now and then

Thus, we need something that is "easy to program" and "runs fast
enough" (and by fast enough we usually mean extremely fast). The tools
of choice seems to be Fortran for the older professors (you can't teach
old dogs new tricks) and MATLAB (perhaps combined with plain C) for the
younger ones (that would e.g. be yours truly). Hiring professional
programmers are usually futile, as they don't understand the problems
we are working with. They can't solve problems they don't understand.

What you really ned to address is something very simple:
Why is Python better a better Matlab than Matlab?
The programs we need to write typically falls into one of three
categories:

1. simulations
2. data analysis
3. experiment control and data aquisition

(that are words that scientists do know)

In addition, there are 10 things you should know about scientific
programming:

1. Time is money. Time is the only thing that a scientist cannot afford
to lose. Licensing fees for Matlab is not an issue. If we can spend
$1,000,000 on specialised equipment we can pay whatever Mathworks or
Lahey charges as well. However, time spent programming are an issue.
(As are time time spend learning a new language.)

2. We don't need fancy GUIs. GUI coding is a waste of time we don't
have. We don't care if Python have fancy GUI frameworks or not.

3. We do need fancy data plotting and graphing. We do need fancy
plotting and graphing that are easy to use - as in Matlab or S-PLUS.

4. Anything that has to do with website development or enterprise class
production quality control are crap that we don't care about.

5. Versioning control? For each program there is only one developer and
a single or a handful users.

6. The prototype is the final version. We are not making software for a
living, we are doing research.

7. "My simulation is running to slowly" is the number ONE complaint.
Speed of excecution is an issue, regardless of what computer science
folks try to tell you. That is why we spend disproportionate amount of
time learning to vectorize Matlab code.

8. "My simulation is running of of memory" is the number TWO complaint.
Matlab is notoriously known for leaking memory and fragmenting the
heap.

9. What are algorithms and data structures? Very few of us knows how to
use a datastructure more complicated than an array. That is why we like
Matlab and Fortran so much.

10. We are novice programmers. We are not passionate programmers. We
take no pride in our work. The easier hack the better. We don't care if
we are doing OOP or not. However, we do hate complicated APIs or APIs
that look funny. We are used to seeing sin(x) in our calculus textbooks
and because of that we don't find Math.Sin(x) particularly elegant --
even though Math.Sin(x) is more OOP and sin(x) clutters the global
namespace.
Now please go ahead and tell me how Python can help me become a better
scientist. And try to steer clear of the computer science buzzwords
that don't mean anyting to me.

Thanks!

Sturla Molden
(neuroscience PhD)









The synergies among these programming modes is in some ways harder to
explain than to experience. The Python novice may nevertheless observe
that a single language can take the place of shell scripts, makefiles,
desktop computation environments, compiled languages to build GUIs, and
scripting languages to build web interfaces. In addition, Python is
useful as a wrapper for Fortran modules, facilitating the
implementation of true test-driven design processes in Fortran models.

Another Python advocacy slogan is "batteries included". The point here
is that (in part because Python is dramatically easier to write than
other languages) there is a very broad range of very powerful standard
libraries that make many tasks which are difficult in other languages
astonishingly easy in Python. For instance, drawing upon the standard
libraries (no additional download required) a portable webserver
(runnable on both Microsoft and Unix-based platforms) can be
implemented in seven lines of code. (See
http://effbot.org/librarybook/simplehttpserver.htm ) Installation of
pure python packages is also very easy, and installation of mixed
language products with a Python component is generally not
significantly harder than a comparable product with no Python
component.

Among the Python components and Python bindings of special interest to
scientists are the elegant and powerful matplotlib plotting package,
which began by emulating and now surpasses the plotting features of
Matlab, SWIG, which allows for runtime interoperability with various
languages, f2py which specifically interoperates with Fortran, NetCDF
libraries (which cope with NetCDF files with dramatically less fuss
than the standard C or Fortran bindings), statistics packages including
bindings to the R language, linear algebra packages, various
platform-specific and portable GUI libraries, genetic algorithms,
optimization libraries, and bindings for high performance differential
equation solvers (notably, using the Argonne National Laboratory
package PetSC). An especially interesting Python trick for runtime
visualization in models that were not designed to support it, pioneered
by David Beazley's SWILL, embeds a web server in your model code.

See especially http://starship.python.net/~hinsen/ScientificPython/ and
http://scipy.org as good starting points to learn about scientific uses
of Python.

mt


Mar 4 '06 #10
On 3 Mar 2006 17:33:31 -0800
"sturlamolden" <st**********@yahoo.no> wrote:
1. Time is money. Time is the only thing that a scientist
cannot afford to lose. Licensing fees for Matlab is not an
issue. If we can spend $1,000,000 on specialised equipment
we can pay whatever Mathworks or Lahey charges as well.
However, time spent programming are an issue. (As are time
time spend learning a new language.)


"that man speaks for himself!" ;-)

Seriously, this depends on the lab. If you're working for
a monster pharmaceutical corp or on a military contract on
"applied" science (meaning there is a definitely payback
expected), then you likely have money to burn. People
working in a academic or non-profit lab on "unsexy"/"pure"
science, likely don't.

Remember that site-licensing usually works on some kind of
"per seat" basis (even if you are lucky enough *not* to have
a "license server" that constantly tracks usage in order to
deny service if and when N+1 users try to use the system,
the fee the site fee is still based on the number
of expected users). The last science facility I worked at
was in considerable debt to a proprietary scientific
software producer and struggling to pay the bills. The
result was that they had fewer licenses than they wanted
and many people simply couldn't use the software when they
wanted.

I'm not sure what happened in the end, because I left for
unrelated reasons before all of that got sorted out, but
Python (with a suitable array of add-ons) was definitely on
the short-list of replacement software (and partly because I
was trying to sell people on it, of course).

In fact, if I had one complaint about Python, it was the
"with a suitable array of add-ons" caveat. The proprietary
alternative had all of that rolled into one package (abeit
it glopped into one massive and arcane namespace), whereas
there was no "Python Data Language" or whatever that
would include all that in one named package that everyone
could recognize (I suppose SciPy is trying to achieve that).

For similar reasons, Space Telescope Science Institute
decided to go full tilt into python development -- they
created "numarray" and "pyraf", and they are the ones paying
for the "chaco" development contract.

Which brings up another point -- whereas with proprietary
software (and stuff written using it, like the IDL astronomy
library) can leave you with an enormous investment in stuff
you can't use, free software development can often be just
as cheap, and you get to keep what you make.

At one point, I was seriously thinking about trying to write
some kind of translator to convert those IDL libs into
python libs (quixotic of me?).

So why rent when you can own?

Scientists certainly do understand all that bit about
"seeing further" because you're "standing on the shoulders
of giants". With proprietary software, the giants keep
getting shot out from under you, which tends to make things
a bit harder to keep up with.

Cheers,
Terry

--
Terry Hancock (ha*****@AnansiSpaceworks.com)
Anansi Spaceworks http://www.AnansiSpaceworks.com

Mar 4 '06 #11
Terry Hancock <ha*****@anansispaceworks.com> wrote:
In fact, if I had one complaint about Python, it was the
"with a suitable array of add-ons" caveat. The proprietary
alternative had all of that rolled into one package (abeit
it glopped into one massive and arcane namespace), whereas
there was no "Python Data Language" or whatever that
would include all that in one named package that everyone
could recognize (I suppose SciPy is trying to achieve that).


I believe the Enthought distribution of Python (for Windows, with a Mac
version planned) is trying to move exactly in that direction, by
packaging up everything and a half (while of course leaving a reasonable
assignment of namespaces from the pieces it's packaging!-). However,
maintaining such a distro, and making it available for a wider variety
of platforms, are heavy, continuing tasks -- unless either firms, such
as Enthought, or volunteers, commit to such tasks, they won't "just
happen".
Alex
Mar 4 '06 #12
There is a range of folks doing scientific programming. Not all of them
are described correctly by your summary, but many are. The article is
aimed not at them, but rather at institutions that develop engineered
Fortran models using multipuurpose teams and formal methods. I
appreciate your comments, because I see that there should be another
article aimed at desktop programmers.

One of the things python addresses best is the division of labor, where
the subtle concepts are available to those who need them and hidden
from those who don't need them. From what I understand of your work
(and what I have seen of the work of two other neuroscientists,
actually) Python would be a good choice for you.

That said, the level of computational skill in many scientists is
alarming. Why do we expect to spend six semesters learning mathematics
and expect to pick up computing "on the side"? It baffles me. Frankly,
saying "I don't need version control" sounds to me no less foolish than
saying "I don't need logarithms". (Perhaps you don't but someday soon
you will.)

"Speed of excecution is an issue, regardless of what computer science
folks try to tell you." strikes me as nothing short of hallucinatory.
No informed person says that speed is never an issue, and a great deal
of effort is spent on speed. Where do you suppose your Fortran
compiler came from in the first place?

For someone without legacy code to worry about, fussing with Fortran
for single-user one-off codes strikes me as a weak choice. If you are
hitting Matlab's performance or memory limits, you should take the time
to learn something about computation, not because you are idle, but
because you are busy. Or if you prefer, because your competitors will
be learning how to be more productive while you put all your efforts
into coping with crude tools.

The peculiar lack of communication between computer scientists and
application scientists is real; but I believe the fault is not all on
one side. The fact that you have a PhD does not prove that you know
everything you need to know, and I strongly recommend you reconsider
this attitude. For one thing, you misjudged which side of the divide I
started on.

Michael Tobis
(While I dislike credentialism on usenet, I will reply in kind. I hold
a Ph.D. in geophysical fluid dynamics.)

Mar 4 '06 #13
sturlamolden wrote:

Typically a scientist need to:

1. do a lot of experiments

2. analyse the data from experiments

3. run a simulation now and then

unless you are a theorist! in that case, I would order this list backwards.


1. Time is money. Time is the only thing that a scientist cannot afford
to lose. Licensing fees for Matlab is not an issue. If we can spend
$1,000,000 on specialised equipment we can pay whatever Mathworks or
Lahey charges as well. However, time spent programming are an issue.
(As are time time spend learning a new language.)

Another person stated that they don't have infinite funds, as implied here. I would
add that, in addition to one's own research, professors must also teach and advise.
I find it very helpful to be able to say to a student, "go download this, and here is
the code I wrote for the work I do". The price is often an impediment for getting
students into research. Often there are site licenses, but they don't work off campus.

2. We don't need fancy GUIs. GUI coding is a waste of time we don't
have. We don't care if Python have fancy GUI frameworks or not.

again, for sharing ideas, GUIs are *necessary*. If you work with people who do less
programming than you, then you need to make an interface to your code that they can
understand. it doesn't have to be fancy, just functional.
3. We do need fancy data plotting and graphing. We do need fancy
plotting and graphing that are easy to use - as in Matlab or S-PLUS.

here, I've found python to be good, but not great. matplotlib (pylab) is a really
great thing, but is not as straightforward as plotting in Matlab. Either, you have a
window which locks the process until you close it, or you do interactive mode, but
the window results disappear if any other window is put on top (like your shell), and
has to be manually redrawn. This makes it far less convenient to deal with in
interactive mode.
4. Anything that has to do with website development or enterprise class
production quality control are crap that we don't care about.

I think it can be pitched as an alternative to shell-scripts, which is a nice economy
of concepts: the language you use for your scientific work, you can also use for your
OS work, and your tinkering.

7. "My simulation is running to slowly" is the number ONE complaint.
Speed of excecution is an issue, regardless of what computer science
folks try to tell you. That is why we spend disproportionate amount of
time learning to vectorize Matlab code.

here, I would plug Pyrex like crazy. to me the Python/Pyrex combination is the
biggest selling point for me to convert my scientific matlab code to Python.
Learning a new API is a drag, and I've found that SWIG is not particularly intuitive
(although convenient, if you have a lot of libraries already written). Pyrex seems
to get the best of all possible worlds: seamless use of python objects, and the
ability to do C-loops for speed, with no API. Making extensions this way is a real joy.

Now please go ahead and tell me how Python can help me become a better
scientist. And try to steer clear of the computer science buzzwords
that don't mean anyting to me.

I have been using Matlab for nearly 10 years. My claim to no-fame is the neural
network simulator Plasticity (http://web.bryant.edu/~bblais/plasticity) which has
taken me years to write. I have some complaints about Matlab, but it has been a
useful tool. Some of my complaints are as follows:

1) Cost. I find that the marketing model for Matlab is annoying. They
nickle-and-dime you, with the base package (educational pricing) at $500 per
machine/operating system/user and then between $200-$500 *per* "toolbox", which adds
up really quick. I can't even buy 1 license for a dual boot, to have Matlab run on
a Linux partition and a Windows partition.

The cost impacts my use of Matlab in the classroom, and with research students.

2) License Manager. The license manager for Matlab is the most inconvenient program
I have ever dealt with. It is incredibly sensitive to the license file, and it
nearly impossible to debug. This has made Matlab one of the hardest programs to
install, for me. The issue that impacts my productivity is the following: the
license key is tied to the network card, on eth0. Thus, if I upgrade my laptop, I
need to contact Mathworks for an updated license key. Also, occasionally, my
operating system decides to name my wireless card eth0, and my wired card eth1.
Nothing else is affected by this, but then I can't run Matlab!

3) Upgrade Version Hell. *Every* time Matlab has upgraded, my program has broken.
Usually something small, but still it is a real pain in the butt. Also, I have to
pay full price for the upgrade, or pay some fee continuously whether there is an
upgrade or not.

I have only been using Python for about 2 months, so I can't speak to some issues,
but what does Python offer me?

1) Free (as in free beer). I've elaborated on this above.

2) Free (as in free speech). I like the fact that I am not burdened by having my
projects tied to something proprietary.

3) Distribution ease. With py2exe, I can distribute on Windows systems which have no
python installed. That's a real plus!

4) Clean programming environment. For teaching, it is nice to use a language which
is so readable.

5) A huge number of built-in, or available, packages for nearly everything.

6) The ability to write portions of code in an optimized, as-fast-as-C, manner.

7) Relatively easy GUI frameworks

I'm sure there are other things, but that's the way I am thinking right now.
--
-----------------

bb****@bryant.edu
http://web.bryant.edu/~bblais

Mar 4 '06 #14
sturlamolden wrote:
Michael Tobis skrev:

Being a scientist, I can tell you that your not getting it right. If
you speak computer science or business talk no scientist are going to
listen. Lets just see how you argue:
I see we've forgone the standard conventions of politeness and gone straight for
the unfounded assumptions. Fantastic.
These include: source and version control and audit trails for runs,
build system management, test specification, deployment testing (across
multiple platforms), post-processing analysis, run-time and
asynchronous visualization, distributed control and ensemble
management.


At this point, no scientist will no longer understand what the heck you
are talking about. All have stopped reading and are busy doing
experiments in the laboratory instead. Perhaps it sound good to a CS
geek, but not to a busy researcher.

Typically a scientist need to:

1. do a lot of experiments

2. analyse the data from experiments

3. run a simulation now and then


Being a one-time scientist, I can tell you that you're not getting it right. You
have an extremely myopic view of what a scientist does. You seem to be under the
impression that all scientists do exactly what you do. When I was in the
geophysics program at Scripps Institution of Oceanography, almost no one was
doing experiments. The closest were the people who were building and deploying
instruments. In our department, typically a scientist would

1. Write grant proposals.

2. Advise and teach students.

3. Analyze the data from the last research cruise/satellite passover/earthquake.

4. Do some simulations.

5. Write a lot of code to do #3 and #4.

There are whole branches of science where the typical scientist usually spends a
lot of his time in #5. Michael Tobis is in one of those branches, and his
article was directed to his peers. As he clearly stated.

You are not from one of those branches, and you have different needs. That's
fine, but please don't call the kettle black.
Thus, we need something that is "easy to program" and "runs fast
enough" (and by fast enough we usually mean extremely fast). The tools
of choice seems to be Fortran for the older professors (you can't teach
old dogs new tricks) and MATLAB (perhaps combined with plain C) for the
younger ones (that would e.g. be yours truly). Hiring professional
programmers are usually futile, as they don't understand the problems
we are working with. They can't solve problems they don't understand.
I call shenanigans. Believe me, I would love it if it were true. I make my
living writing scientific software. For a company where half of us have science
degrees (myself included) and the other half have CS degrees, it would be great
advertising to say that none of those other companies could ever understand the
problems scientists face. But it's just not true.

Scientists are an important part of the process, certainly. They're called
"customers." Their needs drive the whole process. The depth and breadth of their
knowledge of the field and the particular problem are necessary to write good
scientific software. But it usually doesn't take particularly deep or broad
knowledge to write a specific piece of software. Once the scientist can reduce
the problem to a set of requirements, the CS guys are a perfect fit. That's what
a good professional programmer does: take requirements and produce software that
fulfills those requirements. They do the same thing regardless of the field. In
my company, everyone pulls their weight, even the person with the philosphy degree.

At that point, the CS skillset is perfectly suited to writing good scientific
software. Or at least, any given CS-degree person is no less likely to have the
appropriate skillset than a science-degree person. Frequently, they have a much
broader and deeper skillset that is actually useful to writing scientific
software. Most of the scientists I know couldn't write robust floating point
code to save his life. Or his career.
What you really ned to address is something very simple:

Why is Python better a better Matlab than Matlab?

The programs we need to write typically falls into one of three
categories:

1. simulations
2. data analysis
3. experiment control and data aquisition

(that are words that scientists do know)

In addition, there are 10 things you should know about scientific
programming:

1. Time is money. Time is the only thing that a scientist cannot afford
to lose. Licensing fees for Matlab is not an issue. If we can spend
$1,000,000 on specialised equipment we can pay whatever Mathworks or
Lahey charges as well. However, time spent programming are an issue.
(As are time time spend learning a new language.)

2. We don't need fancy GUIs. GUI coding is a waste of time we don't
have. We don't care if Python have fancy GUI frameworks or not.
Uh, time is money? Fighting unusable interfaces, GUI or otherwise, is a waste of
resources. My brother works in biostatistics at the NIH. Every once in a while,
the doctors he works for will ask him to do a particular analysis which requires
him to use a particularly unusable piece of software. Every time, he has to
spend half a day setting up the problem. This is why he's the one who gets to do
it instead of the doctors.

Now, he's considering rewriting the program in Python with a GUI that will
essentially provide a Big Red Go Button (TM) so the doctors can do the analysis
in a fraction of the time it takes now.
3. We do need fancy data plotting and graphing. We do need fancy
plotting and graphing that are easy to use - as in Matlab or S-PLUS.

4. Anything that has to do with website development or enterprise class
production quality control are crap that we don't care about.
There are quite a few scientists who are managing gigantic amounts of data, and
run experiments/observations/whate-have-you so large that they need
multi-institutional participation. Sharing that data in an efficient manner
*does* require good dynamic websites and enterprise class software backing it up.

There are more kinds of scientist in Heaven and Earth than are dreamt of in your
philosophy.
5. Versioning control? For each program there is only one developer and
a single or a handful users.
I used to think like that up until two seconds before I entered this gem:

$ rm `find . -name "*.pyc"`

Okay, I didn't type it exactly like that; I was missing one character. I'll let
you guess which.

This is one thing that a lot of people seem to get wrong: version control is not
a burden on software development. It is a great enabler of software development.
It helps you get your work done faster and easier even if you are a development
team of one. You can delete code (intentionally!) because it's not longer used
in your code, but you won't lose it. You can always look at your history and get
it again. You can make sweeping changes to your code, and if that experiment
fails, you can go back to what was working before. Now you can do this by making
copies of your code, but that's annoying, clumsy, and more effort than it's
worth. Version control makes the process easier and lets you do more interesting
things.

I would go so far as to say that version control enables the application of the
scientific method to software development. When you are in lab, do you say to
yourself, "Nah, I won't write anything in my lab notebook. If the experiment
works at the end of the day, only that result matters"?
6. The prototype is the final version. We are not making software for a
living, we are doing research.
I have lots of research code on my harddrive with decade-long changelogs that
give the lie to that statement. If the code is useful now, it will probably
still be useful in a few years. People will add to it, make suggestions, build
on your work.

This is how science is supposed to work. Practices which encourage this behavior
are good things for science.
7. "My simulation is running to slowly" is the number ONE complaint.
Speed of excecution is an issue, regardless of what computer science
folks try to tell you. That is why we spend disproportionate amount of
time learning to vectorize Matlab code.

8. "My simulation is running of of memory" is the number TWO complaint.
Matlab is notoriously known for leaking memory and fragmenting the
heap.

9. What are algorithms and data structures? Very few of us knows how to
use a datastructure more complicated than an array. That is why we like
Matlab and Fortran so much.
Yes, and this is why you will keep saying, "My simulation is running too
slowly," and "My simulation is running out of memory." All the vectorization you
do won't make a quadratic algorithm run in O(n log(n)) time. Knowing the right
algorithm and the right data structures to use will save you programming time
and execution time. Time is money, remember, and every hour you spend tweaking
Matlab code to get an extra 5% of speed is just so much grant money down the drain.

That said, we have an excellent array object far superior to Matlab's.

http://numeric.scipy.org/
10. We are novice programmers. We are not passionate programmers. We
take no pride in our work. The easier hack the better. We don't care if
we are doing OOP or not. However, we do hate complicated APIs or APIs
that look funny. We are used to seeing sin(x) in our calculus textbooks
and because of that we don't find Math.Sin(x) particularly elegant --
even though Math.Sin(x) is more OOP and sin(x) clutters the global
namespace.

Now please go ahead and tell me how Python can help me become a better
scientist. And try to steer clear of the computer science buzzwords
that don't mean anyting to me.


1. You will probably spend less time writing and running software.

2. If you play your cards right, more people will be able to use and improve
your software.

--
Robert Kern
ro*********@gmail.com

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

Mar 5 '06 #15
Brian Blais wrote:
here, I've found python to be good, but not great. matplotlib (pylab) is a really
great thing, but is not as straightforward as plotting in Matlab. Either, you have a
window which locks the process until you close it, or you do interactive mode, but
the window results disappear if any other window is put on top (like your shell), and
has to be manually redrawn. This makes it far less convenient to deal with in
interactive mode.


All I can say is that I've never seen anything like that behavior. Are you tried
using ipython in pylab mode?

--
Robert Kern
ro*********@gmail.com

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

Mar 5 '06 #16
Robert Kern wrote:

That said, we have an excellent array object far superior to Matlab's.

http://numeric.scipy.org/


I'd like to ask, being new to python, in which ways is this array object far superior
to Matlab's? (I'm not being sarcastic, I really would like to know!)

I've heard similar things about matplotlib, about how it surpasses Matlab's graphics.
I haven't personally experienced this, but I'd like to know in which ways it is.

bb

--
-----------------

bb****@bryant.edu
http://web.bryant.edu/~bblais

Mar 5 '06 #17

Robert Kern wrote:
1. Write grant proposals.

2. Advise and teach students.
Sorry I forgot the part about writing grant applications. As for
teaching students, I have thankfully not been bothered with that too
much.
Yes, and this is why you will keep saying, "My simulation is running too
slowly," and "My simulation is running out of memory." All the vectorization you
do won't make a quadratic algorithm run in O(n log(n)) time. Knowing the right
algorithm and the right data structures to use will save you programming time
and execution time. Time is money, remember, and every hour you spend tweaking
Matlab code to get an extra 5% of speed is just so much grant money down the drain.


Yes, and that is why I use C (that is ISO C99, not ANSI C98) instead of
Matlab for everything except trivial tasks. The design of Matlab's
language is fundamentally flawed. I once wrote a tutorial on how to
implement things like lists and trees in Matlab (using functional
programming, e.g. using functions to represent list nodes), but it's
just a toy. And as Matlab's run-time does reference counting insted of
proper garbage collection, any datastructure more complex than arrays
are sure to leak memory (I believe Python also suffered from this as
some point). Matlab is not useful for anything except plotting data
quickly. And as for the expensive license, I am not sure its worth it.
I have been considering a move to Scilab for some time, but it too
carries the burden of working with a flawed language.

Mar 5 '06 #18
> $ rm `find . -name "*.pyc"`

Ouch. Is that a true story?

While we're remeniscing about bad typos and DEC, I should tell the
story about the guy who clobberred his work because his English wasn't
very strong.

Under RT-11, all file management was handled by a program called PIP.
For example to get a list of files in the current working directory you
would enter PIP *.* /LI . Well this fellow, from one of those countries
where our long "e" sound is their "i" sound, mournfully announced

"I vanted a deerectory so I typed 'PIP *.* /DE' "

That one is true.

mt

Mar 5 '06 #19
Michael Tobis wrote:
$ rm `find . -name "*.pyc"`


Ouch. Is that a true story?


Yup. Fortunately, it was a small, purely personal project, so it was no huge
loss. It was enough for me to start using CVS on my small, purely personal
projects, though!

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Mar 5 '06 #20
sturlamolden <st**********@yahoo.no> wrote:
just a toy. And as Matlab's run-time does reference counting insted of
proper garbage collection, any datastructure more complex than arrays
are sure to leak memory (I believe Python also suffered from this as
some point).
Yes, that was fixed in the move from 1.5.2 to 2.0 (I don't recall if the
intermediate short-lived 1.6 also fixed it, but nobody used that
anyway;-).
Matlab is not useful for anything except plotting data
quickly. And as for the expensive license, I am not sure its worth it.
I have been considering a move to Scilab for some time, but it too
carries the burden of working with a flawed language.


There was a pyscilab once, still around at
<http://pdilib.sourceforge.net/>, but I don't think it ever matured
beyond a "proof of concept" release 0.1 or something.
Alex
Mar 5 '06 #21
sturlamolden wrote:
Robert Kern wrote:
Yes, and this is why you will keep saying, "My simulation is running too
slowly," and "My simulation is running out of memory." All the vectorization you
do won't make a quadratic algorithm run in O(n log(n)) time. Knowing the right
algorithm and the right data structures to use will save you programming time
and execution time. Time is money, remember, and every hour you spend tweaking
Matlab code to get an extra 5% of speed is just so much grant money down the drain.


Yes, and that is why I use C (that is ISO C99, not ANSI C98) instead of
Matlab for everything except trivial tasks. The design of Matlab's
language is fundamentally flawed. I once wrote a tutorial on how to
implement things like lists and trees in Matlab (using functional
programming, e.g. using functions to represent list nodes), but it's
just a toy. And as Matlab's run-time does reference counting insted of
proper garbage collection, any datastructure more complex than arrays
are sure to leak memory (I believe Python also suffered from this as
some point).


Python still uses reference counting and has several very good data structures
more complex than arrays. And yet, most programs don't leak memory.
Matlab is not useful for anything except plotting data
quickly. And as for the expensive license, I am not sure its worth it.
I have been considering a move to Scilab for some time, but it too
carries the burden of working with a flawed language.


And you need to ask why Python is a better Matlab than Matlab?

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Mar 5 '06 #22
Brian Blais wrote:
Robert Kern wrote:
That said, we have an excellent array object far superior to Matlab's.

http://numeric.scipy.org/


I'd like to ask, being new to python, in which ways is this array object far superior
to Matlab's? (I'm not being sarcastic, I really would like to know!)


Matlab takes the view that "everything is a rank-2 matrix of floating point values."

Our arrays have been N-dimensional since day one. They really are arrays, not
matrices. You have complete control over the types.

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Mar 5 '06 #23
Robert Kern wrote:
And you need to ask why Python is a better Matlab than Matlab?

First there are a few things I don't like:

1. Intendation as a part of the syntax, really annoying.

2. The "self.something" syntax is really tedious (look to ruby)!

4. Multithreading and parallel execution is impossible AFAIK because of
the so-called GIL (global interpreter lock). Matlab is perhaps even
worse in this respect.

5. I don't like numpy's array slicing. Array operations should be a
part of the language, as in Matlab, Fortran 90, Ada 95, D, Octave.
And there is a couple of questions I need answered:

1. Can python do "pass by reference"? Are datastructures represented by
references as in Java (I don't know yet).

2. How good is matplotlib/pylab? I tried to install it but only get
error messages so I haven't tested it. But plotting capabilities is
really major issue.

3. Speed. I haven't seen any performance benchmarks that actually deals
with things that are important for scientific programs.

4. Are there "easy to use" libraries containing other stuff important
for scientific programs, e.q. linear algebra (LU, SVD, Cholesky),
Fourier transforms, etc. E.g. in Matlab I can just type,

[u,s,v] = svd(x) % which calls LAPACK linked to ATLAS or
vendor-optimized BLAS

Even though the language itself is very limited this type of library
functionality more than makes up for it.
I have looked for alternatives to Matlab for quite a while, mainly due
to the cost, the ĺpoor speed and poor memory management. I am not sure
it is Python but so far I have not found anything mor promising either.

Mar 5 '06 #24
sturlamolden wrote:
First there are a few things I don't like:

1. Intendation as a part of the syntax, really annoying.
Each to his own. I find having the compiler enforce indentation rules is a
real benefit. There's nothing quite so annoying as having 'coding
standards' you are supposed to be following which aren't enforced (so
nobody else working on the code followed them either). C code which never
executes a for loop because of a spurious ';' can be pretty annoying too.

2. The "self.something" syntax is really tedious (look to ruby)!
I find it really useful to use a similar style when programming in other
languages such as Java or C#. In particular it means that you can
instantly see when you are referring to a member variable (without
having to prefix them all with m_ or some other abomination) and when you
have method arguments which get assigned to member variables you don't have
to think of different names for each.

4. Multithreading and parallel execution is impossible AFAIK because of
the so-called GIL (global interpreter lock). Matlab is perhaps even
worse in this respect.
Multithreading and parallel execution work fine. The only problem is that
you don't get the full benefit when you have multiple processors. This one
will become more of an annoyance in the future though as more systems have
hyperthreading and multi-core processors.

And there is a couple of questions I need answered:

1. Can python do "pass by reference"? Are datastructures represented by
references as in Java (I don't know yet).

Python only does "pass by reference", although it is more normally referred
to as "pass by object reference" to distinguish it from language where the
references refer to variables rather than objects.

What it doesn't do is let you rebind a variable in the caller's scope which
is what many people expect as a consequence of pass by reference. If you
pass an object to a function (and in Python *every* value is an object)
then when you mutate the object the changes are visible to everything else
using the same object. Of course, some objects aren't mutable so it isn't
that easy to tell that they are always passed by reference.
Mar 5 '06 #25
Dennis Lee Bieber wrote:
1. Can python do "pass by reference"? Are datastructures represented by
references as in Java (I don't know yet).

Everything in Python is a reference to an object. I think the
question you want is more on the lines of: Can I change an object that
has been passed? Short answer: depends on what type of object is at the
end of the reference. A mutable container object (list, dictionary,
maybe a class instance) can have its contents changed.


Thank you.

Mar 5 '06 #26
> First there are a few things I don't like:

Hi, I will respond to things that others haven't responded yet
2. How good is matplotlib/pylab? I tried to install it but only get
error messages so I haven't tested it. But plotting capabilities is
really major issue.
I don't know because I haven't managed to get it working either. But
other people have and I guess it should not be so difficult.

I personally use gnuplot-py (gnuplot-py.sourceforge.net) which I adapted
to the new NumPy/SciPy (see below) by searching-and-replacing "Numeric"
by "numpy". It allows to use raw gnuplot commands.
4. Are there "easy to use" libraries containing other stuff important
for scientific programs, e.q. linear algebra (LU, SVD, Cholesky),
Fourier transforms, etc. E.g. in Matlab I can just type,

[u,s,v] = svd(x) % which calls LAPACK linked to ATLAS or
vendor-optimized BLAS


Yes !

There is the excellent SciPy package, which you can get at www.scipy.org

Personnally I use it a lot for linear algebra (linked to
LAPACK/ATLAS/BLAS), but there are also libraries for statistics,
optimization, signal processing, etc.

There has been many changes recently, including package names, so don't
get confused and be sure to get recent versions of NumPy and SciPy ;).

Evan
Mar 6 '06 #27
Robert Kern <ro*********@gmail.com> wrote:
...
just a toy. And as Matlab's run-time does reference counting insted of
proper garbage collection, any datastructure more complex than arrays
are sure to leak memory (I believe Python also suffered from this as
some point).


Python still uses reference counting and has several very good data structures
more complex than arrays. And yet, most programs don't leak memory.


Python uses reference counting *AND* cyclic-garbage collection for the
kind of garbage that wouldn't go away by RC due to reference-cycles
(plus, also, weak references for yet another helper). To leak memory
despite all of that, you really need to do it on purpose (e.g. via a
C-coded container extension-type that does NOT play nice with gc;-).
Alex
Mar 6 '06 #28
sturlamolden wrote:
Robert Kern wrote:

And you need to ask why Python is a better Matlab than Matlab?
First there are a few things I don't like:

1. Intendation as a part of the syntax, really annoying.

Troll. You think this is going away because *you* don't like it? Am I to
presume that you don't bother to indent your C code according to its
nested block structure? If you *do* indent your C code, perhaps you can
explain the additional benefits of the braces?
2. The "self.something" syntax is really tedious (look to ruby)!
This is done because of a preference from explicit references over
implied ones. It does avoid a lot of namespace confusion.

By the way, anyone who can't count shouldn't be criticising programming
languages. What happened to "3"?
4. Multithreading and parallel execution is impossible AFAIK because of
the so-called GIL (global interpreter lock). Matlab is perhaps even
worse in this respect.
Right. So kindly tell us how to write thread-safe code without using a
GIL. This is not an easy problem, and you shouldn't assume that all you
have to do to get rid of the GIL is to wave your magic wand. There are
deep reasons why the GIL is there.
5. I don't like numpy's array slicing. Array operations should be a
part of the language, as in Matlab, Fortran 90, Ada 95, D, Octave.
Slicing *is* a part of the language, inserted into the grammar (as far
as I know) precisely to support the numeric/scientific community.
And there is a couple of questions I need answered:

1. Can python do "pass by reference"? Are datastructures represented by
references as in Java (I don't know yet).
All assignments store references.
2. How good is matplotlib/pylab? I tried to install it but only get
error messages so I haven't tested it. But plotting capabilities is
really major issue.
Good enough to keep you away, apparently ;-) (Sorry, I don't use these
features).
3. Speed. I haven't seen any performance benchmarks that actually deals
with things that are important for scientific programs.
The major fact here is that no matter how fast a language is there is
always a need for more speed in certain areas.

Suffice it to say that Python is being used for a wide range of
scientific and engineering problems to the evident satisfaction of its
users.
4. Are there "easy to use" libraries containing other stuff important
for scientific programs, e.q. linear algebra (LU, SVD, Cholesky),
Fourier transforms, etc. E.g. in Matlab I can just type,

[u,s,v] = svd(x) % which calls LAPACK linked to ATLAS or
vendor-optimized BLAS

Even though the language itself is very limited this type of library
functionality more than makes up for it.
The more people who join in and write libraries to add to the growing
corpus of scientific and engineering libraries the sooner the answer to
this question will be "we have everything you want".

For the moment, however, since apparently Google isn't available where
you are, a quick search for "Python LAPACK" gave

http://mdp-toolkit.sourceforge.net/faq.html

as its first hit. This appears to include information about how to have
LAPACK make use of ATLAS' faster LAPACK routines. Satisfied?

I have looked for alternatives to Matlab for quite a while, mainly due
to the cost, the ĺpoor speed and poor memory management. I am not sure
it is Python but so far I have not found anything mor promising either.


You know, recently the Python community has acquired a reputation in
certain quarters for defensive support of the status quo. With
ill-informed criticism like this from self-confessed beginners it's not
hard to see how this myth has arisen.

I'd be very surprised if Python doesn't already give you 95% of what you
appear to want. If you prejudices about indented code and self-relative
references blind you to the clear advantages of the Python environment
then frankly you are a lost cause, and good riddance.

If, on the other hand, you are prepared to engage the community and do a
little bit of learning rather than just trolling, you may find that one
of the most valuable features of Python is its supportive user base,
whom at the moment you seem to be doing your best to offend.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd www.holdenweb.com
Love me, love my blog holdenweb.blogspot.com

Mar 6 '06 #29
1) indentation:

I claim that this is a trivial matter. Indentation is enforced, but if
you want to end all your blocks with #end, feel free.

Or write a preprocessor to go from your preferred block style to
python's

2) self.something tedious to look at.

Again, you can somewhat work around this if you like. Just use "s" or
"_" instead of "self".

3) missing

4) multithreading and parallel execution impossible

This is simply false, though admittedly the MPI libraries need a little
work. Mike Steder of our group is likely to release an alternative. A
good way to think of Python is as a powerful extension to C. So using
MPI from Python just amounts to setting up a communicator in C and
wrapping the MPI calls.

As for less tightly coupled threads on a single processor, Python is
adept at it. I think the issues with multiple processors are much more
those of a server farm than those of a computational cluster.

We have been encountering no fundamental difficulty in cluster programs
using Python.

5) "I don't like numpy's array slicing" ?

this is unclear. It is somewhat different form Matlab's, but much more
powerful.

1) pass by reference

Python names are all references. The model is a little peculiar to
Fortran and C people, but rather similar to the Java model.

2) matplotlib

A correct install can be difficult, but once it works it rocks. ipython
(a.k.a. pylab) is also a very nice work environment.

3D plots remain unavailable at present.

3) speed

Speed matters less in Python than in other languages because Python
plays so well with others. For many applications, NumPy is fine.
Otherwise write your own C or C++ or F77; building the Python bindings
is trivial. (F9* is problematic, though in fact we do some calling of
F90 from Python using the Babel toolkit)

4) useful libraries

yes. for your svd example see Hinsen's Scientific package. In general,
Python's claim of "batteries included" applies to scientific code.

mt

Mar 6 '06 #30
Dennis Lee Bieber wrote:
I did look at Ruby once... It looked to me like the worst aspects of
PERL grafted onto the worst parts of old Python.
Don't forget that there are portions of Smalltalk syntax
(blocks) added in as well. I guess it could be seen as Perl-NG.
Both the name 'Ruby' and the Ruby syntax seems to suggest that
Matz had the idea to "flirt" a bit with the Perl programmers,
and considering how Perl seems to be in decline today, that
might have been clever from a user-base point of view. Whether
it was really good for the language is another issue. I still
think it's a bit prettier than Perl though.
For CPU-bound number-crunching, perhaps... For I/O-bound jobs, the
GIL is(should be) released when ever a thread is blocked waiting for I/O
to complete.
I think CPU-bound number-crunching was the big deal in this case.
Somehow, I doubt that the OP uses Matlab for I/O-bound jobs. At
least if writing threaded applications becomes less error prone
in competing languages, this might well be the weak point of Python
in the future. I hope to see some clever solution to this from the
Python developers.

It seems the Python attitude to performance has largely been:
Let Python take care of development speed, and let Moore's law
and the hardware manufacturers take care of execution speed. As
it seems now, increases in processing speed the coming years
will largely be through parallell thread. If Python can't utilize
that well, we have a real problem.
5. I don't like numpy's array slicing. Array operations should be a
part of the language, as in Matlab, Fortran 90, Ada 95, D, Octave.


Python is not primarily a mathematics language. It's not a text
processing language either, so no regexp support directly in the
syntax. That might make it less ideal as a Matlab substitute, or
as a sed or awk substitute, but on the other hand, it's useful for
so many other things...
Everything in Python is a reference to an object. I think the
question you want is more on the lines of: Can I change an object that
has been passed?


The key lies in understanding that "a=b" means "bind the local name
(unless declared global) "a" to the object the name "b" refers to.
It never means "copy the content of b into the location of a".
Mar 6 '06 #31
Magnus Lycka wrote:
Dennis Lee Bieber wrote:
I did look at Ruby once... It looked to me like the worst aspects of
PERL grafted onto the worst parts of old Python.


Don't forget that there are portions of Smalltalk syntax
(blocks) added in as well. I guess it could be seen as Perl-NG.


Actually, the Perl part was one of the last steps
in the Ruby recipe according to Matz:

Ruby is a language designed in the following steps:

* take a simple lisp language (like one prior to CL).
* remove macros, s-expression.
* add simple object system (much simpler than CLOS).
* add blocks, inspired by higher order functions.
* add methods found in Smalltalk.
* add functionality found in Perl (in OO way).

So, Ruby was a Lisp originally, in theory.
Let's call it MatzLisp from now on. ;-)

--from http://ruby-talk.org/cgi-bin/scat.rb...by-talk/179642

Regards,
--
Bil
http://fun3d.larc.nasa.gov
Mar 6 '06 #32
Magnus Lycka wrote:
Dennis Lee Bieber wrote: [SNIP]
For CPU-bound number-crunching, perhaps... For I/O-bound jobs, the
GIL is(should be) released when ever a thread is blocked waiting for I/O
to complete.

I think CPU-bound number-crunching was the big deal in this case.
Somehow, I doubt that the OP uses Matlab for I/O-bound jobs. At
least if writing threaded applications becomes less error prone
in competing languages, this might well be the weak point of Python
in the future. I hope to see some clever solution to this from the
Python developers.


I don't disagree with this, but it's largely irrelevant to CPU-bound
number-crunching using numpy and its bretheren. In that case the bulk of
the work is going on in the array extensions, in C, and thus the GIL
should be released. Whether it actually is released, I can't say --
never having been blessed/cursed with a multiproccessing box, I haven't
looked into it.

[SNIP]

-tim

Mar 6 '06 #33
Michael Tobis wrote:
3) speed

Speed matters less in Python than in other languages because Python
plays so well with others. For many applications, NumPy is fine.
Otherwise write your own C or C++ or F77; building the Python bindings
is trivial. (F9* is problematic, though in fact we do some calling of
F90 from Python using the Babel toolkit)


I have met no problems using F90 together with f2py -- in fact usually it
can bind f90 code to python completely automatically with no need to write
extra glue code.

The only problem was setting up Intel Fortran compiler and making it play
along with f2py, but I suppose the compiler business will become easier
when gfortran matures.

--
Pauli Virtanen

Mar 6 '06 #34
> I have met no problems using F90 together with f2py

Thank you for the correction. I should have qualified my statement.

Our group must cope with F90 derived types to wrap a library that we
need. f2py fails to handle this case. While the f2py site alleges that
someone is working on this, I contacted the fellow and he said it was
on hold.

To our knowledge only the (officially unreleased) Babel toolkit can
handle F9* derived types. I would be pleased to know of alternatives.
To be fair, Babel remains in development, and so perhaps it will become
less unwieldy, and the developers have been very helpful to us. Still,
it certainly is not as simple a matter as f2py or swig.

mt

Mar 6 '06 #35

"Magnus Lycka" <ly***@carmen.se> wrote in message
news:du**********@wake.carmen.se...
It seems the Python attitude to performance has largely been:
Let Python take care of development speed, and let Moore's law
and the hardware manufacturers take care of execution speed.
I think this is pretty fair, and yet .... the core Python interpreter has
perhaps doubled in speed (hardware held constant) since some years ago.
And new builtins like enumerate speed up code that needs to enumerate
sequence items (which is not uncommon). And class sets.Set is rewritten in
C as builtin type set primarily for speed.

It is certainly true that Guido regards continued correct performance of
the interperter to be more important that greater speed. Ditto for Python
programs.
it seems now, increases in processing speed the coming years
will largely be through parallell thread. If Python can't utilize
that well, we have a real problem.
I believe it is Guido's current view, perhaps Google's collective view, and
a general *nix view that such increases can just as well come thru parallel
processes. I believe one can run separate Python processes on separate
cores just as well as one can run separate processes on separate chips or
separate machines. Your view has also been presented and discussed on the
pydev list. (But I am not one for thread versus process debate.)
At
least if writing threaded applications becomes less error prone
in competing languages, this might well be the weak point of Python
in the future.
Queue.Queue was added to help people write correct threaded programs.
I hope to see some clever solution to this from the Python developers.


A Python developer is one who helps develop Python. Threading improvements
will have to come from those who want it enough to contribute to the
effort. (There have been some already.)

Terry Jan Reedy


Mar 6 '06 #36
Duncan Booth schrieb:
sturlamolden wrote:
1. Can python do "pass by reference"? Are datastructures represented by
references as in Java (I don't know yet).

Python only does "pass by reference", although it is more normally referred
to as "pass by object reference" to distinguish it from language where the
references refer to variables rather than objects.

What it doesn't do is let you rebind a variable in the caller's scope which
is what many people expect as a consequence of pass by reference. If you
pass an object to a function (and in Python *every* value is an object)
then when you mutate the object the changes are visible to everything else
using the same object. Of course, some objects aren't mutable so it isn't
that easy to tell that they are always passed by reference.


This is hard to understand for an outsider. If you pass an int, a float,
a string or any other "atomic" object to a function you have "pass by
value" semantics. If you put a compound object like a list or a dictionary
or any other object that acts as an editable data container you can return
modified *contents* (list elements etc.) to the caller, exactly like in
Java and different from C/C++.

Peter Maas, AAchen
Mar 6 '06 #37
Steve Holden schrieb:
sturlamolden wrote:
First there are a few things I don't like:

1. Intendation as a part of the syntax, really annoying.

Troll. You think this is going away because *you* don't like it?


You are over-reacting. Keep in mind that sturlamolden has criticized
Python and not you :) I think there is a more convincing reply to
indentation phobia:

It is natural that compiler and programmer agree on how to identify
block structures. Anybody who disagrees should bang his code against
the left side or put everything in one line to get rid of annoying
line breaks. :)

Peter Maas, AAchen
Mar 6 '06 #38
sturlamolden wrote:
Robert Kern wrote:
1. Write grant proposals.

2. Advise and teach students.

Sorry I forgot the part about writing grant applications. As for
teaching students, I have thankfully not been bothered with that too
much.
Yes, and this is why you will keep saying, "My simulation is
running too slowly," and "My simulation is running out of memory."
All the vectorization you do won't make a quadratic algorithm run
in O(n log(n)) time. Knowing the right algorithm and the right data
structures to use will save you programming time and execution
time. Time is money, remember, and every hour you spend tweaking
Matlab code to get an extra 5% of speed is just so much grant money
down the drain.

Yes, and that is why I use C (that is ISO C99, not ANSI C98) instead
of Matlab for everything except trivial tasks. The design of Matlab's
language is fundamentally flawed. I once wrote a tutorial on how to
implement things like lists and trees in Matlab (using functional
programming, e.g. using functions to represent list nodes), but it's
just a toy. And as Matlab's run-time does reference counting insted
of proper garbage collection, any datastructure more complex than
arrays are sure to leak memory (I believe Python also suffered from
this as some point). Matlab is not useful for anything except
plotting data quickly. And as for the expensive license, I am not
sure its worth it. I have been considering a move to Scilab for some
time, but it too carries the burden of working with a flawed
language.


A quick addition to Robert's very reasonable response to you. My point
is that to *trust* a simulation *results* (no matter how fast/slow/etc
you obtained it) you have to explore and manage the "physics" or
"biology" of your code. That's where Python's readability, flexibility,
and dynamism (including on-the-fly model building/testing/correction) as
well as model introspecting and exploration capabilities are of critical
importance and sometimes the indication to a missing link. It does not
hurt to remember that the original idea (by S.Ulam) of a computer was
the idea of an *experimentation environment* (including sampling). It
does not look like the Matlab's strongest point is the feedback-driven
experimentation. Or i'm missing smth about ISO C99?
Val Bykoski
Mar 7 '06 #39
Robert Kern <ro*********@gmail.com> writes:
sturlamolden wrote:
5. Versioning control? For each program there is only one developer and
a single or a handful users.


I used to think like that up until two seconds before I entered this gem:

$ rm `find . -name "*.pyc"`

Okay, I didn't type it exactly like that; I was missing one character. I'll let
you guess which.


I did that once. I ended up having to update decompyle to run with
Python 2.4 :-) Lost comments and stuff, but the code came out great.

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
Mar 7 '06 #40
Michael Tobis:
Python plays so well with others. For many applications, NumPy is fine.
Otherwise write your own C or C++ or F77; building the Python bindings
is trivial.


On Windows I have found that creating such bindings is very very
difficult... I have succed only partially, with C++, and I've had to
compile Python with MinGW (a compilation succed at about 95%). It's not
easy and surely not trivial (for me).

Bye,
bearophile

Mar 7 '06 #41

"Steve Holden" <st***@holdenweb.com> wrote in message
news:ma***************************************@pyt hon.org...
sturlamolden wrote:
Robert Kern wrote:

And you need to ask why Python is a better Matlab than Matlab?


First there are a few things I don't like:

1. Intendation as a part of the syntax, really annoying.

Troll. You think this is going away because *you* don't like it? Am I to
presume that you don't bother to indent your C code according to its
nested block structure? If you *do* indent your C code, perhaps you can
explain the additional benefits of the braces?

Calling people trolls for expressing their honest opinion is probably
the worst thing you can do. Try to be more constructive if you want to
help people, especially with such heated topic as indentation.

My own opinion on this, I think the indentation is probably one
biggest drawback which prevents wider Python acceptance. Indentation
makes all kinds of inlined code extremely clumsy or practically impossible
in Python. OK, I'll stop here, time to be called troll myself :(

Andy.
Mar 7 '06 #42
Terry Reedy wrote:
I believe it is Guido's current view, perhaps Google's collective view, and
a general *nix view that such increases can just as well come thru parallel
processes. I believe one can run separate Python processes on separate
cores just as well as one can run separate processes on separate chips or
separate machines. Your view has also been presented and discussed on the
pydev list. (But I am not one for thread versus process debate.)


That's ok for me. I usually have lots of different things
happening on the same computer, but for someone who writes
an application and want to make his particular program faster,
there is not a lot of support for building simple multi-process
systems in Python. While multi-threading is nasty, it makes
it possible to perform tasks in the same program in parallel.

I could well imagine something similar to Twisted, where the
different tasks handled by the event loop were dispatched to
parallel execution on different cores/CPUs by some clever
mechanism under the hood.

If the typical CPU in X years from now is a 5GHz processor with
16 cores, we probably want a single Python "program" to be able
to use more than one core for CPU intensive tasks.
At
least if writing threaded applications becomes less error prone
in competing languages, this might well be the weak point of Python
in the future.


Queue.Queue was added to help people write correct threaded programs.


What I'm trying to say is that multi-threaded programming
(in all languages) is difficult. If *other* languages manage
to make it less difficult than today, they will achieve a
convenient performance boost that Python can't compete with
when the GIL prevents parallel execution of Python code.

- For many years, CPU performance has increased year after
year through higher and higher processor clock speeds. The
number of instructions through a single processing pipeline
per second has increased. This simple kind of speed increase
is flattening out. The first 1GHz CPUs from Intel appeared
about five years ago. At that time, CPU speed still doubled
every 2 years. With that pace, we would have had 6GHz CPUs
now. We don't! Perhaps someone will make some new invention
and the race will be on again...but it's not the case right
now.

- The hardware trend right now, is to make CPUs allow more
parallel execution. Today, double core CPU's are becoming
common, and in a few years there will be many more cores
on each CPU.

- While most computers have a number of processes running in
parallel, there is often one process / application that is
performance critical on typical computers.

- To utilize the performance in these multi-core processors,
we need to use multi-threading, multiple processes or some
other technology that executes code in parallel.

- I think languages and application design patterns will evolve
to better support this parallel execution. I guess it's only
a few languages such as Erlang that support it well today.

- If Python isn't to get far behind the competition, it has
to manage this shift in how to utilize processor power. I
don't know if this will happen through core language changes,
or via some conveninent modules that makes fork/spawn/whatever
and IPC much more convenient, or if there is something
entirely different waiting around the corner. Something is
needed I think.
Mar 7 '06 #43
> Indentation
makes all kinds of inlined code extremely clumsy or practically impossible
in Python.


This is the only sensible argument against the indentation thing I've
heard. Python squirms about being inlined in a presentation template.
Making a direct competitor to PHP in pure Python is problematic.

While there are workarounds which are not hard to envision, it seems
like the right answer is not to inline small fragments of Python code
in HTML, which is probably the wrong approach for any serious work
anyway. This problem does, however, seem to interfere with adoption by
beginning web programmers, who may conceivably end up in PHP or perhaps
Perl Mason out of an ill-considered expedience.

Why this should matter in this discussion, about scientific
programming, escapes me though.

When you say "all kinds" of inlined code, do you have any other
examples besides HTML?

mt

Mar 7 '06 #44

"Michael Tobis" <mt****@gmail.com> wrote in message
news:11*********************@j33g2000cwa.googlegro ups.com...
Indentation
makes all kinds of inlined code extremely clumsy or practically
impossible
in Python.


This is the only sensible argument against the indentation thing I've
heard. Python squirms about being inlined in a presentation template.
Making a direct competitor to PHP in pure Python is problematic.

While there are workarounds which are not hard to envision, it seems
like the right answer is not to inline small fragments of Python code
in HTML, which is probably the wrong approach for any serious work
anyway. This problem does, however, seem to interfere with adoption by
beginning web programmers, who may conceivably end up in PHP or perhaps
Perl Mason out of an ill-considered expedience.

Why this should matter in this discussion, about scientific
programming, escapes me though.

When you say "all kinds" of inlined code, do you have any other
examples besides HTML?

Makefiles is one example. Shell script containing snippet(s) of
Python code is another one.

At one time I also tried to make a simple "configuration file"
engine based on Python for a big Framework used in one physics lab.
Idea was to have a Python extension for that C++ framework and
to configure the Framework from Python code, like:

# Module means C++ Framework module, not Python

Module1.param1 = "a string"
Module2.paramX = [ 1, 2, 3 ]
# etc., with all Python niceties.

People who were using this Framework were all hard-core physicists,
some of them knew Fortran, many were exposed to C++. There were
few other "languages", some of them home-grown, used for different
tasks, but none of these mentioned languages ever placed so much
significance on the whitespaces. There were some big surprises for
people when they discovered they can't arbitrary indent pieces of
the above configuration files because it is all Python code. Add
here space/tabs controversy if it is not enough yet to confuse
poor physicist fellows :) I think that config file project was killed
later in favor of less restrictive format (I left the lab before that,
can't say for sure.)

Andy.

Mar 8 '06 #45
Andy Salnikov wrote:
"Michael Tobis" <mt****@gmail.com> wrote in message
news:11*********************@j33g2000cwa.googlegro ups.com...
Indentation
makes all kinds of inlined code extremely clumsy or practically
impossible
in Python.


This is the only sensible argument against the indentation thing I've
heard. Python squirms about being inlined in a presentation template.
Making a direct competitor to PHP in pure Python is problematic.

While there are workarounds which are not hard to envision, it seems
like the right answer is not to inline small fragments of Python code
in HTML, which is probably the wrong approach for any serious work
anyway. This problem does, however, seem to interfere with adoption by
beginning web programmers, who may conceivably end up in PHP or perhaps
Perl Mason out of an ill-considered expedience.

Why this should matter in this discussion, about scientific
programming, escapes me though.

When you say "all kinds" of inlined code, do you have any other
examples besides HTML?


Makefiles is one example. Shell script containing snippet(s) of
Python code is another one.

At one time I also tried to make a simple "configuration file"
engine based on Python for a big Framework used in one physics lab.
Idea was to have a Python extension for that C++ framework and
to configure the Framework from Python code, like:

# Module means C++ Framework module, not Python

Module1.param1 = "a string"
Module2.paramX = [ 1, 2, 3 ]
# etc., with all Python niceties.

People who were using this Framework were all hard-core physicists,
some of them knew Fortran, many were exposed to C++. There were
few other "languages", some of them home-grown, used for different
tasks, but none of these mentioned languages ever placed so much
significance on the whitespaces. There were some big surprises for
people when they discovered they can't arbitrary indent pieces of
the above configuration files because it is all Python code. Add
here space/tabs controversy if it is not enough yet to confuse
poor physicist fellows :) I think that config file project was killed
later in favor of less restrictive format (I left the lab before that,
can't say for sure.)

I just hope this remains a "someone made a poor choice of configuration
language and trained the users inadequately" story, and does not
transmogrify into a "Python is bad" story.

You mention makefiles and shell scripts as contexts unsympathetic to
Python's indentation requirements, but frankly you don't see much code
in any language except shell inlined in these contexts.

Given the makefile's requirement that significant leading whitespace be
tabs and not spaces and you have a recipe for disaster inlining any
language.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd www.holdenweb.com
Love me, love my blog holdenweb.blogspot.com

Mar 8 '06 #46
I think I agree with Steve here.

I suspect you should either have sufficiently trained your users in
Python, or have limited them to one-line statements which you could
then strip of leading whitespace before passing them to Python, or even
offered the alternative of one or the other. This would not have been
much extra work.

As for shell scripts generating Python code, I am not sure what you
were trying to do, but if you're going that far why not just replace
the shell script with a python script altogether?

os.system() is your friend.

I also agree with Steve that I can't see what this has to do with
makefiles. (But then I think "make" is a thoroughly bad idea in the
first place, and think os.system() is my friend.)

mt

Mar 8 '06 #47

"Steve Holden" <st***@holdenweb.com> wrote in message
news:ma***************************************@pyt hon.org...
Andy Salnikov wrote:
"Michael Tobis" <mt****@gmail.com> wrote in message
news:11*********************@j33g2000cwa.googlegro ups.com...

When you say "all kinds" of inlined code, do you have any other
examples besides HTML?

Makefiles is one example. Shell script containing snippet(s) of
Python code is another one.

At one time I also tried to make a simple "configuration file"
engine based on Python for a big Framework used in one physics lab.
Idea was to have a Python extension for that C++ framework and
to configure the Framework from Python code, like:

# Module means C++ Framework module, not Python

Module1.param1 = "a string"
Module2.paramX = [ 1, 2, 3 ]
# etc., with all Python niceties.

People who were using this Framework were all hard-core physicists,
some of them knew Fortran, many were exposed to C++. There were
few other "languages", some of them home-grown, used for different
tasks, but none of these mentioned languages ever placed so much
significance on the whitespaces. There were some big surprises for
people when they discovered they can't arbitrary indent pieces of
the above configuration files because it is all Python code. Add
here space/tabs controversy if it is not enough yet to confuse
poor physicist fellows :) I think that config file project was killed
later in favor of less restrictive format (I left the lab before that,
can't say for sure.)

I just hope this remains a "someone made a poor choice of configuration
language and trained the users inadequately" story, and does not
transmogrify into a "Python is bad" story.

It does not, and I did not say it's "bad". But people do percieve it
as at least very weird kind of language in a modern times of all the
"curly brace languages".
You mention makefiles and shell scripts as contexts unsympathetic to
Python's indentation requirements, but frankly you don't see much code in
any language except shell inlined in these contexts.
Shell's strength is in the process spawning/management and input/output
redirection, Python is rather weak in that area but OTOH Python is
strong in processing highly structured and numeric data, where shells
are really weak. I saw lots of awk or sed "code" embedded in scripts
so your claim that nothing except sheel is being inlined does not look
right to me.
Given the makefile's requirement that significant leading whitespace be
tabs and not spaces and you have a recipe for disaster inlining any
language.

I saw makefiles with thousands lines of Perl code in them. I agree this
(Perl) is disaster, but it would probably be better if it was Python code
instead.

Andy.

Mar 8 '06 #48

"Michael Tobis" <mt****@gmail.com> wrote in message
news:11**********************@j52g2000cwj.googlegr oups.com...
I think I agree with Steve here.

I suspect you should either have sufficiently trained your users in
Python, or have limited them to one-line statements which you could
then strip of leading whitespace before passing them to Python, or even
offered the alternative of one or the other. This would not have been
much extra work.

As for shell scripts generating Python code, I am not sure what you
were trying to do, but if you're going that far why not just replace
the shell script with a python script altogether?

os.system() is your friend.

I also agree with Steve that I can't see what this has to do with
makefiles. (But then I think "make" is a thoroughly bad idea in the
first place, and think os.system() is my friend.)

mt


Actually os.system() is rather poor replacement for the shell's
capabilities, and it's _very_ low level, it's really a C-level code
wrapped in Python syntax. Anyway, to do something useful you need
to use all popen() stuff, and this is indeed infinitely complex
compared to the easy shell syntax.

Andy.

Mar 8 '06 #49
Andy Salnikov wrote:
I saw lots of awk or sed "code" embedded in scripts


In my experience, embedding any of make/sh/awk/sed in
any of the others is a nightmare of singlequote/
doublequote/backslash juggling that makes a few
tab/space problems in Python pale by comparison.

--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
Mar 9 '06 #50

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Jeff Rush | last post by:
I'd like to extend an invitation to those who would like to get involved in advocating the use of Python. In August, the PSF hired me, for a 6-mo contract, to coordinate the Python advocacy...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.