473,320 Members | 2,177 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

RE: Python-list Digest, Vol 61, Issue 368

From: "Andy O'Meara" <an****@gmail.com>
Unfortunately, a shared address region doesn't work when you have
large and opaque objects (e.g. a rendered CoreVideo movie in the
QuickTime API or 300 megs of audio data that just went through a
DSP). Then you've got the hit of serialization if you're got
intricate data structures (that would normally would need to be
serialized, such as a hashtable or something). Also, if I may speak
for commercial developers out there who are just looking to get the
job done without new code, it's usually always preferable to just a
single high level sync object (for when the job is complete) than to
Just to chime as a CPython-based ISV from the scientific visualization
realm, we face the same problem & limitations due to lack of threading
(or at least multiple independent interpreters). A typical use case
might be a 1-3 GB dataset (molecular dynamics trajectory and derived
state) subjected to asynchronous random read/write by N threads each
running on one of N cores in parallel.

We get by jettisoning CPython almost entirely and working in C for all
tasks other than the most basic operations: thread creation, workload
scheduling, mutexes, and thread deletion.

The biggest problem is not for the most compute-intensive tasks (where
use of C is justified), but for those relatively short-running but
algorithmically complex tasks which could be done much more easily from
Python than from C (e.g. data organization, U.I. survey/present tasks,
rarely used transformations, ad hoc scripting experiments, etc.).

Cheers,
Warren
Oct 24 '08 #1
0 709

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Logan | last post by:
Several people asked me for the following HOWTO, so I decided to post it here (though it is still very 'alpha' and might contain many (?) mistakes; didn't test what I wrote, but wrote it - more or...
10
by: Berthold Hoellmann | last post by:
Hello, When I use ./configure --with-thread --with-fpectl --with-signal-module \ --with-pymalloc --enable-shared --with-cxx=g++ make test on 2.3.3 I get
2
by: Olaf Meyer | last post by:
I'm having some problems compiling Python 2.3.3 on HP-UX (B.11.00). I've tried sevral different options for the configure script (e.g. enabling/disabling gcc, aCC) but I always get the same problem...
1
by: Jerald | last post by:
Running python 2.3.4 on valgrind (a tool like purify which checks the use of uninitialized memory, etc), gives a lot of errors. See below. jfj@cluster:~/> python -V Python 2.3.4...
3
by: Saravanan | last post by:
Hello, Im running Python Application as a Windows Service (using windows extensions). But, sporadically the application crashes (crash in Python23.dll) and this stops the service. This...
0
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 391 open ( +7) / 3028 closed (+12) / 3419 total (+19) Bugs : 906 open ( -3) / 5519 closed (+19) / 6425 total (+16) RFE : 207 open...
0
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 402 open ( +6) / 3360 closed ( +6) / 3762 total (+12) Bugs : 861 open ( -3) / 6114 closed (+27) / 6975 total (+24) RFE : 228 open...
11
by: Osiris | last post by:
I have these pieces of C-code (NOT C++ !!) I want to call from Python. I found Boost. I have MS Visual Studio 2005 with C++. is this the idea: I write the following C source file:...
1
by: Justin Johnson | last post by:
Hello, I'm trying to build Python 2.5.0 on AIX 5.3 using IBM's compiler (VisualAge C++ Professional / C for AIX Compiler, Version 6). I run configure and make, but makes fails with undefined...
0
by: mg | last post by:
When make gets to the _ctypes section, I am getting the following in my output: building '_ctypes' extension creating build/temp.solaris-2.10-i86pc-2.5/home/ecuser/Python-2.5.1/ Modules/_ctypes...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.