473,385 Members | 1,814 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Poor python and/or Zope performance on Sparc

Hello everybody,

I'm posting this message because I'm quiet frustrated.

We just bought a software from a small software vendor. In the
beginning he hosted our application on a small server at his office. I
think it was a Fujitsu-Siemens x86 running debian Linux. The
performance of the DSL-Line was very poor, so we decided to buy an own
machine to host the application ourselves.

The application is based on the Zope Application server (2.8.8-final
w/ python 2.3.6) along with many other packages like ghostview,
postgres, freetype, python imaging lib, etc... I once saw the
application running at the office of my software vendor and it was
running very well.

So I thought that - when the bottleneck of poor DSL performance has
disappeared - the software will run as fast as at his office. But I
erred. The performance is more than poor.

We have a Sun T1000 with 8 cores and 8 GB of RAM. First, I installed
Solaris 10 because I know this OS better than Debian Linux. Result:
poor Performance. After that I decided to migrate to debian:

root@carmanager uname -a
Linux carmanager 2.6.18-5-sparc64-smp #1 SMP Wed Oct 3 04:16:38 UTC
2007 sparc64 GNU/Linux

Result: Almost even worse. The application is not scaling at all.
Every time you start a request it is hanging around on one cpu and is
devouring it at about 90-100% load. The other 31 CPUs which are shown
in "mpstat" are bored at 0% load.

If anybody needs further information about installed packages, I'll
post it here.

Any hints are appreciated!
Thanks, Joe.

PS: Fortunately the Sun is not bought yet. It's a "try&buy" from my
local dealer. So if there are any hints like "buy a new machine
because Sun is crap" I will _not_ refuse obediance.

Nov 3 '07 #1
4 2368
On Nov 3, 9:35 am, joa2212 <joa2...@yahoo.dewrote:
Result: Almost even worse. The application is not scaling at all.
Every time you start a request it is hanging around on one cpu and is
devouring it at about 90-100% load. The other 31 CPUs which are shown
in "mpstat" are bored at 0% load.
You are probably not aware of Python's Global Interpeter Lock:
http://docs.python.org/api/threads.html.

George

Nov 3 '07 #2
On 3 Nov., 17:19, George Sakkis <george.sak...@gmail.comwrote:
On Nov 3, 9:35 am, joa2212 <joa2...@yahoo.dewrote:
Result: Almost even worse. The application is not scaling at all.
Every time you start a request it is hanging around on one cpu and is
devouring it at about 90-100% load. The other 31 CPUs which are shown
in "mpstat" are bored at 0% load.

You are probably not aware of Python's Global Interpeter Lock:http://docs.python.org/api/threads.html.

George
Hi George,

yes, that's right. I wasn't aware of this. If I understand you
correctly then we have a problem of implementing threads in the
software we are using. But tell me one thing: Why does this software
almost runs like a fool on an Intel machine with a single cpu (perhaps
dual core?) and slow like a snail when it runs on Sparc? It's exactly
the same source code on both platforms. Certainly all relevant
packages (python + zope) were recompiled on the sparc.

Sorry for my questions, I'm really no software developer. I'm just a
little bit helpless because my software vendor can't tell my anything
about by concerns.

Joe.

Nov 3 '07 #3
joa2212 wrote:
We have a Sun T1000 with 8 cores and 8 GB of RAM. First, I installed
Solaris 10 because I know this OS better than Debian Linux. Result:
poor Performance. After that I decided to migrate to debian:
Do you know the architecture of this machine? It's extremely streamlined
for data throughput (IO) at the expense of computational ability. In
particular:

- it runs at a relatively small clock speed (1 GHz - 1.4 GHz)
- it's terrible for floating-point calculations because there is only
one FPU shared by all 32 logical processors

While it will be screamingly fast for serving static content, and pretty
decent for light database jobs, this not an optimal platform for dynamic
web applications, especially for Python since the language's so dynamic
and it doesn't support SMP. Since it's so optimized for a certain
purpose, you can freely consider it a special-purpose machine, rather
than a general-purpose one.

Even if you manage to get Zope to spawn parallel request handlers
(probably via something like fastcgi), if the web application is
CPU-intensive you won't be happy with its performance (for CPU-intensive
tasks you probably don't want to spawn more than 8 handlers, since
that's the number of physical cores in the CPU).

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHLSB+ldnAQVacBcgRAhHTAKCzXw8NvXmpQuoNhl/xVDKlSFDyBQCgjBe0
hSM1O5T5E3dQSdHrF/yhD4A=
=o0GD
-----END PGP SIGNATURE-----

Nov 4 '07 #4
On Nov 3, 2:35 pm, joa2212 <joa2...@yahoo.dewrote:
Result: Almost even worse. The application is not scaling at all.
Every time you start a request it is hanging around on one cpu and is
devouring it at about 90-100% load. The other 31 CPUs which are shown
in "mpstat" are bored at 0% load.
Like others already pointed out: Performance is going to suffer if
you
run something that's single-threaded on a T1000.

We've been using the T1000 for a while (but not for python) and it's
actually really fast. The performance is way better than the 8 cores
imply. In fact, it'll do work that's not too far from 32 times the
work
of a single thread.

Of course, it all depends on what your application is like, but try
experimenting a bit with parallelism. Try changing from 4 up to 64
parallel
working processes to see what kind of effects you'll get.

I'm guessing that whatever caching of persistent data you can do will
be a good thing as well as the disk might end up working really hard.

Nov 5 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

65
by: Anthony_Barker | last post by:
I have been reading a book about the evolution of the Basic programming language. The author states that Basic - particularly Microsoft's version is full of compromises which crept in along the...
19
by: Patrick Useldinger | last post by:
Hi all, after my unsuccessful try to run Apache 2 with mod_python and Python 2.3, I am looking for an alternative approach. My aim is to write a small web-based application: Python - the...
0
by: Joel Burton | last post by:
The newly-formed Zope/Python Users Group of DC announces our upcoming meeting schedule: Mo Jul 19: Zope -- presentation on Archetypes by Joel Burton Th Aug 5: Python -- no topic yet Mo Aug...
1
by: Vbfoo Bar | last post by:
Hello, To implement a Zope intranet on a linux RedHat ES 3, I had to install a Python 2.3.4 that I have compiled myself. I would like to compare the respective performance of this python...
28
by: Admin | last post by:
I am doing some research for a Python framework to build web applications. I have discarted Zope because from what I've read, the learning curve is too steep, and it takes more time to build...
2
by: Charles | last post by:
Dear sirs, I'm doing some investigations on the Python language. I'd like to know if it is possible to compile Python code made for the web (an online database for instance), and to run it using...
12
by: john_sips_tea | last post by:
I've got a fairly substantial webapp written in Java (plus Tomcat, Hibernate, Struts, JSP, MySQL) that is a bit of a bear to work with. I didn't write it. Much of it is only very sparsely...
9
by: Randy.Galbraith | last post by:
I'm investigating the possible use of Mecurial SCM as a replacement for CVS. Mecurial is written in Python. I have a background in GNU/ Linux, Solaris, sparc and Perl. However AIX, powerpc and...
1
by: M.-A. Lemburg | last post by:
On 2008-07-25 08:13, python@bdurham.com wrote: Yes. But then Intel Itanium is being phased out anyway and the AMD64 build works on both Intel and AMD processors. True.
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.