Below is a simple program that will cause python to intermittently
stop executing for a few seconds. it's 100% reproducible on my machine.
I'd be tempted to say this is a nasty garbage collection performance
issue except that there is no major memory to be garbage collected in
this script. I'd be tempted to say it was a unix virtual memory
issue except this is occuring at around 1/5th of my physical memory
size. So something else unexplained is going on
Class Foo instances create and hold a list of size nfoo of integers.
(nfoo =10)
Class Bar instances create and hold a list of size nbar of Foo
objects. (nbar =100)
When the code runs it starts creating and storing Bar objects in a
list while watching for real-time glitches in how long it takes to
create the next Foo object. If this is longer than 1/2 of a second
then it reports it.
On my computer after creating 1500 Bar objects, the rate of
creation of new Foo suddenly has a periodic glitch. This glitch re-
occurs about every 300 Bar Objects, and the duration of the glitch
keeps getting longer--growing to many seconds!!!!
Platform: 800Mhz powermac g 4 1Gb of memory
python: python 2.4.2
Note: since I an using absolute time threshold for reporting the
glitches, the first one may take more iterations before it occurs on
a fast computer. You may need to increase nbar or nfoo.
import sys
from time import time
class Foo(object):
def __init__(me,nfo o):
me.memory = [1]*nfoo
class Bar(object):
def __init__(me,nba r,nfoo):
tgx.set_tag("Ba r") # timer
b = [None]*nbar
for i in xrange(nbar):
b[i]=Foo(nfoo) # make a foo, add it to list
tgx.jump("Bar"+ `i`) #timer
me.b = b #store the list in my instance memory
# just a utility class to time things.
class gtime:
def __init__(me,f=s ys.stderr):
me.last = time()
me.f=f
me.tag = ""
me.ticks = 0
def set_tag(me,tag) :
me.tag = tag
me.ticks = 0
def reset(me):
me.ticks = 0
def jump(me,tag="NO NE",allowed_gap =0.5):
me.t = time()
me.ticks +=1
if me.t-me.last>allowed _gap:
print >>me.f,"Big Gap:",me.t-me.last,"second s
",me.tag,tag,me .ticks
me.last = time()
tgx = gtime() # instance of the timer
# main loop
nbar = 100
nfoo = 10
ad_nauseum = 20000
final = [None]*ad_nauseum
for i in xrange(ad_nause um ):
if i%100 == 0 :print >>sys.stderr,"B ars Made: ",i
final[i] = Bar(nbar,nfoo)
sample Output:
Bars Made: 0
Bars Made: 100
Bars Made: 200
Bars Made: 300
Bars Made: 400
Bars Made: 500
Bars Made: 600
Bars Made: 700
Bars Made: 800
Bars Made: 900
Bars Made: 1000
Bars Made: 1100
Bars Made: 1200
Bars Made: 1300
Bars Made: 1400
Bars Made: 1500
Big Gap: 0.618862867355 seconds Bar Bar5 6
Bars Made: 1600
Bars Made: 1700
Bars Made: 1800
Big Gap: 0.748915195465 seconds Bar Bar76 77
Bars Made: 1900
Bars Made: 2000
Bars Made: 2100
Big Gap: 0.809149980545 seconds Bar Bar45 46
Bars Made: 2200
Bars Made: 2300
Bars Made: 2400
Big Gap: 0.898494958878 seconds Bar Bar15 16
Bars Made: 2500
Bars Made: 2600
Bars Made: 2700
Big Gap: 1.01110696793 seconds Bar Bar86 87
Bars Made: 2800
Bars Made: 2900
Bars Made: 3000
Big Gap: 1.12396192551 seconds Bar Bar55 56
Bars Made: 3100
Bars Made: 3200
Bars Made: 3300
Big Gap: 1.39006495476 seconds Bar Bar25 26
Bars Made: 3400
Bars Made: 3500
Bars Made: 3600
Big Gap: 1.55699706078 seconds Bar Bar96 97
Bars Made: 3700
Bars Made: 3800
Bars Made: 3900
Big Gap: 1.49929594994 seconds Bar Bar65 66
Bars Made: 4000
Bars Made: 4100
Bars Made: 4200
Big Gap: 1.64840602875 seconds Bar Bar35 36
Bars Made: 4300
Bars Made: 4400
Bars Made: 4500
Big Gap: 1.728484869 seconds Bar Bar5 6
Bars Made: 4600
Bars Made: 4700
Bars Made: 4800 5 1588
charlie strauss wrote:
Below is a simple program that will cause python to intermittently
stop executing for a few seconds. it's 100% reproducible on my
machine.
Confirmed with Python 2.4.2 on Windows.
gc.disable() fixes it, so it looks like you found an inefficiency in the
Python's GC. I have no idea whether this would be considered a bug by Python's
developer, but you can try opening a bugreport...
--
Giovanni Bajo
Giovanni Bajo wrote:
charlie strauss wrote:
Below is a simple program that will cause python to intermittently
stop executing for a few seconds. it's 100% reproducible on my
machine.
Confirmed with Python 2.4.2 on Windows.
gc.disable() fixes it, so it looks like you found an inefficiency in the
Python's GC. I have no idea whether this would be considered a bug by Python's
developer, but you can try opening a bugreport...
Reproduced on 2.4.3 and 2.5 on Windows.
Disabling GC fixes the speed problem as Giovanni said, but doesn't
reduce the amount of memory taken (measured by increase in "page file
used" display in Task Manager). At about 520 Mb, this seems rather too
high to me.
Definitely worth reporting, even if the outcome is only(!) a timbottian
dissertation of why it's not a problem -- at least we'd learn something
:-)
Cheers,
John
Below is a simple program that will cause python to intermittently
stop executing for a few seconds. it's 100% reproducible on my
machine.
Confirmed with Python 2.4.2 on Windows.
gc.disable() fixes it, so it looks like you found an inefficiency in the
Python's GC. I have no idea whether this would be considered a bug by
Python's
developer, but you can try opening a bugreport...
Reproduced on 2.4.3 and 2.5 on Windows.
Disabling GC fixes the speed problem as Giovanni said, but doesn't
reduce the amount of memory taken (measured by increase in "page file
used" display in Task Manager). At about 520 Mb, this seems rather too
high to me.
Definitely worth reporting, even if the outcome is only(!) a timbottian
dissertation of why it's not a problem -- at least we'd learn something
:-)
This is because the OP violated the Style Guide (PEP 8) and used 'me'
instead of 'self' as the first argument of instance methods, the
couple of seconds delay in runtime is the revenge of the vicious
interpreter :)
On Sun, 01 Oct 2006 07:13:25 GMT, Giovanni Bajo <no***@sorry.co mwrote:
charlie strauss wrote:
>Below is a simple program that will cause python to intermittently stop executing for a few seconds. it's 100% reproducible on my machine.
Confirmed with Python 2.4.2 on Windows.
And Python 2.3.5 on Linux, amd64. In fact, it causes heavy swapping so it
will disrupt other processes, too.
I didn't read the code, stupid as I am, but I trust that the behavior
doesn't match what the code actually tries to do.
/Jorgen
--
// Jorgen Grahn <grahn@ Ph'nglui mglw'nafh Cthulhu
\X/ snipabacken.dyn dns.org R'lyeh wgah'nagl fhtagn!
[charlie strauss]
>>Below is a simple program that will cause python to intermittently stop executing for a few seconds. it's 100% reproducible on my machine.
[Giovanni Bajo]
>Confirmed with Python 2.4.2 on Windows.
[Jorgen Grahn]
And Python 2.3.5 on Linux, amd64. In fact, it causes heavy swapping so it
will disrupt other processes, too.
I didn't read the code, stupid as I am, but I trust that the behavior
doesn't match what the code actually tries to do.
No, that's what it does: as it goes on, it creates an ever-increasing
and unbounded number of immortal objects and lists. The "time burps"
merely reflect that the more live containers and objects you have, the
longer it takes for cyclic gc to determine that they're not trash. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Sateesh |
last post by:
Hi,
I am a beginner in Python, and am wondering what is it about the indentation
in Python, without which python scripts do not work properly.
Why can't the indentation not so strict so as to give better freedom to the
user?
Is there any plausible reason behind this?
Cheers!
Sateesh
|
by: Karl Ehr |
last post by:
I have written the following simple program to monitor a single URL,
and notify me via email whenever this URL changes. Intermittently,
I raise the following exception:
Traceback (most recent call last):
File "./urlwatch.py", line 34, in ?
server.sendmail(sourceAddress, emailAddress, message)
File "/usr/lib/python2.2/smtplib.py", line 621, in sendmail
if not (200 <= self.ehlo() <= 299):
File "/usr/lib/python2.2/smtplib.py", line...
|
by: Matthias Kaeppler |
last post by:
Hi,
sorry for my ignorance, but after reading the Python tutorial on
python.org, I'm sort of, well surprised about the lack of OOP
capabilities in python. Honestly, I don't even see the point at all of
how OO actually works in Python.
For one, is there any good reason why I should ever inherit from a
class? ^^ There is no functionality to check if a subclass correctly
implements an inherited interface and polymorphism seems to be...
|
by: zxo102 |
last post by:
Hi there,
I have a python application (many python scripts) and I start the
application like this
python myServer.py start
in window. It is running in dos window. Now I would like to put it in
background as NT service. I got a example code: SmallestService.py from
chapter 18 of the book "Python Programming On Win32" by Mark Hammond
etc. The code is as follows and runs well as an NT service.
|
by: Kurt B. Kaiser |
last post by:
Patch / Bug Summary
___________________
Patches : 378 open ( +3) / 3298 closed (+34) / 3676 total (+37)
Bugs : 886 open (-24) / 5926 closed (+75) / 6812 total (+51)
RFE : 224 open ( +7) / 227 closed ( +7) / 451 total (+14)
New / Reopened Patches
______________________
| |
by: Mythmon |
last post by:
I am trying to make a program that will basically simulate a chess
clock in python. To do this I have two threads running, one that
updates the currently running clock, and one that watches for a
keypress. I am using the effbot Console module, and that is where I get
the events for the keypresses. But when I press space it crashes
shortly after. The program stops, the Console closes, and windows says
that the program pythonw.exe has had an...
|
by: Mark Tarver |
last post by:
How do you compare Python to Lisp? What specific advantages do you
think that one has over the other?
Note I'm not a Python person and I have no axes to grind here. This is
just a question for my general education.
Mark
|
by: falloutphil |
last post by:
Hi,
I'm running a CGI script written in python that tars up the photos in user selected directories (via a form). It's running on Apache 1.3.31.1 on Solaris 5.8.
It works well for a small amount of files, but when I try to use it for large archives (typically over 10MB, but this seems to vary) my download stops short. Occasionally I get an error message, this is usually a Broken Pipe (IO Error 32), but sometimes I don't see anything at...
|
by: falloutphil |
last post by:
Hi,
First of all sorry for the double post - this is on the Python page too, but as far as I can see this is an Apache issue now. Mods - feel free to delete the similar titled posts from me on Python, if this is the case (can't seem to do it myself!).
Anyway,
I'm running a CGI script written in python that tars up the photos in user selected directories (via a form). It's running on Apache 1.3.31.1 on Solaris 5.8.
It works well for a...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |