473,573 Members | 3,208 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Are weak refs slower than strong refs?

Are weak refs slower than strong refs? I've been considering making the
"parent" links in BeautifulSoup into weak refs, so the trees will release
immediately when they're no longer needed. In general, all links back
towards the root of a tree should be weak refs; this breaks the loops
that give reference counting trouble.

John Nagle
Feb 25 '07 #1
3 3414
On Feb 24, 10:17 pm, John Nagle <n...@animats.c omwrote:
Are weak refs slower than strong refs? I've been considering making the
"parent" links in BeautifulSoup into weak refs, so the trees will release
immediately when they're no longer needed. In general, all links back
towards the root of a tree should be weak refs; this breaks the loops
that give reference counting trouble.
I've never really benchmarked their overhead. Thinking about how they
work, I wouldn't expect a measurable difference in the time for
dereferencing the weakrefs. But internally objects must track who all
their weak reference holders are, so that will add some cost. I am
guessing if the hierarchy is built once and remains fairly static you
won't see the cost of this management. If the hierarchy is very
dynamic there will be more weakref overhead. On the other hand, you
will be freeing up the work done by the cyclic testing in the garbage
collector.

After doing similar work with weakrefs in hierarchies, I'd say it is
worth doing. If you do the work, it would be interesting to see what
the performance looks like. Be sure there is a gc.collect() in the
timing for the original strong-ref version. :-)

Feb 25 '07 #2
John Nagle <na***@animats. comwrote:
Are weak refs slower than strong refs? I've been considering
making the "parent" links in BeautifulSoup into weak refs, so the
trees will release immediately when they're no longer needed. In
general, all links back towards the root of a tree should be weak
refs; this breaks the loops that give reference counting trouble.
Yes, weak references are slower, but that may not be significant unless
people are following the parent links a lot: I would expect other
processing to make the overhead fairly insignificant.

Why not do a few timing tests with some typical use cases? Here's an
example which does nothing much but create and follow. Typical output:

Using proxy
100 loops, best of 3: 5.6 msec per loop
Call to dummy proxy
100 loops, best of 3: 3.11 msec per loop
Without proxy call
100 loops, best of 3: 2.71 msec per loop

------ timetest.py --------------------------------------------------------
from timeit import Timer, default_timer as timer
from weakref import proxy

class C:
'''Use a weak proxy (or whatever 'proxy' returns)'''
def __init__(self, parent=None):
self.parent = proxy(parent) if parent is not None else parent
self.child = None

def show_parents(se lf):
while self is not None:
# print "show_paren ts", id(self)
self = self.parent

def show_children(s elf):
while self is not None:
# print "show_children" , id(self)
self = self.child
def t1(n):
base = C()
current = base
for i in range(n):
current.child = C(current)
current = current.child
base.show_child ren()
current.show_pa rents()

class D:
'''Strong parent reference'''
def __init__(self, parent=None):
self.parent = parent if parent is not None else parent
self.child = None

def show_parents(se lf):
while self is not None:
# print "show_paren ts", id(self)
self = self.parent

def show_children(s elf):
while self is not None:
# print "show_children" , id(self)
self = self.child

def t2(n):
base = D()
current = base
for i in range(n):
current.child = D(current)
current = current.child
base.show_child ren()
current.show_pa rents()
def timetest(stmt):
repeat = 3
precision = 3
t = Timer(stmt, 'import __main__', timer)

# determine number so that 0.2 <= total time < 2.0
for i in range(1, 10):
number = 10**i
try:
x = t.timeit(number )
except:
t.print_exc()
return 1
if x >= 0.2:
break

try:
r = t.repeat(repeat , number)
except:
t.print_exc()
return 1
best = min(r)
print "%d loops," % number,
usec = best * 1e6 / number
if usec < 1000:
print "best of %d: %.*g usec per loop" % (repeat, precision, usec)
else:
msec = usec / 1000
if msec < 1000:
print "best of %d: %.*g msec per loop" % (repeat, precision,
msec)
else:
sec = msec / 1000
print "best of %d: %.*g sec per loop" % (repeat, precision,
sec)

if __name__=='__ma in__':
print "Using proxy"
timetest('__mai n__.t1(1000)')
print "Call to dummy proxy"
def proxy(x): return x
timetest('__mai n__.t1(1000)')
print "Without proxy call"
timetest('__mai n__.t2(1000)')
-------------------------------------------------------------------------
Feb 25 '07 #3
John Nagle wrote:
Are weak refs slower than strong refs? I've been considering making the
"parent" links in BeautifulSoup into weak refs, so the trees will release
immediately when they're no longer needed. In general, all links back
towards the root of a tree should be weak refs; this breaks the loops
that give reference counting trouble.

John Nagle
I just finished converting BeautifulSoup to use weak back references.
All that's necessary is to make the "parent", "previous", "previousSiblin g",
and "tagStack" links into weak proxy references. Those are the
"backlinks" of the Beautiful Soup tree; once they're weak links,
BeautifulSoup then becomes loop-free and GC in debug mode reports
zero garbage. This is useful, because BeautifulSoup's backlinks create huge
amounts of collectable garbage, leading to frequent GC cycles.

The "weakref" module could use some support functions. If you
try to create a weakref proxy from a weakref proxy, or from "None",
you get an exception, which is not usually what you want.
It's more useful to pass through "None" or an existing weakref proxy.
So I wrote the function below, which makes it much easier to
convert code to use weak proxy references.

"weakref.proxy( )" probably should work that way.
Weakref proxies are supposed to be transparent, but they're not
quite transparent enough.

Patches to BeautifulSoup are below.

John Nagle

59a60,80
import weakref # Weak references for previous, parent, previousSibling
#
# Weakref allocation control
#
# The following links are always weak references, to avoid internal referenc
# require extra garbage collection.
# self.parent
# self.previous
# self.previousSi bling
# These are all "back links".
#
# backref -- create a weak reference as a back pointer
#
# Generates a weakref proxy, but handles input of "none" or an existing weak
#
def backref(p) :
if p == None : # if none
return(None) # then none
if isinstance(p,we akref.ProxyType ) or isinstance(p,we akref.CallableP roxyTyp
return(p)
return(weakref. proxy(p)) # otherwise a new weakref
60a82
#
79,80c101,102
< self.parent = parent
< self.previous = previous
---
self.parent = backref(parent)
self.previous = backref(previou s)
85c107
< self.previousSi bling = self.parent.con tents[-1]
---
self.previousSi bling = backref(self.pa rent.contents[-1])
127c149
< self.nextSiblin g.previousSibli ng = self.previousSi bling
---
self.nextSiblin g.previousSibli ng = backref(self.pr eviousSibling)
157c179
< newChild.parent = self
---
newChild.parent = backref(self)
161c183
< newChild.previo us = self
---
newChild.previo us = backref(self)
164c186
< newChild.previo usSibling = previousChild
---
newChild.previo usSibling = backref(previou sChild)
166c188
< newChild.previo us = previousChild._ lastRecursiveCh ild()
---
newChild.previo us = backref(previou sChild._lastRec ursiveChild())
190c212
< newChild.nextSi bling.previousS ibling = newChild
---
newChild.nextSi bling.previousS ibling = backref(newChil d)
194c216
< newChildsLastEl ement.next.prev ious = newChildsLastEl ement
---
newChildsLastEl ement.next.prev ious = backref(newChil dsLastElemen
1052c1074
< self.tagStack.a ppend(tag)
---
self.tagStack.a ppend(backref(t ag))
1074c1096
< self.previous = o
---
self.previous = backref(o)
1167c1189
< self.previous = tag
---
self.previous = backref(tag)
Feb 25 '07 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

94
4649
by: Gabriel Zachmann | last post by:
Is it correct to say that strong/weak typing does not make a difference if one does not use any pointers (or adress-taking operator)? More concretely, I am thinking particularly of Python vs C++. So, are there any examples (without pointers, references, or adress-taking), which would have a different result in Python and in C++? I would...
3
2349
by: memememe | last post by:
I see weak reference on the .net api, but I do not see soft or phantom, are they supported on .net?
2
1540
by: Matthew Herrmann | last post by:
Hi, I've heard from groups that listeners to event handlers cause references to be kept alive, if the targets are marked to stay alive. I need to make sure that attaching events to objects will not cause them to be kept open. I created a test which has "target" listening to "source" for events. After plugging source into target, I then...
1
7403
by: seyong_choi | last post by:
I want to write a library that has a defined function (special_subroutine in this example) aliased different names (special_subroutine_1 and special_subroutine_2 in this example). i.e., I need to call a function with different names. Can someone give some help? I try the following way, but it did not work. Can someone give me a...
5
1874
by: _dee | last post by:
I'm working on a port of a legacy app originally written in C. Data was compacted into bit fields. Are there any sites or books that cover optimized handling of this type of data? I'd need to develop optimized functions for reading and writing, and given the volume of source, I don't want to end up with pages of obscure code. (Surprisingly,...
1
1762
by: Henri.Chinasque | last post by:
Hi all, I've been considering that my objects should subscribe to an event via a weak reference, however I've found several warnings that this approach comes with concurrency considerations, like the fact that the event handler method on the subscriber could be called and be executing while the object is being garbage collected. Not...
0
7661
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
7977
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
1
7730
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
8026
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6347
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5550
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5252
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3692
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
2163
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.