473,657 Members | 2,430 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Are weak refs slower than strong refs?

Are weak refs slower than strong refs? I've been considering making the
"parent" links in BeautifulSoup into weak refs, so the trees will release
immediately when they're no longer needed. In general, all links back
towards the root of a tree should be weak refs; this breaks the loops
that give reference counting trouble.

John Nagle
Feb 25 '07 #1
3 3417
On Feb 24, 10:17 pm, John Nagle <n...@animats.c omwrote:
Are weak refs slower than strong refs? I've been considering making the
"parent" links in BeautifulSoup into weak refs, so the trees will release
immediately when they're no longer needed. In general, all links back
towards the root of a tree should be weak refs; this breaks the loops
that give reference counting trouble.
I've never really benchmarked their overhead. Thinking about how they
work, I wouldn't expect a measurable difference in the time for
dereferencing the weakrefs. But internally objects must track who all
their weak reference holders are, so that will add some cost. I am
guessing if the hierarchy is built once and remains fairly static you
won't see the cost of this management. If the hierarchy is very
dynamic there will be more weakref overhead. On the other hand, you
will be freeing up the work done by the cyclic testing in the garbage
collector.

After doing similar work with weakrefs in hierarchies, I'd say it is
worth doing. If you do the work, it would be interesting to see what
the performance looks like. Be sure there is a gc.collect() in the
timing for the original strong-ref version. :-)

Feb 25 '07 #2
John Nagle <na***@animats. comwrote:
Are weak refs slower than strong refs? I've been considering
making the "parent" links in BeautifulSoup into weak refs, so the
trees will release immediately when they're no longer needed. In
general, all links back towards the root of a tree should be weak
refs; this breaks the loops that give reference counting trouble.
Yes, weak references are slower, but that may not be significant unless
people are following the parent links a lot: I would expect other
processing to make the overhead fairly insignificant.

Why not do a few timing tests with some typical use cases? Here's an
example which does nothing much but create and follow. Typical output:

Using proxy
100 loops, best of 3: 5.6 msec per loop
Call to dummy proxy
100 loops, best of 3: 3.11 msec per loop
Without proxy call
100 loops, best of 3: 2.71 msec per loop

------ timetest.py --------------------------------------------------------
from timeit import Timer, default_timer as timer
from weakref import proxy

class C:
'''Use a weak proxy (or whatever 'proxy' returns)'''
def __init__(self, parent=None):
self.parent = proxy(parent) if parent is not None else parent
self.child = None

def show_parents(se lf):
while self is not None:
# print "show_paren ts", id(self)
self = self.parent

def show_children(s elf):
while self is not None:
# print "show_children" , id(self)
self = self.child
def t1(n):
base = C()
current = base
for i in range(n):
current.child = C(current)
current = current.child
base.show_child ren()
current.show_pa rents()

class D:
'''Strong parent reference'''
def __init__(self, parent=None):
self.parent = parent if parent is not None else parent
self.child = None

def show_parents(se lf):
while self is not None:
# print "show_paren ts", id(self)
self = self.parent

def show_children(s elf):
while self is not None:
# print "show_children" , id(self)
self = self.child

def t2(n):
base = D()
current = base
for i in range(n):
current.child = D(current)
current = current.child
base.show_child ren()
current.show_pa rents()
def timetest(stmt):
repeat = 3
precision = 3
t = Timer(stmt, 'import __main__', timer)

# determine number so that 0.2 <= total time < 2.0
for i in range(1, 10):
number = 10**i
try:
x = t.timeit(number )
except:
t.print_exc()
return 1
if x >= 0.2:
break

try:
r = t.repeat(repeat , number)
except:
t.print_exc()
return 1
best = min(r)
print "%d loops," % number,
usec = best * 1e6 / number
if usec < 1000:
print "best of %d: %.*g usec per loop" % (repeat, precision, usec)
else:
msec = usec / 1000
if msec < 1000:
print "best of %d: %.*g msec per loop" % (repeat, precision,
msec)
else:
sec = msec / 1000
print "best of %d: %.*g sec per loop" % (repeat, precision,
sec)

if __name__=='__ma in__':
print "Using proxy"
timetest('__mai n__.t1(1000)')
print "Call to dummy proxy"
def proxy(x): return x
timetest('__mai n__.t1(1000)')
print "Without proxy call"
timetest('__mai n__.t2(1000)')
-------------------------------------------------------------------------
Feb 25 '07 #3
John Nagle wrote:
Are weak refs slower than strong refs? I've been considering making the
"parent" links in BeautifulSoup into weak refs, so the trees will release
immediately when they're no longer needed. In general, all links back
towards the root of a tree should be weak refs; this breaks the loops
that give reference counting trouble.

John Nagle
I just finished converting BeautifulSoup to use weak back references.
All that's necessary is to make the "parent", "previous", "previousSiblin g",
and "tagStack" links into weak proxy references. Those are the
"backlinks" of the Beautiful Soup tree; once they're weak links,
BeautifulSoup then becomes loop-free and GC in debug mode reports
zero garbage. This is useful, because BeautifulSoup's backlinks create huge
amounts of collectable garbage, leading to frequent GC cycles.

The "weakref" module could use some support functions. If you
try to create a weakref proxy from a weakref proxy, or from "None",
you get an exception, which is not usually what you want.
It's more useful to pass through "None" or an existing weakref proxy.
So I wrote the function below, which makes it much easier to
convert code to use weak proxy references.

"weakref.proxy( )" probably should work that way.
Weakref proxies are supposed to be transparent, but they're not
quite transparent enough.

Patches to BeautifulSoup are below.

John Nagle

59a60,80
import weakref # Weak references for previous, parent, previousSibling
#
# Weakref allocation control
#
# The following links are always weak references, to avoid internal referenc
# require extra garbage collection.
# self.parent
# self.previous
# self.previousSi bling
# These are all "back links".
#
# backref -- create a weak reference as a back pointer
#
# Generates a weakref proxy, but handles input of "none" or an existing weak
#
def backref(p) :
if p == None : # if none
return(None) # then none
if isinstance(p,we akref.ProxyType ) or isinstance(p,we akref.CallableP roxyTyp
return(p)
return(weakref. proxy(p)) # otherwise a new weakref
60a82
#
79,80c101,102
< self.parent = parent
< self.previous = previous
---
self.parent = backref(parent)
self.previous = backref(previou s)
85c107
< self.previousSi bling = self.parent.con tents[-1]
---
self.previousSi bling = backref(self.pa rent.contents[-1])
127c149
< self.nextSiblin g.previousSibli ng = self.previousSi bling
---
self.nextSiblin g.previousSibli ng = backref(self.pr eviousSibling)
157c179
< newChild.parent = self
---
newChild.parent = backref(self)
161c183
< newChild.previo us = self
---
newChild.previo us = backref(self)
164c186
< newChild.previo usSibling = previousChild
---
newChild.previo usSibling = backref(previou sChild)
166c188
< newChild.previo us = previousChild._ lastRecursiveCh ild()
---
newChild.previo us = backref(previou sChild._lastRec ursiveChild())
190c212
< newChild.nextSi bling.previousS ibling = newChild
---
newChild.nextSi bling.previousS ibling = backref(newChil d)
194c216
< newChildsLastEl ement.next.prev ious = newChildsLastEl ement
---
newChildsLastEl ement.next.prev ious = backref(newChil dsLastElemen
1052c1074
< self.tagStack.a ppend(tag)
---
self.tagStack.a ppend(backref(t ag))
1074c1096
< self.previous = o
---
self.previous = backref(o)
1167c1189
< self.previous = tag
---
self.previous = backref(tag)
Feb 25 '07 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

94
4687
by: Gabriel Zachmann | last post by:
Is it correct to say that strong/weak typing does not make a difference if one does not use any pointers (or adress-taking operator)? More concretely, I am thinking particularly of Python vs C++. So, are there any examples (without pointers, references, or adress-taking), which would have a different result in Python and in C++? I would appreciate all insights or pointers to literature. TIA,
3
2352
by: memememe | last post by:
I see weak reference on the .net api, but I do not see soft or phantom, are they supported on .net?
2
1544
by: Matthew Herrmann | last post by:
Hi, I've heard from groups that listeners to event handlers cause references to be kept alive, if the targets are marked to stay alive. I need to make sure that attaching events to objects will not cause them to be kept open. I created a test which has "target" listening to "source" for events. After plugging source into target, I then let go of source. Since I'm still holding onto target, if delegates were a strong reference, then
1
7407
by: seyong_choi | last post by:
I want to write a library that has a defined function (special_subroutine in this example) aliased different names (special_subroutine_1 and special_subroutine_2 in this example). i.e., I need to call a function with different names. Can someone give some help? I try the following way, but it did not work. Can someone give me a suggestion? #gcc -c mylibrary.c
5
1875
by: _dee | last post by:
I'm working on a port of a legacy app originally written in C. Data was compacted into bit fields. Are there any sites or books that cover optimized handling of this type of data? I'd need to develop optimized functions for reading and writing, and given the volume of source, I don't want to end up with pages of obscure code. (Surprisingly, the original C was pretty clear).
1
1763
by: Henri.Chinasque | last post by:
Hi all, I've been considering that my objects should subscribe to an event via a weak reference, however I've found several warnings that this approach comes with concurrency considerations, like the fact that the event handler method on the subscriber could be called and be executing while the object is being garbage collected. Not being super strong in the multithreaded department, I am having a hard time coming up with other...
0
8402
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8829
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8734
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
7341
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6172
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5633
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4164
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
2733
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
1962
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.