473,406 Members | 2,867 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,406 software developers and data experts.

Thread safe object cache without locking

I'm trying to make a thread safe object cache without locking.

The objects are cached by the id of the data dict given in __new__.
Objects are removed from the cache as soon as they are no longer
referenced. The type of the data must be a Python dict (comes from an
external source).

Here's what I've got so far:

import time
from weakref import ref

cache = {} # {id(data): ref_to_Foo(data), ...}

class Foo(object):

__slots__ = ('__weakref__', '_data')

def __new__(cls, data):
new_self = object.__new__(cls)

def remove(wr, _cache=cache, _data=data):
try:
del _cache[id(_data)]
except KeyError:
pass # should never happen

while 1:
self = cache.setdefault(id(data), ref(new_self, remove))()
try:
self._data = data
return self # this is new_self or a cached instance
except AttributeError:
# happens rarely: got a 'dead' cached instance
time.sleep(0) # small delay before retrying

# Usage example:
data = {'a':1, 'b':2}
a = Foo(data)
b = Foo(data)
assert a is b

I've got couple of assumptions based on I think the code is working
correctly:

a) Python's dict is thread safe, i.e. setdefault can't replace an
existing value until the key has been removed - no matter how many
threads are calling it simultaneously. Additionally, only one thread
succeeds in setting the new value.

b) Unreferenced ref(new_self, remove) objects are garbage collected
without the callbacks being called.

c) The remove() keeps a reference to the data dict. That should prevent
Python from reusing the object id too soon.

Are these assumptions correct? Do you have ideas for improvements
(especially regarding speed)? I'm restricted to Py2.3+ compatibility.

Thanks,
Timo

Sep 15 '05 #1
0 1494

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Roman Suzi | last post by:
(this is a repost with an addition - probably noone noticed my message first time) Hi! Just to be sure, is email package of Python 2.3 thread-safe or not (to use, for example, in...
3
by: Philip V Pham | last post by:
These questions apply to std vector, map, and cout: I am uncertain of the thread safety for reading/writing for std templates. I know if all threads are reading concurrently, it is thread...
5
by: nicolas_riesch | last post by:
Does someone know if the module pytz (http://sourceforge.net/projects/pytz/) is thread-safe ? I have not seen it explicitely stated, and just wanted to be sure, as I want to use it. That's...
1
by: William Sullivan | last post by:
I'm trying to nail down some issues with the cache in my application. Currently, I have an object that stands between my business logic and database logic called CacheLogic (cute, no?). ...
6
by: Anders J | last post by:
Hi We have some code that runs in a EventReceiver ItemAdded handler. The code must be thread-safe since it is iterating a List to find the max number of a column and assigns it + 1 to the Item...
7
by: reyesflaco | last post by:
I am developing an application using asp.net 2.0. I created all my business objects in my app_code folder. As of right now, all my classes are public. In my aspx pages, I am declaring the class...
12
by: Peter K | last post by:
Say I have this class public class Simple { private string name; public string Name { get { return name; } set { name = value; }
5
by: Nuno Magalhaes | last post by:
Hello, Is there any keyword applicable to a class to make it thread-safe? Without having to put lock(this){} in all functions? Thanks.
29
by: NvrBst | last post by:
I've read a bit online seeing that two writes are not safe, which I understand, but would 1 thread push()'ing and 1 thread pop()'ing be thread-safe? Basically my situation is the follows: ...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.