473,385 Members | 1,478 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

cache-like structure

Hi,

I've write a class that actually is data structure with items that
automatically removed from collection when timeout expires. Interface
is pretty simple. First public method is to make a record. And second
(__contains__) is to define if record is actually in table. I prefer
async way to clean out expired records. Thus I have to create timeout
thread every cycle to wait for the next clean-up.

So my questions are:
- is the implementation thread-safe (or I've missed something)
- is there way to avoid creating new timer every cycle
- are there better ways to do this (or any ready recipes)

---
import threading
import time
import collections

class CacheTable(object):
def __init__(self, ttl):
self.records = collections.deque()
self.ttl = ttl
self.timer = None
self.mutex = threading.Lock()

def record(self, item):
self.mutex.acquire()
try:
self.records.append((time.time(), item))
if not self.timer:
self.__settimer(self.ttl)
finally:
self.mutex.release()

def __cleanup(self):
self.mutex.acquire()
try:
current = time.time()
self.timer = None
while self.records:
timestamp, item = self.records.popleft()
if timestamp + self.ttl current:
self.records.appendleft((timestamp, item))
time_to_wait = timestamp + self.ttl - current
self.__settimer(time_to_wait)
break
finally:
self.mutex.release()

def __settimer(self, timeout):
self.timer = threading.Timer(timeout, self.__cleanup)
self.timer.start()

def __contains__(self, value):
self.mutex.acquire()
try:
items = [item for ts, item in self.records]
return items.__contains__(value)
finally:
self.mutex.release()
---

Thanks!
Aug 7 '08 #1
3 1170


konstantin wrote:
Hi,
Hi
>...
- are there better ways to do this (or any ready recipes)

Did you consider using the shed module (not threaded)?

Cheers,

Maxime
Aug 7 '08 #2
On Aug 7, 7:00*pm, maxime <maximestei...@gmail.comwrote:
konstantin wrote:
Hi,
Hi
...
- are there better ways to do this (or any ready recipes)

Did you consider using the shed module (not threaded)?

Cheers,

Maxime
If you need threading, what i would do is implementing a _remove
method like:
def _remove(self, rec): #try not to use __form, almost always an error
self.records.remove(rec)

the Timer would become
Timer(ttl, self._remove, [(t, item)]) # t is the time that you got
before

And if you change the deque for a Queue.Queue, you don't even need a
Lock...

Cheers,

Maxime
Aug 7 '08 #3
And if you change the deque for a Queue.Queue, you don't even need a
Lock...

Cheers,

Maxime
Maxime,

thanks, sched module could be a nice solution. I've never used it
before
but it seems to be the right way.
You guessed right. I want to used it in Queue. But this implementation
does
not guarantee that __cleanup__ won't be invoked while __contains__
processing.
def _remove(self, rec): #try not to use __form, almost always an error
self.records.remove(rec)

the Timer would become
Timer(ttl, self._remove, [(t, item)]) # t is the time that you got
before
This solution has one shortcoming. When record() called many times
during
short period (almost simultaneously) the _remove(item) calls do not
manage to
remove records on time. So I choosed batch cleanup.

But thanks for suggestion! I'll try sched.
Aug 7 '08 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

14
by: Tom.PesterDELETETHISSS | last post by:
Hi, I think this question requires an in depth understanding of how a browser cache works. I hope I can reach an expert here. I may have found a quirk in the asp.net documentation or I don't...
18
by: siddharthkhare | last post by:
Hi All, what is the diference between these two cache control header. no-cache and no-store. I have read the w3.org explanation. So lets say I am using only no-cache ....my understanding is...
19
by: siddharthkhare | last post by:
Hi All, what is the diference between these two cache control header. no-cache and no-store. I have read the w3.org explanation. So lets say I am using only no-cache ....my understanding is...
0
by: mateipuiu | last post by:
When a try to run a client build on 2005, which uses the Microsoft.ApplicationBlocks.Cache.dll reference, when using a Microsoft.ApplicationBlocks.Cache.dll created on Debug mode, the client works...
0
by: =?Utf-8?B?YmlqYXk=?= | last post by:
The type initializer for 'Microsoft.ApplicationBlocks.Cache.CacheService' threw an exception. We migrated our windows application from 1.1 to 2.0. The debug and Release mode of the application...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.