473,748 Members | 4,935 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Distributed Cache Server?

Does anyone know if a "distribute d caching system" has been developed
for use with Python? I've seen mention of memcached, but was really
after something natively python.

Yes, "distribute d caching system" is a bit of a general term, but am
really just talking about something as simple as key + value (arbitrary
class) which can be split over a number of machines in an efficient
manner.

I've started development of such a system, but am wondering if
something already exists out there.

KenF

Nov 6 '05 #1
3 1986
<ke**********@g mail.com> wrote in message news:11******** **************@ g43g2000cwa.goo glegroups.com.. .
Does anyone know if a "distribute d caching system" has been developed
for use with Python?
BitTorrent :-)
Yes, "distribute d caching system" is a bit of a general term, but am
really just talking about something as simple as key + value (arbitrary
class) which can be split over a number of machines in an efficient
manner.


You'll need to define what the sweet spot is that you are aiming for.
Are we talking tens of thousands of keys or billions? How big is the
data (megabytes, gigabytes, terabytes?) Do you need transactional
integrity (eg when are updates seen by other readers)? Do you want
redundancy (data duplicated on multiple machines)? How many machines
are we talking about? Should failure be automatically detected? Is
there a need for security or treating the machines as untrusted?

Roger
Nov 7 '05 #2
ke**********@gm ail.com wrote:
Does anyone know if a "distribute d caching system" has been developed
for use with Python? I've seen mention of memcached, but was really
after something natively python. [...]
Too bad, because memcached implements all the difficult parts already
(like failover if one node dies), has Python bindings:
ftp://ftp.tummy.com/pub/python-memcached/ and just plain works.
[...] I've started development of such a system, but am wondering if
something already exists out there.


I don't know of any pure-Python equivalent.

-- Gerhard
Nov 7 '05 #3
har.... nah, think I'll give BT a miss for this purpose.

Yeah, know it was a loaded question (no specifics), but was really only
thinking out aloud.

The plans are:

- Multiple copies of data on different machines (just thinking 2 copies
of the data)
- Load spread over multiple machines... anything between 2->lots
(again, not being specific).
- Data itself I'm expecting to be low.... a few Gb at tops (but
ofcourse, as this extends I might want to increase the size)
- Haven't thought a LOT about automatic handling of errors...... yet.
- and its assumed to be on a trusted network.

Apart from memcached, I haven't seen anything similar to what I'm after
(plus to be perfectly honest..... I wouldn't mind coding it up anyway)
;)

Was just "feeling about" to see if anything else is out there.

KenF

ps. btw, only part of it is theoretical, a lot of this I've already
coded, but not targetted towards large scale (100's machines etc)... So
am thinking what I'd need to consider to increase this to a larger
scale.


Roger Binns wrote:
<ke**********@g mail.com> wrote in message news:11******** **************@ g43g2000cwa.goo glegroups.com.. .
Does anyone know if a "distribute d caching system" has been developed
for use with Python?


BitTorrent :-)
Yes, "distribute d caching system" is a bit of a general term, but am
really just talking about something as simple as key + value (arbitrary
class) which can be split over a number of machines in an efficient
manner.


You'll need to define what the sweet spot is that you are aiming for.
Are we talking tens of thousands of keys or billions? How big is the
data (megabytes, gigabytes, terabytes?) Do you need transactional
integrity (eg when are updates seen by other readers)? Do you want
redundancy (data duplicated on multiple machines)? How many machines
are we talking about? Should failure be automatically detected? Is
there a need for security or treating the machines as untrusted?

Roger


Nov 8 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2464
by: DotNetJunkies User | last post by:
I am writing a distributed transaction code. My current scenario include a client database(Suppose client- having 4 main database) which can be installed anywhere which would connect to a public database placed at ISP datacentre(Suppose Server- having 2 database). this server database is added to client as linked server through vb.net interface. i have a transmission routine which transmitts the data from the client to Server and update client...
8
3659
by: Rob S | last post by:
I have UDB 8.1 Personal Edition installed. I'm using Development centre to develop JAVA Stored Procedues. I am unable to debug them. I have installed IBM Distributed Debugger and have set DB2ROUTINE_DEBUG=ON. I installed Distributed Debugger after UDB. The Procs will build with debug but no debugger window appears when run in debug mode. I followed the instructions that came with Debugger regarding the installation directory etc.
4
2823
by: tsui | last post by:
Dear all Someone say that past is Client/Server application, and now is distributed application, but i don't understand these terms Is it Client/Server application is mean two tiers, and only one server can not be web farm or cluster, and asp and vb can be a client/server app and is it distributed application is mean more than two tiers, and can more than one server ????????????
0
907
by: Marcin Rzeznicki | last post by:
Hello, Is it possible that CacheDependency does not work as expected when monitoring file changes on distributed file system? I've run into following issue: two asp.net applications are deployed on two different servers. Servers share directory by DFS (operating system is Win2k). Each application establishes CacheDependency on one of shared files. Whenever file is changed only one application (it seems that it is always the one located on...
7
1609
by: ask ksa | last post by:
hi, is anybody familiar with production-ready distributed/clustered cache solution for .net ? Google helped me to found some (like ncache and scale-out-state-server), but they are over-priced thanks anyway for your time reading that
2
3360
by: adigunpalmer | last post by:
Synopsis: Need a .Net cache solution in which the cache can be synchronized across multiple hosts on our server farm. My company is investigating caching as a means to improve performance, by reducing the database as a bottleneck. However due to constraints imposed by our integration with business partners, we need a solution that does not have a single point of failure. In a nutshell: So far we have found a few vendors that offer...
0
8830
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9370
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
9247
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8242
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6796
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6074
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4602
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
2
2782
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2215
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.