473,699 Members | 2,615 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Distributed Cache Server?

Does anyone know if a "distribute d caching system" has been developed
for use with Python? I've seen mention of memcached, but was really
after something natively python.

Yes, "distribute d caching system" is a bit of a general term, but am
really just talking about something as simple as key + value (arbitrary
class) which can be split over a number of machines in an efficient
manner.

I've started development of such a system, but am wondering if
something already exists out there.

KenF

Nov 6 '05 #1
3 1982
<ke**********@g mail.com> wrote in message news:11******** **************@ g43g2000cwa.goo glegroups.com.. .
Does anyone know if a "distribute d caching system" has been developed
for use with Python?
BitTorrent :-)
Yes, "distribute d caching system" is a bit of a general term, but am
really just talking about something as simple as key + value (arbitrary
class) which can be split over a number of machines in an efficient
manner.


You'll need to define what the sweet spot is that you are aiming for.
Are we talking tens of thousands of keys or billions? How big is the
data (megabytes, gigabytes, terabytes?) Do you need transactional
integrity (eg when are updates seen by other readers)? Do you want
redundancy (data duplicated on multiple machines)? How many machines
are we talking about? Should failure be automatically detected? Is
there a need for security or treating the machines as untrusted?

Roger
Nov 7 '05 #2
ke**********@gm ail.com wrote:
Does anyone know if a "distribute d caching system" has been developed
for use with Python? I've seen mention of memcached, but was really
after something natively python. [...]
Too bad, because memcached implements all the difficult parts already
(like failover if one node dies), has Python bindings:
ftp://ftp.tummy.com/pub/python-memcached/ and just plain works.
[...] I've started development of such a system, but am wondering if
something already exists out there.


I don't know of any pure-Python equivalent.

-- Gerhard
Nov 7 '05 #3
har.... nah, think I'll give BT a miss for this purpose.

Yeah, know it was a loaded question (no specifics), but was really only
thinking out aloud.

The plans are:

- Multiple copies of data on different machines (just thinking 2 copies
of the data)
- Load spread over multiple machines... anything between 2->lots
(again, not being specific).
- Data itself I'm expecting to be low.... a few Gb at tops (but
ofcourse, as this extends I might want to increase the size)
- Haven't thought a LOT about automatic handling of errors...... yet.
- and its assumed to be on a trusted network.

Apart from memcached, I haven't seen anything similar to what I'm after
(plus to be perfectly honest..... I wouldn't mind coding it up anyway)
;)

Was just "feeling about" to see if anything else is out there.

KenF

ps. btw, only part of it is theoretical, a lot of this I've already
coded, but not targetted towards large scale (100's machines etc)... So
am thinking what I'd need to consider to increase this to a larger
scale.


Roger Binns wrote:
<ke**********@g mail.com> wrote in message news:11******** **************@ g43g2000cwa.goo glegroups.com.. .
Does anyone know if a "distribute d caching system" has been developed
for use with Python?


BitTorrent :-)
Yes, "distribute d caching system" is a bit of a general term, but am
really just talking about something as simple as key + value (arbitrary
class) which can be split over a number of machines in an efficient
manner.


You'll need to define what the sweet spot is that you are aiming for.
Are we talking tens of thousands of keys or billions? How big is the
data (megabytes, gigabytes, terabytes?) Do you need transactional
integrity (eg when are updates seen by other readers)? Do you want
redundancy (data duplicated on multiple machines)? How many machines
are we talking about? Should failure be automatically detected? Is
there a need for security or treating the machines as untrusted?

Roger


Nov 8 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2457
by: DotNetJunkies User | last post by:
I am writing a distributed transaction code. My current scenario include a client database(Suppose client- having 4 main database) which can be installed anywhere which would connect to a public database placed at ISP datacentre(Suppose Server- having 2 database). this server database is added to client as linked server through vb.net interface. i have a transmission routine which transmitts the data from the client to Server and update client...
8
3650
by: Rob S | last post by:
I have UDB 8.1 Personal Edition installed. I'm using Development centre to develop JAVA Stored Procedues. I am unable to debug them. I have installed IBM Distributed Debugger and have set DB2ROUTINE_DEBUG=ON. I installed Distributed Debugger after UDB. The Procs will build with debug but no debugger window appears when run in debug mode. I followed the instructions that came with Debugger regarding the installation directory etc.
4
2820
by: tsui | last post by:
Dear all Someone say that past is Client/Server application, and now is distributed application, but i don't understand these terms Is it Client/Server application is mean two tiers, and only one server can not be web farm or cluster, and asp and vb can be a client/server app and is it distributed application is mean more than two tiers, and can more than one server ????????????
0
903
by: Marcin Rzeznicki | last post by:
Hello, Is it possible that CacheDependency does not work as expected when monitoring file changes on distributed file system? I've run into following issue: two asp.net applications are deployed on two different servers. Servers share directory by DFS (operating system is Win2k). Each application establishes CacheDependency on one of shared files. Whenever file is changed only one application (it seems that it is always the one located on...
7
1604
by: ask ksa | last post by:
hi, is anybody familiar with production-ready distributed/clustered cache solution for .net ? Google helped me to found some (like ncache and scale-out-state-server), but they are over-priced thanks anyway for your time reading that
2
3355
by: adigunpalmer | last post by:
Synopsis: Need a .Net cache solution in which the cache can be synchronized across multiple hosts on our server farm. My company is investigating caching as a means to improve performance, by reducing the database as a bottleneck. However due to constraints imposed by our integration with business partners, we need a solution that does not have a single point of failure. In a nutshell: So far we have found a few vendors that offer...
0
8615
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9033
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8911
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8882
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
5872
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4375
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4627
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3057
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
2009
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.