473,288 Members | 1,710 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,288 software developers and data experts.

in Ram database?

Does anyone know if there is an "in-ram" dictionary based database
for Python.

I can see some applications where this can really be powerful in
applications requiring 10,000 records or less.

Is such a database available? If so, who is working on it,
or where is it?

I've heard that Python supports a stand-alone database. I looked
for it in the Docs, and see the "dbm" module. I haven't found
any examples of code, so don't know how to use them. I'm one
of those guys that learn through examples, and have very little
luck in figuring out how to use anything unless I can see examples
of their use.

My idea is that ALL datums would be Python Dictionary objects, so
not only can you store data, but python objects as well.

Storage to disk would be through pickled files, or through CSV
format, allowing easy import and export from other databases.

I'm sure something like this exists, I just haven't found it yet,
and if not, then I'm well on my way of implementing one myself.

I kinda like the idea of being able to define smaller tables as
"in-memory" types, and the really large massive ones in other forms
managed by mySQL, PostGres, or bsddb3 databases.

Any leads you guys???

JD

Jul 18 '05 #1
5 2791
>>>>> "John" == John D <li***@webcrunchers.com> writes:
I kinda like the idea of being able to define smaller tables as
"in-memory" types, and the really large massive ones in other forms
managed by mySQL, PostGres, or bsddb3 databases.


Use PySQLite (http://pysqlite.sf.net). Make sure that you have at least
version 2.8.1 of SQLite and give the database name as :memory: (this is not
obvious from the documentation). Since SQLite can handle databases upto 2
terabytes, you use it to store "really massive ones" on disk too :-).

Ganesan

--
Ganesan R
Jul 18 '05 #2
"John D." <li***@webcrunchers.com> writes:
[...]
Is such a database available? If so, who is working on it,
or where is it?
Gadfly, PySQLite.

I've heard that Python supports a stand-alone database. I looked
for it in the Docs, and see the "dbm" module. I haven't found
any examples of code, so don't know how to use them. I'm one
of those guys that learn through examples, and have very little
luck in figuring out how to use anything unless I can see examples
of their use.
These sorts of dbs aren't in-memory. Still, if you want to find some
simple examples, I'm sure a few queries like 'group:comp.lang.python
bsddb example' in Google Groups will find something.

My idea is that ALL datums would be Python Dictionary objects, so
not only can you store data, but python objects as well.
Sounds like you might want ZODB (again, not in-memory). It's more of
an object persistence system + BTrees than it is a DBMS, since there's
no query system. IndexedCatalog is a query system built on top of
ZODB, but it's quite feasible to roll your own simple
application-specific queries with BTrees (and the other data
structures it provides). BTrees are rather like dictionaries, but
they support on-demand loading of data, so not everything has to be in
memory at once.

I kinda like the idea of being able to define smaller tables as
"in-memory" types, and the really large massive ones in other forms
managed by mySQL, PostGres, or bsddb3 databases.


That sounds like a recipe for extra work to me. If you need a big
DBMS, use that for everything.
John
Jul 18 '05 #3

"John D." <li***@webcrunchers.com> wrote in message
news:ma*********************************@python.or g...
Does anyone know if there is an "in-ram" dictionary based database
for Python.

I can see some applications where this can really be powerful in
applications requiring 10,000 records or less.

Is such a database available? If so, who is working on it,
or where is it?

I've heard that Python supports a stand-alone database. I looked
for it in the Docs, and see the "dbm" module. I haven't found
any examples of code, so don't know how to use them. I'm one
of those guys that learn through examples, and have very little
luck in figuring out how to use anything unless I can see examples
of their use.

My idea is that ALL datums would be Python Dictionary objects, so
not only can you store data, but python objects as well.

Storage to disk would be through pickled files, or through CSV
format, allowing easy import and export from other databases.

I'm sure something like this exists, I just haven't found it yet,
and if not, then I'm well on my way of implementing one myself.

I kinda like the idea of being able to define smaller tables as
"in-memory" types, and the really large massive ones in other forms
managed by mySQL, PostGres, or bsddb3 databases.

Any leads you guys???
Look at Prevalyar. While it's a Java project, it might give
you some ideas about what's possible if you forget "data base"
and simply think "persistance."

John Roth
JD

Jul 18 '05 #4
John D. wrote:
Does anyone know if there is an "in-ram" dictionary based database
for Python.

I can see some applications where this can really be powerful in
applications requiring 10,000 records or less.

Is such a database available? If so, who is working on it,
or where is it?

I've heard that Python supports a stand-alone database. I looked
for it in the Docs, and see the "dbm" module. I haven't found
any examples of code, so don't know how to use them. I'm one
of those guys that learn through examples, and have very little
luck in figuring out how to use anything unless I can see examples
of their use.

My idea is that ALL datums would be Python Dictionary objects, so
not only can you store data, but python objects as well.

Storage to disk would be through pickled files, or through CSV
format, allowing easy import and export from other databases.

I'm sure something like this exists, I just haven't found it yet,
and if not, then I'm well on my way of implementing one myself.

I kinda like the idea of being able to define smaller tables as
"in-memory" types, and the really large massive ones in other forms
managed by mySQL, PostGres, or bsddb3 databases.

Any leads you guys???


Why not just use Dictionaries, or an object tree? Why do you want tables or
SQL abstraction if it is "in-memory"? Nothing else but your app will be
able to access the "live" data, anyway.

Jul 18 '05 #5

"Andy Todd" <an****@halfcooked.com> wrote in message
news:ma*********************************@python.or g...
John Roth wrote:
"John D." <li***@webcrunchers.com> wrote in message
news:ma*********************************@python.or g...
Does anyone know if there is an "in-ram" dictionary based database
for Python.

I can see some applications where this can really be powerful in
applications requiring 10,000 records or less.

Is such a database available? If so, who is working on it,
or where is it?

I've heard that Python supports a stand-alone database. I looked
for it in the Docs, and see the "dbm" module. I haven't found
any examples of code, so don't know how to use them. I'm one
of those guys that learn through examples, and have very little
luck in figuring out how to use anything unless I can see examples
of their use.

My idea is that ALL datums would be Python Dictionary objects, so
not only can you store data, but python objects as well.

Storage to disk would be through pickled files, or through CSV
format, allowing easy import and export from other databases.

I'm sure something like this exists, I just haven't found it yet,
and if not, then I'm well on my way of implementing one myself.

I kinda like the idea of being able to define smaller tables as
"in-memory" types, and the really large massive ones in other forms
managed by mySQL, PostGres, or bsddb3 databases.

Any leads you guys???

Look at Prevalyar. While it's a Java project, it might give
you some ideas about what's possible if you forget "data base"
and simply think "persistance."

John Roth
JD


Or you could look at the Python port of Prevayler (well, thats how it
started out);

http://www.orbtech.com/wiki/PyPerSyst


Thanks for the reference.

John Roth
Regards,
Andy
--
-------------------------------------------------------------------------- ------ From the desk of Andrew J Todd esq - http://www.halfcooked.com/

Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Cherrish Vaidiyan | last post by:
sir, The following are the steps that i followed in setting up standby database on Red hat Linux 9. i am using Oracle 9i. i have followed the steps in this site : ...
6
by: Marvin Libson | last post by:
Hi All: I am running DB2 UDB V7.2 with FP11. Platform is Windows 2000. I have created a java UDF and trigger. When I update my database I get the following error: SQL1224N A database...
8
by: Kamlesh | last post by:
Hi, How do I know the physical database path of a database. When I goto the DB2INSTANCE users's directory (/home/db2inst1), I see following folders: /db2inst1/NODE0000/SQL00001...
1
by: pintur | last post by:
The message is: SQL1036C Errore di I/O durante l' accesso al database. SQLSTATE=58030 what is the proble? what for restore tables? thanks
3
by: josh.kuo | last post by:
Sorry about the subject, I can't think of a better one. I recently wrote some PHP classes that I think might be of interest to this group. Since I have been reaping the benefits of reading news...
8
by: morleyc | last post by:
Hi, until recently i was quite happy to add data sources from mssql database in visual studio and drag the datasets directly onto the form this creating a directly editable form which worked well....
0
by: Jack | last post by:
Training Classes for Oracle10g, 9i, 8i Certification training in Oracle10g and 9i: DBA, Developer, Discoverer. training conducted at your location worldwide. Courseware licensing also available....
0
by: Winder | last post by:
Training Classes for Oracle10g, 9i, 8i Certification training in Oracle10g and 9i: DBA, Developer, Discoverer. training conducted at your location worldwide. Courseware licensing also available....
0
by: Laurynn | last post by:
# (ebook - pdf) - programming - mysql - php database applicati # (Ebook - Pdf)Learnkey How To Design A Database - Sql And Crystal Report # (ebook-pdf) E F Codd - Extending the Database Relational...
9
by: Peter Duniho | last post by:
Is there a straightfoward API in .NET that allows for inspection of a database? That is, to look at the structure of the database, without knowing anything in advance about it? For example,...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.