471,353 Members | 1,545 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,353 software developers and data experts.

List of objects X Database

I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?

Thanks!
Eduardo

Oct 3 '07 #1
5 1123

On Oct 3, 2007, at 1:01 PM, MindMaster32 wrote:
I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?
Berkeley DB?
Oct 3 '07 #2
Michael Bentley wrote:
>
On Oct 3, 2007, at 1:01 PM, MindMaster32 wrote:
>I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
No it would not:
<URL:http://docs.python.org/lib/module-sqlite3.html>
(Well, its new in Python 2.5, but you didn't say you were restricted to
some older version.)

/W
Oct 3 '07 #3
On 3 oct, 22:01, MindMaster32 <egerm...@gmail.comwrote:

Hi,

Maybe PyDbLite (http://quentel.pierre.free.fr/PyDbLite/index.html) is
what you need : a single Python module, compatible with Python 2.3+,
that lets you manipulate data in memory

You can manage a database like this :

import PyDbLite
db = PyDbLite.Base("dummy")
db.create("record,"artist","released") # fields are untyped
db.insert("Closer","Joy Division",1980)
db.insert("Different Class","Pulp",1996)
db.commit() # save to disk
print [ r for r in db if r["released"] 1990 ]
print db(artist="Joy Division")

Regards,
Pierre

Oct 3 '07 #4
MindMaster32 wrote:
I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.

Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?
Python 2.5 ships with SQLite and also supports in-memory
storage of the data.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Oct 04 2007)
>>Python/Zope Consulting and Support ... http://www.egenix.com/
mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
__________________________________________________ ______________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
Oct 4 '07 #5
MindMaster32 wrote:
>I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?

Thanks!
Eduardo
What about shelve? It Requires some db support, yes....And what about
just reading the file and making a dict? It is a `csv' like file, so you
can just read it and generate a dict?

Cheers.
Gerardo
Oct 4 '07 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

10 posts views Thread by Der Andere | last post: by
3 posts views Thread by kyle.tk | last post: by
1 post views Thread by matthew.macdonald-wallace | last post: by
reply views Thread by Jeff | last post: by
reply views Thread by XIAOLAOHU | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.