List of objects X Database

[MindMaster32]

I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?

Thanks!
Eduardo

[Michael Bentley]

On Oct 3, 2007, at 1:01 PM, MindMaster32 wrote:

I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?

Berkeley DB?

[Wildemar Wildenburger]

Michael Bentley wrote:

>
On Oct 3, 2007, at 1:01 PM, MindMaster32 wrote:

>I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.

No it would not:
<URL:http://docs.python.org/lib/module-sqlite3.html>
(Well, its new in Python 2.5, but you didn't say you were restricted to
some older version.)

/W

[Pierre Quentel]

On 3 oct, 22:01, MindMaster32 <[e-mail]:

Hi,

Maybe PyDbLite (http://quentel.pierre.free.fr/PyDbLite/index.html) is
what you need : a single Python module, compatible with Python 2.3+,
that lets you manipulate data in memory

You can manage a database like this :

import PyDbLite
db = PyDbLite.Base("dummy")
db.create("record,"artist","released") # fields are untyped
db.insert("Closer","Joy Division",1980)
db.insert("Different Class","Pulp",1996)
db.commit() # save to disk
print [ r for r in db if r["released"] 1990 ]
print db(artist="Joy Division")

Regards,
Pierre

[M.-A. Lemburg]

MindMaster32 wrote:

I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.

Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?

Python 2.5 ships with SQLite and also supports in-memory
storage of the data.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Oct 04 2007)

>>Python/Zope Consulting and Support ... http://www.egenix.com/
mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

__________________________________________________ ______________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611

[Gerardo Herzig]

MindMaster32 wrote:

>I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?

Thanks!
Eduardo

What about shelve? It Requires some db support, yes....And what about
just reading the file and making a dict? It is a `csv' like file, so you
can just read it and generate a dict?

Cheers.
Gerardo