473,216 Members | 1,920 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,216 software developers and data experts.

List of objects X Database

I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?

Thanks!
Eduardo

Oct 3 '07 #1
5 1188

On Oct 3, 2007, at 1:01 PM, MindMaster32 wrote:
I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?
Berkeley DB?
Oct 3 '07 #2
Michael Bentley wrote:
>
On Oct 3, 2007, at 1:01 PM, MindMaster32 wrote:
>I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
No it would not:
<URL:http://docs.python.org/lib/module-sqlite3.html>
(Well, its new in Python 2.5, but you didn't say you were restricted to
some older version.)

/W
Oct 3 '07 #3
On 3 oct, 22:01, MindMaster32 <egerm...@gmail.comwrote:

Hi,

Maybe PyDbLite (http://quentel.pierre.free.fr/PyDbLite/index.html) is
what you need : a single Python module, compatible with Python 2.3+,
that lets you manipulate data in memory

You can manage a database like this :

import PyDbLite
db = PyDbLite.Base("dummy")
db.create("record,"artist","released") # fields are untyped
db.insert("Closer","Joy Division",1980)
db.insert("Different Class","Pulp",1996)
db.commit() # save to disk
print [ r for r in db if r["released"] 1990 ]
print db(artist="Joy Division")

Regards,
Pierre

Oct 3 '07 #4
MindMaster32 wrote:
I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.

Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?
Python 2.5 ships with SQLite and also supports in-memory
storage of the data.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Oct 04 2007)
>>Python/Zope Consulting and Support ... http://www.egenix.com/
mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
__________________________________________________ ______________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
Oct 4 '07 #5
MindMaster32 wrote:
>I am writing a script that has to read data from an ASCII file of
about 50 Mb and do a lot of searches and calculations with that data.
That would be a classic problem solved by the use of a database
(SQLite would suit just fine), but that would require the user to
install more packages other than python itself, and that I am trying
to avoid.
Since the data is not too large, I wonder if there is another way to
store all data in memory and work with it more or less like a
database, doing searches and working with datafields.

This is not clear to me how can be implemented. I thought of creating
a class with the data structure, and creating a list of objects of
that class, each one containing one line of data from the "database".

Any thoughts or suggestions?

Thanks!
Eduardo
What about shelve? It Requires some db support, yes....And what about
just reading the file and making a dict? It is a `csv' like file, so you
can just read it and generate a dict?

Cheers.
Gerardo
Oct 4 '07 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
by: Der Andere | last post by:
I need to implement a sorted (ordered) list. STL has no sorted list type as far as I know. Is there a (straight) way to implement a sorted list using STL? BTW: The type of items in the list will...
3
by: kyle.tk | last post by:
So I have a central list of python objects that I want to be able to share between different process that are possibly on different computers on the network. Some of the processes will add objects...
1
by: zeke | last post by:
I have an Access database that suddenly opens without the list of objects (You know, Tables, Queries, Forms, Reports . .) at the left of the screen. The objects still exist - for instance, I can...
1
by: matthew.macdonald-wallace | last post by:
Hi all, I'm trying to add functionality to an app written in C# so that a list of people is displayed in a list box on the left hand side and then selected individuals can be added to another...
2
by: G Gerard | last post by:
Hello I am trying to connect to a msaccess database and then create a list of the objects in that database (more specifically the tables) and then create a list of the fields (including the...
0
by: Jeff | last post by:
ASP.NET 2.0 My webpage has a GridView showing rows from a table in the database. All these rows are stored in the session object -what's actually stored is generic list of objects, as this:...
1
by: smoothmunkey | last post by:
I want to make a switchboard with a list box to open objects... but i want the list box to be dynamic (as new queries or forms are created, i want them added). so far i have a combo box that selects...
10
by: AZRebelCowgirl73 | last post by:
This is what I have so far: My program! import java.util.*; import java.lang.*; import java.io.*; import ch06.lists.*; public class UIandDB {
0
by: jappenzeller | last post by:
We're using the HttpRuntime Cache for our objects and we're trying to do the following. We pull lists of objects from the database and put them in cache. We put a reference to the list in...
1
by: krishna81m | last post by:
I am unable to create a list of objects dynamically. I run a SQL query on the database and populate objects of types that I also know aprior. I read the query as a string, run it on the database and...
1
isladogs
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...
0
by: veera ravala | last post by:
ServiceNow is a powerful cloud-based platform that offers a wide range of services to help organizations manage their workflows, operations, and IT services more efficiently. At its core, ServiceNow...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: mar23 | last post by:
Here's the situation. I have a form called frmDiceInventory with subform called subfrmDice. The subform's control source is linked to a query called qryDiceInventory. I've been trying to pick up the...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.