anyway to create a table-like object?

[Wensui Liu]

Dear all,

is there a way to create a 2-dimension-table-like object such that I
can reference a whole column directly using something like :
mytable.column1 ?

by the way, can python be used for database programming to pull large
volume data (hundred M or even several Gs) out of large database and
do data manipulation and reporting ? if yes, how is the speed and
efficiency ?

thanks.

wensui

[James Stroud]

Wensui Liu wrote:

Dear all,

is there a way to create a 2-dimension-table-like object such that I
can reference a whole column directly using something like :
mytable.column1 ?

by the way, can python be used for database programming to pull large
volume data (hundred M or even several Gs) out of large database and
do data manipulation and reporting ? if yes, how is the speed and
efficiency ?

thanks.

wensui

I have a table class that works like this:

atable['some_col'] ==list of values
atable[some_row_index] ==list of values
atable[slice_start:slice_end] ==new table
atable[('some_col', 'other_col', 'another_col')] ==3 col. table
atable['some_col', some_row_index] ==one value
atable['some_col', slice_start:slice_end] ==list of values
atable[('some_col', 'other_col'), some_row_index] ==list of 2 vals
atable[('some_col', 'other_col'), slice_start:end]) ==2 col. table

As you can see, it has a lot of flexibility, but may not be the ultimate
in terms of speed (and purists may not like the way __getitem__ returns
data based on context). It is for people who care about convenience over
speed and do not adhere to some puritanical ideal about how __getitem__
should work. There is little typechecking (in accord with my own brand
of puritanical philosophy), so be careful if you have ints as column
headers. The datastructure is implemented as a list of lists (by row)
and the "keys" (column headings) are a list. So size limitations are
limited by memory. Theoretically, the __getitem__ (where all the work
and decisions are done) could be factored out and wrapped around dbi for
hard-core database usage. This might entail some serious creativity.

It also has some other features, such as returning matches, etc. It is
not intended, at all, to rival an actual database such as mysql--the
model is more based on how Joe Users use excel files.

If you are interested, I can send you the module. Beyond actual use in
my own and my wife's work, there has been little testing.

James

[James Stroud]

James Stroud wrote:

Wensui Liu wrote:

>Dear all,

is there a way to create a 2-dimension-table-like object such that I
can reference a whole column directly using something like :
mytable.column1 ?

by the way, can python be used for database programming to pull large
volume data (hundred M or even several Gs) out of large database and
do data manipulation and reporting ? if yes, how is the speed and
efficiency ?

thanks.

wensui

I have a table class that works like this:

atable['some_col'] ==list of values
atable[some_row_index] ==list of values
atable[slice_start:slice_end] ==new table
atable[('some_col', 'other_col', 'another_col')] ==3 col. table
atable['some_col', some_row_index] ==one value
atable['some_col', slice_start:slice_end] ==list of values
atable[('some_col', 'other_col'), some_row_index] ==list of 2 vals
atable[('some_col', 'other_col'), slice_start:end]) ==2 col. table

As you can see, it has a lot of flexibility, but may not be the ultimate
in terms of speed (and purists may not like the way __getitem__ returns
data based on context). It is for people who care about convenience over
speed and do not adhere to some puritanical ideal about how __getitem__
should work. There is little typechecking (in accord with my own brand
of puritanical philosophy), so be careful if you have ints as column
headers. The datastructure is implemented as a list of lists (by row)
and the "keys" (column headings) are a list. So size limitations are
limited by memory. Theoretically, the __getitem__ (where all the work
and decisions are done) could be factored out and wrapped around dbi for
hard-core database usage. This might entail some serious creativity.

It also has some other features, such as returning matches, etc. It is
not intended, at all, to rival an actual database such as mysql--the
model is more based on how Joe Users use excel files.

If you are interested, I can send you the module. Beyond actual use in
my own and my wife's work, there has been little testing.

James

Also, if you want columns as attributes (which may be pretty cool):

def __getattr__(self, att):
if att in self.headers:
return self[att]
else:
return self.__getattribute__(att)

James