473,320 Members | 2,094 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

list() coercion

I have an iterable object. It supports many list-like methods,
specifically __len__. These methods are rather expensive (they result
in database calls, COUNT(*) to be specific), but cheaper than iterating
over the object. Sometimes it is useful to create a list from the
iterator, using list(). However, list() seems to call the object's
__len__, I imagine to pre-allocate space. This is a problem, as
pre-allocation saves much less than is spent doing __len__.

Is there a way I can keep this from happening? Maybe something list()
tries first that I can make fail. (I notice list() catches any
exceptions in __len__ and then will just skip that step)

Ian

Jul 18 '05 #1
5 1836
Ian Bicking wrote:
However, list() seems to call the object's
__len__,

Is there a way I can keep this from happening?


Give your object an __iter__ method that returns
itself.

--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg

Jul 18 '05 #2
On Wed, 2003-07-16 at 22:04, Greg Ewing (using news.cis.dfn.de) wrote:
Ian Bicking wrote:
However, list() seems to call the object's
__len__,
>
Is there a way I can keep this from happening?


Give your object an __iter__ method that returns
itself.


I'm not clear on how that will help...?

It already does have an __iter__ method, but it returns a separate
iterator.

Ian

Jul 18 '05 #3
Ian Bicking wrote:
However, list() seems to call the object's
__len__,

Is there a way I can keep this from happening?

Greg Ewing:
Give your object an __iter__ method that returns
itself.

Ian Bicking wrote:
I'm not clear on how that will help...?

It already does have an __iter__ method, but it returns a separate
iterator.


Just to be explicitly clear

1. You're retrieving record sets from a database.
2. You want to build a list from the database results, one list entry
for each result row.
3. You don't want to have the list preallocated (thus requiring
invocation of __len__()), because the time saving is not worth it
(compared to the cost of the SQL count() operation), since you're
going to be iterating over the record set anyway.

Therefore, use an iterator to build up the list. This will not call
__len__(). Instead, the list will continually be appended to, until
the iterator raise StopIteration. The list may be re-allocated
multiple times, as the number of records retrieved exceeds the
allocation unit size of lists. But this will likely still be lower
cost than your SQL count().

The object returned from the __iter__() method should have a .next()
method which returns the next row in your result set. So if you have
implemented a .next() on your result set object, define your
__.iter__() method as follows

class ResultSet:

def next(self):
"Pseudocode"
result = databasecursor.fetchone()
if result:
return result
else:
raise StopIteration

def __iter__(self):
return self

Have I understood the problem correctly?

--
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/mailto/alan
Jul 18 '05 #4

"Ian Bicking" <ia**@colorstudy.com> wrote in message
news:ma**********************************@python.o rg...
I have an iterable object. It supports many list-like methods,
specifically __len__. These methods are rather expensive (they result
in database calls, COUNT(*) to be specific), but cheaper than iterating
over the object. Sometimes it is useful to create a list from the
iterator, using list(). However, list() seems to call the object's
__len__, I imagine to pre-allocate space. This is a problem, as
pre-allocation saves much less than is spent doing __len__.

Is there a way I can keep this from happening? Maybe something list()
tries first that I can make fail. (I notice list() catches any
exceptions in __len__ and then will just skip that step)


Instead of:

list(yourobj)

use:

list(iter(yourobj))

If that doesn't help, create your own wrapper:

def myiter(it):
for elem in it:
yield it

list(myiter(yourobj))
This idea is to provide 'list' with a wrapper that only supplies
the iter methods and not the len method.
Raymond Hettinger
Jul 18 '05 #5
On Thu, 2003-07-17 at 10:08, Raymond Hettinger wrote:
Instead of:

list(yourobj)

use:

list(iter(yourobj))


Sigh... that's too bad. I'll probably just get rid of the __len__
method instead, as list(yourobj) (list(myobj)?) is a more appealing
idiom than len(somebodysobj).

Ian

Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: massimo | last post by:
Hey, I wrote this program which should take the numbers entered and sort them out. It doesn¹t matter what order, if decreasing or increasing. I guess I'm confused in the sorting part. Anyone...
1
by: Neil Schemenauer | last post by:
The title is perhaps a little too grandiose but it's the best I could think of. The change is really not large. Personally, I would be happy enough if only %s was changed and the built-in was...
1
by: ziga.seilnacht | last post by:
""" I am trying to write some classes representing the quaternion number. I wrote a base class, which implements only the numerical interface, and a few subclasses, which provide methods for their...
43
by: michael.f.ellis | last post by:
The following script puzzles me. It creates two nested lists that compare identically. After identical element assignments, the lists are different. In one case, a single element is replaced. In...
40
by: nufuhsus | last post by:
Hello all, First let me appologise if this has been answered but I could not find an acurate answer to this interesting problem. If the following is true: C:\Python25\rg.py>python Python...
11
by: kudruu | last post by:
Hi, I am trying to find a way to populate a list of active Com ports on a computer. There may be around 30 on one computer and all connected to different Buses but I am looking for one in...
12
by: Godzilla | last post by:
Hello, I'm trying to find a way to convert an integer (8-bits long for starters) and converting them to a list, e.g.: num = 255 numList = with the first element of the list being the...
0
by: mazerj2006 | last post by:
I'm an old hand at python, but totally new to win32com -- I've run into a snag that seems to be related to python's (usually handy) lack of distinction between floats and doubles. Can anyone...
9
by: AGP | last post by:
I've been scratching my head for weeks to understand why some code doesnt work for me. here is what i have: dim sVal as string = "13.2401516" dim x as double x = sVal debug.writeline ( x)
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.