it seems that range() can be really slow:
the following program will run, and the last line shows how long it ran
for:
import time
startTime = time.time()
a = 1.0
for i in range(0, 30000):
if i in range (0, 10000):
a += 1
if not i % 1000: print i
print a, " ", round(time.time () - startTime, 1), "seconds"
---------------------------------
the last line of output is
---------------------------------
10001.0 22.8 seconds
so if i change the line
if i in range (0, 10000):
to
if i >= 0 and i < 10000:
the the last line is
10001.0 0.2 seconds
so approximately, the program ran 100 times faster!
or is there an alternative use of range() or something similar that can
be as fast?
Jul 18 '06
45  8580 
K.S.Sreeram wrote:
Simon Forman wrote:
Nick Craig-Wood wrote:
Sets are pretty fast too, and have the advantage of flexibility in
that you can put any numbers in you like
I know this is self-evident to most of the people reading this, but I
thought it worth pointing out that this is a great way to test
membership in range(lo, hi, step) without doing "the necessary
algebra".
i.e. n in set(xrange(0, 10000, 23)) ...
This is very very misleading... here are some timings :
Yes it is. I'm sorry about that.
python -mtimeit "n=5000" "n in set(xrange(0,10 000))"
1000 loops, best of 3: 1.32 msec per loop
python -mtimeit "n=5000" "n in xrange(0,10000) "
1000 loops, best of 3: 455 usec per loop
python -mtimeit "n=5000" "0 <= n < 10000"
1000000 loops, best of 3: 0.217 usec per loop
sets are fast only if you create them *once* and use them again and
again. even in that case, the sets use up O(n) memory.
That's what I meant. But I didn't state it clearly.
One of the things I like most about python is that it allows you to
specify the problem that you want to solve without a great deal of
difficulty as to *how* to specify it. To me, and perhaps others, "T =
set(xrange(0, 10000, 23))" and "n in T" are somewhat easier to read
and write than "not n % 23 and 0 <= n < 10000", YMMV.
In the given case a set of ~(10000 / 23) ints would not usually be too
burdensome on ram, and the code runs close to the same speed as
compared to the direct calculation:
from timeit import Timer
times = 100000
Max = 10000
n = 5000
T = set(xrange(0, Max, 23))
s1 = 'n in T'
s2 = 'not n %% 23 and 0 <= n < %s' % Max
setup = 'from __main__ import n, T'
S1 = Timer(s1, setup).repeat(n umber=times)
S2 = Timer(s2, setup).repeat(n umber=times)
print "%.3f usec/pass" % (1000000 * min(S1) / times)
print "%.3f usec/pass" % (1000000 * min(S2) / times)
On my machine this printed:
0.476 usec/pass
0.552 usec/pass
>
with comparison operators, you don't need extra memory *and* there is no
pre-computation required.
When I set Max = 100000000 in the above test code there was serious
disk thrashing... ;-)
>
[sreeram;]
FWIW, in production code I would certainly use the comparison
operators. A kilobyte saved is a kilobyte earned.
Peace,
~Simon Su************@ gmail.com wrote:
it seems that range() can be really slow:
if i in range (0, 10000):
My original use was like this:
if i in range (iStart, iEnd):
listData.append (a)
in which iStart is 1000 and iEnd is 1008
so in that case, the program ran fine...
but later on, i wanted to include all data, so I relaxed the range by
setting iStart to 0 and iEnd to 9999 and later on i found that the
program was slow due to this.
So looks like the usage of
if sDay in ("Tue", "Wed", "Thu"):
is more like good use of "in a list" but in range(0,10000) will be a
big search in a list.
Simon Forman wrote:
To me, and perhaps others, "T =
set(xrange(0, 10000, 23))" and "n in T" are somewhat easier to read
and write than "not n % 23 and 0 <= n < 10000", YMMV.
Eh? How is the first easier to read than the second?? You have a nested
function call in the first!
Regardless, testing if a member is part of a ranged set is always going
to be slower. It's the nature of what you're doing. Building a set and
then searching it takes much longer than a single modulus and
subtraction (which is all an integer comparison is).
On 19/07/2006 1:05 AM, Dan Bishop wrote:
Paul Boddie wrote:
>Yes, he wants range to return an iterator, just like xrange more or
less does now. Given that xrange objects support __getitem__, unlike a
lot of other iterators (and, of course, generators), adding
__contains__ wouldn't be much of a hardship. Certainly, compared to
other notational conveniences bounced around on the various development
lists, this one would probably provide an order of magnitude
improvement on the usual bang per buck development ratio.
xrange already has __contains__.
As pointed out previously, xrange is a function and one would not expect
it to have a __contains__ method.
The objects returned by xrange do not (according to my reading of the
2.4.3 version of Objects/rangeobject.c) have a __contains__ method.
I find it difficult to believe that an inefficient __contains__ has been
implemented since.
Perhaps you are unaware that the mere fact that an object supports the
"in" operation does not mean that this support is provided by a
__contains__ method. The following section of the manual may help:
"""
The membership test operators (in and not in) are normally implemented
as an iteration through a sequence. However, container objects can
supply the following special method with a more efficient
implementation, which also does not require the object be a sequence.
__contains__( self, item)
Called to implement membership test operators. Should return true
if item is in self, false otherwise. For mapping objects, this should
consider the keys of the mapping rather than the values or the key-item
pairs.
"""
John Machin wrote:
On 19/07/2006 1:05 AM, Dan Bishop wrote:
xrange already has __contains__.
As pointed out previously, xrange is a function and one would not expect
it to have a __contains__ method.
Well, you pointed out that range is a function, but xrange seems to be
a type...
>>xrange
<type 'xrange'>
>>dir(xrange)
['__class__', '__delattr__', '__doc__', '__getattribute __',
'__getitem__', '__hash__', '__init__', '__iter__', '__len__',
'__new__', '__reduce__', '__reduce_ex__' , '__repr__', '__reversed__',
'__setattr__', '__str__']
No __contains__ method, though, at least in 2.4.1.
The objects returned by xrange do not (according to my reading of the
2.4.3 version of Objects/rangeobject.c) have a __contains__ method.
As confirmed by the above evidence.
I find it difficult to believe that an inefficient __contains__ has been
implemented since.
So do I. As you go on to say, the usual sequence traversal mechanisms
are probably used to support the "in" operator. Whether it's a pressing
matter to add support for a more efficient mechanism depends on how
often people want to use ranges in the way described. Perhaps I'll
write a patch - who knows? ;-)
Paul
tac-tics wrote:
Simon Forman wrote:
To me, and perhaps others, "T =
set(xrange(0, 10000, 23))" and "n in T" are somewhat easier to read
and write than "not n % 23 and 0 <= n < 10000", YMMV.
Eh? How is the first easier to read than the second?? You have a nested
function call in the first!
I find the first form more immediately comprehensible than the latter.
I know what xrange() does, and I know what set() does, and "nested
function calls" give me no trouble, whereas the latter form with a
modulus, negation, and comparisons would take me a bit longer both to
compose and/or understand.
If this is not the case for you then by all means please disregard my
posting. YMMV.
>
Regardless, testing if a member is part of a ranged set is always going
to be slower.
Yes. Roughly 0.0000001 seconds slower on my five year old computer.
I'm not worried.
It's the nature of what you're doing. Building a set and
then searching it takes much longer than a single modulus and
subtraction (which is all an integer comparison is).
Building the set, yes, but searching the set is very close to the same
speed, even for rather large sets. If you were performing the search
30000 times (like in the OP) it would only take about three thousandths
of a second longer, and that's on my old slow computer.
If I were doing this a thousand times more often, or on a range of a
million or more, or in production code, or with ranges that changed
often, then I would certainly take the time to write out the latter
form.
Peace,
~Simon
Paul Boddie wrote:
John Machin wrote:
On 19/07/2006 1:05 AM, Dan Bishop wrote:
>
xrange already has __contains__.
As pointed out previously, xrange is a function and one would not expect
it to have a __contains__ method.
Well, you pointed out that range is a function, but xrange seems to be
a type...
>xrange
<type 'xrange'>
>dir(xrange)
['__class__', '__delattr__', '__doc__', '__getattribute __',
'__getitem__', '__hash__', '__init__', '__iter__', '__len__',
'__new__', '__reduce__', '__reduce_ex__' , '__repr__', '__reversed__',
'__setattr__', '__str__']
No __contains__ method, though, at least in 2.4.1.
The objects returned by xrange do not (according to my reading of the
2.4.3 version of Objects/rangeobject.c) have a __contains__ method.
As confirmed by the above evidence.
I find it difficult to believe that an inefficient __contains__ has been
implemented since.
So do I. As you go on to say, the usual sequence traversal mechanisms
are probably used to support the "in" operator. Whether it's a pressing
matter to add support for a more efficient mechanism depends on how
often people want to use ranges in the way described. Perhaps I'll
write a patch - who knows? ;-)
My mistake. I should have looked at dir(xrange) before posting.
But the point remains that xrange's "implicit __contains__" runs in
linear time when a constant-time algorithm exists.
Paul Boddie <pa**@boddie.or g.ukwrote:
John Machin wrote:
range() and xrange() are functions. You are suggesting that 2
*functions* should acquire a __contains__ method each? I trust not.
Well, range is a function in the current implementation, although its
usage is similar to that one would get if it were a class, particularly
a subclass of list or one providing a list-style interface. With such a
class, you could provide a __contains__ method which could answer the
question of what the range contains based on the semantics guaranteed
by a range (in contrast to a normal list).
You'd also have to override just about every mutating method to switch
back to a "normal" __contains__ (or change self's type on the fly) -- a
pretty heavy price to pay.
I have often noticed that subclassing list, dict and maybe set has this
kind of issue: the need to track every possible change to the object.
Maybe a good mechanism to have for the purpose would be to add to
mutable types a "hook" method, say __mutator__, which gets called either
right before or right after any mutating method (there are different
tradeoffs for before-calls and after-calls), presumably passing along
the *a and **k for generality (although it might be faster for the base
case to avoid that); the base types would have a no-op implementation,
but subtypes could easily override just the hook to facilitate their
task of maintaining extra state (could be as little as a per-instance
flag recording whether the object is guaranteed to be still "pristine") .
At C level, that might be an extra slot tp_mutator, left NULL in base
types to indicate "no mutator-hook method implemented here".
Like any other addition of, or change to, functionality, this would of
course be a proposal for 2.6, since 2.5 is feature-frozen now.
Alex
Alex Martelli wrote:
Paul Boddie <pa**@boddie.or g.ukwrote:
Well, range is a function in the current implementation, although its
usage is similar to that one would get if it were a class, particularly
a subclass of list or one providing a list-style interface. With such a
class, you could provide a __contains__ method which could answer the
question of what the range contains based on the semantics guaranteed
by a range (in contrast to a normal list).
You'd also have to override just about every mutating method to switch
back to a "normal" __contains__ (or change self's type on the fly) -- a
pretty heavy price to pay.
A subclass of list is probably a bad idea in hindsight, due to various
probable requirements of it actually needing to be a list with all its
contents, whereas we wanted to avoid having anything like a list around
until the contents of this "lazy list" were required by the program. If
we really wanted to subclass something, we could consider subclassing
the slice class/type, but that isn't subclassable in today's Python for
some reason, and it doesn't really provide anything substantial,
anyway. However, Python being the language it is, an appropriately
behaving class is quite easily written from scratch.
Paul
Paul Boddie <pa**@boddie.or g.ukwrote:
Alex Martelli wrote:
Paul Boddie <pa**@boddie.or g.ukwrote:
>
Well, range is a function in the current implementation, although its
usage is similar to that one would get if it were a class, particularly
a subclass of list or one providing a list-style interface. With such a
class, you could provide a __contains__ method which could answer the
question of what the range contains based on the semantics guaranteed
by a range (in contrast to a normal list).
You'd also have to override just about every mutating method to switch
back to a "normal" __contains__ (or change self's type on the fly) -- a
pretty heavy price to pay.
A subclass of list is probably a bad idea in hindsight, due to various
probable requirements of it actually needing to be a list with all its
contents, whereas we wanted to avoid having anything like a list around
until the contents of this "lazy list" were required by the program. If
we really wanted to subclass something, we could consider subclassing
the slice class/type, but that isn't subclassable in today's Python for
some reason, and it doesn't really provide anything substantial,
anyway. However, Python being the language it is, an appropriately
behaving class is quite easily written from scratch.
Nevertheless, that class will still need to implement every single
method of the list type; making it a subclass of list has some advantage
in that every such implementation of a method can basically fill the
real list, self.__class__= list, and leave all the rest, forevermore
(explicitly here, implicitly in the future), to class list. Performance
should be much better than by working off semi-deprecated UserList.
A "hook method" __mutator__ (ideally called _before_ in this case), as I
was proposing (for 2.6 or later), would make such approaches way easier
and handier (and would help with most use cases I can think of for
subclassing list, dict or set).
Alex This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Chris Dutrow |
last post by:
I searched around on the net for a bit, couldn't find anything though.
I would like to find some code for a function where I input A Range Of
Integers
For example: Function( 1, 100 );
And the function will return me an array holding a random subset of integers
in that range of a size that I specify
So the Function would Probabaly look something like this:
|
by: illo |
last post by:
hello guys...
a little but very imposrtant question...
I have a broadband connection in my office, and i would like to use this
connection also from my house (150 meters from the office)...
I bought a router and 2 range extenders of belkin....
With the router everything functions very well,
|
by: sameer_deshpande |
last post by:
Hi,
I need to create a partition table but the column on which I need to
create a partition may not have any logical ranges. So while creating
or defining partition function I can not use any range.
like
CREATE PARTITION FUNCTION my_part_func (NUMERIC(7)) AS RANGE LEFT FOR
VALUES (1,100,1000);
|
by: toton |
last post by:
Hi,
I want ro iterate over a few container class within a range
specified, instead of begin & end.
How to construct a range class, which takes start & end, and iteration
is available within that range only. Itaration may be const,
bidiractional, forward or backward.
Say I have a vector or other container class,
like vector<intvec;
and want to return a range class like range(vec.begin()+5,
|
by: Alexander Higgins |
last post by:
Hello,
I would like to thank everyone for there help in advance. I have form
which is using an iframe as a Rich Text Editor. Everything works as
expected in IE but I have two issues with Firefox. I am using the
following to make the frame editable:
tmp=document.getElementById("adeditor").contentWindow.document
tmp.designMode="On";
| | |
by: Joe Goldthwaite |
last post by:
I've been playing with Python a bit. Doing little performance benchmarks and
working with Psyco. It's been fun and I've been learning a lot. For
example, in a previous post, I was looking for a way to dynamically add new
runtime function to a class. Martin told me to use a class instance
variable instead. It turns out that's faster than hard coding a list of
functions. Thanks Martin.
I read that the range function builds a list and...
|
by: Rafael Cunha de Almeida |
last post by:
Hi,
I've found several sites on google telling me that I shouldn't use
rand() % range+1
and I should, instead, use something like:
lowest+int(range*rand()/(RAND_MAX + 1.0))
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
| | |
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
|
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |
