473,554 Members | 2,197 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Comment on PEP-0322: Reverse Iteration Methods

Please comment on the new PEP for reverse iteration methods.
Basically, the idea looks like this:

for i in xrange(10).iter _backwards(): # 9,8,7,6,5,4,3,2 ,1,0
<do something with i>

The HTML version is much more readable than the ReST version.
See:
http://www.python.org/peps/pep-0322.html
Several interesting ideas surfaced in the pre-pep thread:

* Call it ireverse() instead of iter_backwards( ).

Good idea. This is much more pithy.
* Use object properties instead of methods.

I can't yet claim to understand what the author is really
proposing. It has something to do which providing
access to an object that responds to iter, getitem, and
getslice with reversed indices.
* using a single function that looks for an __riter__ magic
method and, if not found, use __getitem__ and __len__
to build a reverse iterator.

A workable version of this was posted.
It saves implementing some object methods at the
expense of adding a new builtin function and of
creating a new magic method name.
It is slower than direct access to the underlying object.
It crashes when applied to an infinite iterator.
It produces bizarre results when applied to mappings.
* special markers for infinite iterators

This idea is interesting but doesn't extend well
when the object is wrapped by another iterator.
Also, there is no automatic way to determine
which generators can be infinite.
Raymond Hettinger
Jul 18 '05
59 4257
[Stephen Horne]
The question is - is the exercise worthwhile. I'm not sure even though
it's my own idea. I think it is nice in its way, and a lot more
flexible than the 'iter_backward' method proposal, but that is quite
likely a bad thing as it is probably massive overkill for the problem
being solved - and in particular the extra flexibility has costs.

What the hell, though - it was fun to play with for a bit!


Yes, it's a one ton solution to a one ounce problem,
but it was an interesting exercise and diversion.

Raymond Hettinger
Jul 18 '05 #11
[sebastien]
So, I'd expect to have a riter() function which would call the
__riter__ special method.
Okay, changed pep to list riter() as an alternative.

And I still think you don't need it often enough to put it in the
builtin namespace, so the function should go in the itertools module.


If you have a magic method, __riter__, then the corresponding
function needs to be a builtin. They go hand in hand. The
core parts of the language need to be usable without having
to know about itertools.
Raymond
Jul 18 '05 #12
On Thu, 25 Sep 2003, Steve Holden wrote:
"Raymond Hettinger" <vz******@veriz on.net> wrote ...
Please comment on the new PEP for reverse iteration methods.


Looks good. But

[...]
It crashes when applied to an infinite iterator.

[...]
Hmm. My little brain is having difficulty imagining anything that won't.
What am I missing?


ideally, riter(riter(som eInfiniteIterat or)) would work, though!

tom

--
I see large and small words on this page, arranged in long rows separated by little dotty characters. Suspect written by little dotty characters, too. -- RonJeffries

Jul 18 '05 #13
Raymond Hettinger:
Beauty and clarity are a matter of taste and we differ here.
To some (including me), s[::-1] looks more like line noise
than s.iter_backward s().
Understood. In all this, let me make it clearthat I
have a preference for a function over a method, but won't
call it a wart of the BFDL thinks otherwise. I point these
out because I'm rather detail oriented.
Also, a variant implementation may be even more concise,


You must be pretty confident to take on the Timbot's code ;-)


Well, I did say "may". I wouldn't really want to use it since
inverting the score before the sort is a tricky thing to see and
somewhat unexpected.

(I'm half expecting some enlightening comment about how
negation has subtle interactions with IEEE 754 math -- though
I thought the point of 754 was to make naive approaches like
mine usually correct. :)
verfiles = os.listdir('/usr/lib/setup')
verfiles = [name for name in verfiles
if name.startswith ("slack-version-")]


I like your version better and recommend you submit it as a patch.
I'll take out the ifilter() comment.


But that goes against the pretty good motto of "if it ain't broke,
don't touch it." Granted, XP-style tests are supposed to
help out with that, but I don't have a /usr/lib/setup where I
can test this.
] random.shuffle( ) uses for i in xrange(len(x)-1, 0, -1)
This isn't a use case. The xrange never returns a 0 while
the iter_backwards one will.


It is an important use case. The replacement code is:

for i in xrange(1,len(x) ). iter_backwards( )


Ahh, right. Didn't think about that. You should include that
wording in the PEP.
Whenever the indices have any complexity, the forwards version,
followed by .iter_backwards () is, IMHO, much easier to mentally verify.
I agree. When I do need to go backwards I often forgot to get
both -1s in place. Ie, I'll do (n, 0) instead of (n-1, -1).
Well, don't forgot much and more, but there's some tension in
my mind as I worry if I got it correct or not.
BTW, don't underestimate the intensity of resistance to adding
new builtins.
For me it's two things: remembering what all these new things
do, and attempting to adhere to the 'don't use variables with
the same name as builtins' guiding philosophy. I don't adhere
to the latter wrt the type named 'file'.
Also, I have a preference for creating something that
is as robust as possible. Making a builtin function that
doesn't apply to all objects; behaves badly with mappings;
and crashes with an infinite iterator is not my idea of robust.
The backup which uses __getitem__ and negative offsets
does work for all objects, behaves exactly as badly as the
existing iter, and doesn't crash with an infinite iterator, no?

It's slow, but if you want the performance, make an __riter__.
I really don't understand the overpowering urge to make this a function
and add a new magic method protocol. Why isn't there a similar rage
against dict.iteritems( )?
Overpowering?

My preference is for a function because of:
- similaraties to iter, which is also a function
- there are many types of iteration orderings, and not all can be
special cased methods.
- I don't see backwards iteration as all that important

As for iteritems, I was annoyed about it because it meant my
dictionary-like code would become more complicated to
implement, until I came across UserDict.DictMi xin, which
lets user-defined code implement just a few methods then
builds the rest from there.

That suggests a UserList.ListMi xin might be useful, but it
isn't obvious to me that it would be.

I also note that items is a method so by similarity it makes
sense that iteritems also be a method. Additionally, many
things offer forward iteration, while only dicts offer items.

Should dicts support riteritems, ritervalues?
Okay, we simply disagree here. That's fine.


Yup. I don't think I have anything useful or even interesting
to offer to this dicusssion. I also think that with these changes,
the PEP is pretty complete and solid.

Andrew
da***@dalkescie ntific.com
Jul 18 '05 #14
On Thu, 25 Sep 2003 00:58:45 GMT, "Raymond Hettinger"
<vz******@veriz on.net> wrote:
Please comment on the new PEP for reverse iteration methods.


One more thought.

Having an iter_backwards method on xrange seems wrong. This suggests
multiple functions/methods...

To generate backward ranges
range_backwards function
xrange_backward s funtion

for i in range_backwards (10) :
...

To iterate an object backwards
either method for appropriate objects, or
iter_backward() function and __iter_backward __ magic method
(I don't mind abbreviating the 'iter', but I think the 'backward'
needs to be kept very obvious and not reduced to an easily missed
'r' prefix).

for i in iter_backwards (seq) :
...

To enumate backwards etc
Add an enumerate_backw ard function which requires the input to
support __len__.

for i in enumerate_backw ards (seq) :
...

For backward slicing
Add a function which translates the slice to a backward equivalent
before applying it...

for i in slice_backwards (seq, start, stop, step) :
...
--
Steve Horne

steve at ninereeds dot fsnet dot co dot uk
Jul 18 '05 #15
"Raymond Hettinger" <vz******@veriz on.net> wrote in message news:<wC******* **********@nwrd ny01.gnilink.ne t>...
If you have a magic method, __riter__, then the corresponding
function needs to be a builtin. They go hand in hand. The
core parts of the language need to be usable without having
to know about itertools.


I don't think so:
We already have this situation for the the copy module:
copy.copy() use the __copy__ magic method and copy.deepcopy() use
__deepcopy__
I've never heard about problems with this behaviour.

In fact it's not the __riter__ magic method which will use riter()
function but the opposite, so an object which define __riter__ doesn't
need to now anything about riter().
Jul 18 '05 #16
Raymond Hettinger wrote:
And I still think you don't need it often enough to put it in the
builtin namespace, so the function should go in the itertools module.


If you have a magic method, __riter__, then the corresponding
function needs to be a builtin. They go hand in hand. The
core parts of the language need to be usable without having
to know about itertools.


I have already seen module copy presented as a counter-example to
your assertion, and I'd like to add module pickle as a second, and
I hope decisive, counter-example. It is, to put it simply, utterly
false that functions corresponding to magic methods "need to be
builtins": it's *perfectly* all right for such functions to live
in standard library modules. Oh, and let's not forget module sets:
the __as_immutable_ _ and __as_temporaril y_immutable__ magic methods,
that are used when making a set of sets, or checking for a set's
membership in another set, can be seen as yet another example (and
in this case there is no wiggling out of it by claiming that the
magicmethod/NON-builtin correspondence is a historical/legacy thing,
as the BDFL approved module sets.py, with just this usage, so very
recently).

I second the motion that function riter, with check for __riter__
and all, should live in module itertools. Reverse iteration is a
RARE need -- far rarer than copying or even pickling -- and there
is no real reason to burden __builtins__ with a function for it.
Alex

Jul 18 '05 #17
I agree. Importing itertools to get at riter()
isn't a significant hardship.

John Roth

"Alex Martelli" <al***@aleax.it > wrote in message
news:8R******** **************@ news2.tin.it...
Raymond Hettinger wrote:
And I still think you don't need it often enough to put it in the
builtin namespace, so the function should go in the itertools module.


If you have a magic method, __riter__, then the corresponding
function needs to be a builtin. They go hand in hand. The
core parts of the language need to be usable without having
to know about itertools.


I have already seen module copy presented as a counter-example to
your assertion, and I'd like to add module pickle as a second, and
I hope decisive, counter-example. It is, to put it simply, utterly
false that functions corresponding to magic methods "need to be
builtins": it's *perfectly* all right for such functions to live
in standard library modules. Oh, and let's not forget module sets:
the __as_immutable_ _ and __as_temporaril y_immutable__ magic methods,
that are used when making a set of sets, or checking for a set's
membership in another set, can be seen as yet another example (and
in this case there is no wiggling out of it by claiming that the
magicmethod/NON-builtin correspondence is a historical/legacy thing,
as the BDFL approved module sets.py, with just this usage, so very
recently).

I second the motion that function riter, with check for __riter__
and all, should live in module itertools. Reverse iteration is a
RARE need -- far rarer than copying or even pickling -- and there
is no real reason to burden __builtins__ with a function for it.
Alex

Jul 18 '05 #18
Alex Martelli <al***@aleax.it > writes:
Raymond Hettinger wrote:
And I still think you don't need it often enough to put it in the
builtin namespace, so the function should go in the itertools module.


If you have a magic method, __riter__, then the corresponding
function needs to be a builtin. They go hand in hand. The
core parts of the language need to be usable without having
to know about itertools.


I have already seen module copy presented as a counter-example to
your assertion, and I'd like to add module pickle as a second, and
I hope decisive, counter-example. It is, to put it simply, utterly
false that functions corresponding to magic methods "need to be
builtins": it's *perfectly* all right for such functions to live
in standard library modules. Oh, and let's not forget module sets:
the __as_immutable_ _ and __as_temporaril y_immutable__ magic methods,
that are used when making a set of sets, or checking for a set's
membership in another set, can be seen as yet another example (and
in this case there is no wiggling out of it by claiming that the
magicmethod/NON-builtin correspondence is a historical/legacy thing,
as the BDFL approved module sets.py, with just this usage, so very
recently).

I second the motion that function riter, with check for __riter__
and all, should live in module itertools. Reverse iteration is a
RARE need -- far rarer than copying or even pickling -- and there
is no real reason to burden __builtins__ with a function for it.


Is there any concern about the fact that a sequence of direction
changes requires building a new iterator object each time under this
proposal? I was going to implement my own bidirectional iteration
protocol by using a prev() method and use an adaptor object to swap
the meaning of next() and prev(). Python iterators (and GoF
iterators) seem poorly suited as pure sequence position indicators
because you can't reposition them without also accessing elements, so
maybe this is not such a serious issue... but my instincts tell me
that the identity of a bidirectional iterator object could be useful
if we allow the same one to be used in both directions.

FWIW, burdening builtins aside I consider the proposed syntax

for i in xrange(n).iter_ backwards():

much uglier than

for i in reverse_view(xr ange(n)):

but far superior to

for i in reverse(xrange( n)):

which implies in-place modification of the sequence.

Also, the idea of denying tuples the ability to reverse iterate seems
arbitrary and capricious.

Cheers,
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
Jul 18 '05 #19
"David Abrahams" <da**@boost-consulting.com> wrote in message
news:uk******** ***@boost-consulting.com. ..
[snip]
FWIW, burdening builtins aside I consider the proposed syntax

for i in xrange(n).iter_ backwards():

much uglier than

for i in reverse_view(xr ange(n)):

but far superior to

for i in reverse(xrange( n)):

which implies in-place modification of the sequence.


How about

from itertools import ireverse
for i in ireverse(xrange (n)):
# suite

ireverse(), like imap(), izip(), etc., suggests that the operation is
iterative, and that no modification of the original sequence will be
performed. Others have suggested riter() (right iteration), in order to form
an association with the iter() builtin. As a matter of taste, I prefer
ireverse().

Sean
Jul 18 '05 #20

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

72
4354
by: Raymond Hettinger | last post by:
Peter Norvig's creative thinking triggered renewed interest in PEP 289. That led to a number of contributors helping to re-work the pep details into a form that has been well received on the python-dev list: http://www.python.org/peps/pep-0289.html In brief, the PEP proposes a list comprehension style syntax for creating fast, memory...
3
1494
by: Christoph Becker-Freyseng | last post by:
Hello, recently there was a lot discussion about a new Path-class. So Gerrit Holl started to write a pre-PEP http://people.nl.linux.org/~gerrit/creaties/path/pep-xxxx.html We tried to summarize the good ideas and suggestions. Some are still missing and concepts of already existing modules have to be added/thought over. Furthermore it...
14
2892
by: Marcin Ciura | last post by:
Here is a pre-PEP about print that I wrote recently. Please let me know what is the community's opinion on it. Cheers, Marcin PEP: XXX Title: Print Without Intervening Space Version: $Revision: 0.0 $
8
1717
by: Micah Elliott | last post by:
I also have this living as a wiki <http://tracos.org/codetag/wiki/Pep> if people would like to add comments there. I might try to capture there feedback from this group anyway. First try at a PEP -- thanks for any feedback! Please add **NOTE:** comments to the bottom of this wiki document using `WikiRestructuredText`:trac:. ...
2
1462
by: Bryan Olson | last post by:
Though I tried to submit a (pre-) PEP in the proper form through the proper channels, it has disappeared into the ether. In building a class that supports Python's slicing interface, http://groups.google.com/group/comp.lang.python/msg/8f35464483aa7d7b I encountered a Python bug, which, upon further discussion, seemed to be a...
18
2108
by: pocmatos | last post by:
Hi all, While I was programming 5 minutes ago a recurring issue came up and this time I'd like to hear some opinions on style. Although they are usually personal I do think that in this case as also to do with making the code easier to read. Imagine a function returning void (for example) and it's body is a big if with lots of special...
3
1481
by: Talin | last post by:
(Note: PEPs in the 3xxx number range are intended for Python 3000, however this particular PEP may be backported if there is support for it.) PEP: 3102 Title: Keyword-Only Arguments Version: $Revision: 46053 $ Last-Modified: $Date: 2006-05-19 22:23:44 -0700 (Fri, 19 May 2006) $ Author: Talin <talin at acm.org> Status: Draft
4
2483
by: Tony Lownds | last post by:
(Note: PEPs in the 3xxx number range are intended for Python 3000) PEP: 3107 Title: Function Annotations Version: $Revision: 53169 $ Last-Modified: $Date: 2006-12-27 20:59:16 -0800 (Wed, 27 Dec 2006) $ Author: Collin Winter <collinw@gmail.com>, Tony Lownds <tony@lownds.com> Status: Draft Type: Standards Track
0
7611
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
8051
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7574
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
6161
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5442
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5162
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3579
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
2026
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
0
850
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.