Templated for-loop fantasy

[algorimancer]

I use the STL collection classes a lot, and frequently find myself
writing something like:

vector<CDataClass>::iterator iter=DataClassCollection.begin();
vector<CDataClass>::iterator iter_end=DataClassCollection.end();
for(;iter!=iter_end;++iter)
{
//doing stuff with iter...
}

I have this fantasy that it ought to be possible to create a template
to replace all of that with something that looks like:

for<CDataClass>(DataClassCollection) dataiteration;
//the for<> template could have a variety of loop types and data items
(iterators, etceteras)
//defined. A simple case that iterates over each element in the
DataClassCollection might
//look like this:
dataiteration.iterate
{
//doing stuff with "dataiteration.iter"
}

And yes, I know about the for_each method. The problem is that
for_each is a bit awkward to use, and I'd like a template construct
which has the capability of operating on a subsequent statement or
block of statements. I'm not aware that such a capabity exists in C++.

Any ideas/suggestions?
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[andy]

algorimancer wrote:

I use the STL collection classes a lot, and frequently find myself
writing something like:

vector<CDataClass>::iterator iter=DataClassCollection.begin();
vector<CDataClass>::iterator iter_end=DataClassCollection.end();
for(;iter!=iter_end;++iter)
{
//doing stuff with iter...
}

Have you looked at boost ForEach macro, scheduled to be part of the
boost distro at some stage?

http://boost-consulting.com/vault/in...ory=Algorithms

If the link doesnt work go to http://www.boost.org. Find link to vault
and look in algorithms directory.

cheers
Andy Little

[James Hopkin]

algorimancer wrote:

I use the STL collection classes a lot, and frequently find myself
writing something like:

vector<CDataClass>::iterator iter=DataClassCollection.begin();
vector<CDataClass>::iterator iter_end=DataClassCollection.end();
for(;iter!=iter_end;++iter)
{
//doing stuff with iter...
}

#include <boost/foreach.hpp>

BOOST_FOREACH(CDataClass& v, DataClassCollection)
{
// do stuff with v
}
James
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[algorimancer]

I had not been aware of the boost FOREACH macro, it looks like it could
do the trick. The boost library seems more accessible than the last
time I looked at it, I'll spend some more time there I think.

Thanks :)

[red floyd]

algorimancer wrote:

I use the STL collection classes a lot, and frequently find myself
writing something like:

vector<CDataClass>::iterator iter=DataClassCollection.begin();
vector<CDataClass>::iterator iter_end=DataClassCollection.end();
for(;iter!=iter_end;++iter)
{
//doing stuff with iter...
}

I have this fantasy that it ought to be possible to create a template
to replace all of that with something that looks like:

for<CDataClass>(DataClassCollection) dataiteration;
//the for<> template could have a variety of loop types and data items
(iterators, etceteras)
//defined. A simple case that iterates over each element in the
DataClassCollection might
//look like this:
dataiteration.iterate
{
//doing stuff with "dataiteration.iter"
}

You've probably thought of this, but...

template <typename Coll, typename Func>
void iterate(Coll& c, Func f)
{
std::for_each(c.begin(), c.end(), f);
}

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[roberts.noah]

algorimancer wrote:

And yes, I know about the for_each method. The problem is that
for_each is a bit awkward to use, and I'd like a template construct
which has the capability of operating on a subsequent statement or
block of statements. I'm not aware that such a capabity exists in C++.

Any ideas/suggestions?

There is also Boost Lambda and Phoenix that let you do things like:

for_each(begin, end, cout << arg1); // Phoenix version...

I've never used either I just mention it as an idea to look into.
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[Valentin Samko]

ro**********@gmail.com wrote:

There is also Boost Lambda and Phoenix that let you do things like:

I wouldn't use these libraries in a production code. For example, I really do not want to
see Boost.Lambda expression in a call stack when I am looking at a core dump of a
production problem. Also, the syntax of any nontrivial Boost.Lambda expression is just
unreadable (as it introduces a new syntax for already existing C++ constructs).
Not to mention that the compiled code is dog slow if inlining is disabled.

--

Valentin Samko - http://www.valentinsamko.com

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[Ivan Kolev]

James Hopkin wrote:

#include <boost/foreach.hpp>

BOOST_FOREACH(CDataClass& v, DataClassCollection)
{
// do stuff with v
}

This is not included in Boost yet. Actually, it wasn't easy to find the
source code - I found it only here:

http://www.nwcpp.org/Meetings/2004/01.html

There's also this page about the library:

http://www.boost.org/regression-logs...l/foreach.html

but I didn't find the source code there.

BOOST_FOREACH is a bit heavy (see the Portability page), actually, as
most Boost libraries. You can define simple macros for your own needs,
like this:

#define for_vector(T,v,i) \
for ( std::vector<T>::iterator i=v.begin(); i != v.end(); ++i )

and use it like this:

for_vector( CDataClass, DataClassCollection, iter )
{
//doing stuff with iter...
}
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[James Hopkin]

Ivan Kolev wrote:

James Hopkin wrote:

#include <boost/foreach.hpp>

BOOST_FOREACH(CDataClass& v, DataClassCollection)
{
// do stuff with v
}
This is not included in Boost yet.

Thanks for pointing that out. I'd forgotten I'd dug it out of the boost
sandbox and imported it into our company copy of boost.


BOOST_FOREACH is a bit heavy (see the Portability page), actually, as
most Boost libraries.

I have to admit, it does pull in a lot of headers. But if you're using
boost regularly, that won't be noticeable, especially with the commonly
used stuff in a pre-compiled header.

You can define simple macros for your own needs,
like this:

#define for_vector(T,v,i) \
for ( std::vector<T>::iterator i=v.begin(); i != v.end(); ++i )

Not such a bad idea, although I'd write it:

#define for_vector(T,v,i) \
for (std::vector<T>::iterator i=v.begin(), _end = v.end(); i != _end;
++i)
Note that a major benefit of BOOST_FOREACH is that it works with any
container, pairs of iterators (e.g. a return value from equal_range)
and built-in arrays.
James
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[Ivan Kolev]

> I have to admit, it does pull in a lot of headers. But if you're using

boost regularly, that won't be noticeable, especially with the commonly
used stuff in a pre-compiled header.
Yes, it depends on whether you're using Boost or not. Once you start,
you get everything new in Boost sort of "for free".
Not such a bad idea, although I'd write it:

#define for_vector(T,v,i) \
for (std::vector<T>::iterator i=v.begin(), _end = v.end(); i != _end;
++i)
Indeed. Though I guess vector::end() should be simple enough to be
inlined, this could be useful for other containers.
Note that a major benefit of BOOST_FOREACH is that it works with any
container, pairs of iterators (e.g. a return value from equal_range)
and built-in arrays.

BOOST_FOREACH is impressive. By coincidence, I was working these days
on something similar, trying to create something like a generic
iterator class, but of course nothing can beat the Boost libraries. So
the news about BOOST_FOREACH was very timely for me - thank you :).
Although our project is not "Boost-enabled" yet, it's good to know what
can be done, and also what can we expect of the next standard. Having
foreach included in C++0x would be great.
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[Thorsten Ottosen]

Ivan Kolev wrote:

Although our project is not "Boost-enabled" yet, it's good to know what
can be done, and also what can we expect of the next standard. Having
foreach included in C++0x would be great.

Please see

http://www.open-std.org/JTC1/SC22/WG...005/n1868.html

-Thorsten

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[Ivan Kolev]

> Please see

http://www.open-std.org/JTC1/SC22/WG...005/n1868.html

Thanks, that looks quite promising.

I have one question though - I wonder if it is possible to achieve
similar syntax for containers which do not use the standard begin/end
approach to container enumeration, and for "multi-containers". Here's
an example of both:

struct Vertex;
struct Face;

class IMesh
{
public:

virtual unsigned numVerts() const = 0;

virtual const Vertex& vert( unsigned i ) const = 0;

virtual unsigned numFaces() const = 0;

virtual const Face& face( unsigned i ) const = 0;

};

It is often more intuitive and useful to enumerate a container using
count()/item(i) methods than begin/end. (Am I wrong about this? Would
begin/end work better for the above example too?)
And as you can see above, sometimes we need "multi-containers", i.e.
containers which hold several collections. It would be nice to be able
to write

IMesh& m = ...;

for ( Vertex v : m )
{
....
}

for ( Face f : m )
{
....
}

with the first loop iterating over mesh vertices and the second over
faces, as expected.

Of course, this wouldn't work if the two collections were of the same
type, which could happen easily in the above example if we had:

typedef Vector3 Vertex;
typedef Vector3 Normal;
struct Face;

class IMesh
{
public:
... // same as above

virtual const Normal& vertNormal( unsigned i ) const = 0;

};

In that case we won't be able to write neither "for ( Vertex v : mesh
)", nor "for ( Normal n : mesh )", although a "strong typedef" could
solve the problem (btw, is there a proposal about "strong typedefs" by
any chance? ;) )

Actually, the first "feature request" about foreach supporting
containers of the count()/item(i) style can be achieved with your
current proposal by writing a special iterator like this:

class ItMeshVert
{
public:

ItMeshVert( const IMesh&, unsigned i );

const Vertex& operator*();

bool operator==( const ItMeshVert& other );

private:

const IMesh& mesh_;

unsigned i_;
}

(implementations are obvious), and defining these two:

ItMeshVert begin( const IMesh& m ) { return ItMeshVert( m, 0 ); }
ItMeshVert end( const IMesh& m ) { return ItMeshVert( m, m.numVerts()
); }

(btw, there seems to be a mistake in the "for-loop requirements" part
of your example for item 4 - all four functions are called "begin").
This would require some extra work on behalf of the container designer
(a non-const version of ItMeshVert may be needed if IMesh allows
non-const access to vertices), but it's not too bad.
However, I don't see how the other "feature request" about
multi-containers could be implemented... (the strong typedefs can help,
but in some cases the collection types are exactly the same, not just
aliases of the same type). Actually, I'm not very sure myself if it is
a good idea... it just sounds good :)

Regards,
Ivan
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[Thorsten Ottosen]

Ivan Kolev wrote:

Please see
http://www.open-std.org/JTC1/SC22/WG...005/n1868.html

Thanks, that looks quite promising.

I have one question though - I wonder if it is possible to achieve
similar syntax for containers which do not use the standard begin/end
approach to container enumeration, and for "multi-containers".

Not as easily as normal standard containers. You would have to write
an iterator adaptor, something that is not that hard if you use

http://www.boost.org/libs/iterator/d...r_adaptor.html
Here's
an example of both:

struct Vertex;
struct Face;

class IMesh
{
public:

virtual unsigned numVerts() const = 0;

virtual const Vertex& vert( unsigned i ) const = 0;

virtual unsigned numFaces() const = 0;

virtual const Face& face( unsigned i ) const = 0;

};

It is often more intuitive and useful to enumerate a container using
count()/item(i) methods than begin/end. (Am I wrong about this? Would
begin/end work better for the above example too?)
Well, the interface above is complete incompatible with generic
programming and so you can't use any of the standard library algorithms.

But it is intuitively enough, though I don't think you gain much. On the
contrary, the interface seems to *require* fast random access to be
efficient.
And as you can see above, sometimes we need "multi-containers", i.e.
containers which hold several collections. It would be nice to be able
to write

IMesh& m = ...;

for ( Vertex v : m )
{
...
}

for ( Face f : m )
{
...
}

with the first loop iterating over mesh vertices and the second over
faces, as expected.
Right. At the surface your interface seems very abstract with a low
coupling. (But in fact, it requires random access for efficient usage.)

So you may as well specify it like this

class IMesh
{
public:

virtual VertexIterator vertexBegin() const = 0;
virtual VertexIterator vertexEnd() const = 0;
...
};

and then call the loop like

for( Vertex v : make_range(m.vertexBegin(), m.vertexEnd()) )
{
...
}

VertexIterator would be an iterator adaptor which internally
store the current index and a reference to IMesh.
Of course, this wouldn't work if the two collections were of the same
type, which could happen easily in the above example if we had:

typedef Vector3 Vertex;
typedef Vector3 Normal;
struct Face;

class IMesh
{
public:
... // same as above

virtual const Normal& vertNormal( unsigned i ) const = 0;

};

In that case we won't be able to write neither "for ( Vertex v : mesh
)", nor "for ( Normal n : mesh )", although a "strong typedef" could
solve the problem (btw, is there a proposal about "strong typedefs" by
any chance? ;) )

Actually, the first "feature request" about foreach supporting
containers of the count()/item(i) style can be achieved with your
current proposal by writing a special iterator like this:

class ItMeshVert
{
public:

ItMeshVert( const IMesh&, unsigned i );

const Vertex& operator*();

bool operator==( const ItMeshVert& other );

private:

const IMesh& mesh_;

unsigned i_;
}

(implementations are obvious), and defining these two:

ItMeshVert begin( const IMesh& m ) { return ItMeshVert( m, 0 ); }
ItMeshVert end( const IMesh& m ) { return ItMeshVert( m, m.numVerts()
); }
Right. I should read the whole mail before answering :-)
(btw, there seems to be a mistake in the "for-loop requirements" part
of your example for item 4 - all four functions are called "begin").
This would require some extra work on behalf of the container designer
(a non-const version of ItMeshVert may be needed if IMesh allows
non-const access to vertices), but it's not too bad.
However, I don't see how the other "feature request" about
multi-containers could be implemented... (the strong typedefs can help,
but in some cases the collection types are exactly the same, not just
aliases of the same type). Actually, I'm not very sure myself if it is
a good idea... it just sounds good :)

The multicontaners could also be supported like this:

class IMesh
{
public:

virtual range<VertexIterator> vertices() const = 0;
virtual range<VertexIterator> faces() const = 0;
...
};

for( Face& f : m.faces() )
;

for( Vertex v : m.verices() )
;
-Thorsten

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[Ivan Kolev]

Thorsten Ottosen wrote:

The multicontaners could also be supported like this:
class IMesh
{
public:
virtual range<VertexIterator> vertices() const = 0;
virtual range<VertexIterator> faces() const = 0;

Thanks, I find the last example close to perfect. And I guess a pair
could be used in place of range (which should be enough in many cases,
mine at least).

Now another question rises from the practice. Sometimes a counter of
the iterations is needed inside the for loop, i.e. when the loop is not
in the classic form "for (i=0;i<n;++i)", which is the case with the new
for style. Here's an example (part of code copying a mesh in some
external format to our IMesh):

IMesh& mesh = ...;
const AnotherKindOfMesh& otherMesh;

for ( unsigned i = 0; i < mesh.numFaces(); ++i )
{
mesh.face(i) = Face( otherMesh.faces[3*i], otherMesh.faces[3*i+1],
otherMesh.faces[3*i+2] );
}

otherMesh stores (triangle) faces in an array of indices to their
vertices. Hence the above 3*i + 0/1/2 indexing. Now if we used the new
for loop over our mesh, we would have to make it like this:

unsigned i = 0;
for ( Face& f : mesh.faces() )
{
f = Face( otherMesh.faces[3*i], otherMesh.faces[3*i+1],
otherMesh.faces[3*i+2] );
++i;
}

which is not bad, except for the injection of i in the outer scope,
when it would better off as a local variable inside the for loop. Maybe
a combination of the old loop style with the new one would solve this?
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]