generator functions in another language

[castironpi]

I'm actually curious if there's a way to write a generator function
(not a generator expression) in C, or what the simplest way to do it
is... besides link the Python run-time.

[Marc 'BlackJack' Rintsch]

On Sat, 03 May 2008 16:39:43 -0700, castironpi wrote:

I'm actually curious if there's a way to write a generator function
(not a generator expression) in C, or what the simplest way to do it
is... besides link the Python run-time.

The reference implementation of Python is written in C, so obviously there
must be a way to write something like generators in C.

Ciao,
Marc 'BlackJack' Rintsch

[Gabriel Genellina]

En Sun, 04 May 2008 01:08:34 -0300, Marc 'BlackJack' Rintsch <bj****@gmx.net escribió:

On Sat, 03 May 2008 16:39:43 -0700, castironpi wrote:

>I'm actually curious if there's a way to write a generator function
(not a generator expression) in C, or what the simplest way to do it
is... besides link the Python run-time.

The reference implementation of Python is written in C, so obviously there
must be a way to write something like generators in C.

Yes and no. Generators are tied to frames, and frames execute Python code, not C. There is no simple way to write generators in C, but there are some generator-like examples in the itertools module.
See this thread http://groups.google.com/group/comp....2f72f2d0e88fc/

--
Gabriel Genellina

[David]

On Sun, May 4, 2008 at 1:39 AM, <ca********@gma il.comwrote:

I'm actually curious if there's a way to write a generator function
(not a generator expression) in C, or what the simplest way to do it
is... besides link the Python run-time.
--

Here's the itertools C code:

http://svn.python.org/view/python/tr...34&view=markup

David.

[castironpi]

On May 4, 12:21*am, "Gabriel Genellina" <[e-mail].a r>
wrote:

En Sun, 04 May 2008 01:08:34 -0300, Marc 'BlackJack' Rintsch <[e-mail] escribió:

On Sat, 03 May 2008 16:39:43 -0700, castironpi wrote:

I'm actually curious if there's a way to write a generator function
(not a generator expression) in C, or what the simplest way to do it
is... besides link the Python run-time.

The reference implementation of Python is written in C, so obviously there
must be a way to write something like generators in C.

Yes and no. Generators are tied to frames, and frames execute Python code,not C. There is no simple way to write generators in C, but there are some generator-like examples in the itertools module.
See this threadhttp://groups.google.c om/group/comp.lang.pytho n/browse_thread/thread/...

--
Gabriel Genellina

Gabriel,
How did your attempt turn out from last May? At first look, it's
outside the scope of Python, but it is not the scope of C
necessarily. Generators offer a lot of simplicity (which I haven't
read about extensively, but am starting to see) that could gain some
reputation for Python. What is the midpoint at which C could meet
Python?

There is no such thing as a 'frame' per se in C; byte code is
integral. As there is no such thing as suspended state without
frames, and no such thing as generators without suspended state. It's
a hard thing to Google for without knowing the prior terminology for
the work that's already been done on them in C. What work is there?
Are any devs interested in pursuing it?

The frame implementation.

http://svn.python.org/projects/pytho.../frameobject.h
http://svn.python.org/projects/pytho.../frameobject.c

The generator code.

http://svn.python.org/projects/pytho...de/genobject.h
http://svn.python.org/projects/pytho...ts/genobject.c

I used Microsoft's search engine (python frame generator
site:svn.python .org , links 3 and 5) to find it.

[Gabriel Genellina]

En Sun, 04 May 2008 08:11:35 -0300, <ca********@gma il.comescribió:

On May 4, 12:21*am, "Gabriel Genellina" <[e-mail].a r>
wrote:

>En Sun, 04 May 2008 01:08:34 -0300, Marc 'BlackJack' Rintsch <[e-mail] escribió:

On Sat, 03 May 2008 16:39:43 -0700, castironpi wrote:

>I'm actually curious if there's a way to write a generator function
(not a generator expression) in C, or what the simplest way to do it
is... besides link the Python run-time.

The reference implementation of Python is written in C, so obviously there
must be a way to write something like generators in C.

Yes and no. Generators are tied to frames, and frames execute Python code, not C. There is no simple way to write generators in C, but there are some generator-like examples in the itertools module.
See this threadhttp://groups.google.c om/group/comp.lang.pytho n/browse_thread/thread/...

Gabriel,
How did your attempt turn out from last May? At first look, it's
outside the scope of Python, but it is not the scope of C
necessarily. Generators offer a lot of simplicity (which I haven't
read about extensively, but am starting to see) that could gain some
reputation for Python. What is the midpoint at which C could meet
Python?

There is no such thing as a 'frame' per se in C; byte code is
integral. As there is no such thing as suspended state without
frames, and no such thing as generators without suspended state. It's
a hard thing to Google for without knowing the prior terminology for
the work that's already been done on them in C. What work is there?
Are any devs interested in pursuing it?

The frame and generator implementations are very tightly coupled to Python code, they aren't useful for implementing generators in C. Don't bother to read them.

Unfortunately, converting a C function into something like a generator isn't as easy as using the "yield" statement... Although the idea is simple, the implementation may be hard sometimes: You have to wrap the function within an object, maintain all state information into that object, and do all computation in the "next" method. Also, __iter__ should return itself so it can be called as an iterator.
(those objects are sometimes called "functors" <http://en.wikipedia.or g/wiki/Function_object not the same meaning as functors in Mathematics)

All examples that I have at hand are propietary code so I can't post them. The itertools module may be used as reference - "cycle" and "chain" are the easiest I think, although they might be *too* easy to understand the structure. "groupby" is a more complex example but at the same time harder to understand. See http://svn.python.org/projects/pytho...rtoolsmodule.c

Ok, I'll try to use Python code as an example. A generator for Fibonacci numbers:

def fibo():
a = b = 1
while True:
a, b = b, a+b
yield b

We can convert that function into this object; it should be written in C, not Python, but the idea is the same:

class fibo:
def __init__(self):
self.a = 1
self.b = 1
def next(self):
temp = self.a + self.b
self.a = self.b
self.b = temp
return temp
def __iter__(self):
return self

It behaves exactly the same as the generator above; we can even use the same code to test it:

pyfor n in fibo():
.... if n>100: break
.... print n
....
2
3
5
8
13
21
34
55
89

Converting that class into C code should be straightforward . And then you have a generator-like function written in C.

--
Gabriel Genellina

[hdante]

On May 4, 8:11*am, castiro...@gmai l.com wrote:

On May 4, 12:21*am, "Gabriel Genellina" <[e-mail].a r>
wrote:

En Sun, 04 May 2008 01:08:34 -0300, Marc 'BlackJack' Rintsch <bj_...@gmx..ne tescribió:

On Sat, 03 May 2008 16:39:43 -0700, castironpi wrote:

>I'm actually curious if there's a way to write a generator function
>(not a generator expression) in C, or what the simplest way to do it
>is... besides link the Python run-time.

The reference implementation of Python is written in C, so obviously there
must be a way to write something like generators in C.

Yes and no. Generators are tied to frames, and frames execute Python code, not C. There is no simple way to write generators in C, but there are some generator-like examples in the itertools module.
See this threadhttp://groups.google.c om/group/comp.lang.pytho n/browse_thread/thread/...

--
Gabriel Genellina

Gabriel,
How did your attempt turn out from last May? *At first look, it's
outside the scope of Python, but it is not the scope of C
necessarily. *Generators offer a lot of simplicity (which I haven't
read about extensively, but am starting to see) that could gain some
reputation for Python. *What is the midpoint at which C could meet
Python?

There is no such thing as a 'frame' per se in C; byte code is
integral. *As there is no such thing as suspended state without
frames, and no such thing as generators without suspended state. *It's
a hard thing to Google for without knowing the prior terminology for
the work that's already been done on them in C. *What work is there?
Are any devs interested in pursuing it?

The frame implementation.

http://svn.python.org/projects/pytho.../frameobject.c

The generator code.

http://svn.python.org/projects/pytho...ts/genobject.c

I used Microsoft's search engine (python frame generator
site:svn.python .org , links 3 and 5) to find it.

Isn't this guy a bot ? :-) It's learning fast. I believe there is a
"frame" in C, composed of its stack and globals. For generators in C,
you may look for "coroutines ". For example, see:

http://www.chiark.greenend.org.uk/~s...oroutines.html

A sample code follows:

#define crBegin static int state=0; switch(state) { case 0:
#define crReturn(i,x) do { state=i; return x; case i:; } while (0)
#define crFinish }
int function(void) {
static int i;
crBegin;
for (i = 0; i < 10; i++)
crReturn(1, i);
crFinish;
}

The "suspended state" is saved in the static variable. It's not
necessary to save the complete "frame", only the suspended state.

[Terry Reedy]

A 'generator function' -- a function that when called returns a generator
object, a specific type of iterator, is a rather Python specific concept.
Better to think, I think, in terms of writing an iterator 'class' (in C,
struct with associated function). Looking at the implementation of two of
the itertools in the link below (previously posted), I could see the
'template' that each was based on and that would be the basis for writing
another. (And I have not programmed C for a decade. Raymond's code is
quite clear.)

=============== =============== ===
[Outlook Express does not 'quote' this post properly]

"Gabriel Genellina" <ga*******@yaho o.com.arwrote in message
news:op******** *******@a98gizw .cpe.telecentro .net.ar...

The itertools module may be used as reference - "cycle" and "chain" are
the easiest I think, although they might be *too* easy to understand the
structure. "groupby" is a more complex example but at the same time harder
to understand. See
http://svn.python.org/projects/pytho...rtoolsmodule.c

Ok, I'll try to use Python code as an example. A generator for Fibonacci
numbers:

def fibo():
a = b = 1
while True:
a, b = b, a+b
yield b

We can convert that function into this object; it should be written in C,
not Python, but the idea is the same:

class fibo:
def __init__(self):
self.a = 1
self.b = 1
def next(self):
temp = self.a + self.b
self.a = self.b
self.b = temp
return temp
def __iter__(self):
return self

=============== ==============
Terry again: one can think of a generator function as an abbreviation of an
iterator class. Calling the gen. func. produces an iterator instance just
like calling the class object.
=============== ==============
[back to Gabriel]
It behaves exactly the same as the generator above; we can even use the
same code to test it:

pyfor n in fibo():
.... if n>100: break
.... print n
....
2
3
5
8
13
21
34
55
89

Converting that class into C code should be straightforward . And then you
have a generator-like function written in C.

=============== =============== ====

[Gabriel Genellina]

En Mon, 05 May 2008 00:09:02 -0300, hdante <hd****@gmail.c omescribió:

Isn't this guy a bot ? :-) It's learning fast. I believe there is a
"frame" in C, composed of its stack and globals. For generators in C,
you may look for "coroutines ". For example, see:

http://www.chiark.greenend.org.uk/~s...oroutines.html

A sample code follows:

#define crBegin static int state=0; switch(state) { case 0:
#define crReturn(i,x) do { state=i; return x; case i:; } while (0)
#define crFinish }
int function(void) {
static int i;
crBegin;
for (i = 0; i < 10; i++)
crReturn(1, i);
crFinish;
}

The "suspended state" is saved in the static variable. It's not
necessary to save the complete "frame", only the suspended state.

Quoting the author himself, "this is the worst piece of C hackery ever seen"

--
Gabriel Genellina

[castironpi]

On May 5, 12:28*am, "Gabriel Genellina" <[e-mail].a r>
wrote:

En Mon, 05 May 2008 00:09:02 -0300, hdante <hda...@gmail.c omescribió:

*Isn't this guy a bot ? :-) It's learning fast. I believe there is a
"frame" in C, composed of its stack and globals. For generators in C,
you may look for "coroutines ". For example, see:

http://www.chiark.greenend.org.uk/~s...oroutines.html

*A sample code follows:

#define crBegin static int state=0; switch(state) { case 0:
#define crReturn(i,x) do { state=i; return x; case i:; } while (0)
#define crFinish }
int function(void) {
* * static int i;
* * crBegin;
* * for (i = 0; i < 10; i++)
* * * * crReturn(1, i);
* * crFinish;
}

*The "suspended state" is saved in the static variable. It's not
necessary to save the complete "frame", only the suspended state.

Quoting the author himself, "this is the worst piece of C hackery ever seen"

--
Gabriel Genellina- Hide quoted text -

- Show quoted text -

At some point, code goes "on" and "off" the processor, which knowledge
I do owe to spending money. Thus, if the group news is a localcy
(other dimension of currency), that's bounce check the house dollar.
What do [second person plural] spend on [vernacular]?

More strictly speaking, processor freely goes on and off binaries. Is
Goto a hardware instruction? Perhaps a Linux-Python-binding could
yield a frame, pun intended. Do [you] have access the file system?

Could yield merely be implemented as multiple strings of processses?
Get the variables on disk. Question is, is it worth optimizing away
from flash rom? There's a gig of chips in a bowling ball.