Problems returning data from embedded Python

[Cromulent]

Okay I'm having a few issues with this and I can't seem to get it
sorted out (most likely due to my inexperience with Python).

Here is my Python code:

def fileInput():

data = []

s = raw_input("Please enter the filename to process (enter full path
if not in current directory): ")

fd = open(s, "r")

fd.readline()
line = fd.readlines()

for record in line:
record = record.strip()
items = record.split(',')

for i in range(1, 5):
items[i] = float(items[i])

items[5] = int(items[5])
data.append(items)

fd.close()

return items

It works perfectly and does exactly what I want it to do. The problem
comes when I try and convert the data I returned from the Python script
and turn it into something useable in C.

I have no idea. The C functions that seem most use deal with Tuples,
when this is a list and I can't see any specific functions for dealing
with lists in the C API. Can anyone give me some pointers in the right
direction at all please? I know the C program has received the data
correctly, I just need to put it in C types.

Thank you.

--
"I disapprove of what you say, but I'll defend to the death your right
to say it." - Voltaire

[Gabriel Genellina]

En Mon, 11 Aug 2008 12:58:00 -0300, Cromulent
<cr*******@justextrememetal.comescribi�:

Okay I'm having a few issues with this and I can't seem to get it sorted
out (most likely due to my inexperience with Python).

Here is my Python code:

def fileInput():

data = []

s = raw_input("Please enter the filename to process (enter full path if
not in current directory): ")

(in general, it's not a good idea mixing the user interfase with data
processing - I'd make the filename a parameter for this function. But this
has nothing to do with your main issue.)

fd = open(s, "r")
fd.readline()
line = fd.readlines()
for record in line:

I'd replace the above two lines with:
for record in fd:

record = record.strip()
items = record.split(',')

for i in range(1, 5):
items[i] = float(items[i])

items[5] = int(items[5])
data.append(items)

fd.close()

return items

`return items` or `return data`? Are you only interested on the *last*
line in the file? If so, I'd skip processing all previous lines...

It works perfectly and does exactly what I want it to do. The problem
comes when I try and convert the data I returned from the Python script
and turn it into something useable in C.

I have no idea. The C functions that seem most use deal with Tuples,
when this is a list and I can't see any specific functions for dealing
with lists in the C API. Can anyone give me some pointers in the right
direction at all please? I know the C program has received the data
correctly, I just need to put it in C types.

Uh? You have a complete API for working with list objects, the functions
named PyList_*
See http://docs.python.org/api/listObjects.html
There is also an abstract layer that works both with lists and tuples:
http://docs.python.org/api/sequence.html
If that's not what you are after, please provide more details...

--
Gabriel Genellina

[Cromulent]

On 2008-08-12 05:37:53 +0100, "Gabriel Genellina"
<ga*******@yahoo.com.arsaid:

En Mon, 11 Aug 2008 12:58:00 -0300, Cromulent
<cr*******@justextrememetal.comescribi�:

><snip>

Uh? You have a complete API for working with list objects, the
functions named PyList_*
See http://docs.python.org/api/listObjects.html
There is also an abstract layer that works both with lists and tuples:
http://docs.python.org/api/sequence.html
If that's not what you are after, please provide more details...

Thank you for the advice.

Bah, just noticed that the very last function on that page you linked
is what I am looking for. I could use PyList_AsTuple and then use
PyArg_ParseTuple to convert it to C types. Why do I always miss the
obvious? You have no idea how long I've spent looking through that API
reference :).
--
"I disapprove of what you say, but I'll defend to the death your right
to say it." - Voltaire

[Cromulent]

On 2008-08-12 05:37:53 +0100, "Gabriel Genellina"
<ga*******@yahoo.com.arsaid:

En Mon, 11 Aug 2008 12:58:00 -0300, Cromulent
<cr*******@justextrememetal.comescribi�:

><snip>

Uh? You have a complete API for working with list objects, the
functions named PyList_*
See http://docs.python.org/api/listObjects.html
There is also an abstract layer that works both with lists and tuples:
http://docs.python.org/api/sequence.html
If that's not what you are after, please provide more details...

Spoke too soon.

Right, I've rewritten the Python program and it returns a tuple of
lists and one integer. Basically as you saw before, the python program
reads a file in, splits it into elements that were separated by a comma.

The new program just puts each element into its own list. Here is a
line from the file I am reading in:

15-Jul-08,37.70,37.70,36.43,36.88,102600

so basically I have 6 lists and one int (which is the total number of
lines read by the program) I then return that tuple to the C program.
After that I call the following:

error = PyArg_ParseTuple(value, "isffffi", &totalLines, &finopen,
&finclose, &finhigh, &finlow, &finvolume);

but that will only give me the first element of the list. What I would
like to do is return the total number of lines read separately, then
use that to allocate a C99 style variable sized array and then loop
through the call to PyArg_ParseTuple to populate the array. The problem
with that is that I don't think PyArg_ParseTuple is designed in that
way. It will put the contents of the list in the variable and not split
it up so that I can populate an array.

Am I making any sense? Probably not, but it is hard to explain.

Thanks for any help.
--
"I disapprove of what you say, but I'll defend to the death your right
to say it." - Voltaire

[Gabriel Genellina]

En Tue, 12 Aug 2008 19:48:54 -0300, Cromulent
<cr*******@justextrememetal.comescribi�:

On 2008-08-12 05:37:53 +0100, "Gabriel Genellina"
<ga*******@yahoo.com.arsaid:

>En Mon, 11 Aug 2008 12:58:00 -0300, Cromulent
<cr*******@justextrememetal.comescribi�:

>><snip>

Uh? You have a complete API for working with list objects, the
functions named PyList_*
See http://docs.python.org/api/listObjects.html
There is also an abstract layer that works both with lists and tuples:
http://docs.python.org/api/sequence.html
If that's not what you are after, please provide more details...

Spoke too soon.

Right, I've rewritten the Python program and it returns a tuple of lists
and one integer. Basically as you saw before, the python program reads a
file in, splits it into elements that were separated by a comma.

The new program just puts each element into its own list. Here is a line
from the file I am reading in:

15-Jul-08,37.70,37.70,36.43,36.88,102600

so basically I have 6 lists and one int (which is the total number of
lines read by the program) I then return that tuple to the C program.

You don't need the integer - it's the list length, and you can easily ask
that value using PyList_Size.
So you are now returning *columns* from the file, ok?

After that I call the following:

error = PyArg_ParseTuple(value, "isffffi", &totalLines, &finopen,
&finclose, &finhigh, &finlow, &finvolume);

but that will only give me the first element of the list. What I would
like to do is return the total number of lines read separately, then use
that to allocate a C99 style variable sized array and then loop through
the call to PyArg_ParseTuple to populate the array. The problem with
that is that I don't think PyArg_ParseTuple is designed in that way. It
will put the contents of the list in the variable and not split it up so
that I can populate an array.

Yes, forget about PyArg_ParseTuple. It's intended to parse function
arguments. Use the appropiate convert function for each object type. For
integers, use PyInt_AsLong; for floats, PyFloat_AsDouble, and so on.
Something like this (untested):

// for a column containing float values:
Py_ssize_t nitems = PyList_Size(the_python_list_of_floats)
// ...allocate the C array...
for (Py_ssize_t i=0; i<nitems; i++) {
PyObject* item = PyList_GetItem(the_python_list_of_floats, i);
your_c_array_of_double[i] = PyFloat_AsDouble(item);
}

--
Gabriel Genellina

[Cromulent]

On 2008-08-13 00:50:11 +0100, "Gabriel Genellina"
<ga*******@yahoo.com.arsaid:

En Tue, 12 Aug 2008 19:48:54 -0300, Cromulent
<cr*******@justextrememetal.comescribi�:

>On 2008-08-12 05:37:53 +0100, "Gabriel Genellina"
<ga*******@yahoo.com.arsaid:

>><snip>

Yes, forget about PyArg_ParseTuple. It's intended to parse function
arguments. Use the appropiate convert function for each object type.
For integers, use PyInt_AsLong; for floats, PyFloat_AsDouble, and so
on. Something like this (untested):

// for a column containing float values:
Py_ssize_t nitems = PyList_Size(the_python_list_of_floats)
// ...allocate the C array...
for (Py_ssize_t i=0; i<nitems; i++) {
PyObject* item = PyList_GetItem(the_python_list_of_floats, i);
your_c_array_of_double[i] = PyFloat_AsDouble(item);
}

Fantastic! Thanks for your help :). You've got me on the right path now.
--
"I disapprove of what you say, but I'll defend to the death your right
to say it." - Voltaire