By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,092 Members | 1,650 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,092 IT Pros & Developers. It's quick & easy.

Is it possible to determine what a function needs for parameters -

P: n/a
Hi all,

Below is a basic threading program. The basic I idea is that I have a
function which needs to be run using a queue of data. Early on I
specified my function needed to only accept basic parameters ( no
postional *args or *kwargs ) but now I am re-writing it and I want to
accept these. Is there anyway to determine what parameters are needed
by a given function and format the arguments given appropriately. My
problem is that I feel my "Kludge"section is just that - a kludge.
While it works for most cases it won't work for all (Try running this
on function4 for example...). I guess I have a problem with multiple
if statements and I think it should just look to see what parameters
are accepted and format them appropriately. Thoughts?

If I am really screwing this up - please help me get back on the right
track. I appreciate all of the help I get and it's great to learn from
the experts.

import random
import threading
import Queue

class WorkerB(threading.Thread):

def __init__(self, *args, **kwargs):
self.id = kwargs.get('id', 0)
self.requestQ = kwargs.get('requestQ', None)
self.function = kwargs.get('function', None)
threading.Thread.__init__(self, name=self.__class__.__name__
+"."+str(self.id))

def run(self):
while 1:
input = self.requestQ.get()
if input is None: break

# How do I look at the function and determine what it
requires then apply that as an input?

# -------- Start Kludge ----------
tu=dic=False
if isinstance(input, tuple):
try:
if isinstance(input[0], tuple): tu = True
elif isinstance(input[0], list): tu=True
except: pass
try:
if isinstance(input[1], dict):dic = True
except: pass
if tu and dic:
print " -Tuple and list found"
result = self.function(*input[0], **input[1])
elif tu and not dic:
print " -Tuple found"
result = self.function(*input[0])
elif isinstance(input, list):
print " -list only found"
result = self.function(*input)
else:
print " -Unknown"
result = self.function(input)

# -------- End Kludge ----------

def function1(arg1):
print arg1

def function2(*a ):
print "args", a

def function3(*a, **kw):
print "args", a
print "kwargs", kw

def function4(arg1, *a, **kw):
print arg1
print "args", a
print "kwargs", kw

def main():
lod = 2
myQ=Queue.Queue()

# A basic example
print "\n== Example 1"
for x in range(lod):myQ.put( random.random() )
myQ.put(None)
a=WorkerB(requestQ=myQ, function=function1).start()

# Throw at is some args
print "\n== Example 2"
for x in range(lod):myQ.put(["a","b","c"])
myQ.put(None)
a=WorkerB(requestQ=myQ, function=function2).start()

# Throw at it both args and kwargs
print "\n== Example 3"
for x in range(lod):myQ.put(((1,2,3),
{"input":random.random(),"loglevel":10}))
myQ.put(None)
a=WorkerB(requestQ=myQ, function=function3).start()

# Throw at it both args and kwargs
print "\n== Example 4 Does nothing!!"
for x in range(lod):myQ.put(("alpha",(1,2,3),
{"input":random.random(),"loglevel":10}))
myQ.put(None)
a=WorkerB(requestQ=myQ, function=function4).start()
if __name__ == '__main__':
main()

May 2 '07 #1
Share this Question
Share on Google+
10 Replies


P: n/a
rh0dium a écrit :
Hi all,

Below is a basic threading program. The basic I idea is that I have a
function which needs to be run using a queue of data. Early on I
specified my function needed to only accept basic parameters ( no
postional *args or *kwargs ) but now I am re-writing it and I want to
accept these. Is there anyway to determine what parameters are needed
by a given function and format the arguments given appropriately.
Yes - using inspect.getargspec. I don't have example code at hand yet,
but it's not really complicated.
May 2 '07 #2

P: n/a
On 2 May 2007 07:22:07 -0700, rh0dium <st**********@gmail.comwrote:
Hi all,

Below is a basic threading program. The basic I idea is that I have a
function which needs to be run using a queue of data. Early on I
specified my function needed to only accept basic parameters ( no
postional *args or *kwargs ) but now I am re-writing it and I want to
accept these. Is there anyway to determine what parameters are needed
by a given function and format the arguments given appropriately. My
problem is that I feel my "Kludge"section is just that - a kludge.
While it works for most cases it won't work for all (Try running this
on function4 for example...). I guess I have a problem with multiple
if statements and I think it should just look to see what parameters
are accepted and format them appropriately. Thoughts?

If I am really screwing this up - please help me get back on the right
track. I appreciate all of the help I get and it's great to learn from
the experts.

import random
import threading
import Queue

class WorkerB(threading.Thread):

def __init__(self, *args, **kwargs):
self.id = kwargs.get('id', 0)
self.requestQ = kwargs.get('requestQ', None)
self.function = kwargs.get('function', None)
threading.Thread.__init__(self, name=self.__class__.__name__
+"."+str(self.id))

def run(self):
while 1:
input = self.requestQ.get()
if input is None: break

# How do I look at the function and determine what it
requires then apply that as an input?

# -------- Start Kludge ----------
tu=dic=False
if isinstance(input, tuple):
try:
if isinstance(input[0], tuple): tu = True
elif isinstance(input[0], list): tu=True
except: pass
try:
if isinstance(input[1], dict):dic = True
except: pass
if tu and dic:
print " -Tuple and list found"
result = self.function(*input[0], **input[1])
elif tu and not dic:
print " -Tuple found"
result = self.function(*input[0])
elif isinstance(input, list):
print " -list only found"
result = self.function(*input)
else:
print " -Unknown"
result = self.function(input)

# -------- End Kludge ----------

def function1(arg1):
print arg1

def function2(*a ):
print "args", a

def function3(*a, **kw):
print "args", a
print "kwargs", kw

def function4(arg1, *a, **kw):
print arg1
print "args", a
print "kwargs", kw

def main():
lod = 2
myQ=Queue.Queue()

# A basic example
print "\n== Example 1"
for x in range(lod):myQ.put( random.random() )
myQ.put(None)
a=WorkerB(requestQ=myQ, function=function1).start()

# Throw at is some args
print "\n== Example 2"
for x in range(lod):myQ.put(["a","b","c"])
myQ.put(None)
a=WorkerB(requestQ=myQ, function=function2).start()

# Throw at it both args and kwargs
print "\n== Example 3"
for x in range(lod):myQ.put(((1,2,3),
{"input":random.random(),"loglevel":10}))
myQ.put(None)
a=WorkerB(requestQ=myQ, function=function3).start()

# Throw at it both args and kwargs
print "\n== Example 4 Does nothing!!"
for x in range(lod):myQ.put(("alpha",(1,2,3),
{"input":random.random(),"loglevel":10}))
myQ.put(None)
a=WorkerB(requestQ=myQ, function=function4).start()
if __name__ == '__main__':
main()

This is far more work than you need. Push an (args, kwargs) tuple into
your arguments queue and call self.function(*args, **kwargs).
May 2 '07 #3

P: n/a
This is far more work than you need. Push an (args, kwargs) tuple into
your arguments queue and call self.function(*args, **kwargs).
No see I tried that and that won't work.

I'm assuming what you are referring to is this (effectively)

Q.put(((),{a:"foo", b:"bar}))

input = Q.get()
self.function( *input[0], **input[1])

This will obviously fail if several conditions aren't met - hence the
kludge. Am I missing something here?

May 2 '07 #4

P: n/a
rh0dium wrote:
>This is far more work than you need. Push an (args, kwargs) tuple into
your arguments queue and call self.function(*args, **kwargs).

No see I tried that and that won't work.
Won't work? How does it fail?
I'm assuming what you are referring to is this (effectively)

Q.put(((),{a:"foo", b:"bar}))

input = Q.get()
self.function( *input[0], **input[1])

This will obviously fail if several conditions aren't met - hence the
kludge. Am I missing something here?

Obviously? Conditions? What conditions?

We do things like this constantly, and in fact, it *does* work.

Please tell us how it fails, or what is unsatisfactory about it.

Gary Herron
May 2 '07 #5

P: n/a
On 2 May 2007 08:13:12 -0700, rh0dium <st**********@gmail.comwrote:
This is far more work than you need. Push an (args, kwargs) tuple into
your arguments queue and call self.function(*args, **kwargs).

No see I tried that and that won't work.

I'm assuming what you are referring to is this (effectively)

Q.put(((),{a:"foo", b:"bar}))

input = Q.get()
self.function( *input[0], **input[1])

This will obviously fail if several conditions aren't met - hence the
kludge. Am I missing something here?
Assuming that the caller correctly pushes arguments, how can this fail?
May 2 '07 #6

P: n/a
On May 2, 7:49 am, Bruno Desthuilliers <bruno.
42.desthuilli...@wtf.websiteburo.oops.comwrote:
Yes - using inspect.getargspec. I don't have example code at hand yet,
but it's not really complicated.
Viola!! Hey this works!! Now I have modified my code to do this - way
cool (still kind of a mess though)

args, varargs, varkw, defs =
inspect.getargspec(self.function)
# Deal with just some args
if varargs is None and varkw is None:
result=self.function(input)
# Deal with *args
if varkw is None and varargs is not None and len(args) >
0:
result=self.function(input[0:-1], *input[-1])
if varkw is None and varargs is not None and len(args)==0:
result=self.function(*input[0])
# Deal with *kwargs
if varkw is not None and varargs is not None and len(args)
0:
result=self.function(input[0:-2], *input[-2],
**input[-1])
if varkw is not None and varargs is not None and
len(args)==0:
result=self.function(*input[-2], **input[-1])
if varkw is not None and varargs is None and len(args) >
0:
result=self.function(input[0:-1], **input[-1])
if varkw is not None and varargs is None and len(args) ==
0:
result=self.function(**input[0])

Now this worked until I threw a function which looked like this

def func5( input1, input2, input3 )
pass

So it barfed because of this..

if varargs is None and varkw is None:
result=self.function(input)

but all of the parameters were lumped as a list so input1 contained
them all...
A small tweak turned into this..
if varargs is None and varkw is None:
if isinstance(input, tuple):
result=self.function(*input)
else:
result=self.function(input)

But now I suppose I need to do this for all of them but that will
break my other logic...

Yuck - I have to be missing something here.
May 2 '07 #7

P: n/a
On May 2, 8:25 am, Gary Herron <gher...@islandtraining.comwrote:
rh0dium wrote:
This is far more work than you need. Push an (args, kwargs) tuple into
your arguments queue and call self.function(*args, **kwargs).
No see I tried that and that won't work.

Won't work? How does it fail?I'm assuming what you are referring to is this (effectively)
Q.put(((),{a:"foo", b:"bar}))
input = Q.get()
self.function( *input[0], **input[1])
This will obviously fail if several conditions aren't met - hence the
kludge. Am I missing something here?

Obviously? Conditions? What conditions?

We do things like this constantly, and in fact, it *does* work.

Please tell us how it fails, or what is unsatisfactory about it.

Gary Herron
Good - It looks like I am the one who is clueless..

If I do this:

def funcA(input):
pass

Then I run the code
for x in range(lod):myQ.put(random.random())
myQ.put(None)
a=WorkerB(requestQ=myQ, function=funcA).start()

This will fail because there isn't an input[1]

May 2 '07 #8

P: n/a
On 2 May 2007 09:41:56 -0700, rh0dium <st**********@gmail.comwrote:
On May 2, 8:25 am, Gary Herron <gher...@islandtraining.comwrote:
rh0dium wrote:
>This is far more work than you need. Push an (args, kwargs) tuple into
>your arguments queue and call self.function(*args, **kwargs).
No see I tried that and that won't work.
Won't work? How does it fail?I'm assuming what you are referring to is this (effectively)
Q.put(((),{a:"foo", b:"bar}))
input = Q.get()
self.function( *input[0], **input[1])
This will obviously fail if several conditions aren't met - hence the
kludge. Am I missing something here?
Obviously? Conditions? What conditions?

We do things like this constantly, and in fact, it *does* work.

Please tell us how it fails, or what is unsatisfactory about it.

Gary Herron

Good - It looks like I am the one who is clueless..

If I do this:

def funcA(input):
pass

Then I run the code
for x in range(lod):myQ.put(random.random())
myQ.put(None)
a=WorkerB(requestQ=myQ, function=funcA).start()

This will fail because there isn't an input[1]

Thats because you put the wrong value into the arguments queue.
For this use case, we define the arguments queue as being a source of 2-tuples,
with an argument list and a kwargs dict. So you have to do:

for x in range(lod):
myQ.put((random.random(), {}))

(don't be afraid of indentation and newlines - I started to modify
your source to provide a working example and got frustrated
reformatting it so I could read it)

Since you've now defined the external interface for your system (pass
it a queue of argument, kwargs tuples and a callable) it's the
responsibility of the caller to correctly satisfy that interface.
May 2 '07 #9

P: n/a
On Wed, 02 May 2007 07:22:07 -0700, rh0dium wrote:
Hi all,

Below is a basic threading program. The basic I idea is that I have a
function which needs to be run using a queue of data. Early on I
specified my function needed to only accept basic parameters ( no
postional *args or *kwargs ) but now I am re-writing it and I want to
accept these. Is there anyway to determine what parameters are needed
by a given function and format the arguments given appropriately.
Is this meant to be just a programming exercise to see how clever you can
be? It's okay if it is, but if it is meant to be something useful,
well, I can't imagine ever being in a situation where I know I have
to pass arguments (say) 4, 5, "Hello", and None to a function, but not
know whether they should be positional arguments or keyword arguments.

Or, to put it another way... the usual procedure is for the developer
(that's you) to read the function API to find out what arguments the
function expects, and how it expects them, and then the developer
modifies the parameters used accordingly.

--
Steven.

May 2 '07 #10

P: n/a
In article <ma***************************************@python. org>,
Chris Mellon <ar*****@gmail.comwrote:
>On 2 May 2007 09:41:56 -0700, rh0dium <st**********@gmail.comwrote:
>On May 2, 8:25 am, Gary Herron <gher...@islandtraining.comwrote:
rh0dium wrote:
May 2 '07 #11

This discussion thread is closed

Replies have been disabled for this discussion.