473,326 Members | 2,061 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

Directly calling threaded class instance methods and attributes

Hi,

I've got a question about whether there are any issues with directly
calling attributes and/or methods of a threaded class instance. I
wonder if someone could give me some advice on this.

Generally, the documentation suggests that queues or similar constructs
should be used for thread inter-process comms. I've had a lot of
success in doing that (generally by passing in the queue during the
__init__ of the thread) and I can see situations where it's pretty much
the only way of doing it. But, there are other situations -
particularly where you've got a "main" thread that creates, and is the
sole communicator with, one or more "worker" threads - where keeping
track of the different queues can get a bit unwieldy.

So I thought about it some more, and came to the conclusion that -
again, in some situations - it could be a lot cleaner if you could call
methods and/or access attributes of a threaded class instance directly.
Here's some example code to show what I'm talking about:

-----------------------------------------------------------------
import threading
from time import sleep

class SimpleThread(threading.Thread):
def __init__(self):
self.total = 0
threading.Thread.__init__(self)

def add(self, number):
self.total += number

def run(self):
while(True):
# In reality, there'd be much more here
sleep(1)

adder = SimpleThread()
adder.start()
for i in range(20):
adder.add(1)
print adder.total
-------------------------------------------------------------------

This example code works. Well, it does for me, anyway :-)

My question is simply, can anyone see any issues with calling methods
and/or attributes of a threaded class instance like this? It looks ok
to me but, as the docs never seem to mention using threads like this,
I'm wondering if I've missed something important. If it helps, I
know the basic considerations of threading, such as locking, exception
handling and so on; I only really need advice on whether there could
be issues with directly calling class instance attributes of a
running thread.

If anyone could let me know either "Yeah, that's fine" or "NO!!! That
can break <foo> / cause a deadlock in <bar> / etc!!!" I'd be much
obliged.

Thanks,
Matthew.
Jul 18 '05 #1
8 2392
Matthew Bell wrote:
import threading
from time import sleep

class SimpleThread(threading.Thread):
defÂ*__init__(self):
self.totalÂ*=Â*0
threading.Thread.__init__(self)

defÂ*add(self,Â*number):
self.totalÂ*+=Â*number

defÂ*run(self):
while(True):
#Â*InÂ*reality,Â*there'dÂ*beÂ*muchÂ*moreÂ*here
sleep(1)

adder = SimpleThread()
adder.start()
for i in range(20):
adder.add(1)
printÂ*adder.total
-------------------------------------------------------------------

This example code works.Â*Â*Well,Â*itÂ*doesÂ*forÂ*me,Â*anywayÂ*:-)

My question is simply, can anyone see any issues with calling methods
and/or attributes of a threaded class instance like this?Â*Â*ItÂ*looksÂ*ok
to me but, as the docs never seem to mention using threads like this,
I'm wondering if I've missed something important.Â*Â*IfÂ*itÂ*helps,Â*I


I know _very_ little about threads, so forgive me if my conclusion that you
know even less is wrong. From what I see in your example you do not have
any data that is shared by multiple threads - total just happens to be
stored in a SimpleThread object but is never accessed by it.

I have tried to desimplify your code a bit

import time
from time import sleep
import threading
class SimpleThread(threading.Thread):
def __init__(self):
self.total = 0
threading.Thread.__init__(self)

def add(self, number):
total = self.total
sleep(.3)
self.total = total + number

def run(self):
for i in range(10):
sleep(.1)
self.add(1)

adder = SimpleThread()
adder.start()
for i in range(10):
adder.add(1)
adder.join()
print "total:", adder.total

and here's the output:

$ python testthread.py
total: 12
$

I don't know whether

self.total += number

is atomic, but even if it were, I wouldn't rely on it.
Conclusion: stick with queues, or wait for an expert's advice - or both :-)

Peter

Jul 18 '05 #2
> My question is simply, can anyone see any issues with calling methods
and/or attributes of a threaded class instance like this? It looks ok
to me but, as the docs never seem to mention using threads like this,
I'm wondering if I've missed something important. If it helps, I
know the basic considerations of threading, such as locking, exception
handling and so on; I only really need advice on whether there could
be issues with directly calling class instance attributes of a
running thread.

If anyone could let me know either "Yeah, that's fine" or "NO!!! That
can break <foo> / cause a deadlock in <bar> / etc!!!" I'd be much
obliged.


Python is very therad-friendly in a way that you don't get SIGSEGVs for
doing this - that means that at least the internal data-structures are
alwasys consistent. As a rule of thumb one can say that every expression is
atomic, and thus leaves the interpreter in a consistent state. But beware!
This is more than one expression:

a = b + c * d

It could be rewritten like this:

h = c* d
a = b + h

which makes it at least two - maybe there are even more. But

l.append(10)

on a list will at least be atomic when the actual appending occurs - thus
its perfectly ok to have 10 workerthreads appending to one list, and one
consumer thread pop'ing values from it.

So your code is perfectly legal, and as long as you are aware that having
more complex operations on objects undergoing can be interrupted at any
time, maybe leaving data inconsistent _from and applications POV_ - then
you'll need explicid sync'ing, by locks, queues or whatever...

--
Regards,

Diez B. Roggisch
Jul 18 '05 #3

li****@cix.co.uk (Matthew Bell) wrote:

Hi,

I've got a question about whether there are any issues with directly
calling attributes and/or methods of a threaded class instance. I
wonder if someone could give me some advice on this.
No problem.

[Snip code and text]
If anyone could let me know either "Yeah, that's fine" or "NO!!! That
can break <foo> / cause a deadlock in <bar> / etc!!!" I'd be much
obliged.


With what you offered, it would not cause a deadlock, though it would
cause what is known as a race condition, where two threads are trying to
modify the same variable at the same time. Note that attributes of a
thread object are merely attributes of an arbitrary Python object, so
nothing special happens with them.
Here is a far more telling example...
import threading
val = 0
def foo(n): ... global val
... for i in xrange(n):
... val += 1
... for i in xrange(10): ... threading.Thread(target=foo, args=(100000,)).start()
... #wait a few seconds... ... val 202229
If there were no race condition, that value should be 1000000. Let us
use locks to fix it.
val2 = 0
lock = threading.Lock()
def goo(n): ... global val2, lock
... for i in xrange(n):
... lock.acquire()
... val2 += 1
... lock.release()
... for i in xrange(10): ... threading.Thread(target=goo, args=(100000,)).start()
... #wait... ... val2

1000000
- Josiah

Jul 18 '05 #4
In article <ma**************************************@python.o rg>,
Josiah Carlson <jc******@uci.edu> wrote:
val2 = 0
lock = threading.Lock()
def goo(n): ... global val2, lock
... for i in xrange(n):
... lock.acquire()
... val2 += 1
... lock.release()
... for i in xrange(10): ... threading.Thread(target=goo, args=(100000,)).start()
... #wait... ... val2

1000000


FWIW, you don't need a global statement for globals you don't assign to,
so you don't need to declare lock global.

Just
Jul 18 '05 #5

Just <ju**@xs4all.nl> wrote:

In article <ma**************************************@python.o rg>,

FWIW, you don't need a global statement for globals you don't assign to,
so you don't need to declare lock global.


I was going to say that I did it for speed, and I could have sworn that
stating something was a global resulted in a fewer namespace lookups,
but testing does not confirm this (it actually refutes it). I guess this
says that I should be aliasing globals when I really care about speed,
making it...
def goo(n):

... global val2
... _lock = lock
... for i in xrange(n):
... _lock.acquire()
... val2 += 1
... _lock.release()
...for a little bit faster (though the locking/unlocking will overwhelm
the actual time spent.

- Josiah

Jul 18 '05 #6
li****@cix.co.uk (Matthew Bell) writes:
(...)
My question is simply, can anyone see any issues with calling methods
and/or attributes of a threaded class instance like this? It looks ok
to me but, as the docs never seem to mention using threads like this,
I'm wondering if I've missed something important. If it helps, I
know the basic considerations of threading, such as locking, exception
handling and so on; I only really need advice on whether there could
be issues with directly calling class instance attributes of a
running thread.


Others have focused more on the locking issues if the methods you use
access data that the separate thread is also accessing, so I'll try to
hit the general question of just sharing the instance itself.

Clearly directly accessing a non-callable attribute has the potential
requirement for locking and/or race conditions. But for callables,
and if I understand what you might be getting at, the answer is
definitely yes. There's absolutely no problem calling methods on a
thread object from separate threads, and even have methods used from
multiple threads simultaneously. The execution flow itself is fine,
but as you note, you have to handle shared data access issues
yourself, to the extent that it applies.

I do think this could simplify your thread communication in some cases
because you can export a more typical "object" interface from your
thread object (at least in the forward direction) rather than having
the user of the thread have to handle queue management.

For example, it's very common for me to have thread objects structured
like (to extend your example):

class SimpleThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
# I often handle the communication queue to the background
# thread internally, so callers need not be aware of it.
self.queue = Queue.Queue()
# Often my thread objects are self starting (so a user is just
# instantiating the object and not knowing a thread is involved)
self.start()

#
# Typically several methods used by the background thread during
# processing.
#
def _spam(self):
pass
def _eggs(self):
pass
def _ham(self):
pass

#
# The main thread itself - processes queue requests
#
def run(self):
while 1:
operation = self.queue.get()
if operation is None:
return

# Perform operation
#
# "Public" (not background thread) operations
#
def shutdown(self):
self.queue.put(None)

def someoperation(self, args):
self.queue.put(("dosomething", args))
So to the outside party, they are just creating an instance of my
object, and using methods on it. The methods happen to then use a queue
internally to get to the processing portion of the object which is in
a background thread, but you don't need to expose that to the caller.

Where this falls down a little is in the result of the processing.
Generally you need to provide for a query mechanism on your object
(which itself might be using an internal queue, or you could just
permit the caller to access an attribute which is the queue), or a
callback system, in which case the caller should clearly be made aware
that the callback will be executing in a separate thread.

Or, if you're using some async, event-driven approach even the thread
is probably completely hideable. For example, with Twisted, your public
thread instance methods can just appear as deferrable methods, using
standard deferreds as return values. Then when the result is ready in the
background thread, you have twisted fire the deferred in the main reactor
loop.

-- David
Jul 18 '05 #7
"Diez B. Roggisch" <de*********@web.de> wrote in message news:<cl*************@news.t-online.com>...
My question is simply, can anyone see any issues with calling methods
and/or attributes of a threaded class instance like this? It looks ok
to me but, as the docs never seem to mention using threads like this,
I'm wondering if I've missed something important.
So your code is perfectly legal, and as long as you are aware that having
more complex operations on objects undergoing can be interrupted at any
time, maybe leaving data inconsistent _from and applications POV_ - then
you'll need explicid sync'ing, by locks, queues or whatever...


Diez,

Thanks for your reply. Basically, then, it seems that there aren't
any special considerations required for directly calling threaded
class instance methods / attributes. I'll still need to take care
of ensuring that the application's data structures are kept
internally consistent (which is fair enough - I was expecting to
have to do that anyway) but it looks like I'm not going to cause
any unusual issues.

If so, that's great - it'll help tidy things up quite nicely in
areas where I've got way too many queues to easily keep track of.

Thanks for your help Diez, and thanks also to everyone else who's
commented. It's always an education!

Matthew.
Jul 18 '05 #8
Josiah Carlson wrote:
def goo(n):


... global val2
... _lock = lock
... for i in xrange(n):
... _lock.acquire()
... val2 += 1
... _lock.release()
...for a little bit faster (though the locking/unlocking will overwhelm
the actual time spent.


If you're intent on making the code less maintainable in
order to achieve tiny improvements in speed, at least
store local references to the entire method, not just
to the object:

global val2
acq = lock.acquire
rel = lock.release
for i in xrange(n):
acq()
val2 += 1
rel()

-Peter
Jul 18 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Brian Munroe | last post by:
I am just starting to learn the OO side of Python scripting, and I am a little confused on the following. Take the following example class: >>> class rectangle(object): z = 1 def...
2
by: Gabriel Genellina | last post by:
Hi In the following code sample, I have: - a Worker class, which could have a lot of methods and attributes. In particular, it has a 'bar' attribute. This class can be modified as needed. - a...
4
by: Neil Zanella | last post by:
Hello, I would like to know whether it is possible to define static class methods and data members in Python (similar to the way it can be done in C++ or Java). These do not seem to be mentioned...
0
by: BergRD | last post by:
Salutations! New to the forums but have gotten many an idea from lurking over the past few months but alas it's time to begin positing; posting to a problem I cannot seem to resolve. This is a...
6
by: Anthony Smith | last post by:
I can call a class using "->", but it complains about the :: I see on the net where :: is used. Is there a good explanation on when to use one over the other or the differences? $help = new...
3
by: Steven D'Aprano | last post by:
I have a class that has a distinct "empty" state. In the empty state, it shouldn't have any data attributes, but it should still have methods. The analogy is with a list: an empty list still has...
2
by: Kirk Strauser | last post by:
I'm trying to write a decorator that would do something like: def trace(before, after): def middle(func): def inner(*args, **kwargs): func.im_self.debugfunction(before) result = func(*args,...
4
by: Travis | last post by:
Is it considered good practice to call a mutator when inside the same class or modify the attribute directly? So if there's a public method SetName() would it be better from say ::Init() to call...
4
by: Kurda Yon | last post by:
Hi, I start to learn the object oriented programing in Python. As far as I understood, every class has a set of corresponding methods and variables. For me it is easy to understand a method as a...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.