473,545 Members | 1,638 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

is +=1 thread safe

Hi,

I have a piece of software which uses threads in very massive way - like
hundreds of them generated every second.

there is also a piece of code which maintains the number of outstanding
threads, simply

counter+=1 is executed when before starting the thread and counter-=1
after it finishes.

all is very simple and by the end of the program life I expect the
counter to zero out.

however I am getting values -1, -2, 1 ,2 ,3 and quite often 0 as expected.

I guarded those statement with Lock.{acquire,r elease} and now it always
returns 0.
But I still can not believe that +=1 is not a thread safe operation.
Any clue?

--

Andy
Jun 27 '08 #1
29 1672
If no other threads will be accessing the counter, there will be no
problem.
Jun 27 '08 #2
AlFire <sp************ ******@ggmail.c omwrote:
But I still can not believe that +=1 is not a thread safe operation.
Any clue?
The statement:

x+=1

is equivalent to:

x = x.__iadd__(1)

i.e. a function call followed by an assignment.

If the object is mutable then this *may* be safe so long as you never do
any other assignment to x (the __iadd__ method will return the object being
mutated so the assignment would in this restricted case be a noop).

Integers are immutable, so the assignment must always rebind x to a new
object. There is no way given Python's semantics that this could be thread
safe.

Generating hundreds of threads is, BTW, a very good way to get poor
performance on any system. Don't do that. Create a few threads and put the
actions for those threads into a Queue. If you want the threads to execute
in parallel investigate using sub-processes.

The threading module already has a function to return the number of Thread
objects currently alive.
Jun 27 '08 #3
AlFire schrieb:
Hi,

I have a piece of software which uses threads in very massive way - like
hundreds of them generated every second.

there is also a piece of code which maintains the number of outstanding
threads, simply

counter+=1 is executed when before starting the thread and counter-=1
after it finishes.

all is very simple and by the end of the program life I expect the
counter to zero out.

however I am getting values -1, -2, 1 ,2 ,3 and quite often 0 as expected.

I guarded those statement with Lock.{acquire,r elease} and now it always
returns 0.
But I still can not believe that +=1 is not a thread safe operation.
don't confuse augmented assignment with incrementation as it is offered
by C (if your data-type actually fits into a single addressable memory
spot, that is)

python's += works like this
a += b <=a = a.__iadd__(b)

Thus you actually get a situation where the expression on the right is
evaluated but not yet assigned - and then another thread can take over
control, computing with the old value of a.

Diez
Jun 27 '08 #4
AlFire wrote:
Hi,
all is very simple and by the end of the program life I expect the
counter to zero out.

however I am getting values -1, -2, 1 ,2 ,3 and quite often 0 as expected.

I guarded those statement with Lock.{acquire,r elease} and now it always
returns 0.
But I still can not believe that +=1 is not a thread safe operation.
Any clue?
"counter += 1" is not an atomic operation, apparently. Sorry.

However, appending and removing from a list or dict are atomic operations,
because if they were not, memory allocation would break.

John Nagle
Jun 27 '08 #5
AlFire wrote:
Hi,

I have a piece of software which uses threads in very massive way -
like hundreds of them generated every second.

there is also a piece of code which maintains the number of
outstanding threads, simply

counter+=1 is executed when before starting the thread and counter-=1
after it finishes.

all is very simple and by the end of the program life I expect the
counter to zero out.

however I am getting values -1, -2, 1 ,2 ,3 and quite often 0 as
expected.

I guarded those statement with Lock.{acquire,r elease} and now it
always returns 0.
But I still can not believe that +=1 is not a thread safe operation.
Any clue?
Of course it's not thread safe. For the same reason and more basic,
even the expression i++ is not thread safe in C++.

Any such calculation, on modern processors, requires three operations:
retrieve value of i into a register,
increment the register
write the value into i.

If a thread is interrupted anywhere within that sequence, and another
thread access i, you have a conflict. (And indeed, hardware interrupts
can occur between any two instructions.)

Gary Herron

Jun 27 '08 #6
On Thu, 01 May 2008 15:33:09 -0700, Gary Herron wrote:
Of course it's not thread safe. For the same reason and more basic,
even the expression i++ is not thread safe in C++.

Any such calculation, on modern processors, requires three operations:
retrieve value of i into a register,
increment the register
write the value into i.
There are no modern processors with an opcode for incrementing a memory
location!? At least my C64 can do that. ;-)

Ciao,
Marc 'BlackJack' Rintsch
Jun 27 '08 #7
AlFire <sp************ ******@ggmail.c omwrites:
But I still can not believe that +=1 is not a thread safe operation.
In CPython I believe it is thread safe, because of the global
interpreter lock. Thread switches can happen only between bytecode
executions, not in the middle of a bytecode, even though executing a
bytecode can take several machine instructions.
Jun 27 '08 #8
Marc 'BlackJack' Rintsch <bj****@gmx.net wrote:
On Thu, 01 May 2008 15:33:09 -0700, Gary Herron wrote:
>Of course it's not thread safe. For the same reason and more basic,
even the expression i++ is not thread safe in C++.

Any such calculation, on modern processors, requires three operations:
retrieve value of i into a register,
increment the register
write the value into i.

There are no modern processors with an opcode for incrementing a memory
location!? At least my C64 can do that. ;-)
The operation i++ in C/C++ implies two things: it increments the memory
location and returns the new result. That means there are at least two ways
to generate code for this:

increment memory
load new value

or

load value
incremenent
store

Neither of these is going to be thread-safe (even on a C64) unless you
protect the operations somehow. The latter is probably preferred as it only
implies two operations to memory whereas the former implies three.

Even on your C64 incrementing a memory location involves both a read and a
write, so if you have a multi-core C64(!) there is still scope for another
processor to get into the middle of that increment instruction.

Jun 27 '08 #9
Paul Rubin <http://ph****@NOSPAM.i nvalidwrites:
AlFire <sp************ ******@ggmail.c omwrites:
>But I still can not believe that +=1 is not a thread safe operation.

In CPython I believe it is thread safe, because of the global
interpreter lock. Thread switches can happen only between bytecode
executions, not in the middle of a bytecode, even though executing a
bytecode can take several machine instructions.
It's safe when writing extensions in C (because of the GIL), but not
when writing Python code. Python switches threads after a number of
bytecode instructions, so Python code behaves somewhat like C
multithreaded code on a single CPU -- although there is no true
parallelism, you still have context switches at arbitrary places to
worry about, and still need locks because of them. Since the +=
operator is not compiled into a single bytecode instruction, it needs
the lock. x+=1 gets compiled into the following bytecode:

3 0 LOAD_GLOBAL 0 (x)
3 LOAD_CONST 1 (1)
6 INPLACE_ADD
7 STORE_GLOBAL 0 (x)

If two threads execute that code simultaneously, a thread switch could
easily occur some time between LOAD_GLOBAL and STORE_GLOBAL, causing
the variable to be incremented by 1 instead of by 2.
Jun 27 '08 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
6632
by: Jonathan Burd | last post by:
Greetings everyone, Here is a random string generator I wrote for an application and I'm wondering about the thread-safety of this function. I was told using static and global variables cause potential problems for thread-safety. So far, I'm only confused. I need a proper explanation for the concept so I can understand how to write...
11
2229
by: dee | last post by:
OleDbCommand class like many .NET classes has the following description in its help file: "Thread Safety Any public static (Shared in Visual Basic) members of this type are safe for multithreaded operations. Any instance members are not guaranteed to be thread safe." I have 2 questions: 1. I thought dynamic variables are thread-safe...
3
2527
by: tcomer | last post by:
Hello! I'm working on an asynchronous network application that uses multiple threads to do it's work. I have a ChatClient class that handles the basic functionality of connecting to a server and sending/ receiving messages. The problem is, some of the ChatClient methods access another ChatWindow class that is derived from a Form and that...
15
2745
by: Laser Lu | last post by:
I was often noted by Thread Safety declarations when I was reading .NET Framework Class Library documents in MSDN. The declaration is usually described as 'Any public static (Shared in Visual Basic) members of this type are thread safe. Any instance members are not guaranteed to be thread safe.' So, does this mean All the static/shared...
1
4602
by: jecheney | last post by:
Hi, Im currently using the following code for reading/writing to a network socket. private StreamReader clientStreamReader; private StreamWriter clientStreamWriter; .... TcpClient tcpClient = new TcpClient(server_host_name, server_port);
13
3568
by: arun.darra | last post by:
Are the following thread safe: 1. Assuming Object is any simple object Object* fn() { Object *p = new Object(); return p; } 2. is return by value thread safe?
12
1669
by: Peter K | last post by:
Say I have this class public class Simple { private string name; public string Name { get { return name; } set { name = value; }
44
7743
by: climber.cui | last post by:
Hi all, Does anyone have experience on the thread-safty issue with malloc()? Some people said this function provided in stdlib.h is not thread- safe, but someone said it is thread safe. Is it possible this function evolves from thread-unsafe to thread-safe in recent years? How could i find out? I am using the C library coming with GNU linux...
13
11424
by: Henri.Chinasque | last post by:
Hi all, I am wondering about thread safety and member variables. If I have such a class: class foo { private float m_floater = 0.0; public void bar(){ m_floater = true; }
29
9096
by: NvrBst | last post by:
I've read a bit online seeing that two writes are not safe, which I understand, but would 1 thread push()'ing and 1 thread pop()'ing be thread-safe? Basically my situation is the follows: --Thread 1-- 1. Reads TCPIP Buffer 2. Adds Buffer to Queue (q.size() to check queue isn't full, and q.push_back(...)) 3. Signals Reading Thread Event &...
0
7393
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
7653
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
1
7411
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7749
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
5965
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5322
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
4942
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3444
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3439
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.