473,289 Members | 2,108 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,289 software developers and data experts.

Why does start_new_thread() create an extra process under Linux?

Running the following under Linux creates
3 processes instead of 2. Once the started
thread exits, 2 processes still remain. Why?
import thread
from thread import start_new_thread

def newthread():
print "child"

.... suitable delay ...

thread.exit()
start_new_thread(newthread, () )
while 1:
pass
Note: am running Linux Kernel 2.6.7 / glibc 2.3.2 (Slackware 10)
Jul 18 '05 #1
5 2686
Jon Perez wrote:
Running the following under Linux creates
3 processes instead of 2. Once the started
thread exits, 2 processes still remain. Why?


Most likely, the "extra" you are seeing is an implementation detail
of your platform's underlying thread library. It probably exists to act
as a scheduler or perform other administrative tasks for the "real"
threads of your application.

Jp

Jul 18 '05 #2
Am Donnerstag, 29. Juli 2004 16:00 schrieb Jp Calderone:
Most likely, the "extra" you are seeing is an implementation detail
of your platform's underlying thread library. It probably exists to act
as a scheduler or perform other administrative tasks for the "real"
threads of your application.


Well, first of all, what the op was seeing wasn't actually what he thought he
was seeing.

In Python there's always the main thread (which is started when python starts
up), and other threads may be started. Thus, if you start two threads in your
program, you'll see three processes in the process list (one for the main
thread, two for the started threads).

But whether these threads will show up as processes depends on the threading
library you use...

LinuxThreads creates a process for each thread that is run. All these
processes share the same memory, although they show up as separate processes
(and actually are, at least for the kernel, they are started by the sys-call
CLONE, which clones a process creating a new process ID, stack and
instruction pointer, but keeping the data and code segment of the cloning
process).

NPTL (Native Posix Threads Library), the "next-generation" threads library for
Linux, handles threads "correctly" in the sense that they are just one
process with separate execution frames but shared memory. NPTL requires
kernel >= 2.5.40-something and a specially adapted glibc. Most new Linux
distributions (>= 9.0 something, debian sid aka. unstable) ship with NPTL
enabled by default, although this creates compatability problems with apps
written for LinuxThreads, as LinuxThreads isn't completely Posix-Threads
compatible (which NPTL is). It also uses some form of syscall, but you'd have
to see the docs for this, I don't know. ps from procps was augmented to
support NPTL threads sometime ago, there's a specific flag you have to
specify to have threads shown.

There are also other Linux threads libraries out there, all of them completely
implemented in user-space, using dispatch/longjmp and other black magic. When
a program uses one of these, you'll also see only one process, although I
don't know any production program that uses one of these threading libraries.

Anyway, hope this clears it up a little...

Heiko.
Jul 18 '05 #3
Am Donnerstag, 29. Juli 2004 17:31 schrieb Heiko Wundram:
Well, first of all, what the op was seeing wasn't actually what he thought
he was seeing.

In Python there's always the main thread (which is started when python
starts up), and other threads may be started. Thus, if you start two
threads in your program, you'll see three processes in the process list
(one for the main thread, two for the started threads).


Forget that, I read the first post wrong. What the op was probably seeing was
output from an NPTL patched ps, which always shows the threads that are
running (not only when asked for it). ps outputs one line (the first) for the
process, all other lines are for each of the threads that are currently
running under this process. So, the following output from ps (actually
ps ax -m on my machine) means that pickup (part of postfix) only runs one
thread (not two processes), and xmms runs five threads.

17539 ? - 0:00 pickup -l -t fifo -u
- - S 0:00 -
18338 ? - 0:02 /usr/bin/xmms
- - S 0:02 -
- - S 0:00 -
- - S 0:00 -
- - S 0:00 -
- - S 0:00 -

I have an NPTL enabled glibc + kernel (without a somewhat strange patch, as
the op seems to have), when I only type ps ax, it'll show up as:

17539 ? S 0:00 pickup -l -t fifo -u
18338 ? S 0:02 /usr/bin/xmms

To see whether you have an NPTL enabled glibc, type /lib/libc.so.6, and it'll
output something like:

....
Available extensions:
....
NPTL 0.61 by Ulrich Drepper
....

Heiko.
Jul 18 '05 #4
Heiko Wundram wrote:
NPTL (Native Posix Threads Library), the "next-generation" threads library for
Linux, handles threads "correctly" in the sense that they are just one
process with separate execution frames but shared memory.
Does this the fact that NPTL threads are 'just one process' mean they
are not created using clone()? Are NPTL threads not scheduled by
the kernel?

If so, then how come NTPL is described as a 1:1 model which afaik
means 1 application thread is mapped to exactly 1 'kernel' scheduled thread
(or lightweight process if you will) which, again afaik, can only be created
via a clone() call (albeit with different flags) and nothing else.

If NTPL threads are scheduled by NPTL code as opposed to kernel code and are
all mapped onto one process started by one clone() call, wouldn't that make it M:1?

What the op was probably seeing was output from an NPTL patched ps, which
always shows the threads that are running (not only when asked for it).


Note that my glibc is not NTPL-enabled, it is the stock 2.3.2 used
in Slackware 10 (although the procps-3.2.1 it uses may already be NPTL-ready),
so this would not seem to be the explanation.

If you start the sample program in my original message and it hasn't launched
a thread yet, ps will only show one running process. The moment it calls
start_new_thread() however, two new processes show up in ps (so that makes
three total)! Once this newly started thread dies, only one process gets removed,
so there will still be two processes running and that's what's puzzling me.

Same thing applies if you start N number of threads. Seems there's always
one extra thread lying around after you call start_new_thread().
Jul 18 '05 #5
Jon Perez <jb********@wahoo.com> writes:
Does this the fact that NPTL threads are 'just one process' mean they
are not created using clone()? Are NPTL threads not scheduled by
the kernel?


they are just hidden from the /proc directory listing.

(erno@fabulous) /home.b/erno % ls -l /proc/`pidof firefox-bin`/task
total 0
dr-xr-xr-x 3 erno erno 0 Jul 30 17:56 28319
dr-xr-xr-x 3 erno erno 0 Jul 30 17:56 31596
dr-xr-xr-x 3 erno erno 0 Jul 30 17:56 31597
dr-xr-xr-x 3 erno erno 0 Jul 30 17:56 31599
(erno@fabulous) /home.b/erno % ls -l /proc/28319 | wc -l
16
(erno@fabulous) /home.b/erno % ls -l /proc|grep -c 28319
0

-- erno
Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Marcus Schneider | last post by:
I use PythonWin on WinXP. Every time I change a module, I have to leave PythonWin and re enter to make it notice I have made changes. I guess this is not the normal way to do that.. do I have to...
3
by: Thomas Schmid | last post by:
Hi there, I wrote a tcp server which listens on a port. When he gets a new connection, he starts a new thread like this: thread.start_new_thread(self.ConnectionHandler, (conn,)) where conn is...
3
by: Konstantin Veretennicov | last post by:
Hi, Just curious: >>> import thread >>> help(thread.start_new_thread) . . . start_new_thread(function, args) . . . Second argument is mandatory. Is it incidental or for a reason?
2
by: greatbooksclassics | last post by:
Open Source DRM? What does everyone think about it? Will Open Source DRM ever catch up to MS DRM? Will DRM ever be integrated into common LAMP applications?...
35
by: Sunil | last post by:
Hi all, I am using gcc compiler in linux.I compiled a small program int main() { printf("char : %d\n",sizeof(char)); printf("unsigned char : ...
9
by: Andy B | last post by:
If I bought one of these boxes/OS combos as a postgresql database server, would postgresql be able to make the best use of it with a huge (e.g. 40GB) database? Box: HP ProLiant DL585, with ...
62
by: robert | last post by:
I'd like to use multiple CPU cores for selected time consuming Python computations (incl. numpy/scipy) in a frictionless manner. Interprocess communication is tedious and out of question, so I...
113
by: John Nagle | last post by:
The major complaint I have about Python is that the packages which connect it to other software components all seem to have serious problems. As long as you don't need to talk to anything outside...
8
by: Nehil | last post by:
When a process is started three segments are created : 1) Text. 2) Stack. 3) Data. The size of First two is fixed and their upperlimit is fixed by the compiler. (Plz correct if i'm wrong) Now...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.