Python threading

[test.07]

I am wondering what happens to a thread in python in relation to
win32com extensions.

If I create a new thread, that uses the Dispatch method from win32com,
what happens to the memory allocated in that thread when the thread is
done. Will the Dispatch release the memory it created, or will the
memory remain?

The problem rises from the fact that Dispatch does not seem to release
memory correctly every time. If I include the commands in a thread by
themselves, will the thread clean up ALL memory it used after it is
done?

I did try the pythoncom.CoUnitialize() to release memory, but it
doesn't seem to work (it does work about 30-45 seconds after the
command is run).

Any input is greatly appreciated (on the thread issue or how to use
the pythoncom.CoUnitiliaze() to make it release memory right away).

Thank you in advance!

[Gabriel Genellina]

En Thu, 08 Mar 2007 14:25:02 -0300, <te*****@gmail.comescribió:

I am wondering what happens to a thread in python in relation to
win32com extensions.

If I create a new thread, that uses the Dispatch method from win32com,
what happens to the memory allocated in that thread when the thread is
done. Will the Dispatch release the memory it created, or will the
memory remain?

The problem rises from the fact that Dispatch does not seem to release
memory correctly every time. If I include the commands in a thread by
themselves, will the thread clean up ALL memory it used after it is
done?

All threads share the same memory space, there is not a "per-thread"
memory allocator, if that's what you are thinking.
Perhaps you hold a reference to some objects in the Thread object? Or you
still keep the Thread object itself?

I did try the pythoncom.CoUnitialize() to release memory, but it
doesn't seem to work (it does work about 30-45 seconds after the
command is run).

I don't understand what you say here.
What means "it doesn't seem to work" and "it does work 30 seconds after"?

Any input is greatly appreciated (on the thread issue or how to use
the pythoncom.CoUnitiliaze() to make it release memory right away).

What memory do you want to release "right away"?

--
Gabriel Genellina

[test.07]

On Mar 8, 6:15 pm, "Gabriel Genellina" <[e-mail]:

En Thu, 08 Mar 2007 14:25:02 -0300, <[e-mail]ó:
All threads share the same memory space, there is not a "per-thread"
memory allocator, if that's what you are thinking.
Perhaps you hold a reference to some objects in the Thread object? Or you
still keep the Thread object itself?

That is what I was thinking. A per-thread memory allocator of some
sort. What I need is run a lot of commands from a COM object that can
be run independently. There commands leak memory, so when the command
finishes, the memory consumption jumps to 800MBs or so. Which begs the
question, if I initialize the COM object in a separate thread and run
the command, does finishing the thread clean up any and all memory the
thread used or not. The output of each thread would actually be a file
on disk, therefore there is no need to pass any data between the
threads and the main program.

I did try the pythoncom.CoUnitialize() to release memory, but it
doesn't seem to work (it does work about 30-45 seconds after the
command is run).

I don't understand what you say here.
What means "it doesn't seem to work" and "it does work 30 seconds after"?

pythoncom.CoUninitialize(), if I am not mistaken, releases a COM
object (and therefore the memory it uses I assume). When the command
is run (in PythonWin for example), the memory is not released but
persists for quite a while. Using my watch and the Task Manager in
Windows I can see that the memory is released approximately 30 seconds
AFTER I run the pythoncom.CoUninitialize() command is run.

What memory do you want to release "right away"?

The memory I want to release is the memory the COM object used (the
one initialized with the win32com Dispatch command). The "right-away"
is not much of an issue, but if I can release it before I run each
command from the COM object that leaks memory, it would be nice.
Running upwards to 800MBs of RAM for one 500 line python script seems
a little bit too much for me.

Thank you for your reply Gabriel. If you have any more input, I would
greatly appreciate it.

[Gabriel Genellina]

En Fri, 09 Mar 2007 00:38:55 -0300, <te*****@gmail.comescribió:

pythoncom.CoUninitialize(), if I am not mistaken, releases a COM
object (and therefore the memory it uses I assume).

Not exactly. *You* must release the COM object (usually assigning None to
all references to it). CoUninitialize releases all system resources held
by the COM application, unloads DLLs, and so; it *may* release all COM
objects but I'm not sure.

When the command
is run (in PythonWin for example), the memory is not released but
persists for quite a while. Using my watch and the Task Manager in
Windows I can see that the memory is released approximately 30 seconds
AFTER I run the pythoncom.CoUninitialize() command is run.

If it is eventually released, you don't have a problem, right?
And, if it will be needed again after a while (when you run the next
command), it looks better this way.
There are many layers on memory management: the OS manages 4K pages that
are mapped on the process address space; the C runtime suballocates such
pages as memory blocks; Python itself has two levels of memory allocation
(and I don't know the details). It's not easy to know exactly how much
memory is actually used; I think the Python runtime, once it allocates a
memory block, never releases it to the OS (remaining as a free block,
available for further usage).

And in general, DLLs are *not* unloaded as soon as its usage count goes to
0; there is certain delay. (I don't remember, but I think there is a
registry key called DLLUnloadTime or something). The idea is to avoid
removing it from memory just to load it again a few moments later, so the
unloading is delayed for a few seconds. You may be seeing this effect.

>What memory do you want to release "right away"?

The memory I want to release is the memory the COM object used (the
one initialized with the win32com Dispatch command). The "right-away"
is not much of an issue, but if I can release it before I run each
command from the COM object that leaks memory, it would be nice.
Running upwards to 800MBs of RAM for one 500 line python script seems
a little bit too much for me.

Let's see if I can understand the problem. You run multiple commands on a
COM object, one after another. The COM object is buggy and leaks some
memory. You run each command on a separate thread, with a CoInitialize(),
waiting for completion, and a CoUninitialize() at the end. Memory usage
grows to 800MB when you invoke each command, but if you wait enough time
after the thread finalizes, memory usage goes down to normal. After that,
you run the next command, and so on.
Or do you wait until the whole program finalizes, and 30 seconds after
that, memory usage drops?

--
Gabriel Genellina