473,230 Members | 1,944 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,230 software developers and data experts.

Memory Usage

rbt
Would a Python process consume more memory on a PC with lots of memory?

For example, say I have the same Python script running on two WinXP
computers that both have Python 2.4.0. One computer has 256 MB of Ram
while the other has 2 GB of Ram. On the machine with less Ram, the
process takes about 1 MB of Ram. On the machine with more Ram, it uses 9
MB of Ram.

Is this normal and expected behavior?

Thanks,

rbt
Jul 18 '05 #1
8 3634
rbt wrote:
Would a Python process consume more memory on a PC with lots of memory?

For example, say I have the same Python script running on two WinXP
computers that both have Python 2.4.0. One computer has 256 MB of Ram
while the other has 2 GB of Ram. On the machine with less Ram, the
process takes about 1 MB of Ram. On the machine with more Ram, it uses 9
MB of Ram.

Is this normal and expected behavior?


It's probably not normal if this is *really* the memory usage, but
I would expect to see such behaviour, given how difficult it is
to measure *actual* memory usage. How are you measuring it?
Just by looking at the Mem Usage column in the Task Manager?

-Peter
Jul 18 '05 #2
rbt
Peter Hansen wrote:
rbt wrote:
Would a Python process consume more memory on a PC with lots of memory?

For example, say I have the same Python script running on two WinXP
computers that both have Python 2.4.0. One computer has 256 MB of Ram
while the other has 2 GB of Ram. On the machine with less Ram, the
process takes about 1 MB of Ram. On the machine with more Ram, it uses
9 MB of Ram.

Is this normal and expected behavior?

It's probably not normal if this is *really* the memory usage, but
I would expect to see such behaviour, given how difficult it is
to measure *actual* memory usage. How are you measuring it?
Just by looking at the Mem Usage column in the Task Manager?

-Peter


That's right. I look at that column. Should I measue mem usage in some
other way?
Jul 18 '05 #3
"rbt" wrote:
For example, say I have the same Python script running on two WinXP computers that both have
Python 2.4.0. One computer has 256 MB of Ram while the other has 2 GB of Ram. On the machine with
less Ram, the process takes about 1 MB of Ram. On the machine with more Ram, it uses 9 MB of Ram.

Is this normal and expected behavior?


1 MB sounds low, 9 MB sounds more reasonable for a script that uses a few mega-
bytes of data. what tool are you using to determine the process size?

</F>

Jul 18 '05 #4
rbt wrote:
Peter Hansen wrote:
I would expect to see such behaviour, given how difficult it is
to measure *actual* memory usage. How are you measuring it?
Just by looking at the Mem Usage column in the Task Manager?


That's right. I look at that column. Should I measue mem usage in some
other way?


Probably, but for a start, have you noticed that even just
switching to another window can drastically affect the
memory apparently used by an application? For example,
running the wxPython demo, clicking on a few controls and
getting memory usage up to 27MB, then minimizing the window
will drop it down aboiut 2MB. Restoring the window will
bring the amount back to only about 4MB, at least until
you click on stuff. Even then, it might climb to only
about 14MB or so.

This particular phenomenon might not be affecting your
application, but it's an indication of how bad this
measurement technique can be.

Inside the Control Panel, you will find "Administrative Tools".
In there is a "Performance" gadget. It's not trivial to
use, and I can't give a tutorial here, but if you can
manage to display the Working Set for the specific Process
in which you are interested, that's probably a better
way to view the information. (To be honest, I think it's
actually this particular value which the "Mem Usage"
field shows in the Task Manager, but at least this way
you get a real-time graph and more precision, and a
record of the usage.) There are also dozens of other
parameters you can examine to help you track down the
actual usage, or to help find out what is going if it turns
out that you really are using such different amounts on
the two machines.

More important than any of this, however, might be making
sure you have similar conditions on the two machines. Are
you terminating as many other processes as you can? Making
sure there is ample Physical Memory Available (see the
Performance tab of Task Manager, for example)? If you
have one of the machines running out of memory because
of other applications running, it is quite possible that
the OS will steal memory from the Python process to feed
the other apps, and that can show up in the working set size.

As I said, this stuff isn't exactly straightforward, so
it's not necessarily surprising you are seeing this behaviour.
There's probably a relatively simple explanation, however,
but it might come only after a bit of exploration.

-Peter
Jul 18 '05 #5
Do you have a page file?

The Mem column should be RAM usage and not total working set. Some of it
could be swapped to the page file. A free tool like process explorer can
give you better informaton than the task manager.

"rbt" <rb*@athop1.ath.vt.edu> wrote in message
news:ct**********@solaris.cc.vt.edu...
Would a Python process consume more memory on a PC with lots of memory?

For example, say I have the same Python script running on two WinXP
computers that both have Python 2.4.0. One computer has 256 MB of Ram
while the other has 2 GB of Ram. On the machine with less Ram, the process
takes about 1 MB of Ram. On the machine with more Ram, it uses 9 MB of
Ram.

Is this normal and expected behavior?

Thanks,

rbt

Jul 18 '05 #6
[<rb*@athop1.ath.vt.edu>]
Would a Python process consume more memory on a PC with lots of
memory?

For example, say I have the same Python script running on two WinXP
computers that both have Python 2.4.0. One computer has 256 MB of Ram
while the other has 2 GB of Ram. On the machine with less Ram, the
process takes about 1 MB of Ram. On the machine with more Ram, it uses
9 MB of Ram.

Is this normal and expected behavior?


[and later confirms he's looking at Mem Usage in Task Manager]

256MB is on the low end for an XP system, and will generally cause
lots of swap space (virtual memory residing on disk -- kinda) to get
used. Mem Usage doesn't count swap space, just RAM currently in use.
Under Task Manager it's better to look at the VM Size column, which
tells roughly how much virtual address space is assigned to the
process, and regardless of how it's currently split between RAM and
disk. You may need to use View -> Select Columns to get this
statistic displayed. If your process is in fact using a fixed amount
of address space, the VM Size column will stay steady but Mem Usage
may jump all over the place as time goes on. In general, I would
expect VM Size to be about the same on your two boxes; I would not
expect Mem Usage to be the same.
Jul 18 '05 #7
In message <ct**********@solaris.cc.vt.edu>, rbt <rb*@athop1.ath.vt.edu>
writes
That's right. I look at that column. Should I measue mem usage in some
other way?


Try VM Validator, a free memory visualization tool from Software
Verification.

http://www.softwareverify.com
http://www.softwareverify.com/vmValidator/index.html

It shows paged memory usage and also Virtual Memory manager usage on
separate tabs. Colour coded visual representation of each 4K page of
memory.

Probably more use on more memory intensive applications than yours, but
may still shed some light all the same. Either launch Python from VM
Validator, or inject VM Validator into your running Python.exe process.

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk
RSI Information: http://www.objmedia.demon.co.uk/rsi.html
Jul 18 '05 #8
Stuart McGarrity wrote:
Do you have a page file?

The Mem column should be RAM usage and not total working set. Some of it
could be swapped to the page file. A free tool like process explorer can
give you better informaton than the task manager.


As Tim pointed out, "View->Select Columns" and activating "VM Size" is enough to
find out how much memory that program actually has *mapped* (rather than
currently loaded into RAM).

Cheers,
Nick.

--
Nick Coghlan | nc******@email.com | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net
Jul 18 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Justice | last post by:
Currently I'm doing some experimenting with the XMLHTTP object in Javascript. Now, the XMLHttp object is asynchronous (at least in this case), and the following code causes a significant memory...
2
by: tomvr | last post by:
Hello I have noticed some 'weird' memory usage in a vb.net windows app The situation is as follows I have an app (heavy on images) with 2 forms (actually there are more forms and on starting...
6
by: Tom | last post by:
We have a VERY simple .NET C# Form Application, that has about a 23MB Memory Footprint. It starts a window runs a process and does a regular expression. I have done a GC.Collect to make sure that,...
2
by: Jarvis | last post by:
I've made a testing program to test the memory usage of some Data Forms. I create a MDI parent form with one single MDI child form, which is a Data Form generated by .NET Data Form Wizard. To...
3
by: Ian Taite | last post by:
Hello, I'm exploring why one of my C# .NET apps has "high" memory usage, and whether I can reduce the memory usage. I have an app that wakes up and processes text files into a database...
20
by: Philip Carnstam | last post by:
How come .Net applications use so much memory? Every application I compile uses at least 10 MB of memory, even the ones consisting of only a form and nothing else. If I minimize them though the...
13
by: Ilias Lazaridis | last post by:
How to detect memory leaks of python programms, which run in an environment like this: * Suse Linux 9.3 * Apache * mod_python The problem occoured after some updates on the infrastructure....
1
by: yzghan | last post by:
Hi all, I feel that my python script is leaking memory. And this is a test I have: log.write(" " + "test() ... memory usage: " + " ".join(repr(i/(1024*1024)) for i in getMemInfo()) + "\n") m...
1
by: Jean-Paul Calderone | last post by:
On Tue, 22 Apr 2008 14:54:37 -0700 (PDT), yzghan@gmail.com wrote: The test doesn't demonstrate any leaks. It does demonstrate that memory usage can remain at or near peak memory usage even after...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.