473,320 Members | 2,117 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Memory Usage

rbt
Would a Python process consume more memory on a PC with lots of memory?

For example, say I have the same Python script running on two WinXP
computers that both have Python 2.4.0. One computer has 256 MB of Ram
while the other has 2 GB of Ram. On the machine with less Ram, the
process takes about 1 MB of Ram. On the machine with more Ram, it uses 9
MB of Ram.

Is this normal and expected behavior?

Thanks,

rbt
Jul 18 '05 #1
8 3641
rbt wrote:
Would a Python process consume more memory on a PC with lots of memory?

For example, say I have the same Python script running on two WinXP
computers that both have Python 2.4.0. One computer has 256 MB of Ram
while the other has 2 GB of Ram. On the machine with less Ram, the
process takes about 1 MB of Ram. On the machine with more Ram, it uses 9
MB of Ram.

Is this normal and expected behavior?


It's probably not normal if this is *really* the memory usage, but
I would expect to see such behaviour, given how difficult it is
to measure *actual* memory usage. How are you measuring it?
Just by looking at the Mem Usage column in the Task Manager?

-Peter
Jul 18 '05 #2
rbt
Peter Hansen wrote:
rbt wrote:
Would a Python process consume more memory on a PC with lots of memory?

For example, say I have the same Python script running on two WinXP
computers that both have Python 2.4.0. One computer has 256 MB of Ram
while the other has 2 GB of Ram. On the machine with less Ram, the
process takes about 1 MB of Ram. On the machine with more Ram, it uses
9 MB of Ram.

Is this normal and expected behavior?

It's probably not normal if this is *really* the memory usage, but
I would expect to see such behaviour, given how difficult it is
to measure *actual* memory usage. How are you measuring it?
Just by looking at the Mem Usage column in the Task Manager?

-Peter


That's right. I look at that column. Should I measue mem usage in some
other way?
Jul 18 '05 #3
"rbt" wrote:
For example, say I have the same Python script running on two WinXP computers that both have
Python 2.4.0. One computer has 256 MB of Ram while the other has 2 GB of Ram. On the machine with
less Ram, the process takes about 1 MB of Ram. On the machine with more Ram, it uses 9 MB of Ram.

Is this normal and expected behavior?


1 MB sounds low, 9 MB sounds more reasonable for a script that uses a few mega-
bytes of data. what tool are you using to determine the process size?

</F>

Jul 18 '05 #4
rbt wrote:
Peter Hansen wrote:
I would expect to see such behaviour, given how difficult it is
to measure *actual* memory usage. How are you measuring it?
Just by looking at the Mem Usage column in the Task Manager?


That's right. I look at that column. Should I measue mem usage in some
other way?


Probably, but for a start, have you noticed that even just
switching to another window can drastically affect the
memory apparently used by an application? For example,
running the wxPython demo, clicking on a few controls and
getting memory usage up to 27MB, then minimizing the window
will drop it down aboiut 2MB. Restoring the window will
bring the amount back to only about 4MB, at least until
you click on stuff. Even then, it might climb to only
about 14MB or so.

This particular phenomenon might not be affecting your
application, but it's an indication of how bad this
measurement technique can be.

Inside the Control Panel, you will find "Administrative Tools".
In there is a "Performance" gadget. It's not trivial to
use, and I can't give a tutorial here, but if you can
manage to display the Working Set for the specific Process
in which you are interested, that's probably a better
way to view the information. (To be honest, I think it's
actually this particular value which the "Mem Usage"
field shows in the Task Manager, but at least this way
you get a real-time graph and more precision, and a
record of the usage.) There are also dozens of other
parameters you can examine to help you track down the
actual usage, or to help find out what is going if it turns
out that you really are using such different amounts on
the two machines.

More important than any of this, however, might be making
sure you have similar conditions on the two machines. Are
you terminating as many other processes as you can? Making
sure there is ample Physical Memory Available (see the
Performance tab of Task Manager, for example)? If you
have one of the machines running out of memory because
of other applications running, it is quite possible that
the OS will steal memory from the Python process to feed
the other apps, and that can show up in the working set size.

As I said, this stuff isn't exactly straightforward, so
it's not necessarily surprising you are seeing this behaviour.
There's probably a relatively simple explanation, however,
but it might come only after a bit of exploration.

-Peter
Jul 18 '05 #5
Do you have a page file?

The Mem column should be RAM usage and not total working set. Some of it
could be swapped to the page file. A free tool like process explorer can
give you better informaton than the task manager.

"rbt" <rb*@athop1.ath.vt.edu> wrote in message
news:ct**********@solaris.cc.vt.edu...
Would a Python process consume more memory on a PC with lots of memory?

For example, say I have the same Python script running on two WinXP
computers that both have Python 2.4.0. One computer has 256 MB of Ram
while the other has 2 GB of Ram. On the machine with less Ram, the process
takes about 1 MB of Ram. On the machine with more Ram, it uses 9 MB of
Ram.

Is this normal and expected behavior?

Thanks,

rbt

Jul 18 '05 #6
[<rb*@athop1.ath.vt.edu>]
Would a Python process consume more memory on a PC with lots of
memory?

For example, say I have the same Python script running on two WinXP
computers that both have Python 2.4.0. One computer has 256 MB of Ram
while the other has 2 GB of Ram. On the machine with less Ram, the
process takes about 1 MB of Ram. On the machine with more Ram, it uses
9 MB of Ram.

Is this normal and expected behavior?


[and later confirms he's looking at Mem Usage in Task Manager]

256MB is on the low end for an XP system, and will generally cause
lots of swap space (virtual memory residing on disk -- kinda) to get
used. Mem Usage doesn't count swap space, just RAM currently in use.
Under Task Manager it's better to look at the VM Size column, which
tells roughly how much virtual address space is assigned to the
process, and regardless of how it's currently split between RAM and
disk. You may need to use View -> Select Columns to get this
statistic displayed. If your process is in fact using a fixed amount
of address space, the VM Size column will stay steady but Mem Usage
may jump all over the place as time goes on. In general, I would
expect VM Size to be about the same on your two boxes; I would not
expect Mem Usage to be the same.
Jul 18 '05 #7
In message <ct**********@solaris.cc.vt.edu>, rbt <rb*@athop1.ath.vt.edu>
writes
That's right. I look at that column. Should I measue mem usage in some
other way?


Try VM Validator, a free memory visualization tool from Software
Verification.

http://www.softwareverify.com
http://www.softwareverify.com/vmValidator/index.html

It shows paged memory usage and also Virtual Memory manager usage on
separate tabs. Colour coded visual representation of each 4K page of
memory.

Probably more use on more memory intensive applications than yours, but
may still shed some light all the same. Either launch Python from VM
Validator, or inject VM Validator into your running Python.exe process.

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk
RSI Information: http://www.objmedia.demon.co.uk/rsi.html
Jul 18 '05 #8
Stuart McGarrity wrote:
Do you have a page file?

The Mem column should be RAM usage and not total working set. Some of it
could be swapped to the page file. A free tool like process explorer can
give you better informaton than the task manager.


As Tim pointed out, "View->Select Columns" and activating "VM Size" is enough to
find out how much memory that program actually has *mapped* (rather than
currently loaded into RAM).

Cheers,
Nick.

--
Nick Coghlan | nc******@email.com | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net
Jul 18 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Justice | last post by:
Currently I'm doing some experimenting with the XMLHTTP object in Javascript. Now, the XMLHttp object is asynchronous (at least in this case), and the following code causes a significant memory...
2
by: tomvr | last post by:
Hello I have noticed some 'weird' memory usage in a vb.net windows app The situation is as follows I have an app (heavy on images) with 2 forms (actually there are more forms and on starting...
6
by: Tom | last post by:
We have a VERY simple .NET C# Form Application, that has about a 23MB Memory Footprint. It starts a window runs a process and does a regular expression. I have done a GC.Collect to make sure that,...
2
by: Jarvis | last post by:
I've made a testing program to test the memory usage of some Data Forms. I create a MDI parent form with one single MDI child form, which is a Data Form generated by .NET Data Form Wizard. To...
3
by: Ian Taite | last post by:
Hello, I'm exploring why one of my C# .NET apps has "high" memory usage, and whether I can reduce the memory usage. I have an app that wakes up and processes text files into a database...
20
by: Philip Carnstam | last post by:
How come .Net applications use so much memory? Every application I compile uses at least 10 MB of memory, even the ones consisting of only a form and nothing else. If I minimize them though the...
13
by: Ilias Lazaridis | last post by:
How to detect memory leaks of python programms, which run in an environment like this: * Suse Linux 9.3 * Apache * mod_python The problem occoured after some updates on the infrastructure....
1
by: yzghan | last post by:
Hi all, I feel that my python script is leaking memory. And this is a test I have: log.write(" " + "test() ... memory usage: " + " ".join(repr(i/(1024*1024)) for i in getMemInfo()) + "\n") m...
1
by: Jean-Paul Calderone | last post by:
On Tue, 22 Apr 2008 14:54:37 -0700 (PDT), yzghan@gmail.com wrote: The test doesn't demonstrate any leaks. It does demonstrate that memory usage can remain at or near peak memory usage even after...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.