473,795 Members | 2,882 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Memory Logging

Hey all,

I'm running some data analysis scripts that, due to the input file
size, tend to be quite memory intensive. I've been reading up on
Python's memory allocation (I'm using 2.4 and am locked into it, so I
don't have the new fixes in 2.5) but because of the nature of its
looping I don't think its losing much memory because of some of the
deallocation issues - it will reuse the ints and lists that Python is
retaining references to.

Because of this I suspect there are other memory leaks, and I'm trying
to find out where; some method through which I could monitor memory
usage as the script runs would be extremely helpful, even if it just
prints the amount of memory python is using every time I call a
particular command.

I've tried using the profile module; the output didn't seem
particularly helpful in terms of memory usage.

In case OS-specific commands are required, all the machines it will
run on are using Windows XP.

Thanks,
-- Aneesh Goel

Jul 24 '07 #1
0 948

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
395
by: Karuppasamy | last post by:
H I am trying to use the Logging Module provided by Microsoft Application Blocks for .Net I installed everything as per the Instructions given in the 'Development Using the Logging Block' But when i am trying to run the sample, i am getting the following error in the Event Viwer Kindly help me on this
1
2326
by: Spur | last post by:
Hi all, I implemented a memory allocation/deallocation class that logs all new/delete calls (overloaded) and remembers for each allocated block where it was allocated from (using a macro that passes __FILE__ and __LINE__). On destruction, it reports all undeleted blocks (memory leaks). The class is implemented in a separate .h/.cpp pair, and should be linked to
4
5207
by: xixi | last post by:
i have a very serious memory problem, we have db2 udb v8.1 load on a HP titanium machine with 4 G memory, it is 64bit machine, currently on DB2 instance , i have three databases, but only one is being used, on the same machine, i have an application server running connect to this DB2 server, every day there are around 12 persons has connection through the app server to db server, the database has 9 G data/index, problem is in the morning...
9
2441
by: CptDondo | last post by:
I am working on an embedded platform which has a block of battery-backed RAM. I need to store various types of data in this block of memory - for example, bitmapped data for control registers, strings for logging, and structures for data points. I want to use one function to read data from this block and one function to write data, for example: sram_read(OBJECT_IDENTIFIER) would return a pointer to the appriate object and
1
1033
by: BuddyWork | last post by:
Hello, If you have the following code public class Logging { public static void LogMessage(string myData) { // code logs some data }
5
1294
by: Jim | last post by:
I am developing an ASP.Net HTTP Module used to log requests and responses. To offload the logging, the module saves the request information to a queue which is accessed by another thread to actually perform the logging. The request information is saved into an object which is then put into the queue and pulled out of the queue by the second thread. The second thread sets the object to null when it is done processing it. Everything is...
6
1778
by: Tina | last post by:
My asp.net app that ran fine on my dev boxes is having problems at my web hoster who is running IIS 6. I'm getting Out of Memory exceptoions. When my web hoster bounces my app, the problem goes away for a couple of days. Sounds like I have a Memory Leak, but my application is managed and garbage collection is automatic, right? How can I track available memory and what kinds of tools are available to shoot this kind of problem? Thanks,...
4
6021
by: Sean Shanny | last post by:
To all, Running into an out of memory error on our data warehouse server. This occurs only with our data from the 'September' section of a large fact table. The exact same query running over data from August or any prior month for that matter works fine which is why this is so weird. Note that June 2004 through today is stored in the same f_pageviews table. Nothing has changed on the server in the last couple of months. I upgraded...
2
5876
by: asterixgallier | last post by:
Hello i'm having problems with log4cxx when using inside an MFC application build with visual studio 2005. The Application starts and works normaly, but while exiting, inside the DEBUG window of Visual Studio, there are printed a lot of Memory Leaks errors: Detected memory leaks! Dumping objects -> {1323} normal block at 0x00BB6F20, 4 bytes long.
3
3686
by: Sam Samson | last post by:
Can any-one recommend some good sites detailing how to code a simple memory profiler? How do they "look under the hood"? I built an uber simple service that tracks the process' memory but I need to "get inside" and look a bit deeper and I don't have a clue where to start ... a commercial memory profiler is not an option due the nature of the environment the application is deployed in .. My app runs fine for weeks (in 8 locations) but...
0
9522
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10443
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10216
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10165
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
1
7543
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5437
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5565
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4113
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
3728
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.