473,387 Members | 1,548 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

memory measurement : different value between top/ps and mallinfo

There's a huge hash table in my application. I use it as a main
memory database. A lot of client write random data into the hash
table, so it call a lot of malloc() / free(). But after it run for
days, I watched it used a lot of memory. Finally it failed to call
malloc (). I use mallinfo() to watch the detail info. I found the
memory usage is different between the RSS value of "top" and
mallinfo().
"top" / "ps" tell me the application memory usage is 1.8G, but
mallinfo() say mallinfo.arena + mallinfo.hblkhd=1G.
Then I use pmap to see detail infomation. The "pmap" report 800M
"anon" blocks.

What's the "anon" blocks that "pmap" reported? Are they memory leak
or fragmentation? How can I avoid it?
Jun 27 '08 #1
6 2900
My software env is SUSE 10 (with GCC and libc).
Jun 27 '08 #2
lxu4net <lx*****@gmail.comwrites:
There's a huge hash table in my application. I use it as a main
memory database. A lot of client write random data into the hash
table, so it call a lot of malloc() / free(). But after it run for
days, I watched it used a lot of memory. Finally it failed to call
malloc (). I use mallinfo() to watch the detail info. I found the
memory usage is different between the RSS value of "top" and
mallinfo().
"top" / "ps" tell me the application memory usage is 1.8G, but
mallinfo() say mallinfo.arena + mallinfo.hblkhd=1G.
Then I use pmap to see detail infomation. The "pmap" report 800M
"anon" blocks.

What's the "anon" blocks that "pmap" reported? Are they memory leak
or fragmentation? How can I avoid it?
The C language doesn't refer to mallinfo, RSS, top, ps, pmap, or
"anon" blocks. Try asking in a forum that deals with your system,
perhaps comp.unix.programmer or one of the Linux groups.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Jun 27 '08 #3
lxu4net wrote:
>
There's a huge hash table in my application. I use it as a main
memory database. A lot of client write random data into the hash
table, so it call a lot of malloc() / free(). But after it run for
days, I watched it used a lot of memory. Finally it failed to call
malloc (). I use mallinfo() to watch the detail info. I found the
memory usage is different between the RSS value of "top" and
mallinfo().

"top" / "ps" tell me the application memory usage is 1.8G, but
mallinfo() say mallinfo.arena + mallinfo.hblkhd=1G. Then I use
pmap to see detail infomation. The "pmap" report 800M "anon" blocks.

What's the "anon" blocks that "pmap" reported? Are they memory leak
or fragmentation? How can I avoid it?
If you have problems with it, why use the non-standard hash table
in the first place? I have a system that is written entirely in
portable standard C, and is available under GPL licensing. If you
don't like GPL you can negotiate a different license with me (which
will no longer be free).

Just try out <http://cbfalconer.home.att.net/download/hashlib.zip>

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.
** Posted from http://www.teranews.com **
Jun 27 '08 #4
I found the mallinfo() only give main arena info. malloc_stats() can
give all arena info.
Jun 27 '08 #5
I found the mallinfo() only give main arena info. malloc_stats() can
give all arena info.
Jun 27 '08 #6
lxu4net wrote:
>
I found the mallinfo() only give main arena info. malloc_stats()
can give all arena info.
Is this intended to have some meaning? From the 'Re' it is
apparantly a response to something, but absolutely nothing is
quoted.

Some useful links on quoting:
<http://www.xs4all.nl/%7ewijnands/nnq/nquote.html>
<http://www.complang.tuwien.ac.at/anton/mail-news-errors.html>
<http://www.netmeister.org/news/learn2quote2.html>
<http://www.star-one.org.uk/computer/format.htm>

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.
** Posted from http://www.teranews.com **
Jun 27 '08 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

8
by: rbt | last post by:
Would a Python process consume more memory on a PC with lots of memory? For example, say I have the same Python script running on two WinXP computers that both have Python 2.4.0. One computer has...
3
by: Sourin | last post by:
Hi all, I am trying to write code for my experiments. My work involves huge datasets, and as such my code needs to be memory efficient. I did some hand calculations regarding the amount of...
50
by: Joerg Schwerdtfeger | last post by:
Hi folks, how can I determine the total main-memory size and the size of free memory available in bytes? I tried to use mallinfo() from malloc.h - resulting some strange values in Windows...
35
by: Alex Martelli | last post by:
Having fixed a memory leak (not the leak of a Python reference, some other stuff I wasn't properly freeing in certain cases) in a C-coded extension I maintain, I need a way to test that the leak is...
26
by: Christophe Lohr | last post by:
Hi, Inside a program, I need to get some statistics about memory usage. I actually need figures given by the "ps -o vsr,rss" command. For the moment, I make a popen("ps...") followed of a...
17
by: christophe.chazeau | last post by:
Hi, I have a problem with a really simple chunk of code which should work but does obviously does not. This chunk of code is just a POC aimed at finding a bug in a larger project in which the...
3
by: asoni | last post by:
Hello Experts, I need some advise in terms of measuring the MAX memory required to execute my tool written in C++ I have this struct which returns my memory usage in C++. int...
3
by: mohi | last post by:
hello everyone , is there any cmmnd in gdb or any other way to find out whats the total dynamically allocated memory a process holds at various instances of execution??? and what can be the...
3
by: kr151080 | last post by:
I need some help with a Measurement Program I Wrote.....I have this code as the main workings of my program: import java.util.*; public class Measurement{ { static Scanner console = new...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.