473,545 Members | 2,004 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Suspected Memory Leak in Multithread queue implmenetation

Hi Guys,

I've done a mulithread queue implementation on stl<queue>, my
developement environment is on VC6 as well as FC3. Let's talks about
the win32 side.

The suspected memory leak is find after I've run through my unit test
cases.

Test Case:
1. start 2 threads, thread A and thread B.
2. thread A enqueue 10000 dummy objects, meanwhile thread B dequeue
10000 dummy objects (delete object at the same time), these operations
are called a "cycle".
3. there is a while loop to loop the cycle, in between each cycle,
there is a 5 seconds sleep time.
4. run this for 10 minutes
5. stop insert any dummy objects

I've run this test case for 15 mins, and monitor its private_bytes,
working_set and virtual_bytes with the windows performance monitor.

My concern after running this test case is that,

1. virtual_bytes increase double its size frequently (seems to me it is
due to the queue memory allocation), and the virtual_bytes never drops
into its original position (I mean at the beginning when the program is
started)
2. both private_bytes and working_set are increasing as well, just the
fact that, duing the 5 seconds wait time, (step 3), these values drop
to nearly zeros, but after the 5 seconds wait time, these values
increase rapidly back to a state where it was dropped. and, of course
keep increasing.
3. during the (step 5), I can tell virtual_bytes is at its max, and
private_bytes and working_set are set at a values (1/3 below the max
point) and won't be able to drop any more.

My questions are:

1. shall windows OS handle the virtual_bytes, private_bytes and
working_bytes dellocation?
2. or is it because there is error on my implementation of multithread
queue?

Thank you very much. I am really appreciate yours help,

Best wishes,

Kit

--------------- My queue --------------------------

#ifndef __QUEUELOCK_HEA DER__
#define __QUEUELOCK_HEA DER__

#ifdef WIN32
# define _WINSOCKAPI_
# include <Windows.h>
#else
extern "C" {
# include <pthread.h>
};
#endif

#include <queue>
#include <iostream>

#define EXIT_IF(EXP) if (EXP) { cout << "Failed qlock: " #EXP "\n";
exit(1); }

#define QUEUE_MAX_SIZE 30000
using namespace std;

/* Queue with a lock, a template extending queue template
*/
template <typename _Tp, typename _Sequence = deque<_Tp
class qlock : public queue<_Tp,_Sequ ence{
typedef typename _Sequence::valu e_type value_type;
typedef typename _Sequence::refe rence reference;
typedef typename _Sequence::cons t_reference const_reference ;
CRITICAL_SECTIO N CriticalSection ;// Lock for modification
HANDLE hSynEvent; // Empty queue condition
public:
qlock() {
// Initialize the critical section one time only.
InitializeCriti calSection(&Cri ticalSection);
hSynEvent = CreateSemaphore (NULL, 0, QUEUE_MAX_SIZE, NULL);
if (hSynEvent == NULL)
{
printf("CreateE vent failed: %d\n", GetLastError()) ;
exit(1);
}
};
~qlock() {
// Release resources used by the critical section object.
DeleteCriticalS ection(&Critica lSection);
EXIT_IF(CloseHa ndle(hSynEvent) );
};
void enqueue(const value_type& __x) {
EnterCriticalSe ction(&Critical Section); // Lock the queue
if (this->size() QUEUE_MAX_SIZE)
{
while(this->size() 1) // Clear the queue
{
if(!this->empty()) {
value_type r;
r = this->front();
this->pop();
if(r != NULL) delete r;
}
}
}
this->push(__x);
// Release Semaphore
ReleaseSemaphor e(hSynEvent, 1, NULL);
LeaveCriticalSe ction(&Critical Section); // Unlock the queue
};
//reference
value_type dequeue(int i=0,int j=0) { // Blocking-call
bool empty = true;
value_type r = {0};
EnterCriticalSe ction(&Critical Section);
empty = this->empty();
if(!empty) {
r = this->front();
this->pop();
}
LeaveCriticalSe ction(&Critical Section);
while (empty) {
// Wait until something is written into the queue
if(WaitForSingl eObject(hSynEve nt, INFINITE) == WAIT_FAILED)
{
printf("WaitFor SingleObject failed (%d)\n", GetLastError()) ;
exit(1);
};
EnterCriticalSe ction(&Critical Section);
empty = this->empty();
if(!empty) {
r = this->front();
this->pop();
}
LeaveCriticalSe ction(&Critical Section);
};
return r;
}
const_reference dequeue() const { // Blocking-call
bool empty = true;
const_reference r;
EnterCriticalSe ction(&Critical Section);
empty = this->empty();
if(!empty) {
r = this->front();
this->pop();
}
LeaveCriticalSe ction(&Critical Section);

while (empty) {
// Wait until something is written into the queue
if(WaitForSingl eObject(hSynEve nt, INFINITE) == WAIT_FAILED)
{
printf("WaitFor SingleObject failed (%d)\n", GetLastError()) ;
LOG(LV_EMERG,"W ait for Single Queue Object failed.",1);
exit(1);
};
EnterCriticalSe ction(&Critical Section);
empty = this->empty();
if(!empty) {
r = this->front();
this->pop();
}
LeaveCriticalSe ction(&Critical Section);
};
return r;
}
int qSize() {
int size = 0;
EnterCriticalSe ction(&Critical Section);
size = this->size();
LeaveCriticalSe ction(&Critical Section);
return size;
}
};

Jan 15 '07 #1
2 5282
tikcireviva wrote:
[snip]
My questions are:

1. shall windows OS handle the virtual_bytes, private_bytes and
working_bytes dellocation?
2. or is it because there is error on my implementation of multithread
queue?
[snip]

This newsgroup is for discussing the standard C++ language proper (see
http://parashift.com/c++-faq-lite/ho....html#faq-5.9). In
particular, we don't discuss Microsoft-specific questions or
multithreading, which is not supported in standard C++ and is thus
necessarily platform-specific. For the former, try
microsoft.publi c.dotnet.langua ges.vc or similar, and for the latter,
try comp.programmin g.threads or similar.

Alternately, if you can rephrase your questions in terms of the
standard C++ language, this would be the right place to ask.

Cheers! --M

Jan 15 '07 #2
Thanks for the advance,

Let me rephrase a little bit for my questions.

1. The stl::queue double its memory for storage when it hits the
maximun size, is there any method to explicitly delete the allocated
memory from stl::queue? Since it consumes the virtual memory without
releaseing it.
2. Would you guys mind critique if I've done anything wrong on my
stl::queue multithread implemenation?

Best wishes,

Kit

mlimber ¼g¹D¡G
tikcireviva wrote:
[snip]
My questions are:

1. shall windows OS handle the virtual_bytes, private_bytes and
working_bytes dellocation?
2. or is it because there is error on my implementation of multithread
queue?
[snip]

This newsgroup is for discussing the standard C++ language proper (see
http://parashift.com/c++-faq-lite/ho....html#faq-5.9). In
particular, we don't discuss Microsoft-specific questions or
multithreading, which is not supported in standard C++ and is thus
necessarily platform-specific. For the former, try
microsoft.publi c.dotnet.langua ges.vc or similar, and for the latter,
try comp.programmin g.threads or similar.

Alternately, if you can rephrase your questions in terms of the
standard C++ language, this would be the right place to ask.

Cheers! --M
Jan 15 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
1125
by: Al Franz | last post by:
I believe there is a memory leak in cPickle. I am using python2.2. I have a parallel code which uses array() and indices() from Numeric to massage data buffers before being sent and received by Pypar. Pypar subsequently uses cPickle to pickle the data. After many hours of execution, my code crashes with one of the following error messages...
8
3394
by: ranjeet.gupta | last post by:
Dear All Is the Root Cause of the Memory corruption is the Memory leak, ?? suppose If in the code there is Memory leak, Do this may lead to the Memory Corruption while executing the program ? In nut shell, what is/are the realtion/s between the Memory Leak and Memory Corruption. Juts Theoritical Assumtion below:
17
4776
by: José Joye | last post by:
Hi, I have implemented a Service that is responsible for getting messages from a MS MQ located on a remote machine. I'm getting memory leak from time to time (???). In some situation, it is easier to reproduce (e.g.: remote machine not available). After about 1 day, I get a usage of 300MB of memory. I have used .NET Memory Profiler tool to...
23
4515
by: James | last post by:
The following code will create memory leaks!!! using System; using System.Diagnostics; using System.Data; using System.Data.SqlClient; namespace MemoryLeak
11
3147
by: tomgee | last post by:
I saw it many times, // T.h: class T { public: static T* instance(); private: T() {}
4
1352
by: Mike C# | last post by:
Hi all, quick question. I have an application I'm running. There are some "fatal" error conditions that could occur during processing, and I want to exit immediately if one comes up. When I put an exit() function call in my code I get a message like this: Detected memory leaks! Dumping objects -> {177} normal block at 0x003864E8, 32...
0
1121
by: tikcireviva | last post by:
Hi Guys, I've done a mulithread queue implementation on stl<queue>, my developement environment is on VC6 as well as FC3. Let's talks about the win32 side. The suspected memory leak is find after I've run through my unit test cases. Test Case:
22
2251
by: Frank Rizzo | last post by:
I have an object tree that is pretty gigantic and it holds about 100mb of data. When I set the top object to null, I expect that the .NET framework will clean up the memory at some point. However, I am looking at the Task Manager and I don't see the MemUsage column decreasing even after an hour or two. I know that TaskManager may not be the...
3
2553
by: Godzilla | last post by:
Hello, I have a program that create and pop an object off a queue, but it is experiencing some memory leakage. I have been unable to detect where the memory leakage occur. The strange thing is when i replace the object creation with a plain integer/string, the leak goes away... Here's the code I used as my test: import Queue
0
7484
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7928
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7440
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
5997
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
0
4963
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3451
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1902
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1030
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
726
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.