I believe I have found a serious performance
problem with heap management in C++ .Net.
I have an application does the following
thousands of times:
vint = new int[nn];
vchar = new char[nn];
The performance of these vector creations varies by
a factor of 40 or more. It is also sensitive to the
size and ratios in which the vectors are created.
Has anyone else had this problem?
I wrote a minimum console application to isolate
and validate the problem. Cycle this program 10 times
or more. The run time is about 0.3 seconds for the
first few iterations, then it increases to 16 seconds
per cycle and remains there.
This is NOT a problem with memory leak or memory
paging (the code is simple enough to rule this out).
I have also validated that no memory is leaking and
no paging occurs.
I speculate that if heap memory gets fragmented in
a certain way, then memory being released is not
properly consolidated, and allocation becomes slow.
I found no obvious way to send this bug report to
Microsoft. Any hints? Would they even want it?
/*******************************************
C++ test program for "new" (constructor).
Test C++ .Net heap memory management.
*******************************************/
#include <windows.h>
#include <conio.h>
#include <stdio.h>
#include <tchar.h>
int _tmain(int argc, char * argv[])
{
int Nv = 500000; // pointer count
int mdd = 200; // max int vector
int mcc = 200; // max. char vector
int Nii = 1; // int arrays to allocate
int Ncc = 50; // char arrays to allocate
void ** pvoid;
int * pint;
char * pchar;
int dd, cc, seed, bytes, ii, jj, check, errs;
double secs = 0.0;
char YN;
start:
printf("\n C++ memory management test \n");
printf("loading memory \n");
seed = 4444;
srand(seed);
bytes = 0;
secs = double(GetTickCount()); // start timer
pvoid = new void * [Nv]; // allocate pointers
bytes += 8 + Nv * 4;
for (ii = 0; ii < Nv; ) // index thru pointers
{
for (jj = ii; (jj < ii + Nii) && (jj < Nv); jj++)
{
dd = 2 + rand() % mdd;
pint = new int[dd]; // allocate ints
pvoid[jj] = (void *) pint; // set pointer
check = jj;
pint[0] = check; // add check data
pint[dd-1] = -check;
bytes += 8 + dd * 4;
}
ii = jj;
for (jj = ii; (jj < ii + Ncc) && (jj < Nv); jj++)
{
cc = 2 + rand() % mcc;
pchar = new char[cc]; // allocate chars
pvoid[jj] = (void *) pchar; // set pointer
check = jj % 128;
pchar[0] = check; // add check data
pchar[cc-1] = -check;
bytes += 8 + cc+2;
}
ii = jj;
}
secs = (double(GetTickCount()) - secs) / 1000.0;
printf("seconds: %.3f bytes: %d \n",secs,bytes);
printf("releasing memory \n");
srand(seed); // re-generate same randoms
errs = 0;
secs = double(GetTickCount());
for (ii = 0; ii < Nv; ) // index thru pointers
{
for (jj = ii; (jj < ii + Nii) && (jj < Nv); jj++)
{
dd = 2 + rand() % mdd;
pint = (int *) pvoid[jj]; // get ints
check = jj; // validate
if (pint[0] != check) errs++;
if (pint[dd-1] != -check) errs++;
delete [] pint; // delete
}
ii = jj;
for (jj = ii; (jj < ii + Ncc) && (jj < Nv); jj++)
{
cc = 2 + rand() % mcc;
pchar = (char *) pvoid[jj]; // get chars
check = jj % 128; // validate
if (pchar[0] != check) errs++;
if (pchar[cc-1] != -check) errs++;
delete [] pchar; // delete
}
ii = jj;
}
delete [] pvoid; // delete array of pointers
secs = (double(GetTickCount()) - secs) / 1000.0;
printf("seconds: %.3f \n",secs);
printf("errors: %d \n\n",errs);
printf("enter Y to test again ");
YN = getche();
if ((YN == 'Y') || (YN == 'y')) goto start;
return 0;
} 7 1732
Michael Cornelison wrote: I believe I have found a serious performance problem with heap management in C++ .Net.
After 100 or more iterations, I didn't notice any increase in running time
for your program.
What compiler version are you using (7.0 or 7.1)? What command-line
options? What OS? Service pack?
-cd
No problems here too.
50 iterations, 0 increase time
VS 7.1, WinXP SP1
Cheers,
Stoyan
"Carl Daniel [VC++ MVP]" <cp******@nospam.mvps.org> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl... Michael Cornelison wrote: I believe I have found a serious performance problem with heap management in C++ .Net.
After 100 or more iterations, I didn't notice any increase in running time for your program.
What compiler version are you using (7.0 or 7.1)? What command-line options? What OS? Service pack?
-cd
Carl,
Thanks for checking this out.
Looks like something is amiss in my system, but it is completely
up to date, and this leaves me clueless.
My system:
- Win XP Pro with all the MS patches installed as of Oct. 1.
- Visual C++ .Net standard edition. There are no updates, I believe.
(Help says it is Version 7.0.9466)
- compiler options:
/D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D"_MBCS"
/GF /FD /EHsc /MT /GS /Gy /YX"stdafx.h" /Fp"./C++ new tester.pch"
/Fo"./" /Fd"./vc70.pdb" /W3 /nologo /c /Wp64 /TP
Are you running the professional edition? Could this be the difference?
Mike C.
"Carl Daniel [VC++ MVP]" <cp******@nospam.mvps.org> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl... Michael Cornelison wrote: I believe I have found a serious performance problem with heap management in C++ .Net.
After 100 or more iterations, I didn't notice any increase in running time for your program.
What compiler version are you using (7.0 or 7.1)? What command-line options? What OS? Service pack?
-cd
Michael Cornelison wrote: Carl,
Thanks for checking this out.
Looks like something is amiss in my system, but it is completely up to date, and this leaves me clueless.
My system: - Win XP Pro with all the MS patches installed as of Oct. 1. - Visual C++ .Net standard edition. There are no updates, I believe. (Help says it is Version 7.0.9466) - compiler options: /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D"_MBCS" /GF /FD /EHsc /MT /GS /Gy /YX"stdafx.h" /Fp"./C++ new tester.pch" /Fo"./" /Fd"./vc70.pdb" /W3 /nologo /c /Wp64 /TP
Are you running the professional edition? Could this be the difference?
I'm running Visual C+ .NET 2003 Enterprise Architect (Version 7.1.3088).
I'm also on Win XP SP1 with up to date patches.
I didn't do an optimized build, so my results should be comparable with
yours (it's the same compiler in standard edition except that standard
edition lacks the optimizer. The libraries are identical).
Perhaps you're seeing something that was fixed in VC7.1? I have another
machine with VC7 and Windows 2000 - I'll give it a try there to see if maybe
it's a VC7 thing.
-cd
Michael Cornelison wrote: Carl,
Thanks for checking this out.
Looks like something is amiss in my system, but it is completely up to date, and this leaves me clueless.
My system: - Win XP Pro with all the MS patches installed as of Oct. 1. - Visual C++ .Net standard edition. There are no updates, I believe. (Help says it is Version 7.0.9466) - compiler options: /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D"_MBCS" /GF /FD /EHsc /MT /GS /Gy /YX"stdafx.h" /Fp"./C++ new tester.pch" /Fo"./" /Fd"./vc70.pdb" /W3 /nologo /c /Wp64 /TP
Are you running the professional edition? Could this be the difference?
I was able to reproduce the behavior you're seeing - the key is /MT. This
repros on Windows 2000 and Windows XP, with VC7 and VC7.1.
Apparently the multi-threaded heap is getting into a very bad way with this
allocation pattern. Windows XP implements something they call the "Low
Fragmentation Heap", which may be helpful in this case. I don't have time
to tryu it at the moment, but you might want to look the LFH up on MSDN and
see if it makes a difference.
-cd
Carl,
Will do. Thanks again. I was able to reproduce the behavior you're seeing - the key is /MT. This repros on Windows 2000 and Windows XP, with VC7 and VC7.1.
Apparently the multi-threaded heap is getting into a very bad way with
this allocation pattern. Windows XP implements something they call the "Low Fragmentation Heap", which may be helpful in this case. I don't have time to tryu it at the moment, but you might want to look the LFH up on MSDN
and see if it makes a difference.
-cd
Carl,
See KB article 323635, "Heap Performance Problems in Windows 2000 and
Windows XP"
I did what this article advised, and my "slowdown" problem has disappeared.
Looks like MS shot themselves in the foot when they removed the small block
heap management within the C runtime library. Fortunately they still allow
it to be re-engaged on request. My understanding is that malloc() can
allocate large blocks and use these for small block allocations which it
manages itself.
Thanks for your help.
Mike C.
"Carl Daniel [VC++ MVP]" <cp******@nospam.mvps.org> wrote in message
news:ON***************@TK2MSFTNGP10.phx.gbl... I was able to reproduce the behavior you're seeing - the key is /MT. This repros on Windows 2000 and Windows XP, with VC7 and VC7.1.
Apparently the multi-threaded heap is getting into a very bad way with
this allocation pattern. Windows XP implements something they call the "Low Fragmentation Heap", which may be helpful in this case. I don't have time to tryu it at the moment, but you might want to look the LFH up on MSDN
and see if it makes a difference.
-cd
This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Jonas Rundberg |
last post by:
Hi
I just started with c++ and I'm a little bit confused where stuff
go...
Assume we have a class:
class test {
private:
int arr;
};
|
by: Aravindakumar Venugopalan |
last post by:
Hi ,
A Windows form application which interacting with the unmanaged C++
codes . In unmanaged c++ code we allocate around 130MB on the heap for
annalysing high resolution images .
Earlier...
|
by: arcticool |
last post by:
I had an interview today and I got destroyed :(
The question was why have a stack and a heap?
I could answer all the practical stuff like value types live on the
stack, enums are on the stack, as...
|
by: sarathy |
last post by:
Hi all,
I need a few clarifications regarding memory allocaion in C++.
I apologize for the lengthy explanation.
1. In C++, Objects are allocated in heap.
What does heap refer to? Is it an area...
|
by: Chris Mullins |
last post by:
We've been using the SSLStream class found in System.Net.Security to build a
giant Sockets server that provides TLS encryption at the channel leve.
Before .Net 2.0, we used an open-source...
|
by: fdmfdmfdm |
last post by:
This is an interview question and I gave out my answer here, could you
please check for me?
Q. What are the memory allocation for static variable in a function,
an automatic variable and global...
|
by: Andy Watson |
last post by:
I have an application that scans and processes a bunch of text files.
The content I'm pulling out and holding in memory is at least 200MB.
I'd love to be able to tell the CPython virtual machine...
|
by: Roman Mashak |
last post by:
Hello,
I'm confused about heap and stack memories management in C language. Most
books explain that local stack variables for each function are automatically
allocated when function starts and...
|
by: kumarmdb2 |
last post by:
Hi guys,
For last few days we are getting out of private memory error. We have a
development environment. We tried to figure out the problem but we
believe that it might be related to the OS...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new...
| |