473,414 Members | 1,944 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,414 software developers and data experts.

my code to prove working set larger than virtual bytes

Hello everyone,
From the definition of working set, it is a subset of virtual pages
resident in physical memory -- from book Windows Internals. It means
working set could not be larger than virtual memory (subset
relationship).

But the following simple code on Windows Server 2003 proves (if you
monitor virtual bytes counter and working set bytes conuter from
perfmon), if we do not unmap the page map file, the working set will
continue to increase (and much larger than virtual bytes) until we
unmap it.

Take a breakpoint before following code section,

<code>
<pre>
// close mapped files to avoid leak
for (sectionIndex = 0; sectionIndex < size/allocSize; sectionIndex+
+)
{
if (map [sectionIndex])
{
UnmapViewOfFile(map [sectionIndex]);
}
}
</pre>
</code>

Any ideas? Does my code break the definition of working set? Why
working set is much larger than virtual bytes?

<code>
<pre>

int main(int argc, char* argv[])
{
LARGE_INTEGER start,end;
LARGE_INTEGER freq;
QueryPerformanceCounter(&start);
QueryPerformanceFrequency(&freq);
MEMORYSTATUS memstat;
void** map;
int sectionIndex = 0;
memstat.dwLength = sizeof(memstat);
GlobalMemoryStatus(&memstat);

// basic file mapping test (512 MB)
long long size = 512*1024*1024;

HANDLE mapping =
CreateFileMapping(NULL,NULL,PAGE_READWRITE|SEC_COM MIT,(DWORD)(size <<
32),DWORD(size),NULL);
if (mapping)
{
// create and destroy temporary views
SYSTEM_INFO sysInfo;
GetSystemInfo(&sysInfo);
const int allocSize = sysInfo.dwAllocationGranularity;

GlobalMemoryStatus(&memstat);

void *mem = new char[allocSize];
memset(mem,0x11,allocSize);

map = (void**) new char [sizeof(void*) * size / allocSize];

for (int i=0; i < 10; i++)
{

sectionIndex = 0;
for (long long offset=0; offset&lt;=size-allocSize; offset
+=allocSize)
{
map [sectionIndex] =
MapViewOfFile(mapping,FILE_MAP_WRITE,(DWORD)(offse t<<32),
(DWORD)offset,allocSize);
if (map [sectionIndex])
{
memcpy(map [sectionIndex],mem,allocSize);
// UnmapViewOfFile(map);
}

sectionIndex++;
} // for (long long offset=0; offset<=size-allocSize; offset
+=allocSize)

// close mapped files to avoid leak
for (sectionIndex = 0; sectionIndex < size/allocSize; sectionIndex+
+)
{
if (map [sectionIndex])
{
UnmapViewOfFile(map [sectionIndex]);
}
}

GlobalMemoryStatus(&memstat);

sectionIndex = 0;
for (long long offset=0; offset <= size-allocSize; offset
+=allocSize)
{
map [sectionIndex] =
MapViewOfFile(mapping,FILE_MAP_READ,(DWORD)(offset << 32),
(DWORD)offset,allocSize);
if (map [sectionIndex])
{
for (int t=0; t < allocSize; t++)
{
if (((char *)(map [sectionIndex]))[t]!=0x11)
{
OutputDebugString("Memory read failed\n");
}
}
}

UnmapViewOfFile(map [sectionIndex]);
}

// close mapped files to avoid leak
/*
for (sectionIndex = 0; sectionIndex < size/allocSize; sectionIndex+
+)
{
if (map [sectionIndex])
{
UnmapViewOfFile(map [sectionIndex]);
}
}
*/

GlobalMemoryStatus(&memstat);
} // for (int i=0; i < 10; i++)

QueryPerformanceCounter(&end);

GlobalMemoryStatus(&memstat);

printf("Time %.3f\n",
double(end.QuadPart-start.QuadPart)/double(freq.QuadPart));
CloseHandle(mapping);
delete[] mem;
GlobalMemoryStatus(&memstat);
} //if (mapping)

return 0;
}
</pre>
</code>
thanks in advance,
George
Jan 17 '08 #1
1 2118
George2 a écrit :
Hello everyone,
From the definition of working set, it is a subset of virtual pages
resident in physical memory -- from book Windows Internals. It means
working set could not be larger than virtual memory (subset
relationship).

But the following simple code on Windows Server 2003 proves (if you
monitor virtual bytes counter and working set bytes conuter from
perfmon), if we do not unmap the page map file, the working set will
continue to increase (and much larger than virtual bytes) until we
unmap it.

Take a breakpoint before following code section,

<code>
<pre>
// close mapped files to avoid leak
for (sectionIndex = 0; sectionIndex < size/allocSize; sectionIndex+
+)
{
if (map [sectionIndex])
{
UnmapViewOfFile(map [sectionIndex]);
}
}
</pre>
</code>

Any ideas? Does my code break the definition of working set? Why
working set is much larger than virtual bytes?

<code>
<pre>

int main(int argc, char* argv[])
{
LARGE_INTEGER start,end;
LARGE_INTEGER freq;
QueryPerformanceCounter(&start);
QueryPerformanceFrequency(&freq);
MEMORYSTATUS memstat;
void** map;
int sectionIndex = 0;
memstat.dwLength = sizeof(memstat);
GlobalMemoryStatus(&memstat);

// basic file mapping test (512 MB)
long long size = 512*1024*1024;

HANDLE mapping =
CreateFileMapping(NULL,NULL,PAGE_READWRITE|SEC_COM MIT,(DWORD)(size <<
32),DWORD(size),NULL);
if (mapping)
{
// create and destroy temporary views
SYSTEM_INFO sysInfo;
GetSystemInfo(&sysInfo);
const int allocSize = sysInfo.dwAllocationGranularity;

GlobalMemoryStatus(&memstat);

void *mem = new char[allocSize];
memset(mem,0x11,allocSize);

map = (void**) new char [sizeof(void*) * size / allocSize];

for (int i=0; i < 10; i++)
{

sectionIndex = 0;
for (long long offset=0; offset&lt;=size-allocSize; offset
+=allocSize)
{
map [sectionIndex] =
MapViewOfFile(mapping,FILE_MAP_WRITE,(DWORD)(offse t<<32),
(DWORD)offset,allocSize);
if (map [sectionIndex])
{
memcpy(map [sectionIndex],mem,allocSize);
// UnmapViewOfFile(map);
}

sectionIndex++;
} // for (long long offset=0; offset<=size-allocSize; offset
+=allocSize)

// close mapped files to avoid leak
for (sectionIndex = 0; sectionIndex < size/allocSize; sectionIndex+
+)
{
if (map [sectionIndex])
{
UnmapViewOfFile(map [sectionIndex]);
}
}

GlobalMemoryStatus(&memstat);

sectionIndex = 0;
for (long long offset=0; offset <= size-allocSize; offset
+=allocSize)
{
map [sectionIndex] =
MapViewOfFile(mapping,FILE_MAP_READ,(DWORD)(offset << 32),
(DWORD)offset,allocSize);
if (map [sectionIndex])
{
for (int t=0; t < allocSize; t++)
{
if (((char *)(map [sectionIndex]))[t]!=0x11)
{
OutputDebugString("Memory read failed\n");
}
}
}

UnmapViewOfFile(map [sectionIndex]);
}

// close mapped files to avoid leak
/*
for (sectionIndex = 0; sectionIndex < size/allocSize; sectionIndex+
+)
{
if (map [sectionIndex])
{
UnmapViewOfFile(map [sectionIndex]);
}
}
*/

GlobalMemoryStatus(&memstat);
} // for (int i=0; i < 10; i++)

QueryPerformanceCounter(&end);

GlobalMemoryStatus(&memstat);

printf("Time %.3f\n",
double(end.QuadPart-start.QuadPart)/double(freq.QuadPart));
CloseHandle(mapping);
delete[] mem;
GlobalMemoryStatus(&memstat);
} //if (mapping)

return 0;
}
</pre>
</code>
thanks in advance,
George
Jan 17 '08 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Mike | last post by:
hopefully this is an easy one to solve and I'm just missing something little... basically, I had a nice little website running with php, apache, etc... that uses imagecreatefromjpeg to create...
2
by: Bryan Olson | last post by:
The current Python standard library provides two cryptographic hash functions: MD5 and SHA-1 . The authors of MD5 originally stated: It is conjectured that it is computationally infeasible to...
171
by: tshad | last post by:
I am just trying to decide whether to split my code and uses code behind. I did it with one of my pages and found it was quite a bit of trouble. I know that most people (and books and articles)...
5
by: Steven T. Hatton | last post by:
I'm engaged in a discussion on the Qt mailing list regarding exceptions. It has been stated that exception handling leads to significantly larger executable images than does 'error-code'...
1
by: anonymous | last post by:
I have written a simple code using inline assembly to compare the individual bytes of two lists (X and Y) and to store the higher byte of each list in a third list (LARGER). The code is as follows...
1
by: George2 | last post by:
Hello everyone, I am using Windows Server 2003 Performance Counter tool to monitor the memory consumed by my process. The interested terms are working set, virtual bytes and private bytes. My...
0
by: George2 | last post by:
Hello everyone, I am using perfmon to watch the working set and virtual bytes, when I do a keyword search in SourceInsight. I found the value of working set is larger than virtual bytes when...
0
by: George2 | last post by:
Hello everyone, Sorry that this question is related to another question I posted some time before because I have some new findings and self-analysis. My question is why sometimes from perfmon...
0
by: George2 | last post by:
Hello everyone, From the definition of working set, it is a subset of virtual pages resident in physical memory -- from book Windows Internals. It means working set could not be larger than...
4
by: David | last post by:
Hi list. Do test-driven development or behaviour-driven development advocate how to do higher-level testing than unit testing? types of testing: unit integration system
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.