Hi,
My production machine has 2G of memory, when aspnet_wp.exe goes up to about
~1.2G of memory usage, I start get out-of-memory exception. Other processes
don't use as much memory and I added all the peak memory usage of all the
processes (including aspnet_wp.exe), it goes up to no more than 1.5.
How is that possible? Would anyone know please help? thanks!
22  2969 
Firstly in Task Manager look at your "Commit Charge". This is the memory
committed to individual processes in total, plus the operating system's
memory requirements.
See how the "total" compares to the "limit". The limit will effectively be
the amount of physical RAM that you have plus your paging space.
"Zen" <ze*@nononospam.com> wrote in message
news:%2****************@TK2MSFTNGP14.phx.gbl... Hi,
My production machine has 2G of memory, when aspnet_wp.exe goes up to
about ~1.2G of memory usage, I start get out-of-memory exception. Other
processes don't use as much memory and I added all the peak memory usage
of all the processes (including aspnet_wp.exe), it goes up to no more than
1.5.
How is that possible? Would anyone know please help? thanks!
"Zen" <ze*@nononospam.com> wrote in message
news:%2****************@TK2MSFTNGP14.phx.gbl...
| Hi,
|
| My production machine has 2G of memory, when aspnet_wp.exe goes up to
about
| ~1.2G of memory usage, I start get out-of-memory exception. Other
processes
| don't use as much memory and I added all the peak memory usage of all the
| processes (including aspnet_wp.exe), it goes up to no more than 1.5.
|
| How is that possible? Would anyone know please help? thanks!
|
|
The amount of RAM is one thing, but the most important is the amount of
Virtual memory your process has allocated.
Each windows process (32 bit not /4GT enabled) has a maximum of 2GB Virtual
Address Space (VAS). This space is used to map the code of the ASP.NET
worker the CLR the C runtime and a couple of other DLL's plus your
application code (the IL and JIT'd code) the native process heaps and the GC
heaps.
So when you say that aspnet_wp.exe goes up to ~1.2GB you have to know
exactly what you are looking at and what tool you are using for this.
Anyway, you should never assume you have 2GB available, you should profile
your application and have a close look at your allocation patterns and the
type of objects you are allocating, take care about large objects (>85kb)
these tend to fragment the large object heap (especially in v1.x), and you
should always be prepared to get OOM exceptions thrown on you especially in
server applications.
Willy.
I used taskmanager that came with window to look at the memory usage info.
Would it help if I add another G or 2 to the server? Apparently each process
can use only up to 2G max of memory, but other items including the OS itself
would occupy a big junk of total memory leaving less memory for the asp.net.
I don't know the answer the question to that because the following 2 items
contradicting to each other:
1) With virtual memory system and enough harddisk space, the asp.net process
is already virtually working with 2G of memory, so adding physical memory
would only speed up (less swapping with page faults) but woudn't help
reducing OOM exception likelihood. My max page file size on harddisk is
1.5G.
2) However, when I moved from 1G to 2G physical memory, the system no longer
ran into OOM as often as before. With 1G, it could never load all my data
into memory for processing without seeing OOMs. Apparently adding more
physical memory did help.
To rephrase my original question, if I can afford any memory size I want,
which amount would be the max I can benefit from? There must be a stopping
point somewhere. I thought it was 2G but I could be wrong and hoping that
I'm wrong because making my code to work with smaller memory amount would be
very costly.
thanks!
zeng
"Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
news:eW**************@TK2MSFTNGP11.phx.gbl...
"Zen" <ze*@nononospam.com> wrote in message
news:%2****************@TK2MSFTNGP14.phx.gbl...
| Hi,
|
| My production machine has 2G of memory, when aspnet_wp.exe goes up to
about
| ~1.2G of memory usage, I start get out-of-memory exception. Other
processes
| don't use as much memory and I added all the peak memory usage of all
the
| processes (including aspnet_wp.exe), it goes up to no more than 1.5.
|
| How is that possible? Would anyone know please help? thanks!
|
|
The amount of RAM is one thing, but the most important is the amount of
Virtual memory your process has allocated.
Each windows process (32 bit not /4GT enabled) has a maximum of 2GB
Virtual
Address Space (VAS). This space is used to map the code of the ASP.NET
worker the CLR the C runtime and a couple of other DLL's plus your
application code (the IL and JIT'd code) the native process heaps and the
GC
heaps.
So when you say that aspnet_wp.exe goes up to ~1.2GB you have to know
exactly what you are looking at and what tool you are using for this.
Anyway, you should never assume you have 2GB available, you should profile
your application and have a close look at your allocation patterns and the
type of objects you are allocating, take care about large objects (>85kb)
these tend to fragment the large object heap (especially in v1.x), and you
should always be prepared to get OOM exceptions thrown on you especially
in
server applications.
Willy.
There is no "Commit Charge" column to choose from in the Task Manager, I'm
using Windows2000.
"Kevin Frey" <ke**********@hotmail.com> wrote in message
news:OK**************@TK2MSFTNGP10.phx.gbl... Firstly in Task Manager look at your "Commit Charge". This is the memory
committed to individual processes in total, plus the operating system's
memory requirements.
See how the "total" compares to the "limit". The limit will effectively be
the amount of physical RAM that you have plus your paging space.
"Zen" <ze*@nononospam.com> wrote in message
news:%2****************@TK2MSFTNGP14.phx.gbl... Hi,
My production machine has 2G of memory, when aspnet_wp.exe goes up to
about ~1.2G of memory usage, I start get out-of-memory exception. Other
processes don't use as much memory and I added all the peak memory usage
of all the processes (including aspnet_wp.exe), it goes up to no more
than 1.5.
How is that possible? Would anyone know please help? thanks!
"Zen" <ze*@nononospam.com> wrote in message
news:uR**************@TK2MSFTNGP14.phx.gbl...
|I used taskmanager that came with window to look at the memory usage info.
|
| Would it help if I add another G or 2 to the server?
No, it won't help you at all, unless you have a high paging rate.
Apparently each process
| can use only up to 2G max of memory, but other items including the OS
itself
| would occupy a big junk of total memory leaving less memory for the
asp.net.
| I don't know the answer the question to that because the following 2 items
| contradicting to each other:
| 1) With virtual memory system and enough harddisk space, the asp.net
process
| is already virtually working with 2G of memory, so adding physical memory
| would only speed up (less swapping with page faults) but woudn't help
| reducing OOM exception likelihood. My max page file size on harddisk is
| 1.5G.
Did you measure the paging rate? Your paging file is too small anyway, it
should be larger than physical memory, and it should be large enough to hold
all 'dirty' datapages of the total of all processes. That means that the OS
can never allocate more data segments (as the GC does) than the size of the
paging file, the system will OOM at allocation time when it's not able to
back-up the dirty pages to the paging file.
| 2) However, when I moved from 1G to 2G physical memory, the system no
longer
| ran into OOM as often as before. With 1G, it could never load all my data
| into memory for processing without seeing OOMs. Apparently adding more
| physical memory did help.
|
Yes, now I see why your page file is 1.5GB only, you didn't change it's size
wen you moved from 1GB to 2GB.
| To rephrase my original question, if I can afford any memory size I want,
| which amount would be the max I can benefit from? There must be a
stopping
| point somewhere. I thought it was 2G but I could be wrong and hoping that
| I'm wrong because making my code to work with smaller memory amount would
be
| very costly.
|
First thing to do is increase your page file size (make it 3GB), and watch
your paging consumption and the paging rate using perfmon (don't use taskman
for this). If you still incur OOM's, profile your application and check your
allocation patterns carefully, watch for containers like ArrayList, they
grow exponentially and the are a pain in the a** when they become larger
than 85KB. Watch your memory fragmentation level when profiling or
debugging. Memory fragmentation cannot be solved by adding memory.
Watch your unmanaged memory counters and your unmanaged resources, make sure
you are disposing correctly.
You should ONLY add RAM when the paging rate is realy too high and becomes a
performance bottleneck. Don't extend above 3GB if you can it's a waste of
money on 32 bit OS.
Willy.
Thanks for you input. Yes, I realized that my page file size was low too, I
was surprised that Win2000 didn't automatically change the page file size to
take advantage of the max page file size set. Occasionally my page swap
rate jumped up to 40-50/sec. I'm still wondering about the max size Windows
can utilize to serve a resource intensive process, a few things I found:
1) On Win2000, max page file is 4G so 4G physical max then.
2) On XP, there isn't limitation - I attempted to set it up to 12G and it
allowed me (I asked me to reboot and I haven't got a chance to do that
though). Does anyone know the limit? Assume that I have $ to max out XP
capability to avoid those page swaps.
-thinh
"Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
news:uw*************@TK2MSFTNGP12.phx.gbl...
"Zen" <ze*@nononospam.com> wrote in message
news:uR**************@TK2MSFTNGP14.phx.gbl...
|I used taskmanager that came with window to look at the memory usage
info.
|
| Would it help if I add another G or 2 to the server?
No, it won't help you at all, unless you have a high paging rate.
Apparently each process
| can use only up to 2G max of memory, but other items including the OS
itself
| would occupy a big junk of total memory leaving less memory for the
asp.net.
| I don't know the answer the question to that because the following 2
items
| contradicting to each other:
| 1) With virtual memory system and enough harddisk space, the asp.net
process
| is already virtually working with 2G of memory, so adding physical
memory
| would only speed up (less swapping with page faults) but woudn't help
| reducing OOM exception likelihood. My max page file size on harddisk is
| 1.5G.
Did you measure the paging rate? Your paging file is too small anyway, it
should be larger than physical memory, and it should be large enough to
hold
all 'dirty' datapages of the total of all processes. That means that the
OS
can never allocate more data segments (as the GC does) than the size of
the
paging file, the system will OOM at allocation time when it's not able to
back-up the dirty pages to the paging file.
| 2) However, when I moved from 1G to 2G physical memory, the system no
longer
| ran into OOM as often as before. With 1G, it could never load all my
data
| into memory for processing without seeing OOMs. Apparently adding more
| physical memory did help.
|
Yes, now I see why your page file is 1.5GB only, you didn't change it's
size
wen you moved from 1GB to 2GB.
| To rephrase my original question, if I can afford any memory size I
want,
| which amount would be the max I can benefit from? There must be a
stopping
| point somewhere. I thought it was 2G but I could be wrong and hoping
that
| I'm wrong because making my code to work with smaller memory amount
would
be
| very costly.
|
First thing to do is increase your page file size (make it 3GB), and watch
your paging consumption and the paging rate using perfmon (don't use
taskman
for this). If you still incur OOM's, profile your application and check
your
allocation patterns carefully, watch for containers like ArrayList, they
grow exponentially and the are a pain in the a** when they become larger
than 85KB. Watch your memory fragmentation level when profiling or
debugging. Memory fragmentation cannot be solved by adding memory.
Watch your unmanaged memory counters and your unmanaged resources, make
sure
you are disposing correctly.
You should ONLY add RAM when the paging rate is realy too high and becomes
a
performance bottleneck. Don't extend above 3GB if you can it's a waste of
money on 32 bit OS.
Willy.
W2K and higher (32 bit OS) supports 16 page files of max. 4GB each. Anyway
you should not set this page file size higher than 3~4GB on a server with
only 2GB RAM.
As I said before a 32 bit windows app. can only use 2GB of Virtual Address
Space, if you think you need this 2GB for one single process, you will
probably need more than 2GB of RAM or you have to accept the possible paging
overhead, but all this is subject to "measure and tune".
Willy.
"Zen" <ze*@nononospam.com> wrote in message
news:ut**************@TK2MSFTNGP14.phx.gbl...
| Thanks for you input. Yes, I realized that my page file size was low too,
I
| was surprised that Win2000 didn't automatically change the page file size
to
| take advantage of the max page file size set. Occasionally my page swap
| rate jumped up to 40-50/sec. I'm still wondering about the max size
Windows
| can utilize to serve a resource intensive process, a few things I found:
| 1) On Win2000, max page file is 4G so 4G physical max then.
| 2) On XP, there isn't limitation - I attempted to set it up to 12G and it
| allowed me (I asked me to reboot and I haven't got a chance to do that
| though). Does anyone know the limit? Assume that I have $ to max out XP
| capability to avoid those page swaps.
|
| -thinh
|
|
| "Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
| news:uw*************@TK2MSFTNGP12.phx.gbl...
| >
| > "Zen" <ze*@nononospam.com> wrote in message
| > news:uR**************@TK2MSFTNGP14.phx.gbl...
| > |I used taskmanager that came with window to look at the memory usage
| > info.
| > |
| > | Would it help if I add another G or 2 to the server?
| >
| >
| > No, it won't help you at all, unless you have a high paging rate.
| >
| > Apparently each process
| > | can use only up to 2G max of memory, but other items including the OS
| > itself
| > | would occupy a big junk of total memory leaving less memory for the
| > asp.net.
| > | I don't know the answer the question to that because the following 2
| > items
| > | contradicting to each other:
| > | 1) With virtual memory system and enough harddisk space, the asp.net
| > process
| > | is already virtually working with 2G of memory, so adding physical
| > memory
| > | would only speed up (less swapping with page faults) but woudn't help
| > | reducing OOM exception likelihood. My max page file size on harddisk
is
| > | 1.5G.
| >
| > Did you measure the paging rate? Your paging file is too small anyway,
it
| > should be larger than physical memory, and it should be large enough to
| > hold
| > all 'dirty' datapages of the total of all processes. That means that the
| > OS
| > can never allocate more data segments (as the GC does) than the size of
| > the
| > paging file, the system will OOM at allocation time when it's not able
to
| > back-up the dirty pages to the paging file.
| >
| >
| >
| >
| > | 2) However, when I moved from 1G to 2G physical memory, the system no
| > longer
| > | ran into OOM as often as before. With 1G, it could never load all my
| > data
| > | into memory for processing without seeing OOMs. Apparently adding
more
| > | physical memory did help.
| > |
| >
| > Yes, now I see why your page file is 1.5GB only, you didn't change it's
| > size
| > wen you moved from 1GB to 2GB.
| >
| > | To rephrase my original question, if I can afford any memory size I
| > want,
| > | which amount would be the max I can benefit from? There must be a
| > stopping
| > | point somewhere. I thought it was 2G but I could be wrong and hoping
| > that
| > | I'm wrong because making my code to work with smaller memory amount
| > would
| > be
| > | very costly.
| > |
| >
| > First thing to do is increase your page file size (make it 3GB), and
watch
| > your paging consumption and the paging rate using perfmon (don't use
| > taskman
| > for this). If you still incur OOM's, profile your application and check
| > your
| > allocation patterns carefully, watch for containers like ArrayList, they
| > grow exponentially and the are a pain in the a** when they become larger
| > than 85KB. Watch your memory fragmentation level when profiling or
| > debugging. Memory fragmentation cannot be solved by adding memory.
| > Watch your unmanaged memory counters and your unmanaged resources, make
| > sure
| > you are disposing correctly.
| > You should ONLY add RAM when the paging rate is realy too high and
becomes
| > a
| > performance bottleneck. Don't extend above 3GB if you can it's a waste
of
| > money on 32 bit OS.
| >
| >
| > Willy.
| >
| >
|
|
Each application/process has its own Virtual Address Space of 2G regardless
of how many other processes are running and even on a dual-processor
machine? Thanks!
-zen
"Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
news:Oy**************@TK2MSFTNGP09.phx.gbl... W2K and higher (32 bit OS) supports 16 page files of max. 4GB each.
Anyway
you should not set this page file size higher than 3~4GB on a server with
only 2GB RAM.
As I said before a 32 bit windows app. can only use 2GB of Virtual Address
Space, if you think you need this 2GB for one single process, you will
probably need more than 2GB of RAM or you have to accept the possible
paging
overhead, but all this is subject to "measure and tune".
Willy.
"Zen" <ze*@nononospam.com> wrote in message
news:ut**************@TK2MSFTNGP14.phx.gbl...
| Thanks for you input. Yes, I realized that my page file size was low
too,
I
| was surprised that Win2000 didn't automatically change the page file
size
to
| take advantage of the max page file size set. Occasionally my page swap
| rate jumped up to 40-50/sec. I'm still wondering about the max size
Windows
| can utilize to serve a resource intensive process, a few things I found:
| 1) On Win2000, max page file is 4G so 4G physical max then.
| 2) On XP, there isn't limitation - I attempted to set it up to 12G and
it
| allowed me (I asked me to reboot and I haven't got a chance to do that
| though). Does anyone know the limit? Assume that I have $ to max out XP
| capability to avoid those page swaps.
|
| -thinh
|
|
| "Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
| news:uw*************@TK2MSFTNGP12.phx.gbl...
| >
| > "Zen" <ze*@nononospam.com> wrote in message
| > news:uR**************@TK2MSFTNGP14.phx.gbl...
| > |I used taskmanager that came with window to look at the memory usage
| > info.
| > |
| > | Would it help if I add another G or 2 to the server?
| >
| >
| > No, it won't help you at all, unless you have a high paging rate.
| >
| > Apparently each process
| > | can use only up to 2G max of memory, but other items including the
OS
| > itself
| > | would occupy a big junk of total memory leaving less memory for the
| > asp.net.
| > | I don't know the answer the question to that because the following 2
| > items
| > | contradicting to each other:
| > | 1) With virtual memory system and enough harddisk space, the asp.net
| > process
| > | is already virtually working with 2G of memory, so adding physical
| > memory
| > | would only speed up (less swapping with page faults) but woudn't
help
| > | reducing OOM exception likelihood. My max page file size on
harddisk
is
| > | 1.5G.
| >
| > Did you measure the paging rate? Your paging file is too small anyway,
it
| > should be larger than physical memory, and it should be large enough
to
| > hold
| > all 'dirty' datapages of the total of all processes. That means that
the
| > OS
| > can never allocate more data segments (as the GC does) than the size
of
| > the
| > paging file, the system will OOM at allocation time when it's not able
to
| > back-up the dirty pages to the paging file.
| >
| >
| >
| >
| > | 2) However, when I moved from 1G to 2G physical memory, the system
no
| > longer
| > | ran into OOM as often as before. With 1G, it could never load all
my
| > data
| > | into memory for processing without seeing OOMs. Apparently adding
more
| > | physical memory did help.
| > |
| >
| > Yes, now I see why your page file is 1.5GB only, you didn't change
it's
| > size
| > wen you moved from 1GB to 2GB.
| >
| > | To rephrase my original question, if I can afford any memory size I
| > want,
| > | which amount would be the max I can benefit from? There must be a
| > stopping
| > | point somewhere. I thought it was 2G but I could be wrong and
hoping
| > that
| > | I'm wrong because making my code to work with smaller memory amount
| > would
| > be
| > | very costly.
| > |
| >
| > First thing to do is increase your page file size (make it 3GB), and
watch
| > your paging consumption and the paging rate using perfmon (don't use
| > taskman
| > for this). If you still incur OOM's, profile your application and
check
| > your
| > allocation patterns carefully, watch for containers like ArrayList,
they
| > grow exponentially and the are a pain in the a** when they become
larger
| > than 85KB. Watch your memory fragmentation level when profiling or
| > debugging. Memory fragmentation cannot be solved by adding memory.
| > Watch your unmanaged memory counters and your unmanaged resources,
make
| > sure
| > you are disposing correctly.
| > You should ONLY add RAM when the paging rate is realy too high and
becomes
| > a
| > performance bottleneck. Don't extend above 3GB if you can it's a waste
of
| > money on 32 bit OS.
| >
| >
| > Willy.
| >
| >
|
|
I was referring to the figures presented on the "Performance" page of Task
Manager, not in the process page.
"Zen" <ze*@nononospam.com> wrote in message
news:OF**************@TK2MSFTNGP09.phx.gbl... There is no "Commit Charge" column to choose from in the Task Manager, I'm
using Windows2000.
"Kevin Frey" <ke**********@hotmail.com> wrote in message
news:OK**************@TK2MSFTNGP10.phx.gbl... Firstly in Task Manager look at your "Commit Charge". This is the memory
committed to individual processes in total, plus the operating system's
memory requirements.
See how the "total" compares to the "limit". The limit will effectively
be the amount of physical RAM that you have plus your paging space.
"Zen" <ze*@nononospam.com> wrote in message
news:%2****************@TK2MSFTNGP14.phx.gbl... Hi,
My production machine has 2G of memory, when aspnet_wp.exe goes up to
about ~1.2G of memory usage, I start get out-of-memory exception. Other
processes don't use as much memory and I added all the peak memory usage
of all the processes (including aspnet_wp.exe), it goes up to no more
than 1.5.
How is that possible? Would anyone know please help? thanks!
ok, found it, and the total is no where near the limit. My limit is 5G,
total about 1.3G, does it tell us anything? By the way, I just ran into
Out-of-memory exceptions again when there were only 1.3G in usage for aspnet
process and even after I increased the page file to 3G min and 4G max. How
could it be possible?
"Kevin Frey" <ke**********@hotmail.com> wrote in message
news:Ov*************@TK2MSFTNGP12.phx.gbl... I was referring to the figures presented on the "Performance" page of Task
Manager, not in the process page.
"Zen" <ze*@nononospam.com> wrote in message
news:OF**************@TK2MSFTNGP09.phx.gbl... There is no "Commit Charge" column to choose from in the Task Manager,
I'm using Windows2000.
"Kevin Frey" <ke**********@hotmail.com> wrote in message
news:OK**************@TK2MSFTNGP10.phx.gbl... Firstly in Task Manager look at your "Commit Charge". This is the memory
committed to individual processes in total, plus the operating system's
memory requirements.
See how the "total" compares to the "limit". The limit will effectively
be the amount of physical RAM that you have plus your paging space.
"Zen" <ze*@nononospam.com> wrote in message
news:%2****************@TK2MSFTNGP14.phx.gbl...
Hi,
My production machine has 2G of memory, when aspnet_wp.exe goes up to
about ~1.2G of memory usage, I start get out-of-memory exception. Other
processes don't use as much memory and I added all the peak memory
usage of all the processes (including aspnet_wp.exe), it goes up to no
more than 1.5.
How is that possible? Would anyone know please help? thanks!
Ok, i just ran into the the Out-of-memory exception again. Aspnet process
doesn't seem to be able to use more than 1.3G at the most. Any other idea?
Thanks!
"Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
news:uw*************@TK2MSFTNGP12.phx.gbl...
"Zen" <ze*@nononospam.com> wrote in message
news:uR**************@TK2MSFTNGP14.phx.gbl...
|I used taskmanager that came with window to look at the memory usage
info.
|
| Would it help if I add another G or 2 to the server?
No, it won't help you at all, unless you have a high paging rate.
Apparently each process
| can use only up to 2G max of memory, but other items including the OS
itself
| would occupy a big junk of total memory leaving less memory for the
asp.net.
| I don't know the answer the question to that because the following 2
items
| contradicting to each other:
| 1) With virtual memory system and enough harddisk space, the asp.net
process
| is already virtually working with 2G of memory, so adding physical
memory
| would only speed up (less swapping with page faults) but woudn't help
| reducing OOM exception likelihood. My max page file size on harddisk is
| 1.5G.
Did you measure the paging rate? Your paging file is too small anyway, it
should be larger than physical memory, and it should be large enough to
hold
all 'dirty' datapages of the total of all processes. That means that the
OS
can never allocate more data segments (as the GC does) than the size of
the
paging file, the system will OOM at allocation time when it's not able to
back-up the dirty pages to the paging file.
| 2) However, when I moved from 1G to 2G physical memory, the system no
longer
| ran into OOM as often as before. With 1G, it could never load all my
data
| into memory for processing without seeing OOMs. Apparently adding more
| physical memory did help.
|
Yes, now I see why your page file is 1.5GB only, you didn't change it's
size
wen you moved from 1GB to 2GB.
| To rephrase my original question, if I can afford any memory size I
want,
| which amount would be the max I can benefit from? There must be a
stopping
| point somewhere. I thought it was 2G but I could be wrong and hoping
that
| I'm wrong because making my code to work with smaller memory amount
would
be
| very costly.
|
First thing to do is increase your page file size (make it 3GB), and watch
your paging consumption and the paging rate using perfmon (don't use
taskman
for this). If you still incur OOM's, profile your application and check
your
allocation patterns carefully, watch for containers like ArrayList, they
grow exponentially and the are a pain in the a** when they become larger
than 85KB. Watch your memory fragmentation level when profiling or
debugging. Memory fragmentation cannot be solved by adding memory.
Watch your unmanaged memory counters and your unmanaged resources, make
sure
you are disposing correctly.
You should ONLY add RAM when the paging rate is realy too high and becomes
a
performance bottleneck. Don't extend above 3GB if you can it's a waste of
money on 32 bit OS.
Willy.
Ok, it looks like you are not carefully reading what I've written or I did
not make myself clear.
Your asp.net worker process can use at most 2GB address space, at a certain
moment in time you have allocated 1.3 GB, and then you try to allocate
another chunk of memory but this fails with an OOM.
The question YOU have to answer is - how large is this chunk of memory you
are trying to allocate, note I say "chunk" of memory because I don't know
what exactly your program is doing, is it creating another object, if yes
what kind of object is it, how large is it? is it allocating a block of heap
memory, how large is this memory block? All I know is that your program is
trying to get memory from it's virtual address space but the space needed is
not available as a contiguous block. The 700 MB free space might (and will)
be so badly fragmented that the memory request for x bytes fails. Question
is HOW LARGE IS X?
To answer all these questions, you'll have to watch your allocation
patterns, you have to look at what your code is doing, what kind of objects
you are creating. The best way to do this is to run a memory profiler
(scitec or the Clrprofiler to name a few) or attach a debugger.
Take a look at your code and check if you aren't using ArrayLists or
DataSet's that can grow exponentially, take care you aren't pinning objects
(PInvoke calls into unmanaged code) etc...
Willy.
"Zen" <ze*@nononospam.com> wrote in message
news:uu**************@TK2MSFTNGP09.phx.gbl...
| Ok, i just ran into the the Out-of-memory exception again. Aspnet process
| doesn't seem to be able to use more than 1.3G at the most. Any other idea?
| Thanks!
|
|
| "Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
| news:uw*************@TK2MSFTNGP12.phx.gbl...
| >
| > "Zen" <ze*@nononospam.com> wrote in message
| > news:uR**************@TK2MSFTNGP14.phx.gbl...
| > |I used taskmanager that came with window to look at the memory usage
| > info.
| > |
| > | Would it help if I add another G or 2 to the server?
| >
| >
| > No, it won't help you at all, unless you have a high paging rate.
| >
| > Apparently each process
| > | can use only up to 2G max of memory, but other items including the OS
| > itself
| > | would occupy a big junk of total memory leaving less memory for the
| > asp.net.
| > | I don't know the answer the question to that because the following 2
| > items
| > | contradicting to each other:
| > | 1) With virtual memory system and enough harddisk space, the asp.net
| > process
| > | is already virtually working with 2G of memory, so adding physical
| > memory
| > | would only speed up (less swapping with page faults) but woudn't help
| > | reducing OOM exception likelihood. My max page file size on harddisk
is
| > | 1.5G.
| >
| > Did you measure the paging rate? Your paging file is too small anyway,
it
| > should be larger than physical memory, and it should be large enough to
| > hold
| > all 'dirty' datapages of the total of all processes. That means that the
| > OS
| > can never allocate more data segments (as the GC does) than the size of
| > the
| > paging file, the system will OOM at allocation time when it's not able
to
| > back-up the dirty pages to the paging file.
| >
| >
| >
| >
| > | 2) However, when I moved from 1G to 2G physical memory, the system no
| > longer
| > | ran into OOM as often as before. With 1G, it could never load all my
| > data
| > | into memory for processing without seeing OOMs. Apparently adding
more
| > | physical memory did help.
| > |
| >
| > Yes, now I see why your page file is 1.5GB only, you didn't change it's
| > size
| > wen you moved from 1GB to 2GB.
| >
| > | To rephrase my original question, if I can afford any memory size I
| > want,
| > | which amount would be the max I can benefit from? There must be a
| > stopping
| > | point somewhere. I thought it was 2G but I could be wrong and hoping
| > that
| > | I'm wrong because making my code to work with smaller memory amount
| > would
| > be
| > | very costly.
| > |
| >
| > First thing to do is increase your page file size (make it 3GB), and
watch
| > your paging consumption and the paging rate using perfmon (don't use
| > taskman
| > for this). If you still incur OOM's, profile your application and check
| > your
| > allocation patterns carefully, watch for containers like ArrayList, they
| > grow exponentially and the are a pain in the a** when they become larger
| > than 85KB. Watch your memory fragmentation level when profiling or
| > debugging. Memory fragmentation cannot be solved by adding memory.
| > Watch your unmanaged memory counters and your unmanaged resources, make
| > sure
| > you are disposing correctly.
| > You should ONLY add RAM when the paging rate is realy too high and
becomes
| > a
| > performance bottleneck. Don't extend above 3GB if you can it's a waste
of
| > money on 32 bit OS.
| >
| >
| > Willy.
| >
| >
|
|
See this info (it's long) http://msdn.microsoft.com/library/de...nitor_perf.asp
A quote:
First, the likelihood of experiencing an OutOfMemoryException begins to
increase dramatically when "Process\Virtual Bytes" is within 600 MB of the
virtual address space limit (generally 2 GB), and secondly, tests have shown
that "Process\Virtual Bytes" is often larger than "Process\Private Bytes" by
no more than 600 MB. This difference is due in part to the MEM_RESERVE
regions maintained by the GC, allowing it to quickly commit more memory when
needed. Taken together this implies that when "Process\Private Bytes" exceeds
800 MB, the likelihood of experiencing an OutOfMemoryException increases.
Long story short, asp.net 1.1 with has a high likelyhood of experiencing
OutOfMemoryExceptions once the aspnet_wp.exe process exceeds 800 MB due to
address space limitations and additional reserve by the GC.
The below applies to IIS 5.0. For IIS 6.0, you would set this up from IIS
Manager Properties -> Recycling -> Maximum used memory. But one of your
other posts indicated you were using Win 2k server, so you don't have IIS 6.0.
The memorylimit defaults to 60% of available RAM, which is why you didn't
have any issues while you only had 1 GB. Now that you have 2 GB,
aspnet_wp.exe won't recycle until it reaches 1.2 GB, which is over the 800 MB
problem point. You'll want to set the memoryLimit in the your machine.config
setting for processModel to a value that doesn't exceed 800 MB. For a 2 GB
machine, 39% will do the trick.
If you have enough RAM to where you have included the /3GB switch to
boot.ini, you can use a value that calculates out to 1,800 MB instead.
I'd also recommend lowering the timout and idelTimout values from Infinite,
to a lower value. This will allow the aspnet_wp.exe process to actually
recycle.
"Zen" wrote: Hi,
My production machine has 2G of memory, when aspnet_wp.exe goes up to about
~1.2G of memory usage, I start get out-of-memory exception. Other processes
don't use as much memory and I added all the peak memory usage of all the
processes (including aspnet_wp.exe), it goes up to no more than 1.5.
How is that possible? Would anyone know please help? thanks!
Note that this whole story is based on version 1.1 of the framework, this
version and version 1.0 had some problems with GC heap fragmentation
especially the LOH. Most of these problems (but not all) were solved with
SP1 of the framework and I suppose the OP runs v1.1 SP1. V2's memory manager
has even better handling of fragmentation built-in, but fragmentation can
never be avoided completely without the help of the application designer.
Note also that the OP suffered from OOM before the threshold was reached and
it was even worse when he ran on 1GB.
Now there are two things you can do:
1. lower the threshold and let the process recycle before an OOM can posible
occur.
2. Run a profiler and inspect (and adapt) your memory allocation profile.
1. IMO is not a valid option, it's plain ridiculous to have 2GB memory in a
server from which only < 600 MB can be used (remember the OP had OOM's with
1GB and 60% threshold).
2. This is what the OP must do first, it makes no sense to recycle a process
to recover from a bug or an application design mistake.
Willy.
"Neyah" <Ne***@discussions.microsoft.com> wrote in message
news:2A**********************************@microsof t.com...
| See this info (it's long)
| http://msdn.microsoft.com/library/de...nitor_perf.asp
|
| A quote:
| First, the likelihood of experiencing an OutOfMemoryException begins to
| increase dramatically when "Process\Virtual Bytes" is within 600 MB of the
| virtual address space limit (generally 2 GB), and secondly, tests have
shown
| that "Process\Virtual Bytes" is often larger than "Process\Private Bytes"
by
| no more than 600 MB. This difference is due in part to the MEM_RESERVE
| regions maintained by the GC, allowing it to quickly commit more memory
when
| needed. Taken together this implies that when "Process\Private Bytes"
exceeds
| 800 MB, the likelihood of experiencing an OutOfMemoryException increases.
|
| Long story short, asp.net 1.1 with has a high likelyhood of experiencing
| OutOfMemoryExceptions once the aspnet_wp.exe process exceeds 800 MB due to
| address space limitations and additional reserve by the GC.
|
| The below applies to IIS 5.0. For IIS 6.0, you would set this up from IIS
| Manager Properties -> Recycling -> Maximum used memory. But one of your
| other posts indicated you were using Win 2k server, so you don't have IIS
6.0.
|
| The memorylimit defaults to 60% of available RAM, which is why you didn't
| have any issues while you only had 1 GB. Now that you have 2 GB,
| aspnet_wp.exe won't recycle until it reaches 1.2 GB, which is over the 800
MB
| problem point. You'll want to set the memoryLimit in the your
machine.config
| setting for processModel to a value that doesn't exceed 800 MB. For a 2
GB
| machine, 39% will do the trick.
|
| If you have enough RAM to where you have included the /3GB switch to
| boot.ini, you can use a value that calculates out to 1,800 MB instead.
|
| I'd also recommend lowering the timout and idelTimout values from
Infinite,
| to a lower value. This will allow the aspnet_wp.exe process to actually
| recycle.
|
| "Zen" wrote:
|
| > Hi,
| >
| > My production machine has 2G of memory, when aspnet_wp.exe goes up to
about
| > ~1.2G of memory usage, I start get out-of-memory exception. Other
processes
| > don't use as much memory and I added all the peak memory usage of all
the
| > processes (including aspnet_wp.exe), it goes up to no more than 1.5.
| >
| > How is that possible? Would anyone know please help? thanks!
| >
| >
| >
thanks for the information. By the way, I'm not clear on how the author of
the article came up with the 800MB as a good threshold for "Process\Private
Bytes"? I restructured my code to reduce memory consumption peak (it's long
to explain it but it including removing a website from the server) and it's
running fine even when the "Private Bytes" goes up to 1.3G. I suspect that
the MEM_RESERVE regions go up for each additional website as well but I
still don't get the 800MB threshold, could someone explain? Thanks!
Also, I was setting the memory max to 80% instead of 60% hoping to solve the
problem. I'm changing it back down to 65% now.
"Neyah" <Ne***@discussions.microsoft.com> wrote in message
news:2A**********************************@microsof t.com... See this info (it's long)
http://msdn.microsoft.com/library/de...nitor_perf.asp
A quote:
First, the likelihood of experiencing an OutOfMemoryException begins to
increase dramatically when "Process\Virtual Bytes" is within 600 MB of the
virtual address space limit (generally 2 GB), and secondly, tests have
shown
that "Process\Virtual Bytes" is often larger than "Process\Private Bytes"
by
no more than 600 MB. This difference is due in part to the MEM_RESERVE
regions maintained by the GC, allowing it to quickly commit more memory
when
needed. Taken together this implies that when "Process\Private Bytes"
exceeds
800 MB, the likelihood of experiencing an OutOfMemoryException increases.
Long story short, asp.net 1.1 with has a high likelyhood of experiencing
OutOfMemoryExceptions once the aspnet_wp.exe process exceeds 800 MB due to
address space limitations and additional reserve by the GC.
The below applies to IIS 5.0. For IIS 6.0, you would set this up from IIS
Manager Properties -> Recycling -> Maximum used memory. But one of your
other posts indicated you were using Win 2k server, so you don't have IIS
6.0.
The memorylimit defaults to 60% of available RAM, which is why you didn't
have any issues while you only had 1 GB. Now that you have 2 GB,
aspnet_wp.exe won't recycle until it reaches 1.2 GB, which is over the 800
MB
problem point. You'll want to set the memoryLimit in the your
machine.config
setting for processModel to a value that doesn't exceed 800 MB. For a 2
GB
machine, 39% will do the trick.
If you have enough RAM to where you have included the /3GB switch to
boot.ini, you can use a value that calculates out to 1,800 MB instead.
I'd also recommend lowering the timout and idelTimout values from
Infinite,
to a lower value. This will allow the aspnet_wp.exe process to actually
recycle.
"Zen" wrote:
Hi,
My production machine has 2G of memory, when aspnet_wp.exe goes up to
about
~1.2G of memory usage, I start get out-of-memory exception. Other
processes
don't use as much memory and I added all the peak memory usage of all the
processes (including aspnet_wp.exe), it goes up to no more than 1.5.
How is that possible? Would anyone know please help? thanks!
Zen,
You should read the article carefully, the threshold is an empirical one
based on experiments using a typical application on a version (v1.1) of the
framework. But it's nothing else than a (dirty) trick, it sets the threshold
at such a low level that it forces the process to recycle before you
possibly have a chance to encounter an OOM exception, it's 'dirty' because
it may hide a bug (or design flaw) in your code and it side-steps the LOH
fragmentation problem typical for v1.0 and v1.1 (non SP1) of the framework.
You did change your code to reduce the memory consumption, but still your
memory consumption reaches 1.3 GB, so I guess you did refactor your code
such that there is less (or no?) fragmentation of the LOH. But again, once
the heap becomes too fragmented you are at risk for OOM's. Now the point is
what do you prefer, process recycling or OOM's, for me the two are equally
bad, you didn't pay for 2GB of memory to only use 60% of it didn't you?
So, I repeat it again, profile your web application take a look at the
allocation pattern, that's the only way to prevent/reduce fragmentation in a
server like application. If you can eliminate fragmentation completely, your
application will live a long, well performing and healthy life :-), if you
can't eliminate fragmentation, you will have to live with process recycling
or OOM's, the choice is yours.
Willy.
"Zen" <ze*@nononospam.com> wrote in message
news:OY**************@tk2msftngp13.phx.gbl...
| thanks for the information. By the way, I'm not clear on how the author
of
| the article came up with the 800MB as a good threshold for
"Process\Private
| Bytes"? I restructured my code to reduce memory consumption peak (it's
long
| to explain it but it including removing a website from the server) and
it's
| running fine even when the "Private Bytes" goes up to 1.3G. I suspect
that
| the MEM_RESERVE regions go up for each additional website as well but I
| still don't get the 800MB threshold, could someone explain? Thanks!
|
| Also, I was setting the memory max to 80% instead of 60% hoping to solve
the
| problem. I'm changing it back down to 65% now.
|
|
| "Neyah" <Ne***@discussions.microsoft.com> wrote in message
| news:2A**********************************@microsof t.com...
| > See this info (it's long)
| > http://msdn.microsoft.com/library/de...nitor_perf.asp
| >
| > A quote:
| > First, the likelihood of experiencing an OutOfMemoryException begins to
| > increase dramatically when "Process\Virtual Bytes" is within 600 MB of
the
| > virtual address space limit (generally 2 GB), and secondly, tests have
| > shown
| > that "Process\Virtual Bytes" is often larger than "Process\Private
Bytes"
| > by
| > no more than 600 MB. This difference is due in part to the MEM_RESERVE
| > regions maintained by the GC, allowing it to quickly commit more memory
| > when
| > needed. Taken together this implies that when "Process\Private Bytes"
| > exceeds
| > 800 MB, the likelihood of experiencing an OutOfMemoryException
increases.
| >
| > Long story short, asp.net 1.1 with has a high likelyhood of experiencing
| > OutOfMemoryExceptions once the aspnet_wp.exe process exceeds 800 MB due
to
| > address space limitations and additional reserve by the GC.
| >
| > The below applies to IIS 5.0. For IIS 6.0, you would set this up from
IIS
| > Manager Properties -> Recycling -> Maximum used memory. But one of your
| > other posts indicated you were using Win 2k server, so you don't have
IIS
| > 6.0.
| >
| > The memorylimit defaults to 60% of available RAM, which is why you
didn't
| > have any issues while you only had 1 GB. Now that you have 2 GB,
| > aspnet_wp.exe won't recycle until it reaches 1.2 GB, which is over the
800
| > MB
| > problem point. You'll want to set the memoryLimit in the your
| > machine.config
| > setting for processModel to a value that doesn't exceed 800 MB. For a 2
| > GB
| > machine, 39% will do the trick.
| >
| > If you have enough RAM to where you have included the /3GB switch to
| > boot.ini, you can use a value that calculates out to 1,800 MB instead.
| >
| > I'd also recommend lowering the timout and idelTimout values from
| > Infinite,
| > to a lower value. This will allow the aspnet_wp.exe process to actually
| > recycle.
| >
| > "Zen" wrote:
| >
| >> Hi,
| >>
| >> My production machine has 2G of memory, when aspnet_wp.exe goes up to
| >> about
| >> ~1.2G of memory usage, I start get out-of-memory exception. Other
| >> processes
| >> don't use as much memory and I added all the peak memory usage of all
the
| >> processes (including aspnet_wp.exe), it goes up to no more than 1.5.
| >>
| >> How is that possible? Would anyone know please help? thanks!
| >>
| >>
| >>
|
|
I didn't indicate that Zen should not profile the application and attempt to
reduce memory consumption. I merely replied to the original question of "How
is this possible?" and directed him to an article that answered the question.
Zen, the relevant portion of the article where they indicated to use 800 MB
is as follows:
First, the likelihood of experiencing an OutOfMemoryException begins to
increase dramatically when "Process\Virtual Bytes" is within 600 MB of the
virtual address space limit (generally 2 GB), and secondly, tests have shown
that "Process\Virtual Bytes" is often larger than "Process\Private Bytes" by
no more than 600 MB. This difference is due in part to the MEM_RESERVE
regions maintained by the GC, allowing it to quickly commit more memory when
needed. Taken together this implies that when "Process\Private Bytes" exceeds
800 MB, the likelihood of experiencing an OutOfMemoryException increases.
You're not really out of memory, you're just out of address space for the
worker process. This could be solved by utilizing the /3GB switch or
switching to a 64 bit machine. You indicated that you don't have enough RAM
to use /3GB, and purchasing a new machine isn't necessarily a real option.
Please note that 2.0 has many optimizations to the way GC works, including
the ability to reuse a fragmented LOH. You can run your 1.1 web application
under 2.0 without having to recompile simply by going into IIS, going into
ASP.NET tab in the properties for the affected IIS application directory, and
changing the dropdown for ASP.NET Version from 1.1.4322 to 2.0.50727.
If you aren't willing or aren't able to have 2.0 installed on your web
server, I would say that your only option is to live with process recycling.
I wouldn't consider constant OOMs with no way to recover from this state as a
valid "choice".
"Willy Denoyette [MVP]" wrote: Zen,
You should read the article carefully, the threshold is an empirical one
based on experiments using a typical application on a version (v1.1) of the
framework. But it's nothing else than a (dirty) trick, it sets the threshold
at such a low level that it forces the process to recycle before you
possibly have a chance to encounter an OOM exception, it's 'dirty' because
it may hide a bug (or design flaw) in your code and it side-steps the LOH
fragmentation problem typical for v1.0 and v1.1 (non SP1) of the framework.
You did change your code to reduce the memory consumption, but still your
memory consumption reaches 1.3 GB, so I guess you did refactor your code
such that there is less (or no?) fragmentation of the LOH. But again, once
the heap becomes too fragmented you are at risk for OOM's. Now the point is
what do you prefer, process recycling or OOM's, for me the two are equally
bad, you didn't pay for 2GB of memory to only use 60% of it didn't you?
So, I repeat it again, profile your web application take a look at the
allocation pattern, that's the only way to prevent/reduce fragmentation in a
server like application. If you can eliminate fragmentation completely, your
application will live a long, well performing and healthy life :-), if you
can't eliminate fragmentation, you will have to live with process recycling
or OOM's, the choice is yours.
Willy.
"Zen" <ze*@nononospam.com> wrote in message
news:OY**************@tk2msftngp13.phx.gbl...
| thanks for the information. By the way, I'm not clear on how the author
of
| the article came up with the 800MB as a good threshold for
"Process\Private
| Bytes"? I restructured my code to reduce memory consumption peak (it's
long
| to explain it but it including removing a website from the server) and
it's
| running fine even when the "Private Bytes" goes up to 1.3G. I suspect
that
| the MEM_RESERVE regions go up for each additional website as well but I
| still don't get the 800MB threshold, could someone explain? Thanks!
|
| Also, I was setting the memory max to 80% instead of 60% hoping to solve
the
| problem. I'm changing it back down to 65% now.
|
|
| "Neyah" <Ne***@discussions.microsoft.com> wrote in message
| news:2A**********************************@microsof t.com...
| > See this info (it's long)
| >
http://msdn.microsoft.com/library/de...nitor_perf.asp
| >
| > A quote:
| > First, the likelihood of experiencing an OutOfMemoryException begins to
| > increase dramatically when "Process\Virtual Bytes" is within 600 MB of
the
| > virtual address space limit (generally 2 GB), and secondly, tests have
| > shown
| > that "Process\Virtual Bytes" is often larger than "Process\Private
Bytes"
| > by
| > no more than 600 MB. This difference is due in part to the MEM_RESERVE
| > regions maintained by the GC, allowing it to quickly commit more memory
| > when
| > needed. Taken together this implies that when "Process\Private Bytes"
| > exceeds
| > 800 MB, the likelihood of experiencing an OutOfMemoryException
increases.
| >
| > Long story short, asp.net 1.1 with has a high likelyhood of experiencing
| > OutOfMemoryExceptions once the aspnet_wp.exe process exceeds 800 MB due
to
| > address space limitations and additional reserve by the GC.
| >
| > The below applies to IIS 5.0. For IIS 6.0, you would set this up from
IIS
| > Manager Properties -> Recycling -> Maximum used memory. But one of your
| > other posts indicated you were using Win 2k server, so you don't have
IIS
| > 6.0.
| >
| > The memorylimit defaults to 60% of available RAM, which is why you
didn't
| > have any issues while you only had 1 GB. Now that you have 2 GB,
| > aspnet_wp.exe won't recycle until it reaches 1.2 GB, which is over the
800
| > MB
| > problem point. You'll want to set the memoryLimit in the your
| > machine.config
| > setting for processModel to a value that doesn't exceed 800 MB. For a 2
| > GB
| > machine, 39% will do the trick.
| >
| > If you have enough RAM to where you have included the /3GB switch to
| > boot.ini, you can use a value that calculates out to 1,800 MB instead.
| >
| > I'd also recommend lowering the timout and idelTimout values from
| > Infinite,
| > to a lower value. This will allow the aspnet_wp.exe process to actually
| > recycle.
| >
| > "Zen" wrote:
| >
| >> Hi,
| >>
| >> My production machine has 2G of memory, when aspnet_wp.exe goes up to
| >> about
| >> ~1.2G of memory usage, I start get out-of-memory exception. Other
| >> processes
| >> don't use as much memory and I added all the peak memory usage of all
the
| >> processes (including aspnet_wp.exe), it goes up to no more than 1.5.
| >>
| >> How is that possible? Would anyone know please help? thanks!
| >>
| >>
| >>
|
|
Inline
Willy.
"Neyah" <Ne***@discussions.microsoft.com> wrote in message
news:35**********************************@microsof t.com...
|I didn't indicate that Zen should not profile the application and attempt
to
| reduce memory consumption. I merely replied to the original question of
"How
| is this possible?" and directed him to an article that answered the
question.
|
Hmm ... I merely replied (to Zen) to say that the article is quite old, it's
based on a version of the CLR that had problems with LOH fragmentation, and
the most importantly, that you should not take such artices as gospel just
use it as a reference but no more, you have to measure/profile yourself (do
your homework), your application might be totally different with different
load patterns and allocation profiles. But, before you do make sure you know
what and how to measure, you should know how Windows virtual memory system
works, how things like Physical RAM, address spaces and virtual memory
relate, and what the role of the GC is in this whole story.
Willy.
When memoryLimit threshold is reached, does Aspnet process know to call
GC.GetTotalMemory(true) to ensure that it's a real violation of the limit
before recycling itself?
thanks!
"Neyah" <Ne***@discussions.microsoft.com> wrote in message
news:35**********************************@microsof t.com... I didn't indicate that Zen should not profile the application and attempt
to
reduce memory consumption. I merely replied to the original question of
"How
is this possible?" and directed him to an article that answered the
question.
Zen, the relevant portion of the article where they indicated to use 800
MB
is as follows:
First, the likelihood of experiencing an OutOfMemoryException begins to
increase dramatically when "Process\Virtual Bytes" is within 600 MB of the
virtual address space limit (generally 2 GB), and secondly, tests have
shown
that "Process\Virtual Bytes" is often larger than "Process\Private Bytes"
by
no more than 600 MB. This difference is due in part to the MEM_RESERVE
regions maintained by the GC, allowing it to quickly commit more memory
when
needed. Taken together this implies that when "Process\Private Bytes"
exceeds
800 MB, the likelihood of experiencing an OutOfMemoryException increases.
You're not really out of memory, you're just out of address space for the
worker process. This could be solved by utilizing the /3GB switch or
switching to a 64 bit machine. You indicated that you don't have enough
RAM
to use /3GB, and purchasing a new machine isn't necessarily a real option.
Please note that 2.0 has many optimizations to the way GC works, including
the ability to reuse a fragmented LOH. You can run your 1.1 web
application
under 2.0 without having to recompile simply by going into IIS, going into
ASP.NET tab in the properties for the affected IIS application directory,
and
changing the dropdown for ASP.NET Version from 1.1.4322 to 2.0.50727.
If you aren't willing or aren't able to have 2.0 installed on your web
server, I would say that your only option is to live with process
recycling.
I wouldn't consider constant OOMs with no way to recover from this state
as a
valid "choice".
"Willy Denoyette [MVP]" wrote:
Zen,
You should read the article carefully, the threshold is an empirical one
based on experiments using a typical application on a version (v1.1) of
the
framework. But it's nothing else than a (dirty) trick, it sets the
threshold
at such a low level that it forces the process to recycle before you
possibly have a chance to encounter an OOM exception, it's 'dirty'
because
it may hide a bug (or design flaw) in your code and it side-steps the LOH
fragmentation problem typical for v1.0 and v1.1 (non SP1) of the
framework.
You did change your code to reduce the memory consumption, but still
your
memory consumption reaches 1.3 GB, so I guess you did refactor your code
such that there is less (or no?) fragmentation of the LOH. But again,
once
the heap becomes too fragmented you are at risk for OOM's. Now the point
is
what do you prefer, process recycling or OOM's, for me the two are
equally
bad, you didn't pay for 2GB of memory to only use 60% of it didn't you?
So, I repeat it again, profile your web application take a look at the
allocation pattern, that's the only way to prevent/reduce fragmentation
in a
server like application. If you can eliminate fragmentation completely,
your
application will live a long, well performing and healthy life :-), if
you
can't eliminate fragmentation, you will have to live with process
recycling
or OOM's, the choice is yours.
Willy.
"Zen" <ze*@nononospam.com> wrote in message
news:OY**************@tk2msftngp13.phx.gbl...
| thanks for the information. By the way, I'm not clear on how the
author
of
| the article came up with the 800MB as a good threshold for
"Process\Private
| Bytes"? I restructured my code to reduce memory consumption peak (it's
long
| to explain it but it including removing a website from the server) and
it's
| running fine even when the "Private Bytes" goes up to 1.3G. I suspect
that
| the MEM_RESERVE regions go up for each additional website as well but I
| still don't get the 800MB threshold, could someone explain? Thanks!
|
| Also, I was setting the memory max to 80% instead of 60% hoping to
solve
the
| problem. I'm changing it back down to 65% now.
|
|
| "Neyah" <Ne***@discussions.microsoft.com> wrote in message
| news:2A**********************************@microsof t.com...
| > See this info (it's long)
| >
http://msdn.microsoft.com/library/de...nitor_perf.asp
| >
| > A quote:
| > First, the likelihood of experiencing an OutOfMemoryException begins
to
| > increase dramatically when "Process\Virtual Bytes" is within 600 MB
of
the
| > virtual address space limit (generally 2 GB), and secondly, tests
have
| > shown
| > that "Process\Virtual Bytes" is often larger than "Process\Private
Bytes"
| > by
| > no more than 600 MB. This difference is due in part to the
MEM_RESERVE
| > regions maintained by the GC, allowing it to quickly commit more
memory
| > when
| > needed. Taken together this implies that when "Process\Private Bytes"
| > exceeds
| > 800 MB, the likelihood of experiencing an OutOfMemoryException
increases.
| >
| > Long story short, asp.net 1.1 with has a high likelyhood of
experiencing
| > OutOfMemoryExceptions once the aspnet_wp.exe process exceeds 800 MB
due
to
| > address space limitations and additional reserve by the GC.
| >
| > The below applies to IIS 5.0. For IIS 6.0, you would set this up
from
IIS
| > Manager Properties -> Recycling -> Maximum used memory. But one of
your
| > other posts indicated you were using Win 2k server, so you don't have
IIS
| > 6.0.
| >
| > The memorylimit defaults to 60% of available RAM, which is why you
didn't
| > have any issues while you only had 1 GB. Now that you have 2 GB,
| > aspnet_wp.exe won't recycle until it reaches 1.2 GB, which is over
the
800
| > MB
| > problem point. You'll want to set the memoryLimit in the your
| > machine.config
| > setting for processModel to a value that doesn't exceed 800 MB. For
a 2
| > GB
| > machine, 39% will do the trick.
| >
| > If you have enough RAM to where you have included the /3GB switch to
| > boot.ini, you can use a value that calculates out to 1,800 MB
instead.
| >
| > I'd also recommend lowering the timout and idelTimout values from
| > Infinite,
| > to a lower value. This will allow the aspnet_wp.exe process to
actually
| > recycle.
| >
| > "Zen" wrote:
| >
| >> Hi,
| >>
| >> My production machine has 2G of memory, when aspnet_wp.exe goes up
to
| >> about
| >> ~1.2G of memory usage, I start get out-of-memory exception. Other
| >> processes
| >> don't use as much memory and I added all the peak memory usage of
all
the
| >> processes (including aspnet_wp.exe), it goes up to no more than 1.5.
| >>
| >> How is that possible? Would anyone know please help? thanks!
| >>
| >>
| >>
|
|
The worker process is an unmanaged host, it doesn't have to call anything to
know if the threshold has been reached, it calls the OS services (and the
CLR services through the hosting interface) for this and exits when a
threshold is reached or when a deadlock is detected etc...
The aspnet ISAPI dll restarts the process when this happens.
Note also that GC.GetTotalMemory only returns the GC allocated memory, and
that's not the threshold we are talking about.
Willy.
"Zen" <ze*@nononospam.com> wrote in message
news:O5**************@TK2MSFTNGP14.phx.gbl...
| When memoryLimit threshold is reached, does Aspnet process know to call
| GC.GetTotalMemory(true) to ensure that it's a real violation of the limit
| before recycling itself?
|
| thanks!
|
| "Neyah" <Ne***@discussions.microsoft.com> wrote in message
| news:35**********************************@microsof t.com...
| >I didn't indicate that Zen should not profile the application and attempt
| >to
| > reduce memory consumption. I merely replied to the original question of
| > "How
| > is this possible?" and directed him to an article that answered the
| > question.
| >
| > Zen, the relevant portion of the article where they indicated to use 800
| > MB
| > is as follows:
| > First, the likelihood of experiencing an OutOfMemoryException begins to
| > increase dramatically when "Process\Virtual Bytes" is within 600 MB of
the
| > virtual address space limit (generally 2 GB), and secondly, tests have
| > shown
| > that "Process\Virtual Bytes" is often larger than "Process\Private
Bytes"
| > by
| > no more than 600 MB. This difference is due in part to the MEM_RESERVE
| > regions maintained by the GC, allowing it to quickly commit more memory
| > when
| > needed. Taken together this implies that when "Process\Private Bytes"
| > exceeds
| > 800 MB, the likelihood of experiencing an OutOfMemoryException
increases.
| >
| > You're not really out of memory, you're just out of address space for
the
| > worker process. This could be solved by utilizing the /3GB switch or
| > switching to a 64 bit machine. You indicated that you don't have enough
| > RAM
| > to use /3GB, and purchasing a new machine isn't necessarily a real
option.
| >
| > Please note that 2.0 has many optimizations to the way GC works,
including
| > the ability to reuse a fragmented LOH. You can run your 1.1 web
| > application
| > under 2.0 without having to recompile simply by going into IIS, going
into
| > ASP.NET tab in the properties for the affected IIS application
directory,
| > and
| > changing the dropdown for ASP.NET Version from 1.1.4322 to 2.0.50727.
| >
| > If you aren't willing or aren't able to have 2.0 installed on your web
| > server, I would say that your only option is to live with process
| > recycling.
| > I wouldn't consider constant OOMs with no way to recover from this state
| > as a
| > valid "choice".
| >
| > "Willy Denoyette [MVP]" wrote:
| >
| >> Zen,
| >>
| >> You should read the article carefully, the threshold is an empirical
one
| >> based on experiments using a typical application on a version (v1.1) of
| >> the
| >> framework. But it's nothing else than a (dirty) trick, it sets the
| >> threshold
| >> at such a low level that it forces the process to recycle before you
| >> possibly have a chance to encounter an OOM exception, it's 'dirty'
| >> because
| >> it may hide a bug (or design flaw) in your code and it side-steps the
LOH
| >> fragmentation problem typical for v1.0 and v1.1 (non SP1) of the
| >> framework.
| >>
| >> You did change your code to reduce the memory consumption, but still
| >> your
| >> memory consumption reaches 1.3 GB, so I guess you did refactor your
code
| >> such that there is less (or no?) fragmentation of the LOH. But again,
| >> once
| >> the heap becomes too fragmented you are at risk for OOM's. Now the
point
| >> is
| >> what do you prefer, process recycling or OOM's, for me the two are
| >> equally
| >> bad, you didn't pay for 2GB of memory to only use 60% of it didn't you?
| >> So, I repeat it again, profile your web application take a look at the
| >> allocation pattern, that's the only way to prevent/reduce fragmentation
| >> in a
| >> server like application. If you can eliminate fragmentation completely,
| >> your
| >> application will live a long, well performing and healthy life :-), if
| >> you
| >> can't eliminate fragmentation, you will have to live with process
| >> recycling
| >> or OOM's, the choice is yours.
| >>
| >> Willy.
| >>
| >>
| >>
| >>
| >>
| >> "Zen" <ze*@nononospam.com> wrote in message
| >> news:OY**************@tk2msftngp13.phx.gbl...
| >> | thanks for the information. By the way, I'm not clear on how the
| >> author
| >> of
| >> | the article came up with the 800MB as a good threshold for
| >> "Process\Private
| >> | Bytes"? I restructured my code to reduce memory consumption peak
(it's
| >> long
| >> | to explain it but it including removing a website from the server)
and
| >> it's
| >> | running fine even when the "Private Bytes" goes up to 1.3G. I
suspect
| >> that
| >> | the MEM_RESERVE regions go up for each additional website as well but
I
| >> | still don't get the 800MB threshold, could someone explain? Thanks!
| >> |
| >> | Also, I was setting the memory max to 80% instead of 60% hoping to
| >> solve
| >> the
| >> | problem. I'm changing it back down to 65% now.
| >> |
| >> |
| >> | "Neyah" <Ne***@discussions.microsoft.com> wrote in message
| >> | news:2A**********************************@microsof t.com...
| >> | > See this info (it's long)
| >> | >
| >> http://msdn.microsoft.com/library/de...nitor_perf.asp
| >> | >
| >> | > A quote:
| >> | > First, the likelihood of experiencing an OutOfMemoryException
begins
| >> to
| >> | > increase dramatically when "Process\Virtual Bytes" is within 600 MB
| >> of
| >> the
| >> | > virtual address space limit (generally 2 GB), and secondly, tests
| >> have
| >> | > shown
| >> | > that "Process\Virtual Bytes" is often larger than "Process\Private
| >> Bytes"
| >> | > by
| >> | > no more than 600 MB. This difference is due in part to the
| >> MEM_RESERVE
| >> | > regions maintained by the GC, allowing it to quickly commit more
| >> memory
| >> | > when
| >> | > needed. Taken together this implies that when "Process\Private
Bytes"
| >> | > exceeds
| >> | > 800 MB, the likelihood of experiencing an OutOfMemoryException
| >> increases.
| >> | >
| >> | > Long story short, asp.net 1.1 with has a high likelyhood of
| >> experiencing
| >> | > OutOfMemoryExceptions once the aspnet_wp.exe process exceeds 800 MB
| >> due
| >> to
| >> | > address space limitations and additional reserve by the GC.
| >> | >
| >> | > The below applies to IIS 5.0. For IIS 6.0, you would set this up
| >> from
| >> IIS
| >> | > Manager Properties -> Recycling -> Maximum used memory. But one of
| >> your
| >> | > other posts indicated you were using Win 2k server, so you don't
have
| >> IIS
| >> | > 6.0.
| >> | >
| >> | > The memorylimit defaults to 60% of available RAM, which is why you
| >> didn't
| >> | > have any issues while you only had 1 GB. Now that you have 2 GB,
| >> | > aspnet_wp.exe won't recycle until it reaches 1.2 GB, which is over
| >> the
| >> 800
| >> | > MB
| >> | > problem point. You'll want to set the memoryLimit in the your
| >> | > machine.config
| >> | > setting for processModel to a value that doesn't exceed 800 MB.
For
| >> a 2
| >> | > GB
| >> | > machine, 39% will do the trick.
| >> | >
| >> | > If you have enough RAM to where you have included the /3GB switch
to
| >> | > boot.ini, you can use a value that calculates out to 1,800 MB
| >> instead.
| >> | >
| >> | > I'd also recommend lowering the timout and idelTimout values from
| >> | > Infinite,
| >> | > to a lower value. This will allow the aspnet_wp.exe process to
| >> actually
| >> | > recycle.
| >> | >
| >> | > "Zen" wrote:
| >> | >
| >> | >> Hi,
| >> | >>
| >> | >> My production machine has 2G of memory, when aspnet_wp.exe goes up
| >> to
| >> | >> about
| >> | >> ~1.2G of memory usage, I start get out-of-memory exception. Other
| >> | >> processes
| >> | >> don't use as much memory and I added all the peak memory usage of
| >> all
| >> the
| >> | >> processes (including aspnet_wp.exe), it goes up to no more than
1.5.
| >> | >>
| >> | >> How is that possible? Would anyone know please help? thanks!
| >> | >>
| >> | >>
| >> | >>
| >> |
| >> |
| >>
| >>
| >>
|
|
According to the documentation that Neyah, when we pass "true" to the
GetTotalMemory() it actually calls other methods until it "seems" nothing
left to be collected (5% stability) then it returns.
"Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
news:uF*************@TK2MSFTNGP11.phx.gbl... The worker process is an unmanaged host, it doesn't have to call anything
to
know if the threshold has been reached, it calls the OS services (and the
CLR services through the hosting interface) for this and exits when a
threshold is reached or when a deadlock is detected etc...
The aspnet ISAPI dll restarts the process when this happens.
Note also that GC.GetTotalMemory only returns the GC allocated memory, and
that's not the threshold we are talking about.
Willy.
"Zen" <ze*@nononospam.com> wrote in message
news:O5**************@TK2MSFTNGP14.phx.gbl...
| When memoryLimit threshold is reached, does Aspnet process know to call
| GC.GetTotalMemory(true) to ensure that it's a real violation of the
limit
| before recycling itself?
|
| thanks!
|
| "Neyah" <Ne***@discussions.microsoft.com> wrote in message
| news:35**********************************@microsof t.com...
| >I didn't indicate that Zen should not profile the application and
attempt
| >to
| > reduce memory consumption. I merely replied to the original question
of
| > "How
| > is this possible?" and directed him to an article that answered the
| > question.
| >
| > Zen, the relevant portion of the article where they indicated to use
800
| > MB
| > is as follows:
| > First, the likelihood of experiencing an OutOfMemoryException begins
to
| > increase dramatically when "Process\Virtual Bytes" is within 600 MB of
the
| > virtual address space limit (generally 2 GB), and secondly, tests have
| > shown
| > that "Process\Virtual Bytes" is often larger than "Process\Private
Bytes"
| > by
| > no more than 600 MB. This difference is due in part to the MEM_RESERVE
| > regions maintained by the GC, allowing it to quickly commit more
memory
| > when
| > needed. Taken together this implies that when "Process\Private Bytes"
| > exceeds
| > 800 MB, the likelihood of experiencing an OutOfMemoryException
increases.
| >
| > You're not really out of memory, you're just out of address space for
the
| > worker process. This could be solved by utilizing the /3GB switch or
| > switching to a 64 bit machine. You indicated that you don't have
enough
| > RAM
| > to use /3GB, and purchasing a new machine isn't necessarily a real
option.
| >
| > Please note that 2.0 has many optimizations to the way GC works,
including
| > the ability to reuse a fragmented LOH. You can run your 1.1 web
| > application
| > under 2.0 without having to recompile simply by going into IIS, going
into
| > ASP.NET tab in the properties for the affected IIS application
directory,
| > and
| > changing the dropdown for ASP.NET Version from 1.1.4322 to 2.0.50727.
| >
| > If you aren't willing or aren't able to have 2.0 installed on your web
| > server, I would say that your only option is to live with process
| > recycling.
| > I wouldn't consider constant OOMs with no way to recover from this
state
| > as a
| > valid "choice".
| >
| > "Willy Denoyette [MVP]" wrote:
| >
| >> Zen,
| >>
| >> You should read the article carefully, the threshold is an empirical
one
| >> based on experiments using a typical application on a version (v1.1)
of
| >> the
| >> framework. But it's nothing else than a (dirty) trick, it sets the
| >> threshold
| >> at such a low level that it forces the process to recycle before you
| >> possibly have a chance to encounter an OOM exception, it's 'dirty'
| >> because
| >> it may hide a bug (or design flaw) in your code and it side-steps the
LOH
| >> fragmentation problem typical for v1.0 and v1.1 (non SP1) of the
| >> framework.
| >>
| >> You did change your code to reduce the memory consumption, but still
| >> your
| >> memory consumption reaches 1.3 GB, so I guess you did refactor your
code
| >> such that there is less (or no?) fragmentation of the LOH. But again,
| >> once
| >> the heap becomes too fragmented you are at risk for OOM's. Now the
point
| >> is
| >> what do you prefer, process recycling or OOM's, for me the two are
| >> equally
| >> bad, you didn't pay for 2GB of memory to only use 60% of it didn't
you?
| >> So, I repeat it again, profile your web application take a look at
the
| >> allocation pattern, that's the only way to prevent/reduce
fragmentation
| >> in a
| >> server like application. If you can eliminate fragmentation
completely,
| >> your
| >> application will live a long, well performing and healthy life :-),
if
| >> you
| >> can't eliminate fragmentation, you will have to live with process
| >> recycling
| >> or OOM's, the choice is yours.
| >>
| >> Willy.
| >>
| >>
| >>
| >>
| >>
| >> "Zen" <ze*@nononospam.com> wrote in message
| >> news:OY**************@tk2msftngp13.phx.gbl...
| >> | thanks for the information. By the way, I'm not clear on how the
| >> author
| >> of
| >> | the article came up with the 800MB as a good threshold for
| >> "Process\Private
| >> | Bytes"? I restructured my code to reduce memory consumption peak
(it's
| >> long
| >> | to explain it but it including removing a website from the server)
and
| >> it's
| >> | running fine even when the "Private Bytes" goes up to 1.3G. I
suspect
| >> that
| >> | the MEM_RESERVE regions go up for each additional website as well
but
I
| >> | still don't get the 800MB threshold, could someone explain? Thanks!
| >> |
| >> | Also, I was setting the memory max to 80% instead of 60% hoping to
| >> solve
| >> the
| >> | problem. I'm changing it back down to 65% now.
| >> |
| >> |
| >> | "Neyah" <Ne***@discussions.microsoft.com> wrote in message
| >> | news:2A**********************************@microsof t.com...
| >> | > See this info (it's long)
| >> | >
| >>
http://msdn.microsoft.com/library/de...nitor_perf.asp
| >> | >
| >> | > A quote:
| >> | > First, the likelihood of experiencing an OutOfMemoryException
begins
| >> to
| >> | > increase dramatically when "Process\Virtual Bytes" is within 600
MB
| >> of
| >> the
| >> | > virtual address space limit (generally 2 GB), and secondly, tests
| >> have
| >> | > shown
| >> | > that "Process\Virtual Bytes" is often larger than
"Process\Private
| >> Bytes"
| >> | > by
| >> | > no more than 600 MB. This difference is due in part to the
| >> MEM_RESERVE
| >> | > regions maintained by the GC, allowing it to quickly commit more
| >> memory
| >> | > when
| >> | > needed. Taken together this implies that when "Process\Private
Bytes"
| >> | > exceeds
| >> | > 800 MB, the likelihood of experiencing an OutOfMemoryException
| >> increases.
| >> | >
| >> | > Long story short, asp.net 1.1 with has a high likelyhood of
| >> experiencing
| >> | > OutOfMemoryExceptions once the aspnet_wp.exe process exceeds 800
MB
| >> due
| >> to
| >> | > address space limitations and additional reserve by the GC.
| >> | >
| >> | > The below applies to IIS 5.0. For IIS 6.0, you would set this up
| >> from
| >> IIS
| >> | > Manager Properties -> Recycling -> Maximum used memory. But one
of
| >> your
| >> | > other posts indicated you were using Win 2k server, so you don't
have
| >> IIS
| >> | > 6.0.
| >> | >
| >> | > The memorylimit defaults to 60% of available RAM, which is why
you
| >> didn't
| >> | > have any issues while you only had 1 GB. Now that you have 2 GB,
| >> | > aspnet_wp.exe won't recycle until it reaches 1.2 GB, which is
over
| >> the
| >> 800
| >> | > MB
| >> | > problem point. You'll want to set the memoryLimit in the your
| >> | > machine.config
| >> | > setting for processModel to a value that doesn't exceed 800 MB.
For
| >> a 2
| >> | > GB
| >> | > machine, 39% will do the trick.
| >> | >
| >> | > If you have enough RAM to where you have included the /3GB switch
to
| >> | > boot.ini, you can use a value that calculates out to 1,800 MB
| >> instead.
| >> | >
| >> | > I'd also recommend lowering the timout and idelTimout values from
| >> | > Infinite,
| >> | > to a lower value. This will allow the aspnet_wp.exe process to
| >> actually
| >> | > recycle.
| >> | >
| >> | > "Zen" wrote:
| >> | >
| >> | >> Hi,
| >> | >>
| >> | >> My production machine has 2G of memory, when aspnet_wp.exe goes
up
| >> to
| >> | >> about
| >> | >> ~1.2G of memory usage, I start get out-of-memory exception.
Other
| >> | >> processes
| >> | >> don't use as much memory and I added all the peak memory usage
of
| >> all
| >> the
| >> | >> processes (including aspnet_wp.exe), it goes up to no more than
1.5.
| >> | >>
| >> | >> How is that possible? Would anyone know please help? thanks!
| >> | >>
| >> | >>
| >> | >>
| >> |
| >> |
| >>
| >>
| >>
|
|
GetTotalMemory(true), calls GC.Collect()/WaitForPendingFinalizers() a couple
of time that's right, but this is not the point, your question was about the
asp.net worker process right?.
The asp.net WP doesn't call GCGetTotalMemory(true), and doesn't care about
GC allocated memory it cares about "private bytes", GC heaps are only a
(large) part of it. The WP will do all it can to prevent recycling, so it
will instruct the CLR to run a full GC, before he ever decides to recycle.
This is not what you should be worried about, what you should do is take
control over your applications memory consumption in order to prevent
recycling, and this starts with profiling.
Again, don't pay to much attention to the article unless you are running
v1.1 without SP1, in which case I would install the SP as soon as possible.
Willy.
"Zen" <ze*@nononospam.com> wrote in message
news:%2***************@TK2MSFTNGP09.phx.gbl...
| According to the documentation that Neyah, when we pass "true" to the
| GetTotalMemory() it actually calls other methods until it "seems" nothing
| left to be collected (5% stability) then it returns.
|
|
| "Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
| news:uF*************@TK2MSFTNGP11.phx.gbl...
| > The worker process is an unmanaged host, it doesn't have to call
anything
| > to
| > know if the threshold has been reached, it calls the OS services (and
the
| > CLR services through the hosting interface) for this and exits when a
| > threshold is reached or when a deadlock is detected etc...
| > The aspnet ISAPI dll restarts the process when this happens.
| > Note also that GC.GetTotalMemory only returns the GC allocated memory,
and
| > that's not the threshold we are talking about.
| >
| > Willy.
| >
| >
| >
| > "Zen" <ze*@nononospam.com> wrote in message
| > news:O5**************@TK2MSFTNGP14.phx.gbl...
| > | When memoryLimit threshold is reached, does Aspnet process know to
call
| > | GC.GetTotalMemory(true) to ensure that it's a real violation of the
| > limit
| > | before recycling itself?
| > |
| > | thanks!
| > |
| > | "Neyah" <Ne***@discussions.microsoft.com> wrote in message
| > | news:35**********************************@microsof t.com...
| > | >I didn't indicate that Zen should not profile the application and
| > attempt
| > | >to
| > | > reduce memory consumption. I merely replied to the original
question
| > of
| > | > "How
| > | > is this possible?" and directed him to an article that answered the
| > | > question.
| > | >
| > | > Zen, the relevant portion of the article where they indicated to use
| > 800
| > | > MB
| > | > is as follows:
| > | > First, the likelihood of experiencing an OutOfMemoryException begins
| > to
| > | > increase dramatically when "Process\Virtual Bytes" is within 600 MB
of
| > the
| > | > virtual address space limit (generally 2 GB), and secondly, tests
have
| > | > shown
| > | > that "Process\Virtual Bytes" is often larger than "Process\Private
| > Bytes"
| > | > by
| > | > no more than 600 MB. This difference is due in part to the
MEM_RESERVE
| > | > regions maintained by the GC, allowing it to quickly commit more
| > memory
| > | > when
| > | > needed. Taken together this implies that when "Process\Private
Bytes"
| > | > exceeds
| > | > 800 MB, the likelihood of experiencing an OutOfMemoryException
| > increases.
| > | >
| > | > You're not really out of memory, you're just out of address space
for
| > the
| > | > worker process. This could be solved by utilizing the /3GB switch
or
| > | > switching to a 64 bit machine. You indicated that you don't have
| > enough
| > | > RAM
| > | > to use /3GB, and purchasing a new machine isn't necessarily a real
| > option.
| > | >
| > | > Please note that 2.0 has many optimizations to the way GC works,
| > including
| > | > the ability to reuse a fragmented LOH. You can run your 1.1 web
| > | > application
| > | > under 2.0 without having to recompile simply by going into IIS,
going
| > into
| > | > ASP.NET tab in the properties for the affected IIS application
| > directory,
| > | > and
| > | > changing the dropdown for ASP.NET Version from 1.1.4322 to
2.0.50727.
| > | >
| > | > If you aren't willing or aren't able to have 2.0 installed on your
web
| > | > server, I would say that your only option is to live with process
| > | > recycling.
| > | > I wouldn't consider constant OOMs with no way to recover from this
| > state
| > | > as a
| > | > valid "choice".
| > | >
| > | > "Willy Denoyette [MVP]" wrote:
| > | >
| > | >> Zen,
| > | >>
| > | >> You should read the article carefully, the threshold is an
empirical
| > one
| > | >> based on experiments using a typical application on a version
(v1.1)
| > of
| > | >> the
| > | >> framework. But it's nothing else than a (dirty) trick, it sets the
| > | >> threshold
| > | >> at such a low level that it forces the process to recycle before
you
| > | >> possibly have a chance to encounter an OOM exception, it's 'dirty'
| > | >> because
| > | >> it may hide a bug (or design flaw) in your code and it side-steps
the
| > LOH
| > | >> fragmentation problem typical for v1.0 and v1.1 (non SP1) of the
| > | >> framework.
| > | >>
| > | >> You did change your code to reduce the memory consumption, but
still
| > | >> your
| > | >> memory consumption reaches 1.3 GB, so I guess you did refactor your
| > code
| > | >> such that there is less (or no?) fragmentation of the LOH. But
again,
| > | >> once
| > | >> the heap becomes too fragmented you are at risk for OOM's. Now the
| > point
| > | >> is
| > | >> what do you prefer, process recycling or OOM's, for me the two are
| > | >> equally
| > | >> bad, you didn't pay for 2GB of memory to only use 60% of it didn't
| > you?
| > | >> So, I repeat it again, profile your web application take a look at
| > the
| > | >> allocation pattern, that's the only way to prevent/reduce
| > fragmentation
| > | >> in a
| > | >> server like application. If you can eliminate fragmentation
| > completely,
| > | >> your
| > | >> application will live a long, well performing and healthy life :-),
| > if
| > | >> you
| > | >> can't eliminate fragmentation, you will have to live with process
| > | >> recycling
| > | >> or OOM's, the choice is yours.
| > | >>
| > | >> Willy.
| > | >>
| > | >>
| > | >>
| > | >>
| > | >>
| > | >> "Zen" <ze*@nononospam.com> wrote in message
| > | >> news:OY**************@tk2msftngp13.phx.gbl...
| > | >> | thanks for the information. By the way, I'm not clear on how the
| > | >> author
| > | >> of
| > | >> | the article came up with the 800MB as a good threshold for
| > | >> "Process\Private
| > | >> | Bytes"? I restructured my code to reduce memory consumption peak
| > (it's
| > | >> long
| > | >> | to explain it but it including removing a website from the
server)
| > and
| > | >> it's
| > | >> | running fine even when the "Private Bytes" goes up to 1.3G. I
| > suspect
| > | >> that
| > | >> | the MEM_RESERVE regions go up for each additional website as well
| > but
| > I
| > | >> | still don't get the 800MB threshold, could someone explain?
Thanks!
| > | >> |
| > | >> | Also, I was setting the memory max to 80% instead of 60% hoping
to
| > | >> solve
| > | >> the
| > | >> | problem. I'm changing it back down to 65% now.
| > | >> |
| > | >> |
| > | >> | "Neyah" <Ne***@discussions.microsoft.com> wrote in message
| > | >> | news:2A**********************************@microsof t.com...
| > | >> | > See this info (it's long)
| > | >> | >
| > | >>
| > http://msdn.microsoft.com/library/de...nitor_perf.asp
| > | >> | >
| > | >> | > A quote:
| > | >> | > First, the likelihood of experiencing an OutOfMemoryException
| > begins
| > | >> to
| > | >> | > increase dramatically when "Process\Virtual Bytes" is within
600
| > MB
| > | >> of
| > | >> the
| > | >> | > virtual address space limit (generally 2 GB), and secondly,
tests
| > | >> have
| > | >> | > shown
| > | >> | > that "Process\Virtual Bytes" is often larger than
| > "Process\Private
| > | >> Bytes"
| > | >> | > by
| > | >> | > no more than 600 MB. This difference is due in part to the
| > | >> MEM_RESERVE
| > | >> | > regions maintained by the GC, allowing it to quickly commit
more
| > | >> memory
| > | >> | > when
| > | >> | > needed. Taken together this implies that when "Process\Private
| > Bytes"
| > | >> | > exceeds
| > | >> | > 800 MB, the likelihood of experiencing an OutOfMemoryException
| > | >> increases.
| > | >> | >
| > | >> | > Long story short, asp.net 1.1 with has a high likelyhood of
| > | >> experiencing
| > | >> | > OutOfMemoryExceptions once the aspnet_wp.exe process exceeds
800
| > MB
| > | >> due
| > | >> to
| > | >> | > address space limitations and additional reserve by the GC.
| > | >> | >
| > | >> | > The below applies to IIS 5.0. For IIS 6.0, you would set this
up
| > | >> from
| > | >> IIS
| > | >> | > Manager Properties -> Recycling -> Maximum used memory. But
one
| > of
| > | >> your
| > | >> | > other posts indicated you were using Win 2k server, so you
don't
| > have
| > | >> IIS
| > | >> | > 6.0.
| > | >> | >
| > | >> | > The memorylimit defaults to 60% of available RAM, which is why
| > you
| > | >> didn't
| > | >> | > have any issues while you only had 1 GB. Now that you have 2
GB,
| > | >> | > aspnet_wp.exe won't recycle until it reaches 1.2 GB, which is
| > over
| > | >> the
| > | >> 800
| > | >> | > MB
| > | >> | > problem point. You'll want to set the memoryLimit in the your
| > | >> | > machine.config
| > | >> | > setting for processModel to a value that doesn't exceed 800 MB.
| > For
| > | >> a 2
| > | >> | > GB
| > | >> | > machine, 39% will do the trick.
| > | >> | >
| > | >> | > If you have enough RAM to where you have included the /3GB
switch
| > to
| > | >> | > boot.ini, you can use a value that calculates out to 1,800 MB
| > | >> instead.
| > | >> | >
| > | >> | > I'd also recommend lowering the timout and idelTimout values
from
| > | >> | > Infinite,
| > | >> | > to a lower value. This will allow the aspnet_wp.exe process to
| > | >> actually
| > | >> | > recycle.
| > | >> | >
| > | >> | > "Zen" wrote:
| > | >> | >
| > | >> | >> Hi,
| > | >> | >>
| > | >> | >> My production machine has 2G of memory, when aspnet_wp.exe
goes
| > up
| > | >> to
| > | >> | >> about
| > | >> | >> ~1.2G of memory usage, I start get out-of-memory exception.
| > Other
| > | >> | >> processes
| > | >> | >> don't use as much memory and I added all the peak memory usage
| > of
| > | >> all
| > | >> the
| > | >> | >> processes (including aspnet_wp.exe), it goes up to no more
than
| > 1.5.
| > | >> | >>
| > | >> | >> How is that possible? Would anyone know please help? thanks!
| > | >> | >>
| > | >> | >>
| > | >> | >>
| > | >> |
| > | >> |
| > | >>
| > | >>
| > | >>
| > |
| > |
| >
| >
|
| This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Kirby Angell |
last post by:
We are porting our applications from Python 2.2 on RH9 to Python 2.3 on
RH9. One of our apps used to be able to create threads with wild
abandon, 800 or more if necessary. With Python 2.3 we are...
|
by: Hal Vaughan |
last post by:
Is there a maximum length for Javascript program lines?
What about strings? Is there a limit on string length?
I found some references that said the maximum string length was 256
characters,...
|
by: Matt |
last post by:
I am running CR 8.5 from a (large) SQL Server db. My aim is to use
crystal enterprise so end users in our sales team can access reports on
demand for a given date range and client so I am trying...
|
by: Dean Slindee |
last post by:
Is there a maximum number of menu items that can be added to a single form?
If there is no maximum, that would help eliminate one possible source of the
problem.
I had six main menu items in the...
|
by: Leroy |
last post by:
Hello,
I have a question regarding the maximum number of parameters that can be
passed to a procedure.
In VB 6 the max was 60.
What is the max for Dot Net?
please and thanks.
|
by: Gerrit |
last post by:
Hi all,
I'm getting an OutOfMemoryException when I initialize a byte array in
C# like this:
Byte test = new Byte;
I'm using ASP.NET 2.0. In ASP.Net 1.1 it works fine. So what am I doing...
|
by: Paul Sijben |
last post by:
I have a server in Python 2.5 that generates a lot of threads. It is
running on a linux server (Fedora Core 6).
The server quickly runs out of threads.
I am seeing the following error.
File...
|
by: jobs |
last post by:
re: Oracle Exceeded maximum idle time
I have asp.net web gridview that is populated by a function that calls
an Oracle procedure that returns a dataset. At most, it might take 2
or 4 seconds to...
|
by: raylopez99 |
last post by:
The maximum int for an array on my machine (a Pentium IV with 2 GB
RAM) is < 330 Million...before you get an "out of memory" exception.
I simply filled an array of this size with ints...I got as...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
| |
