Connecting Tech Pros Worldwide Forums | Help | Site Map

how delete array from memory

tony collier
Guest
 
Posts: n/a
#1: Nov 15 '05
This is an array question but it is based in my web page so i shall give
scenario:

when the client navigates to the page, the page takes some figures and does
loads of calulations in page_load which result in an array with 4.8 million
elements. i then extract a mere handful which is all the client needs and
then return these results to the client. I would therefore like to delete
the array from memory at the end of the page_load event. Can someone
please tell me how to do this. thanks.

Richard A. Lowe
Guest
 
Posts: n/a
#2: Nov 15 '05

re: how delete array from memory


All you have to do is make sure that the array is not being referenced in
the Application or Session objects (that you're not purposely keeping it
alive across page requests) and .NET will decide upon the best time to
collect it. In .NET you don't explicitly control memory; check out the top
hits for .NET memory management at Google:
http://www.google.com/search?hl=en&i...ory+management
for in-depth explainations.

Richard

--
C#, .NET and Complex Adaptive Systems:
http://blogs.geekdojo.net/Richard
"tony collier" <melakka@hotmail.com> wrote in message
news:Xns947A30B5D948Bmelakkahotmailcom@140.99.99.1 30...[color=blue]
> This is an array question but it is based in my web page so i shall give
> scenario:
>
> when the client navigates to the page, the page takes some figures and[/color]
does[color=blue]
> loads of calulations in page_load which result in an array with 4.8[/color]
million[color=blue]
> elements. i then extract a mere handful which is all the client needs and
> then return these results to the client. I would therefore like to delete
> the array from memory at the end of the page_load event. Can someone
> please tell me how to do this. thanks.[/color]


Justin Rogers
Guest
 
Posts: n/a
#3: Nov 15 '05

re: how delete array from memory


But you can control the GC...

GC.Collect(GC.MaxGeneration); // forces a full collect
GC.Collect(GC.GetGeneration(myArray)); // forces a collect up to the generation
your array is in

This isn't recommended, but if you are doing this calculation simultaneously for
many users you may
have a worker process roll-over (denial of service for some short period of
time) before the weight
on the GC forces it to kick in and collect. You need to weight roll-overs using
the process counter
and if they are high then you should manually control the GC since you know the
best time to collect
your objects and the GC only makes guesses.


--
Justin Rogers
DigiTec Web Consultants, LLC.
Blog: http://weblogs.asp.net/justin_rogers


"Richard A. Lowe" <chadich@yumspamyumYahoo.com> wrote in message
news:e0W83pj4DHA.360@TK2MSFTNGP12.phx.gbl...[color=blue]
> All you have to do is make sure that the array is not being referenced in
> the Application or Session objects (that you're not purposely keeping it
> alive across page requests) and .NET will decide upon the best time to
> collect it. In .NET you don't explicitly control memory; check out the top
> hits for .NET memory management at Google:
> http://www.google.com/search?hl=en&i...ory+management
> for in-depth explainations.
>
> Richard
>
> --
> C#, .NET and Complex Adaptive Systems:
> http://blogs.geekdojo.net/Richard
> "tony collier" <melakka@hotmail.com> wrote in message
> news:Xns947A30B5D948Bmelakkahotmailcom@140.99.99.1 30...[color=green]
> > This is an array question but it is based in my web page so i shall give
> > scenario:
> >
> > when the client navigates to the page, the page takes some figures and[/color]
> does[color=green]
> > loads of calulations in page_load which result in an array with 4.8[/color]
> million[color=green]
> > elements. i then extract a mere handful which is all the client needs and
> > then return these results to the client. I would therefore like to delete
> > the array from memory at the end of the page_load event. Can someone
> > please tell me how to do this. thanks.[/color]
>
>[/color]


Richard A. Lowe
Guest
 
Posts: n/a
#4: Nov 15 '05

re: how delete array from memory


> and if they are high then you should manually control the GC since you
know the[color=blue]
> best time to collect
> your objects and the GC only makes guesses.[/color]

Justin, have you implemented and tested this in a real world app? I'm not
trying to be rhetorical - I'm really asking, since I've never had the
experience of using GC outside of simple test/diagnostic apps (and everytime
I have tried to use it other places, people smarter than I have taken me to
task for it :). It still seems risky to me to call the GC, perhaps even
every time this page fires, in light of the potential expense of GC in
ASP.NET especially - but I will defer to our resident gurus.

One alternative for this kind of thing is running code in an AppDomain, then
unloading it, which I gave an example of doing here:
http://blogs.geekdojo.net/richard/ar...12/10/428.aspx

This certainly has some expense to it, but seems to work well enough at
deterministically removing memory.

Richard
--
C#, .NET and Complex Adaptive Systems:
http://blogs.geekdojo.net/Richard
"Justin Rogers" <Justin@games4dotnet.com> wrote in message
news:uZoi2vj4DHA.384@TK2MSFTNGP11.phx.gbl...[color=blue]
> But you can control the GC...
>
> GC.Collect(GC.MaxGeneration); // forces a full collect
> GC.Collect(GC.GetGeneration(myArray)); // forces a collect up to the[/color]
generation[color=blue]
> your array is in
>
> This isn't recommended, but if you are doing this calculation[/color]
simultaneously for[color=blue]
> many users you may
> have a worker process roll-over (denial of service for some short period[/color]
of[color=blue]
> time) before the weight
> on the GC forces it to kick in and collect. You need to weight roll-overs[/color]
using[color=blue]
> the process counter
> and if they are high then you should manually control the GC since you[/color]
know the[color=blue]
> best time to collect
> your objects and the GC only makes guesses.
>
>
> --
> Justin Rogers
> DigiTec Web Consultants, LLC.
> Blog: http://weblogs.asp.net/justin_rogers
>
>
> "Richard A. Lowe" <chadich@yumspamyumYahoo.com> wrote in message
> news:e0W83pj4DHA.360@TK2MSFTNGP12.phx.gbl...[color=green]
> > All you have to do is make sure that the array is not being referenced[/color][/color]
in[color=blue][color=green]
> > the Application or Session objects (that you're not purposely keeping it
> > alive across page requests) and .NET will decide upon the best time to
> > collect it. In .NET you don't explicitly control memory; check out the[/color][/color]
top[color=blue][color=green]
> > hits for .NET memory management at Google:
> >[/color][/color]
http://www.google.com/search?hl=en&i...ory+management[color=blue][color=green]
> > for in-depth explainations.
> >
> > Richard
> >
> > --
> > C#, .NET and Complex Adaptive Systems:
> > http://blogs.geekdojo.net/Richard
> > "tony collier" <melakka@hotmail.com> wrote in message
> > news:Xns947A30B5D948Bmelakkahotmailcom@140.99.99.1 30...[color=darkred]
> > > This is an array question but it is based in my web page so i shall[/color][/color][/color]
give[color=blue][color=green][color=darkred]
> > > scenario:
> > >
> > > when the client navigates to the page, the page takes some figures and[/color]
> > does[color=darkred]
> > > loads of calulations in page_load which result in an array with 4.8[/color]
> > million[color=darkred]
> > > elements. i then extract a mere handful which is all the client needs[/color][/color][/color]
and[color=blue][color=green][color=darkred]
> > > then return these results to the client. I would therefore like to[/color][/color][/color]
delete[color=blue][color=green][color=darkred]
> > > the array from memory at the end of the page_load event. Can someone
> > > please tell me how to do this. thanks.[/color]
> >
> >[/color]
>
>[/color]


Justin Rogers
Guest
 
Posts: n/a
#5: Nov 15 '05

re: how delete array from memory


For about six months we used it in the .NET Terrarium application. Until the
GC caught up with our expectations and we didn't need it anymore. During that
time there was a deterministic location we could call from (between game ticks)
that caused a low impact on our application at hand and so it worked perfectly.

In this guy's case, he has a couple of options:
1. Don't GC at all, and hope that the worker process doesn't role over
constantly
2. Don't GC at all, and hope that the memory used on the array gets re-used for
the next array
3. GC if he finds that either 1 or 2 isn't coming true.

So what I am saying is pick the lesser of several evils depending on his own
goals and
inspection of what is happening on his servers. Generally speaking, #2 should
come into
play for him, since the memory should get re-used after it goes out of scope.
The largest problem
I see is that he is allocating a huge array, and allocating huge arrays
generally means finding
contiguous locations in memory for said arrays. Because of these types of
allocations in addition
to more normal allocations he might find situations where the managed heap grows
uncontrollably (e.g.
very fast in a short amount of time, causing the work process to role)

--
Justin Rogers
DigiTec Web Consultants, LLC.
Blog: http://weblogs.asp.net/justin_rogers


"Richard A. Lowe" <chadich@yumspamyumYahoo.com> wrote in message
news:exNZFGk4DHA.2528@TK2MSFTNGP09.phx.gbl...[color=blue][color=green]
> > and if they are high then you should manually control the GC since you[/color]
> know the[color=green]
> > best time to collect
> > your objects and the GC only makes guesses.[/color]
>
> Justin, have you implemented and tested this in a real world app? I'm not
> trying to be rhetorical - I'm really asking, since I've never had the
> experience of using GC outside of simple test/diagnostic apps (and everytime
> I have tried to use it other places, people smarter than I have taken me to
> task for it :). It still seems risky to me to call the GC, perhaps even
> every time this page fires, in light of the potential expense of GC in
> ASP.NET especially - but I will defer to our resident gurus.
>
> One alternative for this kind of thing is running code in an AppDomain, then
> unloading it, which I gave an example of doing here:
> http://blogs.geekdojo.net/richard/ar...12/10/428.aspx
>
> This certainly has some expense to it, but seems to work well enough at
> deterministically removing memory.
>
> Richard
> --
> C#, .NET and Complex Adaptive Systems:
> http://blogs.geekdojo.net/Richard
> "Justin Rogers" <Justin@games4dotnet.com> wrote in message
> news:uZoi2vj4DHA.384@TK2MSFTNGP11.phx.gbl...[color=green]
> > But you can control the GC...
> >
> > GC.Collect(GC.MaxGeneration); // forces a full collect
> > GC.Collect(GC.GetGeneration(myArray)); // forces a collect up to the[/color]
> generation[color=green]
> > your array is in
> >
> > This isn't recommended, but if you are doing this calculation[/color]
> simultaneously for[color=green]
> > many users you may
> > have a worker process roll-over (denial of service for some short period[/color]
> of[color=green]
> > time) before the weight
> > on the GC forces it to kick in and collect. You need to weight roll-overs[/color]
> using[color=green]
> > the process counter
> > and if they are high then you should manually control the GC since you[/color]
> know the[color=green]
> > best time to collect
> > your objects and the GC only makes guesses.
> >
> >
> > --
> > Justin Rogers
> > DigiTec Web Consultants, LLC.
> > Blog: http://weblogs.asp.net/justin_rogers
> >
> >
> > "Richard A. Lowe" <chadich@yumspamyumYahoo.com> wrote in message
> > news:e0W83pj4DHA.360@TK2MSFTNGP12.phx.gbl...[color=darkred]
> > > All you have to do is make sure that the array is not being referenced[/color][/color]
> in[color=green][color=darkred]
> > > the Application or Session objects (that you're not purposely keeping it
> > > alive across page requests) and .NET will decide upon the best time to
> > > collect it. In .NET you don't explicitly control memory; check out the[/color][/color]
> top[color=green][color=darkred]
> > > hits for .NET memory management at Google:
> > >[/color][/color]
> http://www.google.com/search?hl=en&i...ory+management[color=green][color=darkred]
> > > for in-depth explainations.
> > >
> > > Richard
> > >
> > > --
> > > C#, .NET and Complex Adaptive Systems:
> > > http://blogs.geekdojo.net/Richard
> > > "tony collier" <melakka@hotmail.com> wrote in message
> > > news:Xns947A30B5D948Bmelakkahotmailcom@140.99.99.1 30...
> > > > This is an array question but it is based in my web page so i shall[/color][/color]
> give[color=green][color=darkred]
> > > > scenario:
> > > >
> > > > when the client navigates to the page, the page takes some figures and
> > > does
> > > > loads of calulations in page_load which result in an array with 4.8
> > > million
> > > > elements. i then extract a mere handful which is all the client needs[/color][/color]
> and[color=green][color=darkred]
> > > > then return these results to the client. I would therefore like to[/color][/color]
> delete[color=green][color=darkred]
> > > > the array from memory at the end of the page_load event. Can someone
> > > > please tell me how to do this. thanks.
> > >
> > >[/color]
> >
> >[/color]
>
>[/color]


Richard A. Lowe
Guest
 
Posts: n/a
#6: Nov 15 '05

re: how delete array from memory


Okay, so I would suggest an option 4 - Do all 'excessive' memory work in a
temporary AppDomain, and reclaim the memory when the AppDomain is unloaded.
This is partly speculation on my part, but it seems this would mean
targeted, deterministic removal of only the memory allocated for the 'large
memory process' and leave the GC to do it's work normally on the range of
objects created in a normal ASP.NET page request.

Worth a try, IMHO,
Richard

--
C#, .NET and Complex Adaptive Systems:
http://blogs.geekdojo.net/Richard
"Justin Rogers" <Justin@games4dotnet.com> wrote in message
news:%23uCF0Jm4DHA.2656@tk2msftngp13.phx.gbl...[color=blue]
> For about six months we used it in the .NET Terrarium application. Until[/color]
the[color=blue]
> GC caught up with our expectations and we didn't need it anymore. During[/color]
that[color=blue]
> time there was a deterministic location we could call from (between game[/color]
ticks)[color=blue]
> that caused a low impact on our application at hand and so it worked[/color]
perfectly.[color=blue]
>
> In this guy's case, he has a couple of options:
> 1. Don't GC at all, and hope that the worker process doesn't role over
> constantly
> 2. Don't GC at all, and hope that the memory used on the array gets[/color]
re-used for[color=blue]
> the next array
> 3. GC if he finds that either 1 or 2 isn't coming true.
>
> So what I am saying is pick the lesser of several evils depending on his[/color]
own[color=blue]
> goals and
> inspection of what is happening on his servers. Generally speaking, #2[/color]
should[color=blue]
> come into
> play for him, since the memory should get re-used after it goes out of[/color]
scope.[color=blue]
> The largest problem
> I see is that he is allocating a huge array, and allocating huge arrays
> generally means finding
> contiguous locations in memory for said arrays. Because of these types of
> allocations in addition
> to more normal allocations he might find situations where the managed heap[/color]
grows[color=blue]
> uncontrollably (e.g.
> very fast in a short amount of time, causing the work process to role)
>
> --
> Justin Rogers
> DigiTec Web Consultants, LLC.
> Blog: http://weblogs.asp.net/justin_rogers
>
>
> "Richard A. Lowe" <chadich@yumspamyumYahoo.com> wrote in message
> news:exNZFGk4DHA.2528@TK2MSFTNGP09.phx.gbl...[color=green][color=darkred]
> > > and if they are high then you should manually control the GC since you[/color]
> > know the[color=darkred]
> > > best time to collect
> > > your objects and the GC only makes guesses.[/color]
> >
> > Justin, have you implemented and tested this in a real world app? I'm[/color][/color]
not[color=blue][color=green]
> > trying to be rhetorical - I'm really asking, since I've never had the
> > experience of using GC outside of simple test/diagnostic apps (and[/color][/color]
everytime[color=blue][color=green]
> > I have tried to use it other places, people smarter than I have taken me[/color][/color]
to[color=blue][color=green]
> > task for it :). It still seems risky to me to call the GC, perhaps[/color][/color]
even[color=blue][color=green]
> > every time this page fires, in light of the potential expense of GC in
> > ASP.NET especially - but I will defer to our resident gurus.
> >
> > One alternative for this kind of thing is running code in an AppDomain,[/color][/color]
then[color=blue][color=green]
> > unloading it, which I gave an example of doing here:
> > http://blogs.geekdojo.net/richard/ar...12/10/428.aspx
> >
> > This certainly has some expense to it, but seems to work well enough at
> > deterministically removing memory.
> >
> > Richard
> > --
> > C#, .NET and Complex Adaptive Systems:
> > http://blogs.geekdojo.net/Richard
> > "Justin Rogers" <Justin@games4dotnet.com> wrote in message
> > news:uZoi2vj4DHA.384@TK2MSFTNGP11.phx.gbl...[color=darkred]
> > > But you can control the GC...
> > >
> > > GC.Collect(GC.MaxGeneration); // forces a full collect
> > > GC.Collect(GC.GetGeneration(myArray)); // forces a collect up to the[/color]
> > generation[color=darkred]
> > > your array is in
> > >
> > > This isn't recommended, but if you are doing this calculation[/color]
> > simultaneously for[color=darkred]
> > > many users you may
> > > have a worker process roll-over (denial of service for some short[/color][/color][/color]
period[color=blue][color=green]
> > of[color=darkred]
> > > time) before the weight
> > > on the GC forces it to kick in and collect. You need to weight[/color][/color][/color]
roll-overs[color=blue][color=green]
> > using[color=darkred]
> > > the process counter
> > > and if they are high then you should manually control the GC since you[/color]
> > know the[color=darkred]
> > > best time to collect
> > > your objects and the GC only makes guesses.
> > >
> > >
> > > --
> > > Justin Rogers
> > > DigiTec Web Consultants, LLC.
> > > Blog: http://weblogs.asp.net/justin_rogers
> > >
> > >
> > > "Richard A. Lowe" <chadich@yumspamyumYahoo.com> wrote in message
> > > news:e0W83pj4DHA.360@TK2MSFTNGP12.phx.gbl...
> > > > All you have to do is make sure that the array is not being[/color][/color][/color]
referenced[color=blue][color=green]
> > in[color=darkred]
> > > > the Application or Session objects (that you're not purposely[/color][/color][/color]
keeping it[color=blue][color=green][color=darkred]
> > > > alive across page requests) and .NET will decide upon the best time[/color][/color][/color]
to[color=blue][color=green][color=darkred]
> > > > collect it. In .NET you don't explicitly control memory; check out[/color][/color][/color]
the[color=blue][color=green]
> > top[color=darkred]
> > > > hits for .NET memory management at Google:
> > > >[/color]
> >[/color][/color]
http://www.google.com/search?hl=en&i...ory+management[color=blue][color=green][color=darkred]
> > > > for in-depth explainations.
> > > >
> > > > Richard
> > > >
> > > > --
> > > > C#, .NET and Complex Adaptive Systems:
> > > > http://blogs.geekdojo.net/Richard
> > > > "tony collier" <melakka@hotmail.com> wrote in message
> > > > news:Xns947A30B5D948Bmelakkahotmailcom@140.99.99.1 30...
> > > > > This is an array question but it is based in my web page so i[/color][/color][/color]
shall[color=blue][color=green]
> > give[color=darkred]
> > > > > scenario:
> > > > >
> > > > > when the client navigates to the page, the page takes some figures[/color][/color][/color]
and[color=blue][color=green][color=darkred]
> > > > does
> > > > > loads of calulations in page_load which result in an array with[/color][/color][/color]
4.8[color=blue][color=green][color=darkred]
> > > > million
> > > > > elements. i then extract a mere handful which is all the client[/color][/color][/color]
needs[color=blue][color=green]
> > and[color=darkred]
> > > > > then return these results to the client. I would therefore like[/color][/color][/color]
to[color=blue][color=green]
> > delete[color=darkred]
> > > > > the array from memory at the end of the page_load event. Can[/color][/color][/color]
someone[color=blue][color=green][color=darkred]
> > > > > please tell me how to do this. thanks.
> > > >
> > > >
> > >
> > >[/color]
> >
> >[/color]
>
>[/color]


Closed Thread