Application performance decrease after deleting rows

[NAdir]

Hi, thank you for your help.
My VB.Net application contains a document that the user can refresh at any
time.
The refresh works fine and needs to loop through few datatables (hundreds of
rows). This works fine until I delete some rows in two tables. Just after the
delete if I do the refresh there is a huge memory allocated and the time
needed to perform the refresh increase, the memory and time continue to
increase on each refresh until I save and force garbage collector, after the
GC the memory return to its normal size. I cheked using analysis tool and the
rows that allocated huge memory are 'select' commands and I have no idea how
to solve this problem (the only choice I have is to save and force GC). My
dataset contains only about 20 datatables and few hundreds of rows. I guess
there is a problem with the select after the delete it seems allocating huge
memory.
Thanks

[Cor Ligthert [MVP]]

Nadir,

Every time there are a lot of messages about the taskmanager showing a hugh
amount of memory.

Be aware that the taskmanager is not correct giving memory, while on the
other hand there is no need to be afraid as there is still enough memory.

If you really want to investigate this for whatever reason, than use other
tools. If you search the newsgroup dotnet general for that you find tons of
messages about this.

Cor

"NAdir" <NA***@discussions.microsoft.comschreef in bericht
news:C4**********************************@microsof t.com...

Hi, thank you for your help.
My VB.Net application contains a document that the user can refresh at any
time.
The refresh works fine and needs to loop through few datatables (hundreds
of
rows). This works fine until I delete some rows in two tables. Just after
the
delete if I do the refresh there is a huge memory allocated and the time
needed to perform the refresh increase, the memory and time continue to
increase on each refresh until I save and force garbage collector, after
the
GC the memory return to its normal size. I cheked using analysis tool and
the
rows that allocated huge memory are 'select' commands and I have no idea
how
to solve this problem (the only choice I have is to save and force GC). My
dataset contains only about 20 datatables and few hundreds of rows. I
guess
there is a problem with the select after the delete it seems allocating
huge
memory.
Thanks

[Hugh Janus]

On Nov 14, 5:53 am, "Cor Ligthert [MVP]" <[e-mail]>
wrote:

Nadir,

Every time there are a lot of messages about the taskmanager showing a hugh
amount of memory.

Be aware that the taskmanager is not correct giving memory, while on the
other hand there is no need to be afraid as there is still enough memory.

If you really want to investigate this for whatever reason, than use other
tools. If you search the newsgroup dotnet general for that you find tons of
messages about this.

Cor, explain why you think task manager does not correctly report
memory usage. If you compare it to perfmon you'll see that it does.
Perhaps the problem is that you do not know how to interpret the
information displayed.

[jeff]

or the question asked ...

the OP asked for possible solutions as to why the delete causes lags and
increased memory usage when using a refresh ... not does the task manager
report accurately ... and as far as I know ... the task managers works just
fine for this!.
"Hugh Janus" <my*************@hotmail.comwrote in message
news:11**********************@k70g2000cwa.googlegr oups.com...

On Nov 14, 5:53 am, "Cor Ligthert [MVP]" <[e-mail]>
wrote:

>Nadir,

Every time there are a lot of messages about the taskmanager showing a
hugh
amount of memory.

Be aware that the taskmanager is not correct giving memory, while on the
other hand there is no need to be afraid as there is still enough memory.

If you really want to investigate this for whatever reason, than use
other
tools. If you search the newsgroup dotnet general for that you find tons
of
messages about this.

Cor, explain why you think task manager does not correctly report
memory usage. If you compare it to perfmon you'll see that it does.
Perhaps the problem is that you do not know how to interpret the
information displayed.

[NAdir]

There is a huge memory allocated after deleting rows and executing select
command, this is not normal because not only memory increase in subsequent
refreshs but also the execution time increase. I checked with dev partner
studio and found that I need to save my document (including the data set)
then execute a GC.collect, this will bring the memory to its normal size.
Executing CG.collect without prior save does not free memory. I suspect
something with ADO .NET commands (what happens after executing delete rows
then select).

"jeff" wrote:

or the question asked ...

the OP asked for possible solutions as to why the delete causes lags and
increased memory usage when using a refresh ... not does the task manager
report accurately ... and as far as I know ... the task managers works just
fine for this!.
"Hugh Janus" <my*************@hotmail.comwrote in message
news:11**********************@k70g2000cwa.googlegr oups.com...

On Nov 14, 5:53 am, "Cor Ligthert [MVP]" <[e-mail]>
wrote:

Nadir,

Every time there are a lot of messages about the taskmanager showing a
hugh
amount of memory.

Be aware that the taskmanager is not correct giving memory, while on the
other hand there is no need to be afraid as there is still enough memory.

If you really want to investigate this for whatever reason, than use
other
tools. If you search the newsgroup dotnet general for that you find tons
of
messages about this.

Cor, explain why you think task manager does not correctly report
memory usage. If you compare it to perfmon you'll see that it does.
Perhaps the problem is that you do not know how to interpret the
information displayed.

[Cor Ligthert [MVP]]

A pity Jeff, that it is a very known issue.

http://groups-beta.google.com/group/...rch+this+group

Cor

"jeff" <jhersey at allnorth dottt comschreef in bericht
news:O4**************@TK2MSFTNGP03.phx.gbl...

or the question asked ...

the OP asked for possible solutions as to why the delete causes lags and
increased memory usage when using a refresh ... not does the task manager
report accurately ... and as far as I know ... the task managers works
just fine for this!.
"Hugh Janus" <my*************@hotmail.comwrote in message
news:11**********************@k70g2000cwa.googlegr oups.com...

>On Nov 14, 5:53 am, "Cor Ligthert [MVP]" <[e-mail]>
wrote:

>>Nadir,

Every time there are a lot of messages about the taskmanager showing a
hugh
amount of memory.

Be aware that the taskmanager is not correct giving memory, while on the
other hand there is no need to be afraid as there is still enough
memory.

If you really want to investigate this for whatever reason, than use
other
tools. If you search the newsgroup dotnet general for that you find tons
of
messages about this.

Cor, explain why you think task manager does not correctly report
memory usage. If you compare it to perfmon you'll see that it does.
Perhaps the problem is that you do not know how to interpret the
information displayed.

[TP]

You didn't mention which version of the framework you are
using. Check this article to see if it applies to you:

FIX: The DataTable.Select method leaks memory from the index
when you set some specific filter properties

http://support.microsoft.com/kb/814470

Please post your code so that we can see what you are doing.

Also, be aware that a datatable.select creates indexes as needed
on the fly, and in .NET 2.0 it discards them after the call completes.
If it needs to create indexes each time this will lead to slower
performance and increased memory pressure.

You may want to experiment with using a dataview instead.

Thanks.

-TP

NAdir wrote:

There is a huge memory allocated after deleting rows and executing
select command, this is not normal because not only memory increase
in subsequent refreshs but also the execution time increase. I
checked with dev partner studio and found that I need to save my
document (including the data set) then execute a GC.collect, this
will bring the memory to its normal size. Executing CG.collect
without prior save does not free memory. I suspect something with ADO
.NET commands (what happens after executing delete rows then select).

[TP]

Many of the past posts regarding the task manager can
be misleading. People could get the false impression
that task manager is "defective" or that the numbers it
shows are "meaningless".

It seems to me that things in the group have degraded to:

1. Person asks question regarding leak or possible
high memory use/performance problem

2. Knee-jerk response of "Are you looking at Task
Manager--forget that, it is wrong anyway. Besides,
today's machines have tons of RAM, who cares?"

This is not a good approach, because high memory
use and/or leaks can cause performance problems and
errors, even on machines with relatively large amounts
of RAM. It is important to remember that an app
may be used under terminal services, in which case
sloppy memory/resource use issues are multiplied.

Task manager displays a number of performance
counters in an easy to use format. It displays the correct
values of these performance counters, just as it did *long*
before the first .NET application existed.

What it doesn't show, and never has shown, is the amount
of memory sharing that is occuring, how much of allocated
memory has recently been touched by the process, the
minimum working set that each process requires to perform
well, how much of allocated memory is resident, etc.

Task manager can be very helpful in detecting substantial
memory leaks in .NET apps. For example, it is easy to
create a winforms app with a leak and then use taskman to
view an ever-increasing amount of private bytes (VM Size)
allocated. It is not helpful for very small leaks, for example,
4 bytes every once in a while.

I would strongly recommend that you and anyone else who
would like to understand memory management under windows
read chapter 7 of Windows Internals, by Mark Russinovich &
David Solomon:

http://www.amazon.com/Microsoft-Wind...dp/0735619174/

-TP

Cor Ligthert [MVP] wrote:

A pity Jeff, that it is a very known issue.

http://groups-beta.google.com/group/...rch+this+group

Cor

[NAdir]

Thank you TP,
I could not publish my code because it is long and I use the select is
multiple
subroutines. Your suggestion about the problem with SELECT index maybe the
right problem. I was thinking about a problem with the SELECT OR DELETE
COMMANDS. I will experiement and publish the code. As I described in previous
post: using devPartner software, the test shown a problem with the select
after deleting datatable rows, the memory increase about 20 Mb and time of
executing next selects increases, only after executing accpet changes on the
dataset and GC.collect the memory and time become normal. Thanks again this
was a big issue for my application performance.

"TP" wrote:

Many of the past posts regarding the task manager can
be misleading. People could get the false impression
that task manager is "defective" or that the numbers it
shows are "meaningless".

It seems to me that things in the group have degraded to:

1. Person asks question regarding leak or possible
high memory use/performance problem

2. Knee-jerk response of "Are you looking at Task
Manager--forget that, it is wrong anyway. Besides,
today's machines have tons of RAM, who cares?"

This is not a good approach, because high memory
use and/or leaks can cause performance problems and
errors, even on machines with relatively large amounts
of RAM. It is important to remember that an app
may be used under terminal services, in which case
sloppy memory/resource use issues are multiplied.

Task manager displays a number of performance
counters in an easy to use format. It displays the correct
values of these performance counters, just as it did *long*
before the first .NET application existed.

What it doesn't show, and never has shown, is the amount
of memory sharing that is occuring, how much of allocated
memory has recently been touched by the process, the
minimum working set that each process requires to perform
well, how much of allocated memory is resident, etc.

Task manager can be very helpful in detecting substantial
memory leaks in .NET apps. For example, it is easy to
create a winforms app with a leak and then use taskman to
view an ever-increasing amount of private bytes (VM Size)
allocated. It is not helpful for very small leaks, for example,
4 bytes every once in a while.

I would strongly recommend that you and anyone else who
would like to understand memory management under windows
read chapter 7 of Windows Internals, by Mark Russinovich &
David Solomon:

http://www.amazon.com/Microsoft-Wind...dp/0735619174/

-TP

Cor Ligthert [MVP] wrote:

A pity Jeff, that it is a very known issue.

http://groups-beta.google.com/group/...rch+this+group

Cor