Database question

If I setup a connection and the appropriate items in C# to link a queried
dataset that comes back with 100,000 results to a grid component on a
windows form, does that mean that my program just read 100,000 records over
the network at once? Or does the grid and dataset have built in knowledge
that allows them to retrieve a select few of the records and then retrieves
more as I scroll up and down through the grid?

Thanks,

glenn

re: Database question

glenn wrote:[color=blue]
> If I setup a connection and the appropriate items in C# to link a queried
> dataset that comes back with 100,000 results to a grid component on a
> windows form, does that mean that my program just read 100,000 records over
> the network at once? Or does the grid and dataset have built in knowledge
> that allows them to retrieve a select few of the records and then retrieves
> more as I scroll up and down through the grid?
>
> Thanks,
>
> glenn
>
>[/color]

considering the fact that the dataset is part of the
disconnected ado.net model, i would assume that all records
where transmitted over the network.

i would, however, reconsider my design strategies if my
application receives 100,000 results. what could possibly be
the reason for retrieving 100,000 records from a database
besides exporting them to a file?

is there no way you can reduce the number of records being
retrieved?

evert.

re: Database question

Well, let me explain:

1) Drop down lists that allow you to start typing a customer's phone and it
matches customers as you type. This is a feature easily accomplished in
Delphi as it only read over necessary records to keep things appearing as if
the drop down had 50,000 records in it, but in reality it was only bringing
over enough records to keep the 10 height of the drop down filled with
records.

2) Parts lists. We have a business application that will typically have
over 1 million records in the parts list. Our customers love our current
version of the application because they can scroll up and down in the parts
grid and look for things visually without having to do hundreds of searches
to try to find things. They also can start typing in a part number and the
system will immediately begin narrowing the view of the grid with numbers
that match their entry.

This is all accomplished without much data moving over the wire at all,
because Delphi allows you to read over only enough records to fill the grid.
Its also extremely fast as you can imagine.

This is a feature that allows you to accomplish things that customers love.
From what I'm seeing trying to accomplish such a task in C# and VS 2003 is
next to impossible but I wanted to be sure I understood what was going on as
no Book I've read so far has even bothered to mention how it was handling
things such as this.

A disconnected model works well for Internet applications but its my opinion
that its pretty slack for trying to provide customers with the types of
applications they require. I love C# but am constantly amazed at how
Microsoft could hire Borlands top developers from out from under them and
this is what they would design.

glenn


"Evert Timmer" <reply_to_group_only@newsgroup.com> wrote in message
news:OrtnbKcQFHA.1476@TK2MSFTNGP09.phx.gbl...[color=blue]
> glenn wrote:[color=green]
> > If I setup a connection and the appropriate items in C# to link a[/color][/color]
queried[color=blue][color=green]
> > dataset that comes back with 100,000 results to a grid component on a
> > windows form, does that mean that my program just read 100,000 records[/color][/color]
over[color=blue][color=green]
> > the network at once? Or does the grid and dataset have built in[/color][/color]
knowledge[color=blue][color=green]
> > that allows them to retrieve a select few of the records and then[/color][/color]
retrieves[color=blue][color=green]
> > more as I scroll up and down through the grid?
> >
> > Thanks,
> >
> > glenn
> >
> >[/color]
>
> considering the fact that the dataset is part of the
> disconnected ado.net model, i would assume that all records
> where transmitted over the network.
>
> i would, however, reconsider my design strategies if my
> application receives 100,000 results. what could possibly be
> the reason for retrieving 100,000 records from a database
> besides exporting them to a file?
>
> is there no way you can reduce the number of records being
> retrieved?
>
> evert.[/color]

re: Database question

Well said glenn,

I thought i am only person who thinks outside the box[.Net].
I know there are lot of people here reading this reply but cannot reply as
they don't have answer or they think this is stupid. Let me tell you
something, as you mentioned, Delphi supports loading data as you scroll "I
love that stuff" and Microsoft will never be able to do that rather tell you
that you stupid to download such large dataset and always want
developers/users to do things their way. I dont know how Borland does that
but if they can do I think anyone can do[Microsoft???].

we as developers need to display data to the user and make programs the way
they like. We should have the way to do things the way we[they] like, i know
its not good thing to download 100,000 but if I really have to then i should
be able to do that and .Net should support that.

Anyway, I am a Delphi and now C# developer even though i hate the fact that
i cannot do things the way Delphi does we still have to deal with it. I like
CSharp for its ease of coding but i still wish to see lot of improvements in
..Net.

Arun


"glenn" <ghancock@softeksoftware.com> wrote in message
news:%23GM1c2cQFHA.1096@tk2msftngp13.phx.gbl...[color=blue]
> Well, let me explain:
>
> 1) Drop down lists that allow you to start typing a customer's phone and
> it
> matches customers as you type. This is a feature easily accomplished in
> Delphi as it only read over necessary records to keep things appearing as
> if
> the drop down had 50,000 records in it, but in reality it was only
> bringing
> over enough records to keep the 10 height of the drop down filled with
> records.
>
> 2) Parts lists. We have a business application that will typically have
> over 1 million records in the parts list. Our customers love our current
> version of the application because they can scroll up and down in the
> parts
> grid and look for things visually without having to do hundreds of
> searches
> to try to find things. They also can start typing in a part number and
> the
> system will immediately begin narrowing the view of the grid with numbers
> that match their entry.
>
> This is all accomplished without much data moving over the wire at all,
> because Delphi allows you to read over only enough records to fill the
> grid.
> Its also extremely fast as you can imagine.
>
> This is a feature that allows you to accomplish things that customers
> love.
> From what I'm seeing trying to accomplish such a task in C# and VS 2003 is
> next to impossible but I wanted to be sure I understood what was going on
> as
> no Book I've read so far has even bothered to mention how it was handling
> things such as this.
>
> A disconnected model works well for Internet applications but its my
> opinion
> that its pretty slack for trying to provide customers with the types of
> applications they require. I love C# but am constantly amazed at how
> Microsoft could hire Borlands top developers from out from under them and
> this is what they would design.
>
> glenn
>
>
> "Evert Timmer" <reply_to_group_only@newsgroup.com> wrote in message
> news:OrtnbKcQFHA.1476@TK2MSFTNGP09.phx.gbl...[color=green]
>> glenn wrote:[color=darkred]
>> > If I setup a connection and the appropriate items in C# to link a[/color][/color]
> queried[color=green][color=darkred]
>> > dataset that comes back with 100,000 results to a grid component on a
>> > windows form, does that mean that my program just read 100,000 records[/color][/color]
> over[color=green][color=darkred]
>> > the network at once? Or does the grid and dataset have built in[/color][/color]
> knowledge[color=green][color=darkred]
>> > that allows them to retrieve a select few of the records and then[/color][/color]
> retrieves[color=green][color=darkred]
>> > more as I scroll up and down through the grid?
>> >
>> > Thanks,
>> >
>> > glenn
>> >
>> >[/color]
>>
>> considering the fact that the dataset is part of the
>> disconnected ado.net model, i would assume that all records
>> where transmitted over the network.
>>
>> i would, however, reconsider my design strategies if my
>> application receives 100,000 results. what could possibly be
>> the reason for retrieving 100,000 records from a database
>> besides exporting them to a file?
>>
>> is there no way you can reduce the number of records being
>> retrieved?
>>
>> evert.[/color]
>
>[/color]

re: Database question

What a shock. I can't believe I've found someone to side with me on Delphi
on this forum... :-)

Thanks for the support, but keep in mind that while Microsoft may say its
stupid to read over 100,000 records, that is not at all what we're doing.
We're only reading over records as the user needs more to fill the grid. So
while there may be a million records in the table, we probably on a typical
session would only ready 50 records across to supply what they are looking
at.

Granted if they held down the down arrow key and scrolled through all 1
million then it would eventually download all million, but that would be the
only time that would happen and who would want to sit that long holding a
key?

Thanks,

glenn

re: Database question

Oh, news to me that MS won't violate others patent. I hear database paging
method is implemented in .Net 2.0.






"Otis Mukinfus" <otis@mukinfus.com> wrote in message
news:mar0619a4chsj3jb9ccg65mjfocboimqns@4ax.com...[color=blue]
> On Fri, 15 Apr 2005 11:08:10 -0400, "glenn" <ghancock@softeksoftware.com>
> wrote:
>
> It's possible that Borland has a patent on the way they do this and MS
> can't
> violate their patent.
>
> I seem to remember reading a description of how to do this in Sceppa's
> ADO.NET
> book.
>[color=green]
>>Well, let me explain:
>>
>>1) Drop down lists that allow you to start typing a customer's phone and
>>it
>>matches customers as you type. This is a feature easily accomplished in
>>Delphi as it only read over necessary records to keep things appearing as
>>if
>>the drop down had 50,000 records in it, but in reality it was only
>>bringing
>>over enough records to keep the 10 height of the drop down filled with
>>records.
>>
>>2) Parts lists. We have a business application that will typically have
>>over 1 million records in the parts list. Our customers love our current
>>version of the application because they can scroll up and down in the
>>parts
>>grid and look for things visually without having to do hundreds of
>>searches
>>to try to find things. They also can start typing in a part number and
>>the
>>system will immediately begin narrowing the view of the grid with numbers
>>that match their entry.
>>
>>This is all accomplished without much data moving over the wire at all,
>>because Delphi allows you to read over only enough records to fill the
>>grid.
>>Its also extremely fast as you can imagine.
>>
>>This is a feature that allows you to accomplish things that customers
>>love.
>>From what I'm seeing trying to accomplish such a task in C# and VS 2003 is
>>next to impossible but I wanted to be sure I understood what was going on
>>as
>>no Book I've read so far has even bothered to mention how it was handling
>>things such as this.
>>
>>A disconnected model works well for Internet applications but its my
>>opinion
>>that its pretty slack for trying to provide customers with the types of
>>applications they require. I love C# but am constantly amazed at how
>>Microsoft could hire Borlands top developers from out from under them and
>>this is what they would design.
>>
>>glenn
>>
>>
>>"Evert Timmer" <reply_to_group_only@newsgroup.com> wrote in message
>>news:OrtnbKcQFHA.1476@TK2MSFTNGP09.phx.gbl...[color=darkred]
>>> glenn wrote:
>>> > If I setup a connection and the appropriate items in C# to link a[/color]
>>queried[color=darkred]
>>> > dataset that comes back with 100,000 results to a grid component on a
>>> > windows form, does that mean that my program just read 100,000 records[/color]
>>over[color=darkred]
>>> > the network at once? Or does the grid and dataset have built in[/color]
>>knowledge[color=darkred]
>>> > that allows them to retrieve a select few of the records and then[/color]
>>retrieves[color=darkred]
>>> > more as I scroll up and down through the grid?
>>> >
>>> > Thanks,
>>> >
>>> > glenn
>>> >
>>> >
>>>
>>> considering the fact that the dataset is part of the
>>> disconnected ado.net model, i would assume that all records
>>> where transmitted over the network.
>>>
>>> i would, however, reconsider my design strategies if my
>>> application receives 100,000 results. what could possibly be
>>> the reason for retrieving 100,000 records from a database
>>> besides exporting them to a file?
>>>
>>> is there no way you can reduce the number of records being
>>> retrieved?
>>>
>>> evert.[/color]
>>[/color]
>
> Otis Mukinfus
> http://www.otismukinfus.com[/color]

re: Database question

There is no patent here I do not believe. I think Microsoft is so focused
on disconnected architectures that it just hasn't thought about anything
like this. There are a lot of signs of non thinking in VS 2003 compared to
Delphi. Hopefully 2005 VS will fix them though...

glenn


"Otis Mukinfus" <otis@mukinfus.com> wrote in message
news:mar0619a4chsj3jb9ccg65mjfocboimqns@4ax.com...[color=blue]
> On Fri, 15 Apr 2005 11:08:10 -0400, "glenn" <ghancock@softeksoftware.com>[/color]
wrote:[color=blue]
>
> It's possible that Borland has a patent on the way they do this and MS[/color]
can't[color=blue]
> violate their patent.
>
> I seem to remember reading a description of how to do this in Sceppa's[/color]
ADO.NET[color=blue]
> book.
>[color=green]
> >Well, let me explain:
> >
> >1) Drop down lists that allow you to start typing a customer's phone and[/color][/color]
it[color=blue][color=green]
> >matches customers as you type. This is a feature easily accomplished in
> >Delphi as it only read over necessary records to keep things appearing as[/color][/color]
if[color=blue][color=green]
> >the drop down had 50,000 records in it, but in reality it was only[/color][/color]
bringing[color=blue][color=green]
> >over enough records to keep the 10 height of the drop down filled with
> >records.
> >
> >2) Parts lists. We have a business application that will typically have
> >over 1 million records in the parts list. Our customers love our current
> >version of the application because they can scroll up and down in the[/color][/color]
parts[color=blue][color=green]
> >grid and look for things visually without having to do hundreds of[/color][/color]
searches[color=blue][color=green]
> >to try to find things. They also can start typing in a part number and[/color][/color]
the[color=blue][color=green]
> >system will immediately begin narrowing the view of the grid with numbers
> >that match their entry.
> >
> >This is all accomplished without much data moving over the wire at all,
> >because Delphi allows you to read over only enough records to fill the[/color][/color]
grid.[color=blue][color=green]
> >Its also extremely fast as you can imagine.
> >
> >This is a feature that allows you to accomplish things that customers[/color][/color]
love.[color=blue][color=green]
> >From what I'm seeing trying to accomplish such a task in C# and VS 2003[/color][/color]
is[color=blue][color=green]
> >next to impossible but I wanted to be sure I understood what was going on[/color][/color]
as[color=blue][color=green]
> >no Book I've read so far has even bothered to mention how it was handling
> >things such as this.
> >
> >A disconnected model works well for Internet applications but its my[/color][/color]
opinion[color=blue][color=green]
> >that its pretty slack for trying to provide customers with the types of
> >applications they require. I love C# but am constantly amazed at how
> >Microsoft could hire Borlands top developers from out from under them and
> >this is what they would design.
> >
> >glenn
> >
> >
> >"Evert Timmer" <reply_to_group_only@newsgroup.com> wrote in message
> >news:OrtnbKcQFHA.1476@TK2MSFTNGP09.phx.gbl...[color=darkred]
> >> glenn wrote:
> >> > If I setup a connection and the appropriate items in C# to link a[/color]
> >queried[color=darkred]
> >> > dataset that comes back with 100,000 results to a grid component on a
> >> > windows form, does that mean that my program just read 100,000[/color][/color][/color]
records[color=blue][color=green]
> >over[color=darkred]
> >> > the network at once? Or does the grid and dataset have built in[/color]
> >knowledge[color=darkred]
> >> > that allows them to retrieve a select few of the records and then[/color]
> >retrieves[color=darkred]
> >> > more as I scroll up and down through the grid?
> >> >
> >> > Thanks,
> >> >
> >> > glenn
> >> >
> >> >
> >>
> >> considering the fact that the dataset is part of the
> >> disconnected ado.net model, i would assume that all records
> >> where transmitted over the network.
> >>
> >> i would, however, reconsider my design strategies if my
> >> application receives 100,000 results. what could possibly be
> >> the reason for retrieving 100,000 records from a database
> >> besides exporting them to a file?
> >>
> >> is there no way you can reduce the number of records being
> >> retrieved?
> >>
> >> evert.[/color]
> >[/color]
>
> Otis Mukinfus
> http://www.otismukinfus.com[/color]

re: Database question

Hi,
I have experienced the same performance issues with a listbox connected
to a dataset cotaining a couple of thousands of records. I was thinking
about loading only let's say 40-50 records at the beginning (calling the
Fill() function with number of records as parameter), this would allow
the user to see almost imediately something on the screen. After that I
would start a thread that continues filling the dataset in background.
Is this possible to be impemented in .NET ? Do I have any
restrictions/conflicts at accessing from another thread the dataset
connected to a listbox in a Windows.Form, created in the main() function
? I would put the code of the thread in the same namespace as the main
form, to be able to access all necessary variables. Is this technically
possible ? Or is there some better way to solve this performance problem
(I mean with C# of course, other languages are not a issue for me).
Thanks in advance,
Abra

*** Sent via Developersdex http://www.developersdex.com ***

re: Database question

Actually yes, you could experience a problem trying to do it this way. A
better approach would be to handle it like Delphi does. As you reach the
end of the current record set, it goes out and grabs another 10 or 20
records to continue scrolling through. If the user begins to type, it will
retrieve records to match what they had typed so far so that the list is
always full but never has more than a few records in it at any one time.

What VS is doing is useless in my opinion and takes me back to the 1990's.
I can't believe that they did not spend more time thinking this sort of
thing out. This one thing is going to cause me to have to abandon VS
altogether if I can't find some way to work through it.

Good luck,

glenn

"Abra" <nospam@devdex.com> wrote in message
news:ulEMkvSRFHA.3476@TK2MSFTNGP10.phx.gbl...[color=blue]
> Hi,
> I have experienced the same performance issues with a listbox connected
> to a dataset cotaining a couple of thousands of records. I was thinking
> about loading only let's say 40-50 records at the beginning (calling the
> Fill() function with number of records as parameter), this would allow
> the user to see almost imediately something on the screen. After that I
> would start a thread that continues filling the dataset in background.
> Is this possible to be impemented in .NET ? Do I have any
> restrictions/conflicts at accessing from another thread the dataset
> connected to a listbox in a Windows.Form, created in the main() function
> ? I would put the code of the thread in the same namespace as the main
> form, to be able to access all necessary variables. Is this technically
> possible ? Or is there some better way to solve this performance problem
> (I mean with C# of course, other languages are not a issue for me).
> Thanks in advance,
> Abra
>
> *** Sent via Developersdex http://www.developersdex.com ***[/color]