473,698 Members | 2,403 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

SPEED QUESTION

In my form load()
Dim daSeaInvoice As New SqlDataAdapter( "select * from InvoiceHeader ;select
* from invoicedetail, conSea)
dim myDataSet as new dataset()
daSeaInvoice.fi ll(mydataset).

As I got over 50000 records in invoiceheader and over 200000 record in
invoicedetail
It seems quit slow to load the form...


Nov 20 '05 #1
11 1126
"Agnes" <ag***@dynamict ech.com.hk> wrote in news:OxcbxEedEH A.3476
@tk2msftngp13.p hx.gbl:
As I got over 50000 records in invoiceheader and over 200000 record in
invoicedetail
It seems quit slow to load the form...


Becuase you're loading 50,000 records into a dataset... it's sucking up all
your memory.

I wouldn't use a dataset if there are >20,000 rows.

Use a datareader instead.

--
Lucas Tam (RE********@rog ers.com)
Please delete "REMOVE" from the e-mail address when replying.
http://members.ebay.com/aboutme/coolspot18/
Nov 20 '05 #2
I'll just bet that it does seem quite slow. You're bringing over a ton of
data, little of which the user is ever actually going to be looking at.
So...how about not loading the whole works all at once? Maybe you might load
just the most recent data, perhaps enough to fill your grid or whatever
you're using, and then wait until your user wants to 'page back' or do a
filtered search on records, using some user-specified criteria. Then go back
and load the data of interest. I know it's a little more work, but nothing
comes for free.

No offense intended, but this is kind of like those people who add 100,000
items to a drop-down list box and wonder why it doesn't work too pretty
good.

Trying hard not to be irritable and not quite succeeding,
Tom Dacon
Dacon Software Consulting

"Agnes" <ag***@dynamict ech.com.hk> wrote in message
news:Ox******** ******@tk2msftn gp13.phx.gbl...
In my form load()
Dim daSeaInvoice As New SqlDataAdapter( "select * from InvoiceHeader ;select * from invoicedetail, conSea)
dim myDataSet as new dataset()
daSeaInvoice.fi ll(mydataset).

As I got over 50000 records in invoiceheader and over 200000 record in
invoicedetail
It seems quit slow to load the form...

Nov 20 '05 #3
this time I understand . But I got another question. in my Invoiceheader,I
can load the current month's data
but .. in my invoicedetail, there is no field to store the"date"
How can I select from invoicedetail with the "date condition" ....

Dim daSeaInvoice As New SqlDataAdapter( "select * from InvoiceHeader from
month = currentmonth;se lect * from invoicedetail where ???, conSea)
thanks

"Tom Dacon" <td****@communi ty.nospam> ¦b¶l¥ó
news:ui******** *****@TK2MSFTNG P11.phx.gbl ¤¤¼¶¼g...
I'll just bet that it does seem quite slow. You're bringing over a ton of
data, little of which the user is ever actually going to be looking at.
So...how about not loading the whole works all at once? Maybe you might load just the most recent data, perhaps enough to fill your grid or whatever
you're using, and then wait until your user wants to 'page back' or do a
filtered search on records, using some user-specified criteria. Then go back and load the data of interest. I know it's a little more work, but nothing
comes for free.

No offense intended, but this is kind of like those people who add 100,000
items to a drop-down list box and wonder why it doesn't work too pretty
good.

Trying hard not to be irritable and not quite succeeding,
Tom Dacon
Dacon Software Consulting

"Agnes" <ag***@dynamict ech.com.hk> wrote in message
news:Ox******** ******@tk2msftn gp13.phx.gbl...
In my form load()
Dim daSeaInvoice As New SqlDataAdapter( "select * from InvoiceHeader

;select
* from invoicedetail, conSea)
dim myDataSet as new dataset()
daSeaInvoice.fi ll(mydataset).

As I got over 50000 records in invoiceheader and over 200000 record in
invoicedetail
It seems quit slow to load the form...


Nov 20 '05 #4
Making a guess here, since I don't know your schema, and no guarantees
without compiling and running it, but how about something like:

Dim daSeaInvoice As New SqlDataAdapter(
"select * from InvoiceHeader where month = currentmonth;
select * from invoicedetail where Invoicedetail.H eaderKey in (select
InvoiceHeader.P rimaryKey where month = currentmonth),
conSea)
thanks
The double select on InvoiceHeader kind of hurts if you do it as an ad hoc
query, but it might not be so bad as a stored procedure with month as the
parameter. The stored procedure could probably re-use the first result set's
primary keys instead of running another query.

Tom Dacon
Dacon Software Consulting

"Agnes" <ag***@dynamict ech.com.hk> wrote in message
news:%2******** ********@TK2MSF TNGP12.phx.gbl. .. this time I understand . But I got another question. in my Invoiceheader,I
can load the current month's data
but .. in my invoicedetail, there is no field to store the"date"
How can I select from invoicedetail with the "date condition" ....

Dim daSeaInvoice As New SqlDataAdapter( "select * from InvoiceHeader from
month = currentmonth;se lect * from invoicedetail where ???, conSea)
thanks

"Tom Dacon" <td****@communi ty.nospam> ¦b¶l¥ó
news:ui******** *****@TK2MSFTNG P11.phx.gbl ¤¤¼¶¼g...
I'll just bet that it does seem quite slow. You're bringing over a ton of data, little of which the user is ever actually going to be looking at.
So...how about not loading the whole works all at once? Maybe you might

load
just the most recent data, perhaps enough to fill your grid or whatever
you're using, and then wait until your user wants to 'page back' or do a
filtered search on records, using some user-specified criteria. Then go

back
and load the data of interest. I know it's a little more work, but nothing comes for free.

No offense intended, but this is kind of like those people who add 100,000 items to a drop-down list box and wonder why it doesn't work too pretty
good.

Trying hard not to be irritable and not quite succeeding,
Tom Dacon
Dacon Software Consulting

"Agnes" <ag***@dynamict ech.com.hk> wrote in message
news:Ox******** ******@tk2msftn gp13.phx.gbl...
In my form load()
Dim daSeaInvoice As New SqlDataAdapter( "select * from InvoiceHeader

;select
* from invoicedetail, conSea)
dim myDataSet as new dataset()
daSeaInvoice.fi ll(mydataset).

As I got over 50000 records in invoiceheader and over 200000 record in
invoicedetail
It seems quit slow to load the form...



Nov 20 '05 #5
Hi Agnes,

Can you tell us what is the reason you load that bunch of data.

Keep in mind that a dataset is disconnected so when you use this as well for
updating you will get probably more and more overhead and especially
concurrency problems (when it is a multiuser environment).

Cor
Nov 20 '05 #6
I don't want to load so much data, I am starting my 3rd .net form
Every textbook teach us 'select * from myTable" and the
daInvocie.fill( dsInvoice)
It seems "load all the data first" ,so i am questioned about it.

"Cor Ligthert" <no**********@p lanet.nl> ¦b¶l¥ó
news:%2******** ********@TK2MSF TNGP10.phx.gbl ¤¤¼¶¼g...
Hi Agnes,

Can you tell us what is the reason you load that bunch of data.

Keep in mind that a dataset is disconnected so when you use this as well for updating you will get probably more and more overhead and especially
concurrency problems (when it is a multiuser environment).

Cor

Nov 20 '05 #7
Hi Agnes,

Do it than directly in the right way.

"Select myclientadres, myclientname from invoice
where myclientId = @ myclientId"

And than use the SQLparameters

http://msdn.microsoft.com/library/de...classtopic.asp

I hope this helps?

Cor
Nov 20 '05 #8
Bob
Almost always when presenting data you will have to supply two essential
elements to your user interface: header search and detail presentation.
Supply criteria to the user to search through all those invoice headers,
like date, open/closed status, vendor, etc. (I typically supply a default
condition of showing only the top 300 rows if no search condition is
provided, sorting by create date descending when applicable). When the user
clicks on a row, have another section of your form load the invoice details
for that particular header.

If you still want to show invoice details based on a search (and not an
individual header), the query should look something like this:

SELECT TOP 300 d.*
FROM InvoiceHeader h
INNER JOIN InvoiceDetail d
ON d.InvoiceID = h.InvoiceID
WHERE
h.InvoiceDate = <your date variable>
AND <any other search conditions>
ORDER BY h.InvoiceID, d.InvoiceLineNu mber

I put in the TOP 300 because users don't want to page through a large result
set - that's what a search feature is for. You can tell your users in a note
on the form that the result set has been truncated when necessary ("only the
top 300 search hits shown"). If a user really does want to see all 200,000
rows, this is a report, something to be printed; not a form. As such I never
user a datareader, because my queries never return large result sets.

As a side note, I don't recommend having any SQL at all in compiled code,
use stored procedures if you can; they supply a useful layer of abstraction,
boost performance, and allow you to make DB changes without having to
recompile your deployment. Also, take some time to buy a book or two on SQL
and really learn it. It will really make a big difference.

HTH,
Bob

"Agnes" <ag***@dynamict ech.com.hk> wrote in message
news:%2******** ********@TK2MSF TNGP12.phx.gbl. ..
this time I understand . But I got another question. in my Invoiceheader,I
can load the current month's data
but .. in my invoicedetail, there is no field to store the"date"
How can I select from invoicedetail with the "date condition" ....

Dim daSeaInvoice As New SqlDataAdapter( "select * from InvoiceHeader from
month = currentmonth;se lect * from invoicedetail where ???, conSea)
thanks

"Tom Dacon" <td****@communi ty.nospam> ¦b¶l¥ó
news:ui******** *****@TK2MSFTNG P11.phx.gbl ¤¤¼¶¼g...
I'll just bet that it does seem quite slow. You're bringing over a ton of data, little of which the user is ever actually going to be looking at.
So...how about not loading the whole works all at once? Maybe you might

load
just the most recent data, perhaps enough to fill your grid or whatever
you're using, and then wait until your user wants to 'page back' or do a
filtered search on records, using some user-specified criteria. Then go

back
and load the data of interest. I know it's a little more work, but nothing comes for free.

No offense intended, but this is kind of like those people who add 100,000 items to a drop-down list box and wonder why it doesn't work too pretty
good.

Trying hard not to be irritable and not quite succeeding,
Tom Dacon
Dacon Software Consulting

"Agnes" <ag***@dynamict ech.com.hk> wrote in message
news:Ox******** ******@tk2msftn gp13.phx.gbl...
In my form load()
Dim daSeaInvoice As New SqlDataAdapter( "select * from InvoiceHeader

;select
* from invoicedetail, conSea)
dim myDataSet as new dataset()
daSeaInvoice.fi ll(mydataset).

As I got over 50000 records in invoiceheader and over 200000 record in
invoicedetail
It seems quit slow to load the form...




Nov 20 '05 #9
I have had a similar problem before and used the overloaded dataAdapter.fil l method. You can tell the fill method the starting record and how many records you would like to return.

I use this all the time to show **LOTS** of data to the user, when the user gets close to the end of the data I have loaded I go and pull more data from the DB.

OleDbDataAdapte r1.Fill(DsDataS et1, intStartRec, intNumRecs, "MainDB")

Hardest part of this is figuring out where the user is and loading the appropriate records but still not that hard and it allows you to configure the number of records you are dealing with if performance becomes a problem. I.E. intNumRecs = 1000 or intNumRecs = 10

Regards,
Justin

"Agnes" wrote:
In my form load()
Dim daSeaInvoice As New SqlDataAdapter( "select * from InvoiceHeader ;select
* from invoicedetail, conSea)
dim myDataSet as new dataset()
daSeaInvoice.fi ll(mydataset).

As I got over 50000 records in invoiceheader and over 200000 record in
invoicedetail
It seems quit slow to load the form...


Nov 20 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

13
23057
by: Yang Li Ke | last post by:
Hi guys, Is it possible to know the internet speed of the visitors with php? Thanx -- Yang
34
2461
by: Jacek Generowicz | last post by:
I have a program in which I make very good use of a memoizer: def memoize(callable): cache = {} def proxy(*args): try: return cache except KeyError: return cache.setdefault(args, callable(*args)) return proxy which, is functionally equivalent to
28
2593
by: Maboroshi | last post by:
Hi I am fairly new to programming but not as such that I am a total beginner From what I understand C and C++ are faster languages than Python. Is this because of Pythons ability to operate on almost any operating system? Or is there many other reasons why? I understand there is ansi/iso C and C++ and that ANSI/ISO Code will work on any system If this is the reason why, than why don't developers create specific Python Distrubutions...
11
2729
by: Fred Bennett | last post by:
I have a simulation project in which data can naturally be held in structures for processing. There are calls to multiple functions involved. Execution speed is an issue. Do I take a big hit for usuing structures? Is there a preferred way of passing the structures (or pointers?) that would speed up the program? I'm relatively new to C++. so, if you have time, detail (or references) would be appreciated. In any case any help is much...
2
1178
by: aaron | last post by:
Hello, I have two questions, first some background info: We are running SQL 2000 on a Windows 2003 server for out aircraft parts database software. There are 7 workstations (windows 2000 and XP) that run the parts database.
11
9206
by: Sezai YILMAZ | last post by:
Hello I need high throughput while inserting into PostgreSQL. Because of that I did some PostgreSQL insert performance tests. ------------------------------------------------------------ -- Test schema create table logs ( logid serial primary key, ctime integer not null,
45
2894
by: charles.lobo | last post by:
Hi, I have recently begun using templates in C++ and have found it to be quite useful. However, hearing stories of code bloat and assorted problems I decided to write a couple of small programs to check. What I expected was that there would be minor code bloat and some speed improvement when using templates. However... I wrote a basic list container (using templates), and a list container (using virtual derived classes). I also tried...
1
1448
by: =?ISO-8859-15?Q?Ma=EBl_Benjamin_Mettler?= | last post by:
Hello Python-List I hope somebody can help me with this. I spent some time googling for an answer, but due to the nature of the problem lots of unrelevant stuff shows up. Anyway, I reimplemented parts of TigerSearch ( http://www.ims.uni-stuttgart.de/projekte/TIGER/TIGERSearch/ ) in Python. I am currently writing the paper that goes along with this reimplementation. Part of the paper deals with the
4
12859
by: halimunan | last post by:
Hello all, I have a question, how do i measure the download/upload speed... as if download rate, upload rate or kbps transfer rate. i want to do my own windows application using C# to measure the download speed. the idea is this. 1. i will put a file at a serverA. 2. i download the file 3. i measure the speed of the download(how long it would take me to complete the download)
0
1238
by: JuAn2226 | last post by:
hi this my code Private Sub Form_Load() car_count = 0 Cumulative_Speed = 0 End Sub Private Sub Timer1_Timer() Dim tmpNumber As Integer
0
8680
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8871
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7738
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6528
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5861
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4371
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4622
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
2335
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2007
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.