473,769 Members | 6,473 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Using Dataset for large amounts of data

Hi,
I'm wondering if it is good to use datasets for large amounts of data with
many users. I'm talking tables with 130,000 records and 15 columns. And we
want current data, so no cached data. Right now we are using an
OleDbDataReader and then just doing reader.Read() to go through the
necessary records. So would using a dataset be good, or bad for this
compared to what we are doing?

We're looking at it again to get the features of transactions out of it, but
are not sure if they can handle that much data very well?
Nov 18 '05 #1
3 2274
Holy cow!! Why would you bring back that much data? First thing I would
do, regardless of what you are trying to do, is find a way to greatly limit
that number of records you are returning somehow. Do your records have a
unique key of some sort? And are they ordered somehow, maybe by the unique
key (that would be prefered)?

DataReaders will perform faster than datasets. DataSets are disconnected
from the database and must store all their information in memory.
DataReaders remain connected to the database and pull records from the
database as you request them (i.e. reader.Read()).

HTH,

Raymond Lewallen

"Brent" <b@b.com> wrote in message
news:10******** *****@corp.supe rnews.com...
Hi,
I'm wondering if it is good to use datasets for large amounts of data with
many users. I'm talking tables with 130,000 records and 15 columns. And we
want current data, so no cached data. Right now we are using an
OleDbDataReader and then just doing reader.Read() to go through the
necessary records. So would using a dataset be good, or bad for this
compared to what we are doing?

We're looking at it again to get the features of transactions out of it, but are not sure if they can handle that much data very well?

Nov 18 '05 #2
Cor
Hi Raymond,

I agree completly with the first part of your message. With the second part
I do not (completly).
Holy cow!! Why would you bring back that much data? First thing I would
do, regardless of what you are trying to do, is find a way to greatly limit that number of records you are returning somehow. Do your records have a
unique key of some sort? And are they ordered somehow, maybe by the unique key (that would be prefered)? DataReaders will perform faster than datasets. DataSets are disconnected
from the database and must store all their information in memory.
DataReaders remain connected to the database and pull records from the
database as you request them (i.e. reader.Read()).


It is not said that DataReaders perform faster than using the dataadapter
which uses also the datareader.

Main thing is that it fixes very efficient the data in memory while with a
datareader you need to do that item by item.

The amount of data with a right selection, as you stated before, can often
be that few, that it is probably often almost the same amount of bytes as
the code needed to retrieve it using a datareader, so that can not be a
point. (and in a modern computer absolute not relevant).

Moreover when readed it the dataset becomes very handy to handle.

Just my thought to give the OP another opinion, while I can think about
situations where your statement is right.

Cor




Nov 18 '05 #3
Brent,

I posted arguments and sample code on DataSet vs DataReader on a response to
Cor on this post dated 4/7/2004 08:24 AM CST. Be sure and read that and it
my help further clarify what I failed to completely clarify in my initial
response post yesterday.

Raymond Lewallen

"Brent" <b@b.com> wrote in message
news:10******** *****@corp.supe rnews.com...
Hi,
I'm wondering if it is good to use datasets for large amounts of data with
many users. I'm talking tables with 130,000 records and 15 columns. And we
want current data, so no cached data. Right now we are using an
OleDbDataReader and then just doing reader.Read() to go through the
necessary records. So would using a dataset be good, or bad for this
compared to what we are doing?

We're looking at it again to get the features of transactions out of it, but are not sure if they can handle that much data very well?

Nov 18 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
2342
by: michaaal | last post by:
If I use a form to pass data (say, for example, through a textbox) the data seems to be limited to somewhat smaller amounts. What should I do if I want to pass a large amount of data? For example a list of 200 items?
10
2424
by: Digety | last post by:
We are looking to store a large amount of user data that will be changed and accessed daily by a large number of people. We expect around 6-8 million subscribers to our service with each record being approximately 2000-2500 bytes. The system needs to be running 24/7 and therefore cannot be shut down. What is the best way to implement this? We were thinking of setting up a cluster of servers to hold the information and another cluster...
0
1113
by: CHosoi | last post by:
I want to transfer large amounts of data from an access database to a SQL server database. The schema within the Access database is vastly different to the SQL Server one. Mapping data from access to SQL server is relatively straight forward. What I'm less sure about is the best approach to take to achieve this. Best option I've come up with so far is to use a .NET windows app which uses ADO.NET to read data from access into a typed...
1
5998
by: Bart | last post by:
Dear all, I would like to encrypt a large amount of data by using public/private keys, but I read on MSDN: "Symmetric encryption is performed on streams and is therefore useful to encrypt large amounts of data. Asymmetric encryption is performed on a small number of bytes and is therefore only useful for small amounts of data." There is not possibility to do it? I have tried to encrypt a 300kB file by RSA Algorithm, but I received...
3
224
by: Brent | last post by:
Hi, I'm wondering if it is good to use datasets for large amounts of data with many users. I'm talking tables with 130,000 records and 15 columns. And we want current data, so no cached data. Right now we are using an OleDbDataReader and then just doing reader.Read() to go through the necessary records. So would using a dataset be good, or bad for this compared to what we are doing? We're looking at it again to get the features of...
1
1873
by: JIM.H. | last post by:
Hello, Here is part of my code to retrieve some data from FoxPro and populate DataGrid. It works fine. What is the best and easiest way to import data from Dataset to a table in SQL Server. ' populate dataAdapter and dataset cmdSelect = Conn.CreateCommand cmdSelect.CommandText = sqlStr Da.SelectCommand = cmdSelect
7
10827
by: =?Utf-8?B?TW9iaWxlTWFu?= | last post by:
Hello everyone: I am looking for everyone's thoughts on moving large amounts (actually, not very large, but large enough that I'm throwing exceptions using the default configurations). We're doing a proof-of-concept on WCF whereby we have a Windows form client and a Server. Our server is a middle-tier that interfaces with our SQL 05 database server.
13
5116
by: trpost | last post by:
I am looking for a way to send data from one page to another as POST data without using forms or cURL. I have a php script that is passing a list of cases from on page to another when a link is clicked. This is working fine for the most part as a link, but sometimes the list gets very large and gets cut off. The reason it gets cut off appears to be a limitation on the amount of data that can be passesd in the URI. It looks like I can...
0
844
by: John Sheppard | last post by:
Hello there, I am wanting to pass my datasets by reference to other forms. As soon as I do that I have to reset any binding sources by setting their datasource property to the original datasource. I also find that I cannot then bind directly to my dataset without a binding source, so I have to duplicated all my binding sources on my second form. Is this normal or am I doing something wrong? How do you guys normally pass
0
9589
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10216
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10049
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9997
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
1
7413
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5309
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5448
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3965
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
3565
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.