I have created a DataSet DataTable which is comprised of data from a
number of sources. The resultant datatable is about 500,000 records and
writing them out one at a time takes forever. Does anyone have some
example code or tell me how to dump a datatable to Access.
One problem is that the datatable structure changes and I would like to
create a generic method, which scans the datatable column types and
builds the Access table on the fly. My current method works all the way
upto writing the data out to an Access table which is terribly slow.
Writing the same data in VBA takes about 1 minute to process 400,000
records but under C# it would take hours if I had the patience to let
it run.
The problem that I see is that the OleDbCommand must be opened and
closed during the record write and that must be where it is falling on
its face. If I open the command once, I could not get the data to
change that was being written to the file. It just wrote the first
record over and over. It ignored the changing parameter values. I even
reset the parameters and recreated them to no avail.
Any help especially some sample code would be greatly appreciated!
4  4738 
..NET + OleDb + Microsoft Access = slow
You may need to go with COM and DAO.
--
Robbe Morris - 2004-2006 Microsoft MVP C#
Microsoft .NET Search Engine Scoring Analysis
How does your site rate? http://www.topichound.com
"oopman2002 " <oo********@yah oo.comwrote in message
news:11******** **************@ h48g2000cwc.goo glegroups.com.. .
>I have created a DataSet DataTable which is comprised of data from a
number of sources. The resultant datatable is about 500,000 records and
writing them out one at a time takes forever. Does anyone have some
example code or tell me how to dump a datatable to Access.
One problem is that the datatable structure changes and I would like to
create a generic method, which scans the datatable column types and
builds the Access table on the fly. My current method works all the way
upto writing the data out to an Access table which is terribly slow.
Writing the same data in VBA takes about 1 minute to process 400,000
records but under C# it would take hours if I had the patience to let
it run.
The problem that I see is that the OleDbCommand must be opened and
closed during the record write and that must be where it is falling on
its face. If I open the command once, I could not get the data to
change that was being written to the file. It just wrote the first
record over and over. It ignored the changing parameter values. I even
reset the parameters and recreated them to no avail.
Any help especially some sample code would be greatly appreciated!
That is what I figured except I can't find much in the way of help for
this. I'm familiar with doing it in VB6 but am not sure about the
calling process with c#. Do you have any links that may help?
Thanks
Lyle
Robbe Morris [C# MVP] wrote:
.NET + OleDb + Microsoft Access = slow
You may need to go with COM and DAO.
--
Robbe Morris - 2004-2006 Microsoft MVP C#
Microsoft .NET Search Engine Scoring Analysis
How does your site rate?
http://www.topichound.com
"oopman2002 " <oo********@yah oo.comwrote in message
news:11******** **************@ h48g2000cwc.goo glegroups.com.. .
I have created a DataSet DataTable which is comprised of data from a
number of sources. The resultant datatable is about 500,000 records and
writing them out one at a time takes forever. Does anyone have some
example code or tell me how to dump a datatable to Access.
One problem is that the datatable structure changes and I would like to
create a generic method, which scans the datatable column types and
builds the Access table on the fly. My current method works all the way
upto writing the data out to an Access table which is terribly slow.
Writing the same data in VBA takes about 1 minute to process 400,000
records but under C# it would take hours if I had the patience to let
it run.
The problem that I see is that the OleDbCommand must be opened and
closed during the record write and that must be where it is falling on
its face. If I open the command once, I could not get the data to
change that was being written to the file. It just wrote the first
record over and over. It ignored the changing parameter values. I even
reset the parameters and recreated them to no avail.
Any help especially some sample code would be greatly appreciated!
It is just COM. You include the COM reference
in your .NET project and go.
--
Robbe Morris - 2004-2006 Microsoft MVP C#
Microsoft .NET Search Engine Scoring Analysis
How does your site rate? http://www.topichound.com
"oopman2002 " <oo********@yah oo.comwrote in message
news:11******** **************@ s13g2000cwa.goo glegroups.com.. .
That is what I figured except I can't find much in the way of help for
this. I'm familiar with doing it in VB6 but am not sure about the
calling process with c#. Do you have any links that may help?
Thanks
Lyle
Robbe Morris [C# MVP] wrote:
>.NET + OleDb + Microsoft Access = slow
You may need to go with COM and DAO.
--
Robbe Morris - 2004-2006 Microsoft MVP C#
Microsoft .NET Search Engine Scoring Analysis
How does your site rate?
http://www.topichound.com
"oopman2002 " <oo********@yah oo.comwrote in message
news:11******* *************** @h48g2000cwc.go oglegroups.com. ..
>I have created a DataSet DataTable which is comprised of data from a
number of sources. The resultant datatable is about 500,000 records and
writing them out one at a time takes forever. Does anyone have some
example code or tell me how to dump a datatable to Access.
One problem is that the datatable structure changes and I would like to
create a generic method, which scans the datatable column types and
builds the Access table on the fly. My current method works all the way
upto writing the data out to an Access table which is terribly slow.
Writing the same data in VBA takes about 1 minute to process 400,000
records but under C# it would take hours if I had the patience to let
it run.
The problem that I see is that the OleDbCommand must be opened and
closed during the record write and that must be where it is falling on
its face. If I open the command once, I could not get the data to
change that was being written to the file. It just wrote the first
record over and over. It ignored the changing parameter values. I even
reset the parameters and recreated them to no avail.
Any help especially some sample code would be greatly appreciated!
Robbe,
First thanks for responding and I'm sorry but I'm not as familiar with
this as you are and it doesn't seem as easy to me as including a COM
reference.
1 ) I'm assuming that when you say a COM reference you are meaning
ADODB
2 ) If you look at the available options, I'm not sure which one to
use.
For example: ADODB._Connecti on, .Connection, .Connection15,
..ConnectionCla ss
Similarily for recordset _Recordset, .Recordset, .Recordset15, 20, 21,
..RecordsetClas s
3 ) By trial and error, I've managed to get a program running but I
don't know if I've created problems waiting to surface or not.
4 ) What would be nice would be a simple program that wrote some stuff
out to an Access database.
Thanks again and sorry to bitch but this has been a really frustrating
experience.
Lyle
Robbe Morris [C# MVP] wrote:
It is just COM. You include the COM reference
in your .NET project and go.
--
Robbe Morris - 2004-2006 Microsoft MVP C#
Microsoft .NET Search Engine Scoring Analysis
How does your site rate?
http://www.topichound.com
"oopman2002 " <oo********@yah oo.comwrote in message
news:11******** **************@ s13g2000cwa.goo glegroups.com.. .
That is what I figured except I can't find much in the way of help for
this. I'm familiar with doing it in VB6 but am not sure about the
calling process with c#. Do you have any links that may help?
Thanks
Lyle
Robbe Morris [C# MVP] wrote:
.NET + OleDb + Microsoft Access = slow
You may need to go with COM and DAO.
--
Robbe Morris - 2004-2006 Microsoft MVP C#
Microsoft .NET Search Engine Scoring Analysis
How does your site rate?
http://www.topichound.com
"oopman2002 " <oo********@yah oo.comwrote in message
news:11******** **************@ h48g2000cwc.goo glegroups.com.. .
I have created a DataSet DataTable which is comprised of data from a
number of sources. The resultant datatable is about 500,000 records and
writing them out one at a time takes forever. Does anyone have some
example code or tell me how to dump a datatable to Access.
One problem is that the datatable structure changes and I would like to
create a generic method, which scans the datatable column types and
builds the Access table on the fly. My current method works all the way
upto writing the data out to an Access table which is terribly slow.
Writing the same data in VBA takes about 1 minute to process 400,000
records but under C# it would take hours if I had the patience to let
it run.
The problem that I see is that the OleDbCommand must be opened and
closed during the record write and that must be where it is falling on
its face. If I open the command once, I could not get the data to
change that was being written to the file. It just wrote the first
record over and over. It ignored the changing parameter values. I even
reset the parameters and recreated them to no avail.
Any help especially some sample code would be greatly appreciated!
This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: VM |
last post by:
When I display data to a Windows datagrid I usually fill the underlying
table (in another class) and then, once it contains all the data, I attach
it to the grid. But there are some processes that involve large amounts of
data so the user won't be able to see anything in the grid until after the
table's already filled up (usually 40-60 secs with these large processes).
Is it possible (without changing the process of filling the table...
|
by: VM |
last post by:
What's the fastest way to insert data into an Access table? I'm currently
using ExecuteNonQuery but it seems to take too long (should filling an mdb
table be slower than filling a datatable?). So this is my code to insert
several rows:
string strDSN = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" +
Application.StartupPath + \\ZMData\\audit.mdb;
OleDbCommand myCommInsert = new OleDbCommand();
myCommInsert.Connection = new...
|
by: authorking |
last post by:
I use the following code to insert a data record in to a datatable of an
access database.But every time I execute the command, there will rise an
exception and the insert operation can't be completed.
here's the code:
System.Data.OleDb.OleDbConnection hist_cnn = new
System.Data.OleDb.OleDbConnection();
System.Data.OleDb.OleDbCommand hist_command = new
System.Data.OleDb.OleDbCommand();
|
by: VMI |
last post by:
For some reason, the process of retrieving data (about 20 records) from an
Access table that has 400K records to a dataTable is taking over 3 mins. to
complete. Below is my code to connect to the DB and query the table. The
table "audit" primary key is "Line".
Another weird thing (but I guess that's another post) is that, while it's
doing the dataset Fill, my PC is slowed done substantially. But I don't
know why that would happen since...
|
by: egoldthwait |
last post by:
I need to convert a 17mb access 2000 db to Oracle and house it in a
Citrix farm. The issue: we have never converted an Access Db to Oracle
but can probably use Oracle's Workbench to assist with this. Also - the
citrix folks do not want us to keep the FE in Access as the queries and
other activities consume a lot of power. The users will be in 3
different offices across the globe all accessing the 1 Oracle DB in
Citrix.
Does anyone have...
| | |
by: onnodb |
last post by:
Hi all,
While working on an Access UI to a MySQL database (which should be a
reasonable, low-cost, flexible interface to the DB, better than
web-based, much less costly than a full-fledged .NET app or so.... is
it?), I tried measuring the bandwith consumed by the
Access/MyODBC/MySQL link, which came out to be, er, quite high.
I fancied it would be interesting to look at the queries Access throws
at MySQL through the ODBC link, so I...
|
by: Brad Baker |
last post by:
I'm trying to write a simple asp.net page which updates some data in a SQL
database. At the top of the page I have the following code:
<%@ Page Language="C#" Debug="true" %>
<%@ import namespace="System.Data" %>
<%@ import namespace="System.Data.SqlClient" %>
<script language="c#" runat="server">
public void Page_Load(object sender, EventArgs e) {
|
by: SaltyBoat |
last post by:
Needing to import and parse data from a large PDF file into an Access
2002 table: I start by converted the PDF file to a html file. Then
I read this html text file, line by line, into a table using a code
loop and an INSERT INTO query. About 800,000 records of raw text.
Later, I can then loop through and parse these 800,000 strings into
usable data using more code.
The problem I have is that the conversion of the text file, using a...
|
by: jehugaleahsa |
last post by:
Hello:
What is the point of using a DataTable in ASP .NET? We are unsure how
you can use them without 1) rebuilding them every postback, or 2)
taking up precious memory. We are not sure how to store a DataTable in
any other way outside of our servers. In doing so, we leave ourselves
open to large memory requirements. Furthermore, most web pages do not
really support multiple changes per transaction. In other words, when
the user submits...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
| | |
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| | |
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |
