473,394 Members | 1,841 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,394 software developers and data experts.

Algorithm for faster search in DataTable

I read claims from a file into a DataTable and DataRow and then use
the data adapter's Update method to push those claims into the
database.
I import a few thousand claims at a time and before adding a claim to
the DataTable I need to check if a claim with the same pk has already
been added to the DataTable. If not I run a stored Proc to do the same
check in the database. Pseudocode looks loke this:
-----------------------------------------------
for(int i = 0;i< tblClaims.Rows.Count;i++)
{
//look for claim_id and others(PK is 6 fields)

//if found return
}
//If not found look in the database
-----------------------------------------------
As you can imagine, more claims added to the table means that many
more loops. This is slowing my code down. What is a good algorithm I
can use so that I cut down the number of loops?
Nov 16 '05 #1
1 12051
Rathtap <am****@yahoo.com> wrote:
I read claims from a file into a DataTable and DataRow and then use
the data adapter's Update method to push those claims into the
database.
I import a few thousand claims at a time and before adding a claim to
the DataTable I need to check if a claim with the same pk has already
been added to the DataTable. If not I run a stored Proc to do the same
check in the database. Pseudocode looks loke this:
-----------------------------------------------
for(int i = 0;i< tblClaims.Rows.Count;i++)
{
//look for claim_id and others(PK is 6 fields)

//if found return
}
//If not found look in the database
-----------------------------------------------
As you can imagine, more claims added to the table means that many
more loops. This is slowing my code down. What is a good algorithm I
can use so that I cut down the number of loops?


If you use a DataView with a RowFilter which looks for things, I
believe it will do a fair amount of work to make things quicker if
you've specified the primary key in the schema.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 16 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
by: pembed2003 | last post by:
Hi all, I asked this question in the C group but no one seems to be interested in answering it. :-( Basically, I wrote a search and replace function so I can do: char source = "abcd?1234?x";...
6
by: richard.pasco | last post by:
Hey all I am trying to write a script that allows users to search through a database of names. But rather than give a search string and then return all those that match, I would like it to...
28
by: joshc | last post by:
If I have an array of data that I know to be sorted in increasing order, and the array is less than 50 elements, and I want to find the first element greater than a certain value, is a simple...
60
by: Julie | last post by:
What is the *fastest* way in .NET to search large on-disk text files (100+ MB) for a given string. The files are unindexed and unsorted, and for the purposes of my immediate requirements, can't...
0
by: SeanEByrne | last post by:
Hi everybody, i have a bit of java programming knowledge but i'd like to find a way how to write a program that searchs a drive for files of a certain type and of a certain size?? I mainly need help...
6
by: pj | last post by:
Hi, I 'm currently writing a program that performs transliteration (i.e., converts greek text written using the english alphabet to "pure" greek text using the greek alphabet) as part of my...
37
by: mazwolfe | last post by:
I'm new here, so excuse me if my style is incorrect. Can anyone come up with a better method for this calculation? Code: int is_leap(int year) { switch (year % 19) { case 0: case 3: case 6:...
3
by: Vincent SHAO | last post by:
Search engine have to record all of the query string. Now i have a search engine log which contains 10 milllion query strings, but almost of them are repeated, not more than 3 million of them are...
2
by: Oreber | last post by:
HELP.... I am trying to read a table that has 598,865 records. The problem is when I try to access it by year. When I use the following query it is fast : SELECT table_data.name,...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.