473,573 Members | 2,845 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Need Help On Best Querying ( is LINQ work with Huge amount of data..)

Dear Group members,
I am new to LINQ, pls help on the deeling with huge amount of data
with the C# stand Alone application.

I have two file, which contains more then 2 lacs lines in every file

suppose file1 like ...

999361894 422021100257001
899119011 422021100344217
899373022 422021100262179
892044443 422021100426945
899292491 422021000154860
----
----

and file2 looks like

92000000,422021 100420675
92000002,422021 100420403
92000003,422021 100420614
92000004,422021 100425785
92000005,422021 100422232
---
----

values under each file divided by different set of delimiter, and this
delimiter can be changed... ie some files came with some other
delimiter with more fields also.

My motive is to import different kind of formated files to my
application, and generate some auto-summarized report like ... 99999
is missed in file 2, and 99999 no is missed in file 1, and for this
id 999999, values are not matching ..

for this thing, i succeeded agenest the import various typed files to
my data grid.
Remain-thing, major thing is compare the files and generate summary
report, kindly remember i don't have Database support for my
application, and all data came with different formates and data will
be came like text files.

So, that i am starting with small comparison with LINQ,
like
DataTable dt1;//hold's the first file data
DataTable dt2; // hold's the second file data.

When i try to find out the records which are having dt1 and not in
dt2. with LINQ

var test= from k in dt1.AsEnumerabl e() where !(from t in
dt2.AsEnumerabl e() select
Field<string>(0 )).Contains(k.F ield<string>(0) ) select k;

myResult_DataGr id.Datasource = test.AsDataView ();
These steps running from server from mints, still application is look
like executing same statment.

i don't know what is the cause of hangout the application, it's look
like a problem with the huge amount of data, ofcource it nearly
having 2lac's X 2 lacs

is there any other solutions, to do this type of thigs? is there any
problem in my LINQ statement?

pls let me know , if anybody having the solutions on this type of
programs.
Thanks and Regards
ranganadh Kodali
Mar 16 '08 #1
0 1352

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

16
3001
by: D Witherspoon | last post by:
I am developing a Windows Forms application in VB.NET that will use .NET remoting to access the data tier classes. A very simple way I have come up with is by creating typed (.xsd) datasets. For example dsParts.xsd and including that in the data tier. I then will create a class that looks like this Public Class CPart Inherits dsParts
19
4081
by: James Fortune | last post by:
I have a lot of respect for David Fenton and Allen Browne, but I don't understand why people who know how to write code to completely replace a front end do not write something that will automate the code that implements managing unbound controls on forms given the superior performance of unbound controls in a client/server environment. I can...
6
2893
by: No Spam Man | last post by:
Hi, I am not a programmer and in the past have only created very simple, flat file databases. Although I'm a newbie, I think I could probably figure out basic Boolean logic and financial functions. The database I want to create would import quite a few records -- maybe as many as 100,000. I would be importing them from an Internet source...
9
1647
by: Algonquin J. Calhoun | last post by:
I've developed an application that exceeds the Access 2GB limit. This application was developed as a prototype and the users have found it very useful. Approximately 45,000 records are added to the database each day. Every 3 months I have to archive some of the older detail rows to allow continued running of the application against the most...
29
2331
by: calvert4rent | last post by:
I need to some sort of data type that will hold a listing of ID's and their counts/frequency. As I do some processing I get an ID back which I need to store and keep an accurate count for how many times that ID is listed. For example I might get a listing of these numbers: 1,4,6,23,6,2,54,1,6,23,2,4,1,6 I will get those numbers one at a...
3
2791
by: Dean Slindee | last post by:
I'm confused. After all these years of promoting 3-tier architecture, isn't LINQ moving us back to a 2-tier architecture (no stored procedures?). Are there scenarios where LINQ would be a better architecture than a "legacy" 3-tier? Another question, concerning binding: Would LINQ be used in addition to binding, instead of binding, or...
16
2032
by: pereges | last post by:
ok so i have written a program in C where I am dealing with huge data(millions and lots of iterations involved) and for some reason the screen tends to freeze and I get no output every time I execute it. However, I have tried to reduce the amount of data and the program runs fine. What could possibly be done to resolve this ?
26
2331
by: Ravindra.B | last post by:
I have declared a global variable which is array of pointers and allocated memory for each array variable by using malloc. Some thing similar to below... static char *arr; main() { int i;
2
1726
by: Neil Chambers | last post by:
All, I have a class describing various actions to take against a LINQ to SQL datasource. What are the pros/cons of instantiating the LINQ object either in the root of the class (for lack of a better description) or within each of the methods? I am naturally drawn to instantiating the LINQ object in the root but some examples I've seen...
0
7765
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
8000
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
8194
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7767
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
6398
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5578
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
3729
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3729
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
0
1034
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.