473,320 Members | 1,969 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

duplicate rows but no key on the tables


Dear All,

I have a table with 10 billion records but there are no key on it. I cannot
build a key on it as it is the data source.

However, the data source exits the duplicated rows.

I have used the DTS to transform the data into a new table and delete the
duplicated rows. As there are 10 billion records, i need to divide it into 3
parts and also the process lasts for 6 hours each part.

I want to ask is there any other good methods to slove my problem??

Thx
Esther
Jul 23 '05 #1
1 1402
esther s via SQLMonster.com (fo***@SQLMonster.com) writes:
I have a table with 10 billion records but there are no key on it. I
cannot build a key on it as it is the data source.

However, the data source exits the duplicated rows.

I have used the DTS to transform the data into a new table and delete
the duplicated rows. As there are 10 billion records, i need to divide
it into 3 parts and also the process lasts for 6 hours each part.

I want to ask is there any other good methods to slove my problem??


Eliminating duplicates from 10 milliard(*) rows is nothing for the
impatient. I'm happy that I don't have to play that game.

(*) I assume. 10 billion rows as in what I mean with billion would be
really dauting...

--
Erland Sommarskog, SQL Server MVP, es****@sommarskog.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techinf...2000/books.asp
Jul 23 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: John Steen | last post by:
This is probably a silly question to most of you, but I'm in the process of splitting off years from a large DB to several smaller ones. Some of the existing smaller DBs already have most of the...
18
by: Elroyskimms | last post by:
I have a table using an identity column as its Primary Key and two columns (table reduced for simplicity) EmployeeNumber and ArrivalTime. CREATE TABLE ( IDENTITY (1, 1) NOT NULL , (10)...
8
by: Kragen Sitaker | last post by:
ERROR: Cannot insert a duplicate key into unique index pg_class_relname_nsp_index We've been getting this error in our application every once in a while --- typically once an hour to once a day,...
0
by: Chris Becker | last post by:
I want to create a TableCell object once and add it to many tables: Dim cell As New TableCell cell.Style.Add("background-color", s) cell.Style.Add("cursor", "pointer") cell.Width =...
12
by: Jared Carr | last post by:
First I wish I knew how this was caused but here is our problem. Sometime in the recent past we got a duplicate table. Here is the result of a pg_dump with a pg_restore for just that table. ...
0
by: phillip.s.powell | last post by:
CREATE TABLE /*! IF NOT EXISTS */ blah SELECT * FROM blah1 ON DUPLICATE KEY UPDATE unique_key=unique_key UNION SELECT * FROM blah2 ON DUPLICATE KEY UPDATE unique_key=unique_key TYPE=MyISAM ...
5
by: dale.zjc | last post by:
I've got the following table data: 11652 5.99 11652 0.14 12996 5.03 12996 0.12 12996 7.00 And I need to write a query to return only rows 2 and 4, since the remaining rows have duplicate...
10
by: Backwards | last post by:
Hello all, I'll start by explaining what my app does so not to confuss you when i ask my question. ☺ I have a VB.Net 2.0 app that starts a process (process.start ...) and passes a prameter...
2
by: Rich | last post by:
Hello, I am just checking if there is a property or technique for displaying or retrieving from a dataTable the top 1 row(s) for rows containing duplicate keys (IDs). I have to pull data from a...
0
by: sree078 | last post by:
Hi I want to delete the duplicate rows from two tables and get the resultant non-duplicate rows from both the tables into another table
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.