473,382 Members | 1,635 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,382 software developers and data experts.

Removing the duplicate rows from a database

16
If you have a problem in deleting duplicate rows in a table if the table does not have any primary key, then here is a tip to do that.

Deleting duplicate rows when there is no primary key

Problem

Every once in awhile a table gets created without a primary key and duplicate records get entered. The problem gets even worse when you have two identical rows in the table and there is no way to distinguish between the two rows. So how do you delete the duplicate record?

Solution

One option that SQL Server gives you is the ability to set ROWCOUNT which limits the numbers of records affected by a command. The default value is 0 which means all records, but this value can be set prior to running a command. So let's create a table and add 4 records with one duplicate record.

Create a table called duplicateTest and add 4 records.

CREATE TABLE dbo.duplicateTest

(

[ID] [int] ,

[FirstName] [varchar](25),

[LastName] [varchar](25)

) ON [PRIMARY]



INSERT INTO dbo.duplicateTest VALUES(1, 'Bob','Smith')

INSERT INTO dbo.duplicateTest VALUES(2, 'Dave','Jones')

INSERT INTO dbo.duplicateTest VALUES(3, 'Karen','White')

INSERT INTO dbo.duplicateTest VALUES(1, 'Bob','Smith')

If we select all data we get the following:

SELECT * FROM dbo.duplicateTest

ID FirstName LastName

1 Bob Smith

2 Dave Jones

3 Karen White

1 Bob Smith



If we try to select the record for Bob Smith will all of the available values such as the following query:

SELECT * FROM dbo.duplicateTest WHERE ID = 1 AND FirstName = 'Bob' AND LastName = 'Smith'

We still get 2 rows of data:

ID FirstName LastName

1 Bob Smith

1 Bob Smith



So to delete the duplicate record with SQL Server 2000 and 2005 we can use the SET ROWCOUNT command to limit the number of rows affected by a query. By setting it to 1 we can just delete one of these rows in the table. Note: the select commands are just used to show the data prior and after the delete occurs.

SELECT * FROM dbo.duplicateTest



SET ROWCOUNT 1

DELETE FROM dbo.duplicateTest WHERE ID = 1

SET ROWCOUNT 0



SELECT * FROM dbo.duplicateTest



With SQL Server 2005 we can also use the TOP command when we issue the delete, such as the following. Note: the select commands are just used to show the data prior and after the delete occurs.

SELECT * FROM dbo.duplicateTest

DELETE TOP(1) FROM dbo.duplicateTest WHERE ID = 1

SELECT * FROM dbo.duplicateTest



So as you can see with SQL Server 2005 there are two options to allow you to delete duplicate identical rows of data in your tables.

Here is one note from Microsoft about using SET ROWCOUNT:

Using SET ROWCOUNT will not affect DELETE, INSERT, and UPDATE statements in the next release of SQL Server. Avoid using SET ROWCOUNT together with DELETE, INSERT, and UPDATE statements in new development work, and plan to modify applications that currently use it. Also, for DELETE, INSERT, and UPDATE statements that currently use SET ROWCOUNT, we recommend that you rewrite them to use the TOP syntax.

Next Steps

Take a look how the ROWCOUNT command can be used to affect the results of your query

Also take a look at the TOP command and changes that have been implemented with SQL Server 2005

Start using TOP instead of ROWCOUNT for SQL Server 2005 and later

Hope this will be useful.

Best Regards,
Bangaar
Mar 9 '07 #1
1 5329
bartonc
6,596 Expert 4TB
Nice solution... Funny thing is this is not a DB forum.
Mar 9 '07 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

2
by: Iain | last post by:
Hi I have inherited a web app with the following table structure, and need to produce a table without any duplicates. Email seems like the best unique identifier - so only one of each e-mail...
0
by: amber | last post by:
Can someone tell me what I may be doing wrong here... I'm trying to remove rows from a table in a dataset that have the same data. My code is pasted below. The 2nd time it runs through the code,...
4
by: Drew | last post by:
I have a permission tracking app that I am working on, and I have made the insert page for it. I am having issues on how to prevent duplicates from getting entered. Currently the interface for...
3
by: dan graziano | last post by:
Hi, How do you suggest is the best way to check for duplicate rows in an access table. And once one knows if there are duplicates, to remove all but one. In my access table, there are 5...
4
by: sri2097 | last post by:
Hi all, I'm storing number of dictionary values into a file using the 'cPickle' module and then am retrieving it. The following is the code for it - # Code for storing the values in the file...
5
by: asgars | last post by:
i have two tables, tab1 having N1 col and tab2 N2 col. now N1 is subset of N2. I need the information from tab2 (having N2) of all rows having the matching entry in N1 in tab1. For this i am...
10
by: Backwards | last post by:
Hello all, I'll start by explaining what my app does so not to confuss you when i ask my question. ☺ I have a VB.Net 2.0 app that starts a process (process.start ...) and passes a prameter...
5
by: Manish | last post by:
The topic is related to MySQL database. Suppose a table "address" contains the following records ------------------------------------------------------- | name | address | phone |...
0
by: B.N.Prabhu | last post by:
Hi, I have a DataTable with several rows. Its having 20 Columns. when i click the Insert button then i have to check the Database Rows. Whether these new rows are already available in the...
5
jamesd0142
by: jamesd0142 | last post by:
My manager and I where looking at some complex code to eliminate duplicate records in a database table. then it hit me how its done easily... so i thought i'd share it... In English:...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.