473,597 Members | 2,459 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

How to copy and compare data?

347 Contributor
I have a query that I run daily against a database that is dumped to me nightly. I want to be able to populate a new database called Newclients and avoid duplicates. This is the query that I'm using:

Expand|Select|Wrap|Line Numbers
  1. insert into msbtotal.dbo.newclients
  2. SELECT tcms_members.dbo.memberdata2.* FROM tcms_members.dbo.memberdata2 left outer join 
  3. msbtotal.dbo.memberdata on tcms_members.dbo.memberdata2.id = 
  4. msbtotal.dbo.memberdata.id where msbtotal.dbo.memberdata.id is
  5. null
  6. GO
  7. SET QUOTED_IDENTIFIER OFF 
  8. GO
  9. SET ANSI_NULLS ON 
  10. GO
  11.  

Here is some of my sample data:

6008272 Doctor A Doctor A, MD CHP UT SOUTHWESTERN PSYCHIATRY RES (512) 333-3333 (512) 333-3333 Sep 25 1977 12:00AM 1212 MockingBird Lane Austin TX 78737 2010-10-01 09:52:38.000 2005-06-06 00:00:00.000 NULL
6008272 Doctor A Doctor A, MD CHP UT SOUTHWESTERN PSYCHIATRY RES (512) 333-3333 (512) 333-3333 Sep 25 1977 12:00AM 1212 MockingBird Lane Austin TX 78737 2010-10-01 09:52:38.000 2005-06-06 00:00:00.000 NULL
6008272 Doctor A Doctor A, MD CHP UT SOUTHWESTERN PSYCHIATRY RES (512) 333-3333 (512) 333-3333 Sep 25 1977 12:00AM 1212 MockingBird Lane Austin TX 78737 2010-10-01 09:52:38.000 2005-06-06 00:00:00.000 NULL
6008272 Doctor A Doctor A, MD CHP UT SOUTHWESTERN PSYCHIATRY RES (512) 333-3333 (512) 333-3333 Sep 25 1977 12:00AM 1212 MockingBird Lane Austin TX 78737 2010-10-01 09:52:38.000 2005-06-06 00:00:00.000 NULL
6009837 Some Person Some Person, MD, MPH P UT SOUTHWESTERN PSYCHIATRY RES (512) 555-5555 (512) 555-5555 Jun 26 1975 12:00AM 3501 Residence Ave. Austin TX 78731 2010-09-29 11:53:30.000 2006-06-09 00:00:00.000 2007-11-28 00:00:00.000
6009837 Some Person Some Person, MD, MPH P UT SOUTHWESTERN PSYCHIATRY RES (512) 555-5555 (512) 555-5555 Jun 26 1975 12:00AM 3501 Residence Ave. Austin TX 78731 2010-09-29 11:53:30.000 2006-06-09 00:00:00.000 2007-11-28 00:00:00.000

As you can see, there is duplicate data, and I need only one copy of each "new" record in my database. Can someone please assist me on this.

Thank you

Doug
Nov 5 '10 #1
3 2069
ck9663
2,878 Recognized Expert Specialist
Check for existence of the record before you insert it. Could you give us the header/column names of your data and which columns do you use to identify duplicate?

~~ CK
Nov 5 '10 #2
dougancil
347 Contributor
msbtotal.dbo.me mberdata.id is what I would need to check against for duplicates. They are not auto generated by the server and are just characters. I hope that's the information that you're asking for.
Nov 5 '10 #3
dougancil
347 Contributor
Ck,

I modified my query like this:

insert into msbtotal.dbo.ne wclients
SELECT distinct tcms_members.db o.memberdata2.* FROM tcms_members.db o.memberdata2 left outer join
msbtotal.dbo.me mberdata on tcms_members.db o.memberdata2.i d =
msbtotal.dbo.me mberdata.id where msbtotal.dbo.me mberdata.id is
null
GO
SET QUOTED_IDENTIFI ER OFF
GO
SET ANSI_NULLS ON
GO

That should work.
Nov 5 '10 #4

Sign in to post your reply or Sign up for a free account.

Similar topics

22
24642
by: Bryan Guilliams | last post by:
I'm trying to come up with an elegant, simple way to compare two consecutive values from the same table. For instance: SELECT TOP 2 datavalues FROM myTable ORDER BY timestamp DESC That gives me the two latest values. I want to test the rate of change of these values. If the top row is a 50% increase over the row below it, I'll execute some special logic.
3
4688
by: 21novembre | last post by:
Hi all, I made a question several days before to describe my strange trouble of mysqldump. But I still can't figour it out. Well, I just want to ask another question whether I could just backup my databases by copying the data folder to some place? Then if I meet some disaster, could I just copy the backup folder back to recover my databases? Thank you. Zh.y
8
1866
by: Riegnman | last post by:
Hey guys, I'm in need of a little help. I am very new to access but have been trying to learn. My problem is as follows. . . We have time clocks that dump the badge punches into a .log file on one of the servers. I then use access to import these .log files and separate the data into various tables so that I can manipulate it. The problem is that the time clock just records punches and not "in" and "out" times. The data that comes...
2
1521
by: Ryan Adler | last post by:
I have a terrible headache from this problem. Perhaps someone can help. My XML is formatted like this: <events> <event> <starts-at>123456<starts-at/> <event> ... <events/>
2
2269
by: Nick Hodge \(MVP\) | last post by:
Hi I currently have a gridview connected to a SQLDatasource. (This data comes from a remote iSeries). It shows catalogue sales as they build during the day (works great). I have a local SQL server with the budget data for each catalogue/day and wanted to compare the gridview data with this to display KPI traffic lights as the sales build. (This is used across the organisation and is connected 'live') One issue is that all catalogue...
0
1224
by: comp21 | last post by:
Hi, Now, I have retrieved or imported data from excel spreadsheet to vb6 application. Now I want to compare this part of data with the one already existing in mssql2000 server(table name being same). If some data which is there in excel spread sheet not to be found in mssql2000 server then those data has to be stored in a seperate table in mssql server. Can anyone please help me with the code....
2
1532
by: getteddy | last post by:
hi i am working on array , i am facing a problem in comparing data base element with array element if not (objrs("namephase")=("despArray(ctr,0)+despArray(ctr,3)")) then or If (StrComp(objRS("UserName")),("despArray(ctr,0)+despArray(ctr,3)")) , vbTextCompare) = 0) Then please help me
1
1315
by: Patrick C | last post by:
hey everyone, i'm going to be comparing data in a list. Just basic >= or <= type stuff. However, the list i have somtimes starts with 'n/a'. That is somtimes it starts like this: data = or data = or
2
7195
by: O.B. | last post by:
When using Marshal to copy data from a byte array to the structure below, only the first byte of the "other" array is getting copied from the original byte array. What do I need to specify to get Marshal.PtrToStructure to copy the all the data into the "other" array? unsafe public struct DeadReckoning {
11
10984
by: jennifersalcido | last post by:
Hello All, I am working with two tables that contain inventory data: 1) INVENTORY contains item_number, description, etc. This data is relatively straight-forward, one record per item_number. 2) INV_CROSS_REF contains item_number, and a data column containing either a scancode or a manufacturer part number. The type column indicates what type of data is in the data column (4=scancode, 5=mfr pn). Finally, because the software allows...
0
8276
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8381
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8259
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
6698
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5436
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
3932
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2408
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1495
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
1243
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.