473,406 Members | 2,713 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,406 software developers and data experts.

Issues with loading 124 million records

I am trying to load from a cursor which selects 124 million
records...but while loading it completes the build phase and it stuck
...I think...at the delete phase(which in turn looks like doing the
relational Integrity thing) according my DBA at around 4%...Is there a
way to speed this up and try to get it done sooner...This has been
running for almost 2 hrs now and seems no near to completion...
Any suggestions are welcome...

Mar 29 '06 #1
2 1608
What your note tries to say is that you are "stuck" in the delete phase of
the load.
This has nothing to do with "relational integrity"!
It means that you have indexes that are unique (at least one) and that the
delete phase of the load has identifie duplicate keys. Since the load and
build phase are done, it is deleting the rows from the table, writing error
messages to the message file and "logging" (I didn't want to use capitals)
the deletes.

maybe your source table where the cursor is running has a lot of duplicates
for the target index !!! But if you are "stuck" in the delete phase, that's
what it is doing.
HTH, Pierre.

--
Pierre Saint-Jacques
SES Consultants Inc.
514-737-4515
"technocrat" <su************@gmail.com> a écrit dans le message de news:
11**********************@i39g2000cwa.googlegroups. com...
I am trying to load from a cursor which selects 124 million
records...but while loading it completes the build phase and it stuck
..I think...at the delete phase(which in turn looks like doing the
relational Integrity thing) according my DBA at around 4%...Is there a
way to speed this up and try to get it done sooner...This has been
running for almost 2 hrs now and seems no near to completion...
Any suggestions are welcome...


Mar 30 '06 #2
yeah i figured that out...the source table had almost everyrecord
duplciate so its going on deleting those...and its taking a long
time..anyway thatnks pierre!

Apr 3 '06 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Philipp K. Janert, Ph.D. | last post by:
Dear All! I am trying to load a relatively large table (about 1 Million rows) into an sqlite table, which is kept in memory. The load process is very slow - on the order of 15 minutes or so. ...
7
by: Warren Wright | last post by:
Hello, We maintain a 175 million record database table for our customer. This is an extract of some data collected for them by a third party vendor, who sends us regular updates to that data...
2
by: PeteCresswell | last post by:
This is pursuant to another thread I tried to start, but can't find on my server. I've got to store rolling rates of return for investment funds - calendar year, quarterly, and monthly To cut...
5
by: John Richardson | last post by:
I've been bothered for some time about my DataGrid not populating my rows very quickly. I have about 10K rows loading into the grid. I create a datatable dt with 2 columns, an ID and a display. ...
3
by: Vivek Khera | last post by:
I have some historic data that I want to analyze. To do this I set up postgres on a spare box I picked up for cheap, which just lucked into having tons of RAM (1.5G). I set up postgres to use...
17
by: Lauren Quantrell | last post by:
Using MS Access 2K, I have a client with a number of seperate customer tables for each country, approx 50 tables, stored on a SQL Server backend. I cleaned up the data in the tables and inserted...
4
by: serge | last post by:
I am running a query in SQL 2000 SP4, Windows 2000 Server that is not being shared with any other users or any sql connections users. The db involves a lot of tables, JOINs, LEFT JOINs, UNIONS...
1
by: achurin | last post by:
Hi everybody, I am a rookie at the forum stuff. Please don't bash me too badly. I have created an app a few years back to store some records in a DB. According our calculations we were never to...
28
by: vikas000000a | last post by:
Hi all, I am stuck in VB once again. I think, VB is something about which you cann't say at anytime for sure that I have mastered it completely. Anyways, my issue is stated below: I am making a...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.