473,721 Members | 1,763 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

bulk load very slow

Hi,

I am running into a problem ( only in this particular instance).

I insert one row into a table - takes a fraction of a second
try to do a bulk load of the same --takes 10 seconds.

Anything that I need to be looking into to identify why bulk load
takes a long time?

Thanks,
Sumanth

Jul 13 '06 #1
5 2865
Try testing with bigger volume of data atleast 100,000 rows to see if
there is a problem. Bulk load is something you should do for highvolume
of data. And it needs a X lock on the table.

Sumanth wrote:
Hi,

I am running into a problem ( only in this particular instance).

I insert one row into a table - takes a fraction of a second
try to do a bulk load of the same --takes 10 seconds.

Anything that I need to be looking into to identify why bulk load
takes a long time?

Thanks,
Sumanth
Jul 13 '06 #2

I was trying this as a test. Have been using bulk load to load large
volumes. Taking more than 10 seconds to upload a single row
only on this particular instance indicates to me something is amiss, the
very same operation takes less than a second on other instances.
Any pointers on what parameters I need to look into ?

Thanks,
Sumnath

"Visu" <Vi*******@gmai l.comwrote in message
news:11******** **************@ 75g2000cwc.goog legroups.com...
Try testing with bigger volume of data atleast 100,000 rows to see if
there is a problem. Bulk load is something you should do for highvolume
of data. And it needs a X lock on the table.

Sumanth wrote:
>Hi,

I am running into a problem ( only in this particular instance).

I insert one row into a table - takes a fraction of a second
try to do a bulk load of the same --takes 10 seconds.

Anything that I need to be looking into to identify why bulk load
takes a long time?

Thanks,
Sumanth

Jul 13 '06 #3
What are using to load ? Informatica?

Sumanth wrote:
I was trying this as a test. Have been using bulk load to load large
volumes. Taking more than 10 seconds to upload a single row
only on this particular instance indicates to me something is amiss, the
very same operation takes less than a second on other instances.
Any pointers on what parameters I need to look into ?

Thanks,
Sumnath

"Visu" <Vi*******@gmai l.comwrote in message
news:11******** **************@ 75g2000cwc.goog legroups.com...
Try testing with bigger volume of data atleast 100,000 rows to see if
there is a problem. Bulk load is something you should do for highvolume
of data. And it needs a X lock on the table.

Sumanth wrote:
Hi,

I am running into a problem ( only in this particular instance).

I insert one row into a table - takes a fraction of a second
try to do a bulk load of the same --takes 10 seconds.

Anything that I need to be looking into to identify why bulk load
takes a long time?

Thanks,
Sumanth
Jul 13 '06 #4

sas to db2.

Thanks,
Sumanth

"Visu" <Vi*******@gmai l.comwrote in message
news:11******** **************@ h48g2000cwc.goo glegroups.com.. .
What are using to load ? Informatica?

Sumanth wrote:
>I was trying this as a test. Have been using bulk load to load large
volumes. Taking more than 10 seconds to upload a single row
only on this particular instance indicates to me something is amiss, the
very same operation takes less than a second on other instances.
Any pointers on what parameters I need to look into ?

Thanks,
Sumnath

"Visu" <Vi*******@gmai l.comwrote in message
news:11******* *************** @75g2000cwc.goo glegroups.com.. .
Try testing with bigger volume of data atleast 100,000 rows to see if
there is a problem. Bulk load is something you should do for highvolume
of data. And it needs a X lock on the table.

Sumanth wrote:
Hi,

I am running into a problem ( only in this particular
instance).

I insert one row into a table - takes a fraction of a second
try to do a bulk load of the same --takes 10 seconds.

Anything that I need to be looking into to identify why bulk
load
takes a long time?

Thanks,
Sumanth

Jul 13 '06 #5
Ian
Visu wrote:
Try testing with bigger volume of data atleast 100,000 rows to see if
there is a problem. Bulk load is something you should do for highvolume
of data. And it needs a X lock on the table.
Visu is correct.

The amount of time required to initiate a load will also depend on
whether you are in a partitioned (DPF) environment -- if so, there is
additional overhead involved in setting up the communications and
processes at all partitions involved in the load. For a large
partitioned environment, the setup can be significant (esp. if you
are comparing to a non-partitioned database).
Jul 14 '06 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
1573
by: Clive Page | last post by:
I am trying to use the geometry data types introduced in version 4.1-alpha and the spatial indexing seems to work well. But I have a large volume of data to load, and from the documentation all I can see is that I have to load using a set of INSERT (or UPDATE) commands, which are rather verbose, and which produce a lot of messages each time a row is inserted: INSERT INTO tm VALUES ( 0.01082, 0.08536, ...
2
8664
by: David Sharp | last post by:
I've been doing some experiments with speeding up copying tables of approximately 1 million rows between databases using BCP and BULK INSERT. I noticed that the total time for removing the indexes (non-clustered) and then recreating them after the BULK INSERT was significantly less than just doing the BULK INSERT with the indexes left there, even though I specified TABLOCK. I would have expected SQL Server not to update the index until...
2
16506
by: php newbie | last post by:
Hello, I am trying to load a simple tab-delimited data file to SQL Server. I created a format file to go with it, since the data file differs from the destination table in number of columns. When I execute the query, I get an error saying that only sysadmin or bulkadmin roles are allowed to use the BULK INSERT statement. So, I proceeded with the Enterprise Manager to grant myself those roles. However, I could not find sysadmin or...
20
8573
by: akej via SQLMonster.com | last post by:
Hi, i have table with 15 columns CREATE TABLE . ( PRIMARY KEY , NULL , NULL , NULL , NULL , (50) NULL , NULL
3
33234
by: moonriver | last post by:
Right now I develop an application to retrieve over 30,000 records from a binary file and then load them into a SQL Server DB. So far I load those records one by one, but the performance is very poor: it takes more than 6 hours to finish the loading. So could I make use of the bulk-insert mechanism of SQL Server to reduce the loading time in C# codes? or other performance improvement solutions?
9
10034
by: David Rysdam | last post by:
I have a large amount of data that I copy in and out of Sybase very often. Now I also want to copy this data in and out of postgres. I have an existing script that creates the entire database(s) from scratch in Sybase and then uses the Sybase bulk copy tool "bcp" to copy the data in. I already did a brute force port of this script to postgres once, but I'm trying to do it more elegantly now that I know what issues I'm going to run...
11
28342
by: Ted | last post by:
OK, I tried this: USE Alert_db; BULK INSERT funds FROM 'C:\\data\\myData.dat' WITH (FIELDTERMINATOR='\t', KEEPNULLS, ROWTERMINATOR='\r\n');
2
15120
by: nano | last post by:
Does sql server have a way to handle errors in a sproc which would allow one to insert rows, ignoring rows which would create a duplicate key violation? I know if one loops one can handle the error on a row by row basis. But is there a way to skip the loop and do it as a bulk insert? It's easy to do in Access, but I'm curious to know if SQL Server proper can handle like this. I am guessing that a looping operation would be slower to...
5
6851
by: yeoj13 | last post by:
Hello, I have a db2load script I'm using to populate a large table. Ideally, my target table is required to have "Not Null" constraints on a number of different columns. I've noticed a huge performance hit when I load the target table with "Not Null" constraints as compared to loading a target table without the constraints.
0
8840
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8730
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9215
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
9064
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
6669
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5981
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4484
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
3189
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
2130
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.