473,657 Members | 2,559 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Insertion and Updates on 20.000.000 tuples table.

Hi,
I have a table with 20.000.000 of tuples.
I have been monitoring the performance of the insertion and updates,
but not convince me at all.
The table have 30 columns, what and 12 of it, are calcultated column.

The test that i do was this:

1 Insertion with all the columns and calculing the calcultated columns
in the insertion sentence.

1 insertion and all the columns calculated in @vars..

1 insertion with the basic fields, and 10 updates.

And the result was that the last test was the most performant.

What is your opinion?

Jun 23 '06 #1
4 1583
Andrix (el********@gma il.com) writes:
I have a table with 20.000.000 of tuples.
I have been monitoring the performance of the insertion and updates,
but not convince me at all.
The table have 30 columns, what and 12 of it, are calcultated column.

The test that i do was this:

1 Insertion with all the columns and calculing the calcultated columns
in the insertion sentence.

1 insertion and all the columns calculated in @vars..

1 insertion with the basic fields, and 10 updates.

And the result was that the last test was the most performant.

What is your opinion?


That your posting is not very clear. I would take "calculated columns"
to mean "computed columns", but since you can't insert explicit values
in computed columns that does not really fit.

Why not post the code you used, so it's easier to understand what you
are talking about.

--
Erland Sommarskog, SQL Server MVP, es****@sommarsk og.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx
Jun 23 '06 #2
Hi.

The test that i do, were this:
select @PK = .....
INSERT INTO shared_calc
VALUES (@PK,10,20,223, "calculo trivial",...... . ,@imp * @ohter, ....)

I mean that in the same Insert sentence, i do all the calcs to insert
in the table.

the other,
was
select @cal1 = @imp * @other
select @cal2 = @imp * @other - @umbral

select @PK = .....
INSERT INTO shared_calc
VALUES (@PK,10,20,223, "calculo trivial",...... . ,@calc1,@calc2, ....)

and the last one was:
select @PK = .....
INSERT INTO shared_calc
VALUES (@PK,10,20,223, "calculo trivial",...... . )

select @cal1 = @imp * @other
select @cal2 = @imp * @other - @umbral

update shared_calc_imp
set calc1 = @calc1
where pk = @PK

update shared_calc_imp
set calc2 = @calc2
where pk = @PK

thanks!

Andrix.
Erland Sommarskog wrote:
Andrix (el********@gma il.com) writes:
I have a table with 20.000.000 of tuples.
I have been monitoring the performance of the insertion and updates,
but not convince me at all.
The table have 30 columns, what and 12 of it, are calcultated column.

The test that i do was this:

1 Insertion with all the columns and calculing the calcultated columns
in the insertion sentence.

1 insertion and all the columns calculated in @vars..

1 insertion with the basic fields, and 10 updates.

And the result was that the last test was the most performant.

What is your opinion?


That your posting is not very clear. I would take "calculated columns"
to mean "computed columns", but since you can't insert explicit values
in computed columns that does not really fit.

Why not post the code you used, so it's easier to understand what you
are talking about.

--
Erland Sommarskog, SQL Server MVP, es****@sommarsk og.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx


Jun 25 '06 #3
Do not store calculations in a table. You can do the calculations in a
VIEW, in the application or in computed columns (a proprietary
shorthand) for a VIEW. This will save you disk space of course. It
will also run faster, since reading from a disk is very slow compared
to math done in main storage. But the real benefit is data integrity,
which your approach will destroy.

Jun 26 '06 #4
Andrix (el********@gma il.com) writes:
The test that i do, were this:
select @PK = .....
INSERT INTO shared_calc
VALUES (@PK,10,20,223, "calculo trivial",...... . ,@imp * @ohter, ....)

I mean that in the same Insert sentence, i do all the calcs to insert
in the table.

the other,
was
select @cal1 = @imp * @other
select @cal2 = @imp * @other - @umbral

select @PK = .....
INSERT INTO shared_calc
VALUES (@PK,10,20,223, "calculo trivial",...... . ,@calc1,@calc2, ....)

and the last one was:
select @PK = .....
INSERT INTO shared_calc
VALUES (@PK,10,20,223, "calculo trivial",...... . )

select @cal1 = @imp * @other
select @cal2 = @imp * @other - @umbral

update shared_calc_imp
set calc1 = @calc1
where pk = @PK

update shared_calc_imp
set calc2 = @calc2
where pk = @PK


And you are saying that the last test had the best performance?
Surprising.

But how many times did you run each test? Did you ensure that there
was no other load on the server? Did you work with the same table
that you just added to each time? Or did you recreate the table for
each test? And what were the results in numbers? Were the huge
differences?

Running performance tests requires care to avoid sources of error. One
very important is that any measurement below 50 ms contains to much
white noise to be reliable.

--
Erland Sommarskog, SQL Server MVP, es****@sommarsk og.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx
Jun 26 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
1530
by: Bernhard Schmidt | last post by:
------=_NextPart_000_0030_01C34C51.B8F4D1A0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable hi dear mysql list members =20 i have setup a mysql database 4.013 with innodb enabled. when i set the =
12
5151
by: Dan Greenblatt | last post by:
I am writing some software that, among other things, needs to track the state of database tables. This includes occasionally checking the table to see what records or added, modified, or deleted. The added and modified parts of this have not proven too difficult to implement, but is there an easy/elegant way to determine what records have been deleted from a mySQL table ?? I'm very to mySQL, and after reading through the docs about logs,...
4
2147
by: Jason | last post by:
Looking for some insight from the professionals about how they handle row inserts. Specifically single row inserts through a stored procedure versus bulk inserts. One argument are people who say all inserts (and updates and deletions I guess) should go through stored procedures. The reasoning is that the developers that code the client side have no reason to understand HOW the data is stored, just that it is. Another problem is an insert...
10
2724
by: Anton.Nikiforov | last post by:
Dear all, i have a problem with insertion data and running post insert trigger on it. Preambula: there is a table named raw: ipsrc | cidr ipdst | cidr bytes | bigint time | timestamp Triggers:
2
2014
by: Ben | last post by:
I'm designing a system where I'll be making frequent updates to rows, but some columns will change far less frequently than others. All columns will be read with equal frequency, though probably by means of a materialized view. Updates will happen via a stored proc. Which makes the most sense? 1. Blindly overwrite the value for all columns, even columns that haven't changed.
0
3843
by: polocar | last post by:
Hi, I have noticed a strange behaviour of CurrencyManager objects in C# (I use Visual Studio 2005 Professional Edition). Suppose that you have a SQL Server database with 2 tables called "Cities" and "Persons", and that: "Cities" has 2 fields called "IDCity" and "NameCity" "Persons" has 3 fields called "IDPerson", "NamePerson" and "IDCityAddress" with "IDCity" and "IDCityAddress" fields relationed with the classical father-child relation...
20
14385
by: technocraze | last post by:
Hi guys & commnunity experts, Does anyone knw how to go about checking for existing data in an MS Acess table? I have tried out the following code using vb but doesnt seem to work that well? Can anyone ps take a look at this code / logic and if possible point out the error or make correction? What i need is to check for existing studentId and name before insertion. If the inserted value existed in the table, message "duplicate" else...
3
5991
by: Subrat Das | last post by:
Hi, I have a java application which calls a stored procedure to insert data into a table.Multiple threads of java call the same procedure at the same time. Sometimes it happens that few threads send the same data as parameter to the procedure. In that case i get the error : "Cannot insert duplicate key row in object ...." The functionality of the proc is - It checks if a given data (Based on Unique key) is present in the table or not. If...
0
980
by: John K Masters | last post by:
I am fairly new to Python and am trying to get to grips with pysqlite2. SELECT via connection.cursor. But I have not, despite frantic googling, found how to INSERT a list of tuples into a sqlite table. If I convert the tuple to a string and concatenate it to the 'INSERT INTO table etc.' string then it works, but only if all the tuple values are strings and then only if all the table fields are of type TEXT. Is it possible to, and if so...
0
8305
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8825
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8732
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8605
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
5632
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4302
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2726
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
1953
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
2
1611
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.