473,326 Members | 2,111 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

How to have a batch transaction?

Sri
I am inserting records into a table - around 1 million records. I want
to do the insert 10000 records at a time using TRAN. My insert
statement is very simple:
BEGIN TRAN T1
Insert INTO TABLE A
(ColA, ColB, ColC)
Select B.TBLBCOLA, B.TBLBCOLB, B.TBLBCOLC
FROM tbltable B
WHERE blah balh...etc.

COMMIT TRAN T1
Any hep or link on the syntax will be helpful!
Thanks
Sri

Jul 23 '05 #1
6 3459
Why would you do this? Your insert..select statement is already atomic
and doesn't need a transaction. Besides, you're not checking for
errors and rolling back so it's useless.

Jul 23 '05 #2
Sri
I have to do this in batches because my log ran out of space in tempdb.

Jul 23 '05 #3
You can't do it inside a single transaction, because this won't free-up
any space in the tran log until the whole process has completed.

As Gary says, if you're splitting the insert into multiple batches,
there's no point in placing each one in iots own transaction because
it's atomic.

If you can delete the records from TableB after you've inserted into
TableA, then life is comparatively simple:
declare @counter int
declare @rows_affected int

set rowcount 10000

Insert INTO TABLE A
(ColA, ColB, ColC)
Select B.TBLBCOLA, B.TBLBCOLB, B.TBLBCOLC
FROM tbltable B
WHERE blah balh...etc

select @rows_affected = @@ROWCOUNT

delete Table B
WHERE blah balh...etc

while @rows_affected > 0
begin

Insert INTO TABLE A
(ColA, ColB, ColC)
Select B.TBLBCOLA, B.TBLBCOLB, B.TBLBCOLC
FROM tbltable B
WHERE blah balh...etc

select @rows_affected = @@ROWCOUNT

delete Table B
WHERE blah balh...etc

end

set rowcount 0
This will repeatedly insert 10000 records into table A, then delete the
same from Table B until there are no more records left that match the
WHERE clause. The final iteration of the loop will probably affect <
10000 records.

If you can't delete from tableB, you can use the same principle, but
will have to compare the PK values in TableA with the max equivalent
for those records just inserted into table B, something like:

(Assuming colA is the PK):

set rowcount 10000

Insert INTO TABLE A
(ColA, ColB, ColC)
Select B.TBLBCOLA, B.TBLBCOLB, B.TBLBCOLC
FROM tbltable B
WHERE blah balh...etc

while @@rowcount > 0
begin

select @max_A = max(colA) from TableA

Insert INTO TABLE A
(ColA, ColB, ColC)
Select B.TBLBCOLA, B.TBLBCOLB, B.TBLBCOLC
FROM tbltable B
WHERE blah balh...etc
and ColA > @max_A

end

Jul 23 '05 #4
Doing it in batches is fine, that's not what I was complaining about.
It's using the BEGIN TRAN..COMMIT TRAN, which you don't need unless
you're doing multiple commands and actually checking for errors.
Transactions should be implicit with atomic statements.

Jul 23 '05 #5
Sri
Rather...I thought a batch process might help since this log space ran
out comes up after I fire my query 2 hrs ago.

Jul 23 '05 #6
On 8 Feb 2005 09:34:02 -0800, Sri wrote:
I am inserting records into a table - around 1 million records. I want
to do the insert 10000 records at a time using TRAN. My insert
statement is very simple:
BEGIN TRAN T1
Insert INTO TABLE A
(ColA, ColB, ColC)
Select B.TBLBCOLA, B.TBLBCOLB, B.TBLBCOLC
FROM tbltable B
WHERE blah balh...etc.

COMMIT TRAN T1
Any hep or link on the syntax will be helpful!


Hi Sri,

You'll need something like this generic format:

SET ROWCOUNT 10000
WHILE 1 = 1
BEGIN
INSERT INTO TableA (ColA, ColB, ColC)
SELECT b.ColA, b.ColB, b.ColC
FROM TableB AS b
LEFT JOIN TableA AS a
ON a.ColA = b.ColA
AND a.ColB = b.ColB
-- Replace the above with whichever column (or combination of columns)
-- suffices to uniquely identify exactly one row in the data.
WHERE a.ColA IS NULL
-- Use any column from table A that will never contain NULL
-- The left join with a test for NULL in a non NULLable column
-- is a trick; it has the same effect as a NOT EXISTS subquery,
-- but is often quicker
IF @@ROWCOUNT = 0 BREAK
BACKUP LOG MyDatabase
END
SET ROWCOUNT 0

The "backup log" is necessary - else, the logged events will still be kept
in the log. If you don't need recoverability during this operation (i.e.
it suffices to restore an old full or incremental backup and repeat the
mega insert process), you can also use BACKUP LOG MyDatebase WITH NO_LOG.
If your database is in the simple recovery model, you can omit this step.

If you used BACKUP LOG MyDatabase WITH NO_LOG, then you should take a full
or incremental database backup directly after this process is finished.

Best, Hugo
--

(Remove _NO_ and _SPAM_ to get my e-mail address)
Jul 23 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

16
by: Robbie | last post by:
Hi All This is a belter that my little brain can't handle. Basically I have 1 SQL table that contains the following fields: Stock Code Stock Desc Reference Transaction Date
5
by: Tyler Hudson | last post by:
After reading Erland Sommarskog's most enlightening articles on SQL Server 2000's error handling capabilities (http://www.sommarskog.se/error-handling-I.html and...
1
by: JT | last post by:
can someone point me in the right direction for learning more about how to go about sending a batch of sql statements from asp to sql server, and having errors appropriately handled? for...
0
by: Krishna | last post by:
Hi Guys, I saw somuch discussions on this topic but not found the proper answers - but after a struggle I found the following answer - here is the code snippet. Protected Friend Function...
4
by: acantatore | last post by:
Hello! I'm looking for patterns, best practices, examples - reference material - that apply to Batch Processing, that is: Basic job control (start and stop) Job partitioning Parallel...
0
by: Sharath | last post by:
"Inspired" by the huge success of our first two automation fast track batches We are forced to start third fast track automation batch ...
0
by: Sharath | last post by:
We are glad to inform you that "Inspired" by the huge success of our first three automation fast track batches We are forced to start fourth fast track automation batch ...
0
by: Sharath | last post by:
We are glad to inform you that "Inspired" by the huge success of our first four automation fast track batches We are forced to start fifth fast track automation batch ...
3
by: John | last post by:
Hi. I have a number of batch jobs that are ran nightly on our Windows 2000 based Oracle 8.1.7 (soon to be 9i) server. I have these designed just right, so the Windows Scheduled Tasks runs them...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.