473,403 Members | 2,270 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,403 software developers and data experts.

Deleteing large bulks of data

Brief background:

We are using SQL Server 2000, and one of the tables stores user
sessions details (each time our users logs into our system we insert a
new record in the session table, and each time user logs out from our
system we insert another record in the same table).
SESSION_ID is the primary key and it is clustered index.
The system produces 5 million session records/day.

The problem:

Each day we transfer the session data (delta only) to other machine and
we want to delete bulk of ~5 million sessions. This should happend
without any interfering of our customers activity ( in the same time,
we should not block the table - new sessions should be created).

What is the best way to perform such task ?

Feb 2 '06 #1
4 1276
generally truncate table xxx ...will be MUCH faster than delete. Be
aware of some of the logging issues associated with truncate table
before you do this. Search your BOL for "truncate table".

MJKulangara
http://sqladventures.blogspot.com

Feb 2 '06 #2
Truncating table will delete the entire table, and this is not what we
want. We are looing for a method to delete specific sessions (by
specifying the exact sessions ID's).

Feb 2 '06 #3
ro*******@hotmail.com (ro*******@hotmail.com) writes:
We are using SQL Server 2000, and one of the tables stores user
sessions details (each time our users logs into our system we insert a
new record in the session table, and each time user logs out from our
system we insert another record in the same table).
SESSION_ID is the primary key and it is clustered index.
The system produces 5 million session records/day.

The problem:

Each day we transfer the session data (delta only) to other machine and
we want to delete bulk of ~5 million sessions. This should happend
without any interfering of our customers activity ( in the same time,
we should not block the table - new sessions should be created).

What is the best way to perform such task ?


If I understand this correctly, you want to delete the main bulk of the
five million rows, but keep some of them.

I would consider doing something like:

1) Rename the table.
2) Create a new table with the same schema.
3) Insert the rows you want to keep from the old table to the new table.
4) Drop the old table.

You would need to put 1) and 2) into a transaction. During this
transactions logins would be blocked, but it would be a matter of
centiseconds.

If you can find a method to define a clean cut a head, then you could
consider partitioned views. That is, you would have a set of table
that are united in a view, and a CHECK constraint defining which
intervals that go into which table. Insertions would be into the view.
You would transfer one table a time, and then truncate and finally
redefine it to fit another slot in the future.

--
Erland Sommarskog, SQL Server MVP, es****@sommarskog.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx
Feb 2 '06 #4
Thanks. We'll try this method.

Feb 3 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

11
by: CSN | last post by:
Is it possible to iterate over an array in plpgsql? Something like: function insert_stuff (rel_ids int) .... foreach rel_ids as id insert into table (rel_id, val) values (id, 5);
6
by: Greg | last post by:
I am working on a project that will have about 500,000 records in an XML document. This document will need to be queried with XPath, and records will need to be updated. I was thinking about...
24
by: Salad | last post by:
Every now and then I see ads that state something like "Experience with Large Databases ...multi-gig...blah-de-blah" And I have to laugh. What's the difference between a large or small database? ...
5
by: Louis LeBlanc | last post by:
Hey folks. I'm new to the list, and not quite what you'd call a DB Guru, so please be patient with me. I'm afraid the lead up here is a bit verbose . . . I am working on an application that...
11
by: Macca | last post by:
Hi, I'm writing an application that will pass a large amount of data between classes/functions. In C++ it was more efficient to send a pointer to the object, e.g structure rather than passing...
10
by: msnnews.msn.com | last post by:
hi there, i've got a photo gallery that programatically creates a subfolder for each gallery, if i delete the album in my site, it also deletes the subfolder, but resets my website and i lose...
6
by: Nick Keighley | last post by:
Hi, Is this code fundamentally broken? class B { } class D: public B {
6
by: sara440 | last post by:
i want to take backup of one table(tables's data) of database, on hard disk or cd.i have a button "backup" on my form that should ask for backup when clicked.(it should allow to take backup on Cd). ...
25
by: tekctrl | last post by:
Anyone: I have a simple MSAccess DB which was created from an old ASCII flatfile. It works fine except for something that just started happening. I'll enter info in a record, save the record,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.