I have a sproc that returns data from a table using a simple SELECT. There
are quite a few rows returned, e.g. ~150,000.
In my first application, I use a reader on the sproc and iterate through the
rows, writing them out to a file. This takes about 5 minutes until I close
the reader.
Whilst this is going on, I have another application that is trying to insert
rows into the table. Normally, the inserts happen straight away, but when
the reader is open each insert takes a very long time to complete.
I realise that this is not an unreasonable thing to happen, given that I am
trying to write to the table whilst reading from it, but the rows being
written will never be included in the WHERE clause in my select statement,
and even if they were (which they won't), I wouldn't want them included in
the selected rows.
Is there a way to read rows so that inserts can still occur without
blocking?
I am using VB.NET in VS2005, and SQL Server 2005.
TIA
Charles
Jun 27 '08
18 1700
I did read up about this, but it suggested that the default is row level locking,
It is not that easy. First, SQL Server decided itself whether to lock row, page, or table level.
Factors involved in this decision is selectivity of query and concurrent users. Also, something that
start up as row level can during execution escalate to table level.
But even with row level, you are not helped if SQL Server need to look at every row. How do SQL
Server know if a row satisfy your criteria without looking at it first? See my other post for
elaboration about this topic.
--
Tibor Karaszi, SQL Server MVP http://www.karaszi.com/sqlserver/default.asp http://sqlblog.com/blogs/tibor_karaszi
"Charles Law" <bl***@nowhere. comwrote in message news:uE******** ******@TK2MSFTN GP04.phx.gbl...
Hi Jack
I did read up about this, but it suggested that the default is row level locking, which seemed to
be the one I'd want anyway, so I haven't attempted to change it. I have just looked again, and I
see there is more to this than I first thought, so I will read up.
Cheers
Charles
"Jack Jackson" <jj******@cinno vations.netwrot e in message
news:55******** *************** *********@4ax.c om...
>On Thu, 19 Jun 2008 02:17:57 +0100, "Charles Law" <bl***@nowhere. com> wrote:
>>>I have a sproc that returns data from a table using a simple SELECT. There are quite a few rows returned, e.g. ~150,000.
In my first application, I use a reader on the sproc and iterate through the rows, writing them out to a file. This takes about 5 minutes until I close the reader.
Whilst this is going on, I have another application that is trying to insert rows into the table. Normally, the inserts happen straight away, but when the reader is open each insert takes a very long time to complete.
I realise that this is not an unreasonable thing to happen, given that I am trying to write to the table whilst reading from it, but the rows being written will never be included in the WHERE clause in my select statement, and even if they were (which they won't), I wouldn't want them included in the selected rows.
Is there a way to read rows so that inserts can still occur without blocking?
I am using VB.NET in VS2005, and SQL Server 2005.
TIA
Charles You might take a look at the Transact-SQL statement SET TRANSACTION ISOLATION LEVEL.
Hi Tibor
Yes, I was obviously viewing this too simplistically. I have a couple of
Kalen Delaney's books, and there are some good topics in them that prove how
naive I was.
Cheers.
Charles
"Tibor Karaszi" <ti************ *************** @hotmail.nomail .comwrote in
message news:8A******** *************** ***********@mic rosoft.com...
>I did read up about this, but it suggested that the default is row level locking,
It is not that easy. First, SQL Server decided itself whether to lock row,
page, or table level. Factors involved in this decision is selectivity of
query and concurrent users. Also, something that start up as row level can
during execution escalate to table level.
But even with row level, you are not helped if SQL Server need to look at
every row. How do SQL Server know if a row satisfy your criteria without
looking at it first? See my other post for elaboration about this topic.
--
Tibor Karaszi, SQL Server MVP http://www.karaszi.com/sqlserver/default.asp http://sqlblog.com/blogs/tibor_karaszi
"Charles Law" <bl***@nowhere. comwrote in message
news:uE******** ******@TK2MSFTN GP04.phx.gbl...
>Hi Jack
I did read up about this, but it suggested that the default is row level locking, which seemed to be the one I'd want anyway, so I haven't attempted to change it. I have just looked again, and I see there is more to this than I first thought, so I will read up.
Cheers
Charles
"Jack Jackson" <jj******@cinno vations.netwrot e in message news:55******* *************** **********@4ax. com...
>>On Thu, 19 Jun 2008 02:17:57 +0100, "Charles Law" <bl***@nowhere. com> wrote:
I have a sproc that returns data from a table using a simple SELECT. There are quite a few rows returned, e.g. ~150,000.
In my first application, I use a reader on the sproc and iterate through the rows, writing them out to a file. This takes about 5 minutes until I close the reader.
Whilst this is going on, I have another application that is trying to insert rows into the table. Normally, the inserts happen straight away, but when the reader is open each insert takes a very long time to complete.
I realise that this is not an unreasonable thing to happen, given that I am trying to write to the table whilst reading from it, but the rows being written will never be included in the WHERE clause in my select statement , and even if they were (which they won't), I wouldn't want them included in the selected rows.
Is there a way to read rows so that inserts can still occur without blocking?
I am using VB.NET in VS2005, and SQL Server 2005.
TIA
Charles
You might take a look at the Transact-SQL statement SET TRANSACTION ISOLATION LEVEL.
Yes, Kalen's books, along with Books Online and possibly some hints mentioned in this thread should
get you going on this. :-)
--
Tibor Karaszi, SQL Server MVP http://www.karaszi.com/sqlserver/default.asp http://sqlblog.com/blogs/tibor_karaszi
"Charles Law" <bl***@nowhere. comwrote in message news:ug******** ******@TK2MSFTN GP06.phx.gbl...
Hi Tibor
Yes, I was obviously viewing this too simplistically. I have a couple of Kalen Delaney's books,
and there are some good topics in them that prove how naive I was.
Cheers.
Charles
"Tibor Karaszi" <ti************ *************** @hotmail.nomail .comwrote in message
news:8A******** *************** ***********@mic rosoft.com...
>>I did read up about this, but it suggested that the default is row level locking,
It is not that easy. First, SQL Server decided itself whether to lock row, page, or table level. Factors involved in this decision is selectivity of query and concurrent users. Also, something that start up as row level can during execution escalate to table level.
But even with row level, you are not helped if SQL Server need to look at every row. How do SQL Server know if a row satisfy your criteria without looking at it first? See my other post for elaboration about this topic.
-- Tibor Karaszi, SQL Server MVP http://www.karaszi.com/sqlserver/default.asp http://sqlblog.com/blogs/tibor_karaszi
"Charles Law" <bl***@nowhere. comwrote in message news:uE******** ******@TK2MSFTN GP04.phx.gbl...
>>Hi Jack
I did read up about this, but it suggested that the default is row level locking, which seemed to be the one I'd want anyway, so I haven't attempted to change it. I have just looked again, and I see there is more to this than I first thought, so I will read up.
Cheers
Charles
"Jack Jackson" <jj******@cinno vations.netwrot e in message news:55****** *************** ***********@4ax .com... On Thu, 19 Jun 2008 02:17:57 +0100, "Charles Law" <bl***@nowhere. com> wrote:
>I have a sproc that returns data from a table using a simple SELECT. There >are quite a few rows returned, e.g. ~150,000. > >In my first application, I use a reader on the sproc and iterate through the >rows, writing them out to a file. This takes about 5 minutes until I close >the reader. > >Whilst this is going on, I have another application that is trying to insert >rows into the table. Normally, the inserts happen straight away, but when >the reader is open each insert takes a very long time to complete. > >I realise that this is not an unreasonable thing to happen, given that I am >trying to write to the table whilst reading from it, but the rows being >written will never be included in the WHERE clause in my select statement, >and even if they were (which they won't), I wouldn't want them included in >the selected rows. > >Is there a way to read rows so that inserts can still occur without >blocking ? > >I am using VB.NET in VS2005, and SQL Server 2005. > >TIA > >Charles >
You might take a look at the Transact-SQL statement SET TRANSACTION ISOLATION LEVEL.
Hi Andrew
Because of the speed of the connection, I think the extended times are
simply because of the time it takes to transfer that many records down the
wire. The file write is almost certainly not the slow bit.
If the sproc were to create the file then it would end up server-side, and I
need it client-side. Copying it would take much longer than the current 5
minutes.
Charles
"Andrew Morton" <ak*@in-press.co.uk.inv alidwrote in message
news:6b******** *****@mid.indiv idual.net...
Charles Law wrote:
>I have a sproc that returns data from a table using a simple SELECT. There are quite a few rows returned, e.g. ~150,000.
In my first application, I use a reader on the sproc and iterate through the rows, writing them out to a file. This takes about 5 minutes until I close the reader.
If you are doing
while moreRecords
begin
read record
write record to file
end
then maybe the writing to disk is the slow step, so how about
while moreRecords
begin
read record
append record to stringbuilder
end
write stringbuilder to file
?
I take it getting the sp to write to a file is not an option?
Andrew
Charles Law wrote:
I have a sproc that returns data from a table using a simple SELECT. There
are quite a few rows returned, e.g. ~150,000.
In my first application, I use a reader on the sproc and iterate through the
rows, writing them out to a file. This takes about 5 minutes until I close
the reader.
Whilst this is going on, I have another application that is trying to insert
rows into the table. Normally, the inserts happen straight away, but when
the reader is open each insert takes a very long time to complete.
I realise that this is not an unreasonable thing to happen, given that I am
trying to write to the table whilst reading from it, but the rows being
written will never be included in the WHERE clause in my select statement,
and even if they were (which they won't), I wouldn't want them included in
the selected rows.
Is there a way to read rows so that inserts can still occur without
blocking?
I am using VB.NET in VS2005, and SQL Server 2005.
TIA
Charles
As already has been mentioned, there is different levels of locking. As
you are reading so many lines, the row locks will probably escalate into
page locks or a table lock to preserve resources. When that happens, it
will also lock other rows than the ones that you have selected.
You can specify (ROWLOCK) in your query. That should keep the database
from escalating the locks.
--
Göran Andersson
_____ http://www.guffa.com
On Jun 18, 8:17 pm, "Charles Law" <bl...@nowhere. comwrote:
I have a sproc that returns data from a table using a simple SELECT. There
are quite a few rows returned, e.g. ~150,000.
In my first application, I use a reader on the sproc and iterate through the
rows, writing them out to a file. This takes about 5 minutes until I close
the reader.
Whilst this is going on, I have another application that is trying to insert
rows into the table. Normally, the inserts happen straight away, but when
the reader is open each insert takes a very long time to complete.
I realise that this is not an unreasonable thing to happen, given that I am
trying to write to the table whilst reading from it, but the rows being
written will never be included in the WHERE clause in my select statement,
and even if they were (which they won't), I wouldn't want them included in
the selected rows.
Is there a way to read rows so that inserts can still occur without
blocking?
I am using VB.NET in VS2005, and SQL Server 2005.
TIA
Charles
Consider using snapshot isolation, so that readers do not block
writers.
Hi Göran
Thanks for the tip. That fits in with what I am seeing. I will give it a
try.
Charles
"Göran Andersson" <gu***@guffa.co mwrote in message
news:eU******** ******@TK2MSFTN GP05.phx.gbl...
Charles Law wrote:
>I have a sproc that returns data from a table using a simple SELECT. There are quite a few rows returned, e.g. ~150,000.
In my first application, I use a reader on the sproc and iterate through the rows, writing them out to a file. This takes about 5 minutes until I close the reader.
Whilst this is going on, I have another application that is trying to insert rows into the table. Normally, the inserts happen straight away, but when the reader is open each insert takes a very long time to complete.
I realise that this is not an unreasonable thing to happen, given that I am trying to write to the table whilst reading from it, but the rows being written will never be included in the WHERE clause in my select statement, and even if they were (which they won't), I wouldn't want them included in the selected rows.
Is there a way to read rows so that inserts can still occur without blocking?
I am using VB.NET in VS2005, and SQL Server 2005.
TIA
Charles
As already has been mentioned, there is different levels of locking. As
you are reading so many lines, the row locks will probably escalate into
page locks or a table lock to preserve resources. When that happens, it
will also lock other rows than the ones that you have selected.
You can specify (ROWLOCK) in your query. That should keep the database
from escalating the locks.
--
Göran Andersson
_____ http://www.guffa.com
Hi Alex
I have just been reading about snapshot isolation and it also looks like a
likely candidate, as are some of the other suggestions.
Thanks.
Charles
"Alex Kuznetsov" <al****@gmail.c omwrote in message
news:f5******** *************** ***********@a70 g2000hsh.google groups.com...
On Jun 18, 8:17 pm, "Charles Law" <bl...@nowhere. comwrote:
>I have a sproc that returns data from a table using a simple SELECT. There are quite a few rows returned, e.g. ~150,000.
In my first application, I use a reader on the sproc and iterate through the rows, writing them out to a file. This takes about 5 minutes until I close the reader.
Whilst this is going on, I have another application that is trying to insert rows into the table. Normally, the inserts happen straight away, but when the reader is open each insert takes a very long time to complete.
I realise that this is not an unreasonable thing to happen, given that I am trying to write to the table whilst reading from it, but the rows being written will never be included in the WHERE clause in my select statement, and even if they were (which they won't), I wouldn't want them included in the selected rows.
Is there a way to read rows so that inserts can still occur without blocking?
I am using VB.NET in VS2005, and SQL Server 2005.
TIA
Charles
Consider using snapshot isolation, so that readers do not block
writers.
Charles,
I thought that I had read once from William Vaughn that in fact the
datareader is processing a big resultset, which is simply build in one time.
Therefore the problem seems strange for me.
Try it in the AdoNet newsgroup, this is honey for Bill.
Cor
"Charles Law" <bl***@nowhere. comschreef in bericht
news:Oa******** ******@TK2MSFTN GP05.phx.gbl...
Hi Kerry
The two applications are on different machines, connected by a fairly slow
link, which is why it takes so long to iterate through the rows returned
by the reader. The database server is clustered on another machine, which
is on a Gb link to the second application, but the first (reader)
application is at the other end of the slow connection.
I could run both on the same machine, but then I would have to transfer
the resulting file over the slow link, and that would take longer in real
time.
Charles
"Kerry Moorman" <Ke**********@d iscussions.micr osoft.comwrote in message
news:6A******** *************** ***********@mic rosoft.com...
>Charles,
Are the 2 applications on the same machine or different machines?
Is the database server on a different machine than the applications?
Kerry Moorman
"Charles Law" wrote:
>>I have a sproc that returns data from a table using a simple SELECT. There are quite a few rows returned, e.g. ~150,000.
In my first application, I use a reader on the sproc and iterate through the rows, writing them out to a file. This takes about 5 minutes until I close the reader.
Whilst this is going on, I have another application that is trying to insert rows into the table. Normally, the inserts happen straight away, but when the reader is open each insert takes a very long time to complete.
I realise that this is not an unreasonable thing to happen, given that I am trying to write to the table whilst reading from it, but the rows being written will never be included in the WHERE clause in my select statement, and even if they were (which they won't), I wouldn't want them included in the selected rows.
Is there a way to read rows so that inserts can still occur without blocking?
I am using VB.NET in VS2005, and SQL Server 2005.
TIA
Charles This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Chris Ochs |
last post by:
I want to do a series of inserts within a single transaction block, but with
postgresql if one insert fails, the whole block is aborted. Is there any
way to get around this behavior so that postgresql won't abort the entire
transaction if a single insert returns an error?
Chris
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
|
by: Charles Law |
last post by:
I have a sproc that returns data from a table using a simple SELECT. There
are quite a few rows returned, e.g. ~150,000.
In my first application, I use a reader on the sproc and iterate through the
rows, writing them out to a file. This takes about 5 minutes until I close
the reader.
Whilst this is going on, I have another application that is trying to insert
rows into the table. Normally, the inserts happen straight away, but when...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
| |
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| |