473,320 Members | 2,117 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Linq Over Dataset/SQL Server in terms of Performance, Concurrency

Hi Experts,

I am doing a prototype of providing data access (read, write & search)
through Web Service. We observed that the data storing in SQL Server 2005,
the memory size is always within 250MB. Our aim is to support ~50K
concurrency users.

After investigation, we are thinking to use In-memory database for achieving
higher goal. One of choices to use LINQ as data structure while I would like
to ask some considerations in the meanwhile.

- How is the performance, Concurrency Issues & Memory Management using LINQ
over Dataset and SQL server ?

Many Thanks.

Jun 12 '07 #1
4 3564
On Jun 12, 9:23 am, Wilson C.K. Ng
<WilsonC...@discussions.microsoft.comwrote:
I am doing a prototype of providing data access (read, write & search)
through Web Service. We observed that the data storing in SQL Server 2005,
the memory size is always within 250MB. Our aim is to support ~50K
concurrency users.

After investigation, we are thinking to use In-memory database for achieving
higher goal. One of choices to use LINQ as data structure while I would like
to ask some considerations in the meanwhile.

- How is the performance, Concurrency Issues & Memory Management using LINQ
over Dataset and SQL server ?
The memory requirements will very much depend on the data. I've
recently been working on a project with a 7GB database which fits into
~100MB in memory due to a few bits of cunning. Other cases might be
worse than the database.

What I'd be worrying about is the writing and searching side of
things. Would you write to memory and also write back to the database
at the same time? Do you have multiple servers, and would they have to
synchronize? (50K concurrent users sounds like a lot for one server,
but it depends on what you really mean by concurrent users.)

In terms of concurrency, Joe Duffy is working on a "PLINQ" project to
utilitise concurrency effectively within LINQ, but I don't know many
details about it.

Sorry not to have more hard facts to give you, but it really depends
immensely on what your data is like and exactly what you need to do
with it.

Jon

Jun 12 '07 #2
Hard to say much without knowing the underying data and it's organization.
For 50k and more users (concurrent=resource consuming?), the overall
architecture becomes quite critical.

Some, hopefully, useful pointers:

http://blogs.msdn.com/wriju/archive/...rformance.aspx
http://www.davepeck.org/oldessay/lin...tion-perf.html
http://www.bluebytesoftware.com/blog...b90c6442b.aspx

"Wilson C.K. Ng" <Wi********@discussions.microsoft.comha scritto nel
messaggio news:78**********************************@microsof t.com...
Hi Experts,

I am doing a prototype of providing data access (read, write & search)
through Web Service. We observed that the data storing in SQL Server 2005,
the memory size is always within 250MB. Our aim is to support ~50K
concurrency users.

After investigation, we are thinking to use In-memory database for
achieving
higher goal. One of choices to use LINQ as data structure while I would
like
to ask some considerations in the meanwhile.

- How is the performance, Concurrency Issues & Memory Management using
LINQ
over Dataset and SQL server ?

Many Thanks.
Jun 12 '07 #3
Thanks experts, let me tell more about the story.

Scenarios:
There are a few (one or two) tables storing some information of servers (ip,
name, other fields).
The system allows other users add/remove/search by keywords for the table(s).
It is estimated that there would be 50K concurrency users.
The content of table(s) is changing while the total size of table is always
within 250MB (when we store it in SQL Server).

Background:
Our previous project is to use SQL Server05 while the bottomneck is at the
ORM stuff in connecting to SQL Server. So, I would like to investigate if it
is in the case of in-memory db.

And I would like to ask about performance, Concurrency Issues & Memory
Management of LINQ over SQL/DataSet.
Many Thanks.

"Laura T." wrote:
Hard to say much without knowing the underying data and it's organization.
For 50k and more users (concurrent=resource consuming?), the overall
architecture becomes quite critical.

Some, hopefully, useful pointers:

http://blogs.msdn.com/wriju/archive/...rformance.aspx
http://www.davepeck.org/oldessay/lin...tion-perf.html
http://www.bluebytesoftware.com/blog...b90c6442b.aspx

"Wilson C.K. Ng" <Wi********@discussions.microsoft.comha scritto nel
messaggio news:78**********************************@microsof t.com...
Hi Experts,

I am doing a prototype of providing data access (read, write & search)
through Web Service. We observed that the data storing in SQL Server 2005,
the memory size is always within 250MB. Our aim is to support ~50K
concurrency users.

After investigation, we are thinking to use In-memory database for
achieving
higher goal. One of choices to use LINQ as data structure while I would
like
to ask some considerations in the meanwhile.

- How is the performance, Concurrency Issues & Memory Management using
LINQ
over Dataset and SQL server ?

Many Thanks.

Jun 12 '07 #4
I don't think that Linq for DataSets is going to help you here, due to
what I believe (based on what you said) the frequently changing content of
the tables. If you load the database into memory, you still will have to
persist the changes to the dataset in memory back to the database, and at
the least, you are going to have the same overhead on update operations that
you would have if you were just using the database.

On the select side, you might get a performance increase, but honestly,
with that many concurrent users, you will have to manage the concurrency to
updating and selecting that data which to me seems like a database is much
better suited for.

I think that you should take a look at how you are querying the database
and see what optimizations you can make there. You would be surprized at
the kinds of optimizations you can make using indexes, stored procedures,
changing the logic in queries, etc, etc.

--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

"Wilson C.K. Ng" <Wi********@discussions.microsoft.comwrote in message
news:40**********************************@microsof t.com...
Thanks experts, let me tell more about the story.

Scenarios:
There are a few (one or two) tables storing some information of servers
(ip,
name, other fields).
The system allows other users add/remove/search by keywords for the
table(s).
It is estimated that there would be 50K concurrency users.
The content of table(s) is changing while the total size of table is
always
within 250MB (when we store it in SQL Server).

Background:
Our previous project is to use SQL Server05 while the bottomneck is at the
ORM stuff in connecting to SQL Server. So, I would like to investigate if
it
is in the case of in-memory db.

And I would like to ask about performance, Concurrency Issues & Memory
Management of LINQ over SQL/DataSet.
Many Thanks.

"Laura T." wrote:
>Hard to say much without knowing the underying data and it's
organization.
For 50k and more users (concurrent=resource consuming?), the overall
architecture becomes quite critical.

Some, hopefully, useful pointers:

http://blogs.msdn.com/wriju/archive/...rformance.aspx
http://www.davepeck.org/oldessay/lin...tion-perf.html
http://www.bluebytesoftware.com/blog...b90c6442b.aspx

"Wilson C.K. Ng" <Wi********@discussions.microsoft.comha scritto nel
messaggio news:78**********************************@microsof t.com...
Hi Experts,

I am doing a prototype of providing data access (read, write & search)
through Web Service. We observed that the data storing in SQL Server
2005,
the memory size is always within 250MB. Our aim is to support ~50K
concurrency users.

After investigation, we are thinking to use In-memory database for
achieving
higher goal. One of choices to use LINQ as data structure while I would
like
to ask some considerations in the meanwhile.

- How is the performance, Concurrency Issues & Memory Management using
LINQ
over Dataset and SQL server ?

Many Thanks.


Jun 12 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

8
by: Bruce Stockwell | last post by:
the setup: Webservice/WinClient application/SQL server. VS.Net (visual basic) winform wizard creates a simple form with load cancel cancelall and datagrid bound to a simple Dataset with one...
14
by: diskoduro | last post by:
Hi!! Years ago I built a database to control the production of a little factory. The users wanted to work in a Windows Net workgroup so I created an mdb with all the tables and data an after...
14
by: Bihn | last post by:
I was reading about datareader which is said to be slimmer & faster then dataset. Since the datareader have to go fetching the dat from the database every time it need it, the data it gets then...
15
by: Joseph Geretz | last post by:
I'm a bit puzzled by the current recommendation not to send Datasets or Datatables between application tiers. http://support.microsoft.com/kb/306134 ...
7
by: Ronald S. Cook | last post by:
I've always been taught that stored procedures are better than writing SQL in client code for a number of reasons: - runs faster as is compiled and lives on the database server - is the more...
2
by: =?Utf-8?B?VmFuZXNzYQ==?= | last post by:
I want to develop a datagridview allowing column selection and filtering, like this: http://www.codeproject.com/KB/grid/GridFilter.aspx This example is using dataset. My question is if: a) I...
4
by: cj | last post by:
Can I use LINQ in VB .Net 2008 to get to a Visual FoxPro Table? I've seen demos of it with SQL Server but they've used it with a server browser and I don't think that'd work when I need to see VFP...
1
by: Frederik | last post by:
Hi all, Am I correct when stating that LINQ replaces somewhat DataTables? I have done some reading concerning LINQ, but I'm still puzzled as to whether I should use LINQ or not. My application...
2
by: Andy B | last post by:
Hi... I have an sql server 2005 database that has a table called TestXml in it. The table has the columns ID (identity, int) and MyXml(xml). I created a stored proc on the server to insert a row...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.