WCF and Service Broker Architecture

[Karch]

Lets say I have a number of web servers that all feed data to a common SQL
Server database, which processes and sends to a master SQL Server. I am
looking at a solution that involves Service Broker at the higher levels and
that all works fine. My question is: what is the recommended way to get data
from the web server to the first-tier Service Broker? I've done a lot of
searching and reading and have not found any definitive answers. But my
ideas are:

1) Write directly to the database from the web server.
I don't really like this solution because it violates the concepts OO design
and abstraction. It also doesn't give me any assurances that the data gets
moved, other than maybe some logging. It also shifts alot of the processing
onto IIS, which I would like to be as "light" as possible, simply servicing
requests.
2) Write to a local queue managed managed by a WCF service hosted in a
Windows Service.
This gives me some abstraction, but also adds some layers that may be
unnecessary. And, if I'm not mistaken, it forces me to use a local MSMQ if I
want to make sure that the messages are durable. And there is still the
question of writing to the queue - seems like I still need to do this from
the IIS hosted web application or use memory-mapping or something. Maybe
someone could elaborate on this one.
3) Install SQL Express on the web servers and use Service Broker all the way
up the line.
This seems a bit of a hack to me - to install SQL Express on the web
servers - but would work. Also, does anyone know how this impacts licensing;
is it really truly free to install Express anywhere, even if its 50-100 web
servers?

Beyond the options above, what is the recommended way to traverse that first
segment to Service Broker if one needs durable messages? What if volatile
messages are okay - should it be done a different way? I feel pretty
comfortable with Service Broker once it gets the initial data - my issue is
getting the data in the pipeline to start.

Thanks
K

[Roger Wolter[MSFT]]

How does you middle tier normally talk to the database? If it uses TSQL
over normal SQL Client or OLEDB connections then this is the way to do the
Service Broker connection. Because the connection is transactional, if the
connection is lost the SEND will roll back and you can send it again.

If you decide to use SQL Express in the middle tier, you gain a little more
fault tolerance at the expense of a little more complexity. If your middle
tier can continue to queue messages while the database is down for
maintenance or the network isn't working, your application is significantly
more fault tolerant. I've always been a fan of reliability and fault
tolerance over OO purity and isolation.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

"Karch" <no****@absotutely.comwrote in message
news:O9**************@TK2MSFTNGP04.phx.gbl...

Lets say I have a number of web servers that all feed data to a common SQL
Server database, which processes and sends to a master SQL Server. I am
looking at a solution that involves Service Broker at the higher levels
and that all works fine. My question is: what is the recommended way to
get data from the web server to the first-tier Service Broker? I've done a
lot of searching and reading and have not found any definitive answers.
But my ideas are:

1) Write directly to the database from the web server.
I don't really like this solution because it violates the concepts OO
design and abstraction. It also doesn't give me any assurances that the
data gets moved, other than maybe some logging. It also shifts alot of the
processing onto IIS, which I would like to be as "light" as possible,
simply servicing requests.
2) Write to a local queue managed managed by a WCF service hosted in a
Windows Service.
This gives me some abstraction, but also adds some layers that may be
unnecessary. And, if I'm not mistaken, it forces me to use a local MSMQ if
I want to make sure that the messages are durable. And there is still the
question of writing to the queue - seems like I still need to do this from
the IIS hosted web application or use memory-mapping or something. Maybe
someone could elaborate on this one.
3) Install SQL Express on the web servers and use Service Broker all the
way up the line.
This seems a bit of a hack to me - to install SQL Express on the web
servers - but would work. Also, does anyone know how this impacts
licensing; is it really truly free to install Express anywhere, even if
its 50-100 web servers?

Beyond the options above, what is the recommended way to traverse that
first segment to Service Broker if one needs durable messages? What if
volatile messages are okay - should it be done a different way? I feel
pretty comfortable with Service Broker once it gets the initial data - my
issue is getting the data in the pipeline to start.

Thanks
K

[Roger Wolter[MSFT]]

By the way, here's a little extra reading on the subject:
http://msdn2.microsoft.com/en-us/arc.../bb245677.aspx
http://msdn2.microsoft.com/en-us/library/aa964144.aspx
--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

"Roger Wolter[MSFT]" <rw*****@online.microsoft.comwrote in message
news:FF**********************************@microsof t.com...

How does you middle tier normally talk to the database? If it uses TSQL
over normal SQL Client or OLEDB connections then this is the way to do the
Service Broker connection. Because the connection is transactional, if
the connection is lost the SEND will roll back and you can send it again.

If you decide to use SQL Express in the middle tier, you gain a little
more fault tolerance at the expense of a little more complexity. If your
middle tier can continue to queue messages while the database is down for
maintenance or the network isn't working, your application is
significantly more fault tolerant. I've always been a fan of reliability
and fault tolerance over OO purity and isolation.

--
This posting is provided "AS IS" with no warranties, and confers no
rights.
Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

"Karch" <no****@absotutely.comwrote in message
news:O9**************@TK2MSFTNGP04.phx.gbl...

>Lets say I have a number of web servers that all feed data to a common
SQL Server database, which processes and sends to a master SQL Server. I
am looking at a solution that involves Service Broker at the higher
levels and that all works fine. My question is: what is the recommended
way to get data from the web server to the first-tier Service Broker?
I've done a lot of searching and reading and have not found any
definitive answers. But my ideas are:

1) Write directly to the database from the web server.
I don't really like this solution because it violates the concepts OO
design and abstraction. It also doesn't give me any assurances that the
data gets moved, other than maybe some logging. It also shifts alot of
the processing onto IIS, which I would like to be as "light" as possible,
simply servicing requests.
2) Write to a local queue managed managed by a WCF service hosted in a
Windows Service.
This gives me some abstraction, but also adds some layers that may be
unnecessary. And, if I'm not mistaken, it forces me to use a local MSMQ
if I want to make sure that the messages are durable. And there is still
the question of writing to the queue - seems like I still need to do this
from the IIS hosted web application or use memory-mapping or something.
Maybe someone could elaborate on this one.
3) Install SQL Express on the web servers and use Service Broker all the
way up the line.
This seems a bit of a hack to me - to install SQL Express on the web
servers - but would work. Also, does anyone know how this impacts
licensing; is it really truly free to install Express anywhere, even if
its 50-100 web servers?

Beyond the options above, what is the recommended way to traverse that
first segment to Service Broker if one needs durable messages? What if
volatile messages are okay - should it be done a different way? I feel
pretty comfortable with Service Broker once it gets the initial data - my
issue is getting the data in the pipeline to start.

Thanks
K

[Karch]

OK, makes sense. Yes, in this case, my middle tier would be talking TSQL
over SQL Client. The transactional connection is good, but it only lets me
retry, while I would prefer to have the messages queued as you mention in
the second scenario. So, is this a recommended approach - to use SQL Express
along with the middle tier? Something about it seems like overkill. Does it
make any sense to use WCF with private queues on the web server and pick it
up on the middle tier, or is this just overcomplicating the situation? I
would like to have some fault tolerance and reliability on the web servers,
but it just seems strange to deploy SQL Express to them when maybe some
other transport might be more efficient.

"Roger Wolter[MSFT]" <rw*****@online.microsoft.comwrote in message
news:FF**********************************@microsof t.com...

How does you middle tier normally talk to the database? If it uses TSQL
over normal SQL Client or OLEDB connections then this is the way to do the
Service Broker connection. Because the connection is transactional, if
the connection is lost the SEND will roll back and you can send it again.

If you decide to use SQL Express in the middle tier, you gain a little
more fault tolerance at the expense of a little more complexity. If your
middle tier can continue to queue messages while the database is down for
maintenance or the network isn't working, your application is
significantly more fault tolerant. I've always been a fan of reliability
and fault tolerance over OO purity and isolation.

--
This posting is provided "AS IS" with no warranties, and confers no
rights.
Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

"Karch" <no****@absotutely.comwrote in message
news:O9**************@TK2MSFTNGP04.phx.gbl...

>Lets say I have a number of web servers that all feed data to a common
SQL Server database, which processes and sends to a master SQL Server. I
am looking at a solution that involves Service Broker at the higher
levels and that all works fine. My question is: what is the recommended
way to get data from the web server to the first-tier Service Broker?
I've done a lot of searching and reading and have not found any
definitive answers. But my ideas are:

1) Write directly to the database from the web server.
I don't really like this solution because it violates the concepts OO
design and abstraction. It also doesn't give me any assurances that the
data gets moved, other than maybe some logging. It also shifts alot of
the processing onto IIS, which I would like to be as "light" as possible,
simply servicing requests.
2) Write to a local queue managed managed by a WCF service hosted in a
Windows Service.
This gives me some abstraction, but also adds some layers that may be
unnecessary. And, if I'm not mistaken, it forces me to use a local MSMQ
if I want to make sure that the messages are durable. And there is still
the question of writing to the queue - seems like I still need to do this
from the IIS hosted web application or use memory-mapping or something.
Maybe someone could elaborate on this one.
3) Install SQL Express on the web servers and use Service Broker all the
way up the line.
This seems a bit of a hack to me - to install SQL Express on the web
servers - but would work. Also, does anyone know how this impacts
licensing; is it really truly free to install Express anywhere, even if
its 50-100 web servers?

Beyond the options above, what is the recommended way to traverse that
first segment to Service Broker if one needs durable messages? What if
volatile messages are okay - should it be done a different way? I feel
pretty comfortable with Service Broker once it gets the initial data - my
issue is getting the data in the pipeline to start.

Thanks
K

[Roger Wolter[MSFT]]

WCF with private queues will complicate the scenario and introduce
distributed transactions which have performance and reliability issues also.
I would be inclined to use Service Broker all the way out to the web server
layer and skip the middle layer if possible. With CLR stored procedures
it's generally not unreasonable to put enough logic into an activated stored
procedure to do the data validation that normally happens on the middle
tier - but that's a decision you will have to make based on the size and
complexity of your business logic.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

"Karch" <no****@absotutely.comwrote in message
news:%2******************@TK2MSFTNGP05.phx.gbl...

OK, makes sense. Yes, in this case, my middle tier would be talking TSQL
over SQL Client. The transactional connection is good, but it only lets me
retry, while I would prefer to have the messages queued as you mention in
the second scenario. So, is this a recommended approach - to use SQL
Express along with the middle tier? Something about it seems like
overkill. Does it make any sense to use WCF with private queues on the web
server and pick it up on the middle tier, or is this just overcomplicating
the situation? I would like to have some fault tolerance and reliability
on the web servers, but it just seems strange to deploy SQL Express to
them when maybe some other transport might be more efficient.

"Roger Wolter[MSFT]" <rw*****@online.microsoft.comwrote in message
news:FF**********************************@microsof t.com...

>How does you middle tier normally talk to the database? If it uses TSQL
over normal SQL Client or OLEDB connections then this is the way to do
the Service Broker connection. Because the connection is transactional,
if the connection is lost the SEND will roll back and you can send it
again.

If you decide to use SQL Express in the middle tier, you gain a little
more fault tolerance at the expense of a little more complexity. If your
middle tier can continue to queue messages while the database is down for
maintenance or the network isn't working, your application is
significantly more fault tolerant. I've always been a fan of reliability
and fault tolerance over OO purity and isolation.

--
This posting is provided "AS IS" with no warranties, and confers no
rights.
Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

"Karch" <no****@absotutely.comwrote in message
news:O9**************@TK2MSFTNGP04.phx.gbl...

>>Lets say I have a number of web servers that all feed data to a common
SQL Server database, which processes and sends to a master SQL Server. I
am looking at a solution that involves Service Broker at the higher
levels and that all works fine. My question is: what is the recommended
way to get data from the web server to the first-tier Service Broker?
I've done a lot of searching and reading and have not found any
definitive answers. But my ideas are:

1) Write directly to the database from the web server.
I don't really like this solution because it violates the concepts OO
design and abstraction. It also doesn't give me any assurances that the
data gets moved, other than maybe some logging. It also shifts alot of
the processing onto IIS, which I would like to be as "light" as
possible, simply servicing requests.
2) Write to a local queue managed managed by a WCF service hosted in a
Windows Service.
This gives me some abstraction, but also adds some layers that may be
unnecessary. And, if I'm not mistaken, it forces me to use a local MSMQ
if I want to make sure that the messages are durable. And there is still
the question of writing to the queue - seems like I still need to do
this from the IIS hosted web application or use memory-mapping or
something. Maybe someone could elaborate on this one.
3) Install SQL Express on the web servers and use Service Broker all the
way up the line.
This seems a bit of a hack to me - to install SQL Express on the web
servers - but would work. Also, does anyone know how this impacts
licensing; is it really truly free to install Express anywhere, even if
its 50-100 web servers?

Beyond the options above, what is the recommended way to traverse that
first segment to Service Broker if one needs durable messages? What if
volatile messages are okay - should it be done a different way? I feel
pretty comfortable with Service Broker once it gets the initial data -
my issue is getting the data in the pipeline to start.

Thanks
K