Select between raw socket programming vs. WCF

[Hao]

We are doing very intensive network communication. We collect data from
thousands of electric devices every minutes. The devices understand both
socket and web service. They are either embeded linux based system or
Windows based servers. WCF, as I understand, has overhead vs. native socket
..Net programming. Should I continue to use socket programming or look into
the new WCF model?
Thanks.
Hao

[Steven Cheng[MSFT]]

Hello Hao,

Regarding on your application scenario, since the electric devices are of
different heterogenous platform, therefore, normally we have two
considerable approachs:

1. Use raw socket and deine your own communication protocol to transfer
data between client and server application.

2. Use interopable XML webservice to transfer data

For WCF, it does support XML webservice conform interface/endpoint,
however, to make the webservice communiation interopable, you need to use
basic http protocol/channel. The problem here is that XML webservice over
http will always transfer data in plain XML text format. Thus, any binary
data will need to be encoded and decoded at webservice's server/client
side. Also, such binary-XML encoding will increase the size of the payload.
So for your case, as you said the operations would be performed
frequently, I suggest you still keep using the socket approach.

If you have any further questions or any other concerns, please feel free
to post here.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead

=============== =============== =============== =====

Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.

=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[Hao]

I understand and completly agree that socket programming is the most
efficient. Of cause the web service is more open to third party components
and are easier to integrate.
We already spent about a year building and released a new server using web
service to transfer data between .Net and Windows/Linux gsoap engines from
all devices. It works for small to medium data load. We are now
re-architecture because we realized the network traffic is usually
overloaded. Before we go back to sock programming, which we have to develop
our own protocol, we want to know if HTTP compression is valuable in
compacting data? Our data is alreasy strongly typed by gsoap in all devices.
Does WCF support compression? What is the typicall HTTP compression rate? A
lot of our data are strings and floats.
Thanks.
Hao

"Steven Cheng[MSFT]" <st*****@online .microsoft.comw rote in message
news:zm******** ******@TK2MSFTN GHUB02.phx.gbl. ..

Hello Hao,

Regarding on your application scenario, since the electric devices are of
different heterogenous platform, therefore, normally we have two
considerable approachs:

1. Use raw socket and deine your own communication protocol to transfer
data between client and server application.

2. Use interopable XML webservice to transfer data

For WCF, it does support XML webservice conform interface/endpoint,
however, to make the webservice communiation interopable, you need to use
basic http protocol/channel. The problem here is that XML webservice over
http will always transfer data in plain XML text format. Thus, any binary
data will need to be encoded and decoded at webservice's server/client
side. Also, such binary-XML encoding will increase the size of the
payload.
So for your case, as you said the operations would be performed
frequently, I suggest you still keep using the socket approach.

If you have any further questions or any other concerns, please feel free
to post here.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead

=============== =============== =============== =====

Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.

=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no
rights.

[=?Utf-8?B?Sm9obg==?=]

Hao,

Another option you might want to look at is Efficient XML, the
cross-platform binary XML protocol being standardized by the W3C. I work for
the company that developed it and we have implementations for both .NET and
Linux. The compression ratios are quite good. See
http://www.agiledelta.com/product_efx.html.

Hope this is helpful!,

John

"Hao" wrote:

I understand and completly agree that socket programming is the most
efficient. Of cause the web service is more open to third party components
and are easier to integrate.
We already spent about a year building and released a new server using web
service to transfer data between .Net and Windows/Linux gsoap engines from
all devices. It works for small to medium data load. We are now
re-architecture because we realized the network traffic is usually
overloaded. Before we go back to sock programming, which we have to develop
our own protocol, we want to know if HTTP compression is valuable in
compacting data? Our data is alreasy strongly typed by gsoap in all devices.
Does WCF support compression? What is the typicall HTTP compression rate? A
lot of our data are strings and floats.
Thanks.
Hao

"Steven Cheng[MSFT]" <st*****@online .microsoft.comw rote in message
news:zm******** ******@TK2MSFTN GHUB02.phx.gbl. ..

Hello Hao,

Regarding on your application scenario, since the electric devices are of
different heterogenous platform, therefore, normally we have two
considerable approachs:

1. Use raw socket and deine your own communication protocol to transfer
data between client and server application.

2. Use interopable XML webservice to transfer data

For WCF, it does support XML webservice conform interface/endpoint,
however, to make the webservice communiation interopable, you need to use
basic http protocol/channel. The problem here is that XML webservice over
http will always transfer data in plain XML text format. Thus, any binary
data will need to be encoded and decoded at webservice's server/client
side. Also, such binary-XML encoding will increase the size of the
payload.
So for your case, as you said the operations would be performed
frequently, I suggest you still keep using the socket approach.

If you have any further questions or any other concerns, please feel free
to post here.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead

=============== =============== =============== =====

Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.

=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no
rights.

[Steven Cheng[MSFT]]

Thanks for your reply Hao,

For WCF, so far there hasn't built-in compression setting(for http or
non-http channel). If your service will transfer some binary data, one
thing you could have a look is the MTOM encoding in WCF(also supported in
WSE 3.0):

Here are some web articles mentioned something about large data transfer in
WCF:

#Moving lots of data
http://wcf.netfx3.com/blogs/wcf_team...21/Moving-lots
-of-data.aspx

#Choosing a Message Encoder
http://msdn2.microsoft.com/en-us/library/aa751889.aspx

#Large Data and Streaming
http://msdn2.microsoft.com/en-us/library/ms733742.aspx

Hope this helps.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no rights.

[Steven Cheng[MSFT]]

Hello Hao,

How are you doing on this issue, have you got any further ideas or does the
information in my last reply helps a little also? If there is still
anything we can help, please don't hesitate to post here.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no rights.

[Hao]

Hi, Steven,
I spent a few days testing various bindings in WCF and found that using MTOM
achieves the best results in my case. I setup many clients built on WCF
running full speed to push data to my server via MTOM. The server consumed
about 8M bytes per second of data, which seems to be pretty good since my
LAN speed is only 100Mbps.
The next step is to figure out if gsoap on linux side can work with WCF via
MTOM. I am sure byte array works, but I like to see rich objects can be
serialized and deserialized automtically through MTOM. I know gsoap supports
MTOM. We alreasy passed the rich objects between gsoap and .Net 2.0 via ASMX
web services.
Any hint there? I have to wait for a few days until another developer has
time to develop the gsoap side.
Thanks.
Hao

"Steven Cheng[MSFT]" <st*****@online .microsoft.comw rote in message
news:Te******** ********@TK2MSF TNGHUB02.phx.gb l...

Hello Hao,

How are you doing on this issue, have you got any further ideas or does
the
information in my last reply helps a little also? If there is still
anything we can help, please don't hesitate to post here.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no
rights.

[Steven Cheng[MSFT]]

Thanks for your followup Hao,

Sure, if possible, using MTOM is the preferred choice. For normal byte
array transfer in webservice, they're encoded through base64 encoding
format which will add 33% overhead. And MTOM encoding will try keep the
original binary data as much as possible so as to have the minest size.

I'm not familar with gsoap, however, if MTOM is supported, I think you're
certainly recommend to use it for communicating with WCF webservice
endpoint.

Please feel free to post here if you get any further progress or there is
anything else we can help you.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no rights.

[Hao]

My client and service are configured to use MTOM encoding and
basicHttpBindin g. After I inspect the network traffic, I am surprised to see
the rich objects are actually transferred as XML data. Did I set the
encoding wrong? I though MTOM encoding will choose either base64 or binary
raw format to transfer data as attachment. Maybe I misunderstood the MTOM.
Does MTOM work for rich objects such as array of structures as well as byte
arrays?
Thanks.
Hao

"Steven Cheng[MSFT]" <st*****@online .microsoft.comw rote in message
news:zT******** ******@TK2MSFTN GHUB02.phx.gbl. ..

Thanks for your followup Hao,

Sure, if possible, using MTOM is the preferred choice. For normal byte
array transfer in webservice, they're encoded through base64 encoding
format which will add 33% overhead. And MTOM encoding will try keep the
original binary data as much as possible so as to have the minest size.

I'm not familar with gsoap, however, if MTOM is supported, I think you're
certainly recommend to use it for communicating with WCF webservice
endpoint.

Please feel free to post here if you get any further progress or there is
anything else we can help you.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no
rights.

[Steven Cheng[MSFT]]

Thanks for your reply Hao,

For service that enable MTOM binding, it will only apply on those data that
are declare as binary stream, such as byte[] (byte array). For other text
type or custom class object, they'll still be serialized as XML/text
stream(over webservice http transport binding).

#Large Data and Streaming
http://msdn2.microsoft.com/en-us/library/ms733742.aspx

For example, given the following contract:

[DataContract]
class MyClass
{
[DataMember]
byte[] rawData;
[DataMember]
string textData
}

only "rawData" (byte array) will benefit the MTOM encoding. MTOM do not
know anything about custom data type serialization/deserialization , it only
care about encoding binary data as raw byte stream format and wrapper them
with a multi-part MIME package.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no rights.