473,657 Members | 2,825 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Parsing XML over TCP/IP

Sorry if this has been posted before...

I am receiving XML messages over a TCP client. Messages vary in size and
sometimes can arrive in groups. Thus, I am not guaranteed to receive a full
message in a single read from my socket.

I am loading each message into an XmlDocument after it arrives. I need a
way to make sure I have a complete XML message before I load it into an
XmlDocument or it will throw an exception. I am currently finding complete
messages by turning each read into a string and manually look for closing
tags. Surely there is an easier way to accomplish this, perhaps with an
XmlReader or XmlTextReader?

Thanks
Thomas
Nov 11 '05 #1
6 13491
Thomas Polan wrote:
Sorry if this has been posted before...

I am receiving XML messages over a TCP client. Messages vary in size and
sometimes can arrive in groups. Thus, I am not guaranteed to receive a full
message in a single read from my socket.

I am loading each message into an XmlDocument after it arrives. I need a
way to make sure I have a complete XML message before I load it into an
XmlDocument or it will throw an exception. I am currently finding complete
messages by turning each read into a string and manually look for closing
tags. Surely there is an easier way to accomplish this, perhaps with an
XmlReader or XmlTextReader?


I see two separate problems here - partial documents and grouped documents.
Former one is simple - just create XmlTextReader over the stream and it'll
read it till EOF. But if second document would arrive immediately after first
one XmlTextReader will throw XmlException, because that breaks XML
well-formdness rules. So you need some way to separate XML documents in a
stream, e.g. put \0 between them - it'll stop XmlTextReader just as EOF has
been encounered and then you can check if the stream is still can be read and
start another XmlTextReader.
--
Oleg Tkachenko
http://www.tkachenko.com/blog
Multiconn Technologies, Israel

Nov 11 '05 #2
What I'm really looking for is an XmlReader that blocks on a network stream
until data arrives. When xml arrives, it will read until it finds a
complete record and then notify me. I'm trying to avoid having to touch or
modify the stream at all.

Thomas

"Oleg Tkachenko" <oleg@NO_SPAM_P LEASEtkachenko. com> wrote in message
news:%2******** ********@TK2MSF TNGP09.phx.gbl. ..
Thomas Polan wrote:
Sorry if this has been posted before...

I am receiving XML messages over a TCP client. Messages vary in size and sometimes can arrive in groups. Thus, I am not guaranteed to receive a full message in a single read from my socket.

I am loading each message into an XmlDocument after it arrives. I need a way to make sure I have a complete XML message before I load it into an
XmlDocument or it will throw an exception. I am currently finding complete messages by turning each read into a string and manually look for closing tags. Surely there is an easier way to accomplish this, perhaps with an
XmlReader or XmlTextReader?
I see two separate problems here - partial documents and grouped

documents. Former one is simple - just create XmlTextReader over the stream and it'll
read it till EOF. But if second document would arrive immediately after first one XmlTextReader will throw XmlException, because that breaks XML
well-formdness rules. So you need some way to separate XML documents in a
stream, e.g. put \0 between them - it'll stop XmlTextReader just as EOF has been encounered and then you can check if the stream is still can be read and start another XmlTextReader.
--
Oleg Tkachenko
http://www.tkachenko.com/blog
Multiconn Technologies, Israel

Nov 11 '05 #3
Thomas Polan wrote:
What I'm really looking for is an XmlReader that blocks on a network stream
until data arrives. When xml arrives, it will read until it finds a
complete record and then notify me.

There is no such thing as "complete record" in XML. XmlReader reads to
the end of file or end of stream, so the following
<first-doc>
</first-doc>
<second-doc>
</second-doc>
will be treated as non wellformed XML document with 2 root element
nodes. If your XML documents don't come with xml declarations, you can
read above as single XML fragment by XmlTextReader, but then you have to
separate them yourself, actually it could be done based on Depth property.
--
Oleg Tkachenko
http://www.tkachenko.com/blog
Multiconn Technologies, Israel

Nov 11 '05 #4
Hello,
Sorry if this has been posted before...

I am receiving XML messages over a TCP client. Messages vary in size
and sometimes can arrive in groups. Thus, I am not guaranteed to
receive a full message in a single read from my socket.

I am loading each message into an XmlDocument after it arrives. I
need a way to make sure I have a complete XML message before I load
it into an XmlDocument or it will throw an exception. I am currently
finding complete messages by turning each read into a string and
manually look for closing tags. Surely there is an easier way to
accomplish this, perhaps with an XmlReader or XmlTextReader?


im working on the same stuff. First i took XMLDocument to validate if have a
complete node that i can parse. But the XmlDocument is very very slow.
Especially on compact Framework. Thats the Reason why im working at the
moment at a better and faster way with the XmlReader or XmlTextReader. You
must Read in a XmlReader and check the depth of the Reader. Then you know
when you have a complete node. The XmlReader must be bound to a Stream. My
problem is that i found no way for now to create a XmlDocument from the
Reader when i know that i have a complete now since this position.
Below is the code of my old Socket Queue class. xmlReader is a small wrapper
class here for the XmlDocument. Can send it to you when it helps.

Alex

public class SocketQueue
{
// static variable where we spool the socket Data when Streams
// are not partial
private static string m_XML = "";

public SocketQueue(str ing strSocket, Session s)
{
XML_Node xmlReader = new XML_Node(m_XML + strSocket);

if ( xmlReader.Valid == true )
{
// We have one valid XML Node
// send it to the parser

s.RaiseOnIncomi ngXML(m_XML + strSocket);
Parser parse = new Parser(m_XML + strSocket, s);
m_XML = "";
}
else
{
//XML_Node xmlReader2 = new XML_Node("<dumm y>" + strSocket +
"</dummy>");
XML_Node xmlReader2 = new XML_Node("<dumm y>" + m_XML + strSocket +
"</dummy>");
if(xmlReader2.V alid == true)
{
// we have more than 1 node
// get all childnodes and send them to the Parser
m_XML = "";

for (int i = 0; i < xmlReader2.XMLO bj.ChildNodes[0].ChildNodes.Cou nt;
i++)
{

s.RaiseOnIncomi ngXML(xmlReader 2.XMLObj.ChildN odes[0].ChildNodes[i].OuterXml)
;
Parser parse = new
Parser(xmlReade r2.XMLObj.Child Nodes[0].ChildNodes[i].OuterXml,s);
}
}
else
{
// we have no complete XML Node
m_XML = m_XML + strSocket;

}
}
}

}
}
Nov 11 '05 #5
Oleg,
Thank you for your suggestions. Using the depth property, I am able to
identify when I have a complete node.

Alexander,
I do not have my XmlReader bound to my TCP stream. Rather I point the
XmlReader to a string buffer. When I receive data from the server, I cast
the read into a global string variable. Each time I get data from the
socket, I concatenate this to my global string (I suppose I could be using a
stream instead of a string). I then try to look for a complete node. If I
find a complete node, I spawn a thread and call a procedure that loads the
complete node into an XmlDocument. Afterwards, I clear that xml from the
buffer and try to find another complete node until the XmlReader returns an
error. Here is my code:

'XmlFrag -> global string var

'Each time I receive data from the socket:

XmlFrag = XmlFrag & str 'str = read from socket converted to string
Do While XmlFrag <> "" 'Will run in loop until I parse the entire buffer
or XmlReader throws an exception
parseXML
End While

Private sub parseXml
Try
Dim nt as NameTable = new NameTable
Dim nsmgr as XmlNamespaceMan ager = new XmlNamespaceMan ager(nt)
Dim context as XmlParserContex t = new XmlParserContex t(Nothing, nsmgr,
Nothing, XmlSpace.None)
Dim reader as XmlTextReader = new XmlTextReader(X mlFrag,
XmlNodeType.Ele ment, context)
Dim sXML as string = ""

While (reader.Read())
Select Case reader.NodeType
Case XmlNodeType.Ele ment
If reader.Depth = 0 then 'If I am at the beginning of a
complete node
sXML = reader.ReadOute rXml() 'Try to read to the end
of the node, if an error is thrown, it means there was not a complete node
in the read
ThreadPool.Queu eUserWorkerItem (AddressOf routeXMLString,
sXML) 'spawn thread to deal with xml (load into XmlDocument)
XmlFrag = XmlFrag.Remove( 0, sXML.length) 'Clear
string of parsed xml
reader.Close()
Exit While
End if
End Select
End While
Catch ex As Exception
Throw New Exception(ex.Me ssage)
End Try

End Sub

You also might want to check out this SAX parser for .NET at:
http://www.xmlforasp.net/codeSection.aspx?csID=36

-Thomas

"Alexander Gnauck" <al*********@gm x.de> wrote in message
news:bg******** ****@ID-112594.news.uni-berlin.de...
Hello,
Sorry if this has been posted before...

I am receiving XML messages over a TCP client. Messages vary in size
and sometimes can arrive in groups. Thus, I am not guaranteed to
receive a full message in a single read from my socket.

I am loading each message into an XmlDocument after it arrives. I
need a way to make sure I have a complete XML message before I load
it into an XmlDocument or it will throw an exception. I am currently
finding complete messages by turning each read into a string and
manually look for closing tags. Surely there is an easier way to
accomplish this, perhaps with an XmlReader or XmlTextReader?
im working on the same stuff. First i took XMLDocument to validate if have

a complete node that i can parse. But the XmlDocument is very very slow.
Especially on compact Framework. Thats the Reason why im working at the
moment at a better and faster way with the XmlReader or XmlTextReader. You
must Read in a XmlReader and check the depth of the Reader. Then you know
when you have a complete node. The XmlReader must be bound to a Stream. My
problem is that i found no way for now to create a XmlDocument from the
Reader when i know that i have a complete now since this position.
Below is the code of my old Socket Queue class. xmlReader is a small wrapper class here for the XmlDocument. Can send it to you when it helps.

Alex

public class SocketQueue
{
// static variable where we spool the socket Data when Streams
// are not partial
private static string m_XML = "";

public SocketQueue(str ing strSocket, Session s)
{
XML_Node xmlReader = new XML_Node(m_XML + strSocket);

if ( xmlReader.Valid == true )
{
// We have one valid XML Node
// send it to the parser

s.RaiseOnIncomi ngXML(m_XML + strSocket);
Parser parse = new Parser(m_XML + strSocket, s);
m_XML = "";
}
else
{
//XML_Node xmlReader2 = new XML_Node("<dumm y>" + strSocket +
"</dummy>");
XML_Node xmlReader2 = new XML_Node("<dumm y>" + m_XML + strSocket +
"</dummy>");
if(xmlReader2.V alid == true)
{
// we have more than 1 node
// get all childnodes and send them to the Parser
m_XML = "";

for (int i = 0; i < xmlReader2.XMLO bj.ChildNodes[0].ChildNodes.Cou nt; i++)
{

s.RaiseOnIncomi ngXML(xmlReader 2.XMLObj.ChildN odes[0].ChildNodes[i].OuterXml) ;
Parser parse = new
Parser(xmlReade r2.XMLObj.Child Nodes[0].ChildNodes[i].OuterXml,s);
}
}
else
{
// we have no complete XML Node
m_XML = m_XML + strSocket;

}
}
}

}
}

Nov 11 '05 #6
hi Thomas,

You also might want to check out this SAX parser for .NET at:
http://www.xmlforasp.net/codeSection.aspx?csID=36


the Link to the Sax Parser looks very good.
Your code works too. I think you restart always the reader from the
beginning of the string. The advantage of a stream is that you can add bytes
to the stream without restarting the reader. So it would be a bit faster.
But the problem i have is when i call reader.ReadOute rXml() in the while
Reader.Read() loop then the reader stops. In my application i could also
have 2 or more complete nodes to parse in the socket stream. Then the reader
shouldnt stop. So i need a way to get the currentNode as string or XML
document without stopping the reader.

Alex
Nov 11 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

8
9436
by: Gerrit Holl | last post by:
Posted with permission from the author. I have some comments on this PEP, see the (coming) followup to this message. PEP: 321 Title: Date/Time Parsing and Formatting Version: $Revision: 1.3 $ Last-Modified: $Date: 2003/10/28 19:48:44 $ Author: A.M. Kuchling <amk@amk.ca> Status: Draft Type: Standards Track
2
3946
by: Cigdem | last post by:
Hello, I am trying to parse the XML files that the user selects(XML files are on anoher OS400 system called "wkdis3"). But i am permenantly getting that error: Directory0: \\wkdis3\ROOT\home Canonicalpath-Directory4: \\wkdis3\ROOT\home\bwe\ You selected the file named AAA.XML getXmlAlgorithmDocument(): IOException Not logged in
16
2878
by: Terry | last post by:
Hi, This is a newbie's question. I want to preload 4 images and only when all 4 images has been loaded into browser's cache, I want to start a slideshow() function. If images are not completed loaded into cache, the slideshow doesn't look very nice. I am not sure how/when to call the slideshow() function to make sure it starts after the preload has been completed.
0
4117
by: Pentti | last post by:
Can anyone help to understand why re-parsing occurs on a remote database (using database links), even though we are using a prepared statement on the local database: Scenario: ======== We have an schema (s1) on an Oracle 9i database with database links pointing to a schema (s2) on another Oracle 9i database.
9
4054
by: ankitdesai | last post by:
I would like to parse a couple of tables within an individual player's SHTML page. For example, I would like to get the "Actual Pitching Statistics" and the "Translated Pitching Statistics" portions of Babe Ruth page (http://www.baseballprospectus.com/dt/ruthba01.shtml) and store that info in a CSV file. Also, I would like to do this for numerous players whose IDs I have stored in a text file (e.g.: cobbty01, ruthba01, speaktr01, etc.)....
5
4297
by: randy | last post by:
Can some point me to a good example of parsing XML using C# 2.0? Thanks
3
4373
by: toton | last post by:
Hi, I have some ascii files, which are having some formatted text. I want to read some section only from the total file. For that what I am doing is indexing the sections (denoted by .START in the file) with the location. And for a particular section I parse only that section. The file is something like, .... DATAS
13
4489
by: Chris Carlen | last post by:
Hi: Having completed enough serial driver code for a TMS320F2812 microcontroller to talk to a terminal, I am now trying different approaches to command interpretation. I have a very simple command set consisting of several single letter commands which take no arguments. A few additional single letter commands take arguments:
7
2401
by: Daniel Fetchinson | last post by:
Many times a more user friendly date format is convenient than the pure date and time. For example for a date that is yesterday I would like to see "yesterday" instead of the date itself. And for a date that was 2 days ago I would like to see "2 days ago" but for something that was 4 days ago I would like to see the actual date. This is often seen in web applications, I'm sure you all know what I'm talking about. I'm guessing this...
1
4382
by: eyeore | last post by:
Hello everyone my String reverse code works but my professor wants me to use pop top push or Stack code and parsing code could you please teach me how to make this code work with pop top push or Stack code and parsing code my professor i does not like me using buffer reader on my code and my professor did even give me an example code for parsing as well as pop push top or Stack code and i don't know how to do this code into parsing and pop push...
0
8403
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8833
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8737
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8610
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
6174
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4168
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
2735
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
1967
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
2
1730
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.