473,287 Members | 1,800 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

Encoding XML troubles

Hi,

I am trying to write a generic RSS/Atom/OPML feed client. The problem
is, that those xml feeds may have different encodings:

- <?xml version="1.0" encoding="ISO-8859-1" ?>...
- <?xml version="1.0" encoding="utf-8" ?>...
- ...

I am using the WebRequest functionality to get the feeds. So, my code
looks simplified like this:

WebRequest req = WebRequest.Create(url);
StreamReader reader = new StreamReader(..., Encoding.Default);
string result = readerEnc.ReadToEnd();

As you can see on the second line, I can (or must, because utf-8 is
default) already define the encoding type of the expected stream.
However, as I do not now the encoding type while fetching the xml
stream, I use Encoding.Default.

And now, I am in the middle of the problem: I like to read the result
xml string, get the encoding type and re-encode result string with the
correct encoding type. Otherwise, all special characters are not
readable or missing in the result string.

I have unlukely tried following work-arounds:
- convert directly the result xml string from Encoding.Default to XML
Encoding Type:
result = this.convertString(result, Encoding.Default,
Encoding.GetEncoding(myEncodingStringFromXMLFile)) ;

The convertString function uses similar code as the convert example on
msdn: http://msdn.microsoft.com/library/de...classtopic.asp
--> did not work - characters remained as they where before

- Creating a second StreamReader instance with the right encoding:
StreamReader reader2 = new StreamReader(...,
Encoding.GetEncoding(myEncodingStringFromXMLFile)) ;
string result = readerEnc.ReadToEnd();
--> did not work - it seems, that the ResponseStream from the
WebRequest class can only be read once! I am getting an error when
trying to modify the Position attribute on the stream (Another guy had
exactly the same problem:
http://groups.google.ch/groups?hl=de...f67a0c2&rnum=1)

Is there another solution, than fetching the URL twice? Do I miss some
basic functionalities? Thanks for your help...

Greets,

Phil
Nov 17 '05 #1
4 8786
fitsch wrote:
Hi,

I am trying to write a generic RSS/Atom/OPML feed client. The problem
is, that those xml feeds may have different encodings:

- <?xml version="1.0" encoding="ISO-8859-1" ?>...
- <?xml version="1.0" encoding="utf-8" ?>...
- ...

I am using the WebRequest functionality to get the feeds. So, my code
looks simplified like this:

WebRequest req = WebRequest.Create(url);
StreamReader reader = new StreamReader(..., Encoding.Default);
string result = readerEnc.ReadToEnd();

As you can see on the second line, I can (or must, because utf-8 is
default) already define the encoding type of the expected stream.
However, as I do not now the encoding type while fetching the xml
stream, I use Encoding.Default.
Note that Encoding.Default is your OS default character set, and no
magic catch all encoding. This step will already render a lot of XML
input useless.
And now, I am in the middle of the problem: I like to read the result
xml string, get the encoding type and re-encode result string with the
correct encoding type. Otherwise, all special characters are not
readable or missing in the result string.
Once it's a string, it's a string. You must re-*de*code bytes.
I have unlukely tried following work-arounds:
- convert directly the result xml string from Encoding.Default to XML
Encoding Type:
result = this.convertString(result, Encoding.Default,
Encoding.GetEncoding(myEncodingStringFromXMLFile)) ;

The convertString function uses similar code as the convert example on
msdn:
http://msdn.microsoft.com/library/de...ry/en-us/cpref
/html/frlrfsystemtextencodingclasstopic.asp --> did not work -
characters remained as they where before

- Creating a second StreamReader instance with the right encoding:
StreamReader reader2 = new StreamReader(...,
Encoding.GetEncoding(myEncodingStringFromXMLFile)) ;
string result = readerEnc.ReadToEnd();
--> did not work - it seems, that the ResponseStream from the
WebRequest class can only be read once! I am getting an error when
trying to modify the Position attribute on the stream (Another guy had
exactly the same problem:
http://groups.google.ch/groups?hl=de...f67a0c2&rnum=1)

Is there another solution, than fetching the URL twice? Do I miss some
basic functionalities? Thanks for your help...


The functionality to safely decode XML content is already available in
the BCL. Just use an XmlTextReader.

Cheers,
--
http://www.joergjooss.de
mailto:ne********@joergjooss.de
Nov 17 '05 #2
fitsch wrote:
Hi,

I am trying to write a generic RSS/Atom/OPML feed client. The problem
is, that those xml feeds may have different encodings:

- <?xml version="1.0" encoding="ISO-8859-1" ?>...
- <?xml version="1.0" encoding="utf-8" ?>...
- ...

I am using the WebRequest functionality to get the feeds. So, my code
looks simplified like this:

WebRequest req = WebRequest.Create(url);
StreamReader reader = new StreamReader(..., Encoding.Default);
string result = readerEnc.ReadToEnd();

As you can see on the second line, I can (or must, because utf-8 is
default) already define the encoding type of the expected stream.
However, as I do not now the encoding type while fetching the xml
stream, I use Encoding.Default.
Note that Encoding.Default is your OS default character set, and no
magic catch all encoding. This step will already render a lot of XML
input useless.
And now, I am in the middle of the problem: I like to read the result
xml string, get the encoding type and re-encode result string with the
correct encoding type. Otherwise, all special characters are not
readable or missing in the result string.
Once it's a string, it's a string. You must re-*de*code bytes.
I have unlukely tried following work-arounds:
- convert directly the result xml string from Encoding.Default to XML
Encoding Type:
result = this.convertString(result, Encoding.Default,
Encoding.GetEncoding(myEncodingStringFromXMLFile)) ;

The convertString function uses similar code as the convert example on
msdn:
http://msdn.microsoft.com/library/de...ry/en-us/cpref
/html/frlrfsystemtextencodingclasstopic.asp --> did not work -
characters remained as they where before

- Creating a second StreamReader instance with the right encoding:
StreamReader reader2 = new StreamReader(...,
Encoding.GetEncoding(myEncodingStringFromXMLFile)) ;
string result = readerEnc.ReadToEnd();
--> did not work - it seems, that the ResponseStream from the
WebRequest class can only be read once! I am getting an error when
trying to modify the Position attribute on the stream (Another guy had
exactly the same problem:
http://groups.google.ch/groups?hl=de...f67a0c2&rnum=1)

Is there another solution, than fetching the URL twice? Do I miss some
basic functionalities? Thanks for your help...


The functionality to safely decode XML content is already available in
the BCL. Just use an XmlTextReader.

Cheers,
--
http://www.joergjooss.de
mailto:ne********@joergjooss.de
Nov 17 '05 #3
fitsch <fi****@bluewin.ch> wrote:
I am trying to write a generic RSS/Atom/OPML feed client. The problem
is, that those xml feeds may have different encodings:

- <?xml version="1.0" encoding="ISO-8859-1" ?>...
- <?xml version="1.0" encoding="utf-8" ?>...
- ...

I am using the WebRequest functionality to get the feeds. So, my code
looks simplified like this:

WebRequest req = WebRequest.Create(url);
StreamReader reader = new StreamReader(..., Encoding.Default);
string result = readerEnc.ReadToEnd();


Why bother reading it as a string? The best solution is to get the
stream and pass it directly to XmlTextReader - then the XmlTextReader,
which knows how to deal with the encoding part of the XML declaration,
can do the right thing.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 17 '05 #4
fitsch <fi****@bluewin.ch> wrote:
I am trying to write a generic RSS/Atom/OPML feed client. The problem
is, that those xml feeds may have different encodings:

- <?xml version="1.0" encoding="ISO-8859-1" ?>...
- <?xml version="1.0" encoding="utf-8" ?>...
- ...

I am using the WebRequest functionality to get the feeds. So, my code
looks simplified like this:

WebRequest req = WebRequest.Create(url);
StreamReader reader = new StreamReader(..., Encoding.Default);
string result = readerEnc.ReadToEnd();


Why bother reading it as a string? The best solution is to get the
stream and pass it directly to XmlTextReader - then the XmlTextReader,
which knows how to deal with the encoding part of the XML declaration,
can do the right thing.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 17 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Xaver Hinterhuber | last post by:
Hello pythonistas, I program a class which stores the source code for an output page in a string. At request time it compiles it, executes it and returns the result. I now have upgraded the...
7
by: Erik Bethke | last post by:
Hello All, I have found much help in the google archives but I am still stuck... here is my code snippet: path = os.getcwd() path = path.decode('UTF8') Now the trouble is I am getting that...
9
by: Ksenia Marasanova | last post by:
Hi, I have a little problem with encoding. Was hoping maybe anyone can help me to solve it. There is some amount of data in a database (PG) that must be inserted into Excel sheet and emailed....
0
by: lmckaha | last post by:
Hi, Mysql version: 3.23.49 Solaris version: 2.7 gcc compiler version: 2.95.2 Python : 2.2.2 I'm evaluating the C and C++ API to decide which one to bye but I have many troubles.
6
by: jmgonet | last post by:
Hello everybody, I'm having troubles loading a Xml string encoded in UTF-8. If I try this code: ------------------------------ XmlDocument doc=new XmlDocument(); String s="<?xml...
0
by: fitsch | last post by:
Hi, I am trying to write a generic RSS/Atom/OPML feed client. The problem is, that those xml feeds may have different encodings: - <?xml version="1.0" encoding="ISO-8859-1" ?>... - <?xml...
0
by: Michal | last post by:
I have troubles with instaling .Net Framework 2.0 (Beta 2 - 2.0.50215). The main instalation went just fine, troubles begin with my attemt to run aspnet_regiss.exe -i. Asp.Net is instaled into IIS...
1
by: Brad Wood | last post by:
I am testing a web service that returns an XML doc as a string. I went through troubles when developing it whereby the saved output of the service would not display in Internet Explorer due to...
12
by: Atlas | last post by:
I'm working on a multilanguage ASP/HTML site using a IIS6 web server. It perfectly works with two languages (english and italian) in this way: - basically the same ASP code for every language -...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.