Set Request.ContentEncoding from code?

[Ricky K. Rasmussen]

Hello NG,

I'm having some encoding trouble using my own RequestHandler:

Since the RequestHandler must be able to serve content with diffrent
encoding, I set the Response.Charset and Response.ContentEncoding to the
encoding of current content. Furthermore I set the Content-Type META tag of
the html page to use the same charset as well.

And this all works fine... I get my pages served and both the HTTP headers
and the HTML page has the right encoding applied... Yieppie!!!

But then... when I try to submit a FORM on one of my pages, all my special
characters (like "ãøåöäé") gets lost in the process.

The only way I can preserve the special characters, is by setting the
requestEncoding attribute (specified in <globalization> in web.config) to
the same encoding as I the current Response.ContentEncoding. And that is a
bad solution since all of my requests could be in different encodings.

It doesn't work if i try to set the Response.ContentEncoding to the same as
the Request.ContentEncoding. It's as if the Request already has been decoded
using the wrong format (specified in <globalization> in web.config).

Can anyone please help? I'm I on the wrong track? Is it impossible to serve
diffrent request/response encodings through the same web site? Or am I just
doing it the wrong way?

Thanks for your time,

Ricky

[Steven Cheng[MSFT]]

Hi Ricky,

From your description, you have met some problem manually setting the
Request's ContentEncoding in ASP.NET via code at runtime, you found it only
works when you specifying them in web.config however what you want it set
different value for each comming request rather than all the same in
web.config , yes?

As for this problem, here are my understanding and suggestions:
The Request/Response 's ContentEncoding set in web.config's Globalization
section is the default setting for the asp.net webapplication. Generally
the comming webrequest is using the encoding from the client(default
setting in browser), if there is no from client, the default setting in
web.config will be used. And also, the Asp.net 's Request and Response
object ContentEncoding can help use manually set them via code. And the
Response's ContentEncoding is nothing particular that we can just set it
before the response content return to client. However, the Request's
ContentEncoding will be a bit different , because when the request comming
, the asp.net runtime will soon checking the ContentEncoding (from client ,
if not, use the default in web.config), So if we set it later in Page's
processing life cycle, it's too late. We must set it before the request is
being processing. In ASP.NET there're several Events during each request's
processing , such as BeginRequest, AuthenticateRequest, EndRequest ...
And the "BeginREquest" is exactly the one we have to use, you can hook this
event in the application 's Global object(Global.asax.cs) or make a custom
HttpModule to handle it. Here are some related reference in MSDN:

#HttpApplication.BeginRequest Event
http://msdn.microsoft.com/library/en...webhttpapplica
tionclassbeginrequesttopic.asp?frame=true

#Custom HttpModule Example
http://msdn.microsoft.com/library/en...tomhttpmodules.
asp?frame=true

Also, following is a former thread discussing on a similiar issue, you may
also have a look:
#query string encoding/decoding
http://groups.google.com/groups?hl=e...readm=PLNLjDdA
EHA.1288%40cpmsftngxa06.phx.gbl&rnum=1&prev=/groups%3Fhl%3Den%26lr%3D%26ie%3
DUTF-8%26oe%3DUTF-8%26q%3Dquerystring%2Bsteven%2Bcheng

Hope helps. Thanks.
Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

Get Preview at ASP.NET whidbey
http://msdn.microsoft.com/asp.net/whidbey/default.aspx

[Ricky K. Rasmussen]

Thanks for you reply Steven. It works just fine setting the
Request.ContentEncoding in the Application BeginRequest event.
Now my only trouble is how do I select the right encoding for the Request?
Please let me elaborate:

1. I have a RequestHandler that serves content using diffrent encoding. So
when a danish page is being served the "ISO-8859-1" encoding is used by
setting the Response.ContentEncoding property accordingly.

2. The problem occurrs when a user submits form data from a page using
encoding other than the default defined in web.config (usually "utf-8").
Since the request is always beeing decoded using the defaut encoding this
results in characters beeing lost in the process.

3. The solution is to set the Request.ContentEncoding in the Application
BeginRequest event. But how do i know what encoding to use in this context?
Are there any way to tell which encoding the browser used when sending the
request?

Since I have to tell the browser which encoding to use in both the HTTP
Header and the HTML Content-Type Meta tag, the least it could do was to
return the favour and tell me which encoding it used when POSTing data to
me.

Thanks for your time,
Ricky

[Steven Cheng[MSFT]]

Hi Ricky,

Thanks for your followup. As for the new question, here are my
understandings:

Generally, all the infos we can ge from the client are all set in the Http
Request 's Header area , the Request object's
UserAgent, Request.ContentType,Request.Browser... etc
We can use the following code to loop through the comming request's header
collection:
foreach(string key in Request.Headers.Keys)
{
Response.Write("<br>"+key+ " : " + Request.Headers[key]);
}

There are some optional items such ad
ACCEPT-LANGUAGE
ACCEPT-ENCODING
ACCEPT-CHARSET

which indicate what kind of language/encoding type / charset the client
browser support. But I've not found anything which directly tell us what
kind of encoding the clieht is using. I think this is determined by the
client machine's OS. And for those new OS such as NT or XP which support
unicode will automatically use the UTF-8 or the certain charset
/contenttype from the serveside when first time getting the page from
server. I think on those old OS such as win98 which doesn't use Unicode may
have different behavior, you may have a test.

Also, you can use the above means to loop the Request's header collection
to see whether there is anyitem which is useful for your determination of
the contentEncoding to change in BeginRequest. If not, I'm afraid we
haven't any means to get anyother clientside's info so far.

Please have a check and if you have any questions, please feel free to post
here. Thanks.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

Get Preview at ASP.NET whidbey
http://msdn.microsoft.com/asp.net/whidbey/default.aspx

[Steven Cheng[MSFT]]

Hi Ricky,

Have you had a chance to check out the suggestions in my last reply or have
you got any further ideas on this issue? If you have anything unclear or if
there're anything else we can help, please feel free to post here. Thanks.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

Get Preview at ASP.NET whidbey
http://msdn.microsoft.com/asp.net/whidbey/default.aspx

[Ricky K. Rasmussen]

Hi Steven,

Please forgive me for forgetting to write back with an update on my issue.

I tried looking at the Request object and the informatin found in the
Headers collection, but it provided me with no usefull information when
deciding what encoding should be applied to it.

The most usefull header must be the ACCEPT-CHARSET, but IE never provided me
with this one and Mozilla only returned a list of supported encoding
formats.

I guess this proves that you can't really use multiple encodings on one
ASP.NET application.

Cause as soon as you set the Response Encoding you'll need to decode
subsequent requests as well by applying the right ContentEncoding. And since
this can only be done in the Applications BegingRequest event (before any
process of the request, and creation of session) you'll have no idea what
encoding to use.

I'm still a bit frustrated about it, but for know my solution has been to
duplicate my ASP.NET application to multiple IIS websites and then applied
diffrent encoding to each of these by setting it in the web.config. Anyone
should be able to see that this is a bad solution.

Thanks for your time,
Ricky

[Steven Cheng[MSFT]]

Hi Ricky,

Thanks for the followup. Generally all the existing browsers won't send the
content-encoding header in request , some will send the ACCEPT-CHARSET but
which have nothing to do with the actual request or response's content. And
there also nothing on the contentencoding in the W3C's http specification.
So the problem is exising on all the browsers since the browser will
encoded the posted request's content. Here is a certain http1.1 content
encoding reference:

http://i4net.tv/marticle/mod_gzip/encoding.htm

Also, since the UTF-8 is currently the most widely used encoding type which
can represent all the datas from different district. So most browser and
serverside web application use this as the default encoding to process the
request's content. Anyway, I'll consult some further experts to see whether
they have any ideas on this issue and will update you if I got any addition
infos. Thanks.
Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

Get Preview at ASP.NET whidbey
http://msdn.microsoft.com/asp.net/whidbey/default.aspx

[Steven Cheng[MSFT]]

Hi Ricky,

Regarding on the problem, I'm still consulting and will update you as soon
as possible. Thanks for your understanding.

Regards,

Steven Cheng
Microsoft Online Support

[Steven Cheng[MSFT]]

Hi Ricky,

I'm sorry for keeping you waiting for so long time. After further
consulting on some experts, they also confirm that the standard request
dosn't contain such a field which represent the request data's encoding
charset. And we could not get the exactly encoding charset from any header
items but looking at the bytes. And they also suggest that the
ACCEPT- LANGUAGES header is helpful on determining the encoding charset to
use, just check this value in BeginRequest event and set the appropriate
ContentEncoding. Thanks.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

Get Preview at ASP.NET whidbey
http://msdn.microsoft.com/asp.net/whidbey/default.aspx