HTTPWebRequest unescapes automatically

[problem]

I am trying to use HTTPWebRequest to make a server side HTTP request to get response XML from a vendor. The vendor expects a bunch of querystring parameters, some of which have special characters in their values.

For some reason HTTPWebRequest automatically unescapes these values in the querystring of the URL and messes up the end result. I then tried to use the System.URI.URI object and used the dontEscape=true constructor. But then later realized that this constructor has become obsolete since .NET 2.0.

Does anyone have any solution for this? Is there an alternative to calling HTTPWebRequest to make a server side HTTP GET request?

Thanks in advance.

[Plater]

URL's should always be sent escaped (ie: a %20 instead of a space), I'm fairly sure it's part of the RFC for the protocol.
However, there are Escape() and Unescape() functions you might be able to use to your advantage. As well being able to piece all of the query string parameters back together if need be.

[problem]

The querystring is correctly escaped/unescaped. If I try to escape it again and send, the HTTPWebRequest call does not unescape it at all. For some reason it is acting wierd. Below is the value that I need to pass in the querystring.

zxeti%8Mfjus%2Nfxgdvetsa%7F057.0.1.4%5Cekefqrab%9Mhfl%1Fx-sytz-dvet-sfkgnus%7NVcha-Aioyu%546A%2MNycrolc%362I3.0%2M%2811cqwabdlkoe%264L%1EMUSP%2E5.0%264L%1EWkxopgv%5BPD%3L8.4%454C%5KVV3%363E%5B.XPU%5KFLT%3C1.0.4523%2812%4Mcfcxuwsdi%8Munoexkydo%1Fvgbdjyq%0C7.6%2Fbhsuszo%0Lgi43%4K4012561056962-%3Dfhaviqx%8M8.8%4Mlddlxqif%8DPHL9C157052M77E1L2P9K663744PI670%2Nrs%6F43%3Dan%0C40%3MA-Lnuz-Vcha-Hgkofbv%0CWcps-Dphnv%4B+Pxcinvl%2I3.0+(npwsjwidvp%3E+PSKO+7.3%6B+Gtonrfv+PD+6.4%6B+CG2%6K+.POE+MOA+1.1.5351)::32B960B0DEC4AFE72915AE7AA584BA62

[Plater]

Now I am confused at what the problem is?

Take that string as exactly as you want it to appear, and put it into the httpWebRequest object. The object will escape it, send it, and the server will unescape it back to its original form.
I don't think you have to manually escape/unescape it.

Other then that I am lost at what the problem actually is.


If all else fails, bust out a Socket and send it yourself.

[problem]

Well sorry to confuse you. Let me try to explain again.

In the server side code of my application I make a HTTPWebRequest to a 3rd party vendor which uses HTTP Get to serve an XML formatted result. One of the parameters in this querystring is a long encoded string. If I open up a browser window and paste in this encoded string along with the URL, it works fine and returns the expected XML. But if I try to make the same call through HTTPWebRequest the result is an error. Hence the only reason it doesnt work is that the encoded string is messed up along the way (confirmed by intercepting the call through Fiddler). The encoded string is unescaped by the call.

I tried to HttpUtility.UrlEncode before passing it to WebRequest.Create method, but the same error. Then I tried UrlDecode and that didnt work either.

Does that make sense now?

[TRScheel]

The querystring is correctly escaped/unescaped. If I try to escape it again and send, the HTTPWebRequest call does not unescape it at all. For some reason it is acting wierd. Below is the value that I need to pass in the querystring.

zxeti%8Mfjus%2Nfxgdvetsa%7F057.0.1.4%5Cekefqrab%9Mhfl%1Fx-sytz-dvet-sfkgnus%7NVcha-Aioyu%546A%2MNycrolc%362I3.0%2M%2811cqwabdlkoe%264L%1EMUSP%2E5.0%264L%1EWkxopgv%5BPD%3L8.4%454C%5KVV3%363E%5B.XPU%5KFLT%3C1.0.4523%2812%4Mcfcxuwsdi%8Munoexkydo%1Fvgbdjyq%0C7.6%2Fbhsuszo%0Lgi43%4K4012561056962-%3Dfhaviqx%8M8.8%4Mlddlxqif%8DPHL9C157052M77E1L2P9K663744PI670%2Nrs%6F43%3Dan%0C40%3MA-Lnuz-Vcha-Hgkofbv%0CWcps-Dphnv%4B+Pxcinvl%2I3.0+(npwsjwidvp%3E+PSKO+7.3%6B+Gtonrfv+PD+6.4%6B+CG2%6K+.POE+MOA+1.1.5351)::32B960B0DEC4AFE72915AE7AA584BA62

Curiously, is there a maxium size? I always thought there was, and I would be sure this breaks that maximum size if there was ( I could swear its 255 characters...).

Just a thought, and I am not sure of it to begin with.

[Plater]

Curiously, is there a maxium size? I always thought there was, and I would be sure this breaks that maximum size if there was ( I could swear its 255 characters...).

Just a thought, and I am not sure of it to begin with.

The RFC for HTTP has a specific max line length I believe. (But it might just be SMTP that has that)

But regardless if it gets pasted into the URL bar just fine, it should work in the request object.

If there's not a security issue, would we be able to get the code/address for doing this (maybe send either me or TR a PM if that's better) so we can fiddle with it?
I admit, this has me curious.

That same text you posted, as URLEncoded
[notag]
zxeti%8Mfjus%2Nfxgdvetsa%7F057.0.1.4%5Cekefqrab%9M %20%20hfl%1Fx-sytz-dvet-sfkgnus%7NVcha-Aioyu%546A%2MNycrolc%362I3.0%2M%2811cqwabdlkoe%264 %20%20L%1EMUSP%2E5.0%264L%1EWkxopgv%5BPD%3L8.4%454 C%5KVV%20%203%363E%5B.XPU%5KFLT%3C1.0.4523%2812%4M cfcxuwsdi%8M%20%20unoexkydo%1Fvgbdjyq%0C7.6%2Fbhsu szo%0Lgi43%4K40125%20%2061056962-%3Dfhaviqx%8M8.8%4Mlddlxqif%8DPHL9C157052M77E1L2P9 %20%20K663744PI670%2Nrs%6F43%3Dan%0C40%3MA-Lnuz-Vcha-Hgkofbv%0CWcps-Dphnv%4B+Pxcinvl%2I3.0+(npwsjwidvp%3E+PSKO+7.3%6B+ %20%20Gtonrfv+PD+6.4%6B+CG2%6K+.POE+MOA+1.1.5351): :32B96%20%200B0DEC4AFE72915AE7AA584BA62
[/notag]

[problem]

Am sorry you wont be able to execute it on your end. The vendor has IP security for the HTTP Get call.

Here is the HTTPWebRequest code if that helps.

        string infospaceUrl = "http://www.something.com/page=thelongstring";
        try
        {
            UTF8Encoding encoding = new UTF8Encoding();
            byte[] encodedPayload = encoding.GetBytes("");
 
            // Set up our connection
            Uri uri = new Uri(HttpUtility.UrlPathEncode(infospaceUrl));
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
            request.Method = WebRequestMethods.Http.Get;
            request.ContentType = "text/xml";
            request.ContentLength = encodedPayload.Length;
 
            // Get the Response
            using (StreamReader responseStream = new StreamReader(request.GetResponse().GetResponseStream(), Encoding.UTF8))
            {
                context.Response.Write(responseStream.ReadToEnd());
            }
        }
        catch (Exception httpexception)
        {
            throw new HttpException(httpexception.Message + " " + infospaceUrl);
        }