UTF8>UNICODE

[Meelis Lilbok]

Hi

My ASP pages uses UTF-8 encoding.

How to convert UTF-8 text from Request.Form("text") to UNICODE for searching
frm MSSQL Database?

Best regards;
Meelis

[Anthony Jones]

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:%2****************@TK2MSFTNGP04.phx.gbl...

Hi

My ASP pages uses UTF-8 encoding.

How to convert UTF-8 text from Request.Form("text") to UNICODE for searching frm MSSQL Database?

Best regards;
Meelis

x = Request.Form("text").

x now contains a Unicode string

When passing to a ADO command object parameter make sure the parameter type
is adVarWChar.

Anthony.

[Meelis Lilbok]

>

x = Request.Form("text").

Nope, x is in UTF-8 format! Thats the problem

I use activex dll and API calls to convert UTF-8 to UNICODE, but where use
of activex is disabled this will not work

Meelis

[Anthony Jones]

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:%2******************@TK2MSFTNGP02.phx.gbl...

x = Request.Form("text").

Nope, x is in UTF-8 format! Thats the problem

I use activex dll and API calls to convert UTF-8 to UNICODE, but where use
of activex is disabled this will not work

Meelis

VBScript supports only one string format and that is Unicode.

I suspect that the form submission is using UTF-8 but the server side script
doesn't know that and is treating it as ISO-8859-1 or the like. Hence you
are getting a Unicode string that contains a series of UTF-8 encodings.

What is the character encoding of page that contains the text control?

Does the page actually inform the client of the character encoding used for
the page?

What method is used to submit the form GET or POST?

What is the Enctype of the form?

Is AcceptCharset specified for the Form?

What Browser are you using?

Anthony.

[Meelis Lilbok]

> What is the character encoding of page that contains the text control?
UTF-8

Does the page actually inform the client of the character encoding used
for
the page? Yes

What method is used to submit the form GET or POST? POST
What is the Enctype of the form?
None, because page encoding is UTF-8
Is AcceptCharset specified for the Form? No
What Browser are you using?

IE6

Meelis

[Meelis Lilbok]

For example

If i enter into text box estonian word "väike"
and submit form to antoher pages search.asp
and read Request.Form("text")
i get väike (UTF-8)

Meelis


"Anthony Jones" <An*@yadayadayada.com> wrote in message
news:ul**************@TK2MSFTNGP03.phx.gbl...

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:%2******************@TK2MSFTNGP02.phx.gbl...

>
> x = Request.Form("text").

Nope, x is in UTF-8 format! Thats the problem

I use activex dll and API calls to convert UTF-8 to UNICODE, but where
use
of activex is disabled this will not work

Meelis

VBScript supports only one string format and that is Unicode.

I suspect that the form submission is using UTF-8 but the server side
script
doesn't know that and is treating it as ISO-8859-1 or the like. Hence you
are getting a Unicode string that contains a series of UTF-8 encodings.

What is the character encoding of page that contains the text control?

Does the page actually inform the client of the character encoding used
for
the page?

What method is used to submit the form GET or POST?

What is the Enctype of the form?

Is AcceptCharset specified for the Form?

What Browser are you using?

Anthony.

[Anthony Jones]

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:em**************@TK2MSFTNGP02.phx.gbl...

For example

If i enter into text box estonian word "väike"
and submit form to antoher pages search.asp
and read Request.Form("text")
i get väike (UTF-8)

Having looked into it a bit more it would seem that the forms approach just
isn't compatible with UTF-8 or unicode. There doesn't seem to be a way to
inform the server of the actual charset used to encode the form values.

I'm actually quite amazed at this.

What do you actually need to do?

Do you need to support input characters beyond ISO-8859-1? If not I would
suggest you ditch UTF-8 and use ISO-8859-1 everywhere instead.

Other wise it is possible to do the decoding in VBScript yourself but it's
really messy. A small VB6 component would make this a lot easier.

Ditching Forms may be another option and post XML instead. (This is what I
do, I don't use forms)

Anthony.

[Meelis Lilbok]

Hi

cant use ISO-8859-1, beacuse i need support cyrillic chars too.
its easier to use my activex dll with convert functions :))
Best Regadrs;
Meelis


"Anthony Jones" <An*@yadayadayada.com> wrote in message
news:%2***************@TK2MSFTNGP05.phx.gbl...

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:em**************@TK2MSFTNGP02.phx.gbl...

For example

If i enter into text box estonian word "väike"
and submit form to antoher pages search.asp
and read Request.Form("text")
i get väike (UTF-8)

Having looked into it a bit more it would seem that the forms approach
just
isn't compatible with UTF-8 or unicode. There doesn't seem to be a way to
inform the server of the actual charset used to encode the form values.

I'm actually quite amazed at this.

What do you actually need to do?

Do you need to support input characters beyond ISO-8859-1? If not I would
suggest you ditch UTF-8 and use ISO-8859-1 everywhere instead.

Other wise it is possible to do the decoding in VBScript yourself but it's
really messy. A small VB6 component would make this a lot easier.

Ditching Forms may be another option and post XML instead. (This is what
I
do, I don't use forms)

Anthony.

[Egbert Nierop \(MVP for IIS\)]

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:%2****************@TK2MSFTNGP04.phx.gbl...

Hi

My ASP pages uses UTF-8 encoding.

How to convert UTF-8 text from Request.Form("text") to UNICODE for
searching frm MSSQL Database?

use at the first line of your ASP page
<% codepage=65001%>

--
compatible web farm Session replacement for Asp and Asp.Net
http://www.nieropwebconsult.nl/asp_session_manager.htm

[Anthony Jones]

"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:ud**************@TK2MSFTNGP05.phx.gbl...

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:%2****************@TK2MSFTNGP04.phx.gbl...

Hi

My ASP pages uses UTF-8 encoding.

How to convert UTF-8 text from Request.Form("text") to UNICODE for
searching frm MSSQL Database?
use at the first line of your ASP page
<% codepage=65001%>

did you mean:-

<%@ codepage=65001 %>

I don't think that helps. The value of session.codepage doesn't seem to
impact the assumptions made by server about the encoding of the request
data.
--
compatible web farm Session replacement for Asp and Asp.Net
http://www.nieropwebconsult.nl/asp_session_manager.htm

[Egbert Nierop \(MVP for IIS\)]

"Anthony Jones" <An*@yadayadayada.com> wrote in message
news:es**************@TK2MSFTNGP04.phx.gbl...

"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:ud**************@TK2MSFTNGP05.phx.gbl...

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:%2****************@TK2MSFTNGP04.phx.gbl...

> Hi
>
> My ASP pages uses UTF-8 encoding.
>
> How to convert UTF-8 text from Request.Form("text") to UNICODE for
> searching frm MSSQL Database?
use at the first line of your ASP page
<% codepage=65001%>

did you mean:-

<%@ codepage=65001 %>

I don't think that helps. The value of session.codepage doesn't seem to
impact the assumptions made by server about the encoding of the request
data.

however you are wrong :)

This really is saying that all input Request.* and output (response.write)
processes UTF-8 format.

--
compatible web farm Session replacement for Asp and Asp.Net
http://www.nieropwebconsult.nl/asp_session_manager.htm

[Meelis Lilbok]

Hi Egbert
Problem is not displayng UTF-8, all pages are using UTF-8
Problem is when i wanna make a query from MSSQL server, then i must convert
UTF-8 to UNICODE.

And <% codepage=65001%> does not work on IIS4 :)

And this is only possible when i use ActiveX DLL with MultiByteToWidechar
and WideCharToMultybite API's.

Meelis


"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:eY**************@TK2MSFTNGP03.phx.gbl...

"Anthony Jones" <An*@yadayadayada.com> wrote in message
news:es**************@TK2MSFTNGP04.phx.gbl...

"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:ud**************@TK2MSFTNGP05.phx.gbl...

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:%2****************@TK2MSFTNGP04.phx.gbl...
> Hi
>
> My ASP pages uses UTF-8 encoding.
>
> How to convert UTF-8 text from Request.Form("text") to UNICODE for
> searching frm MSSQL Database?

use at the first line of your ASP page
<% codepage=65001%>

did you mean:-

<%@ codepage=65001 %>

I don't think that helps. The value of session.codepage doesn't seem to
impact the assumptions made by server about the encoding of the request
data.

however you are wrong :)

This really is saying that all input Request.* and output (response.write)
processes UTF-8 format.

--
compatible web farm Session replacement for Asp and Asp.Net
http://www.nieropwebconsult.nl/asp_session_manager.htm

[Egbert Nierop \(MVP for IIS\)]

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:Od**************@TK2MSFTNGP04.phx.gbl...

Hi Egbert
Problem is not displayng UTF-8, all pages are using UTF-8
Problem is when i wanna make a query from MSSQL server, then i must
convert UTF-8 to UNICODE.

And <% codepage=65001%> does not work on IIS4 :)
Why didn't you say so.
IIS4 indeed does not support that. Or better said, Oleautomation does not
support, so ADO and others do not support that either.
I'd really work on asking your boss upgrading! Because, if you need to
convert it manually, it will be a hard job, you'll end up converting all SQL
data / user-input data etc!

And this is only possible when i use ActiveX DLL with MultiByteToWidechar
and WideCharToMultybite API's. Meelis


"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:eY**************@TK2MSFTNGP03.phx.gbl...

"Anthony Jones" <An*@yadayadayada.com> wrote in message
news:es**************@TK2MSFTNGP04.phx.gbl...

"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:ud**************@TK2MSFTNGP05.phx.gbl...

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:%2****************@TK2MSFTNGP04.phx.gbl...
> Hi
>
> My ASP pages uses UTF-8 encoding.
>
> How to convert UTF-8 text from Request.Form("text") to UNICODE for
> searching frm MSSQL Database?

use at the first line of your ASP page
<% codepage=65001%>
did you mean:-

<%@ codepage=65001 %>

I don't think that helps. The value of session.codepage doesn't seem to
impact the assumptions made by server about the encoding of the request
data.

however you are wrong :)

This really is saying that all input Request.* and output
(response.write) processes UTF-8 format.

--
compatible web farm Session replacement for Asp and Asp.Net
http://www.nieropwebconsult.nl/asp_session_manager.htm

[Meelis Lilbok]

Yeah i know

Some our clients still!! use IIS4 and then i use again my ActiveX DLL to
convert all strings to UTF-8, works fine ;)
Meelis

"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:Oq**************@TK2MSFTNGP05.phx.gbl...

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:Od**************@TK2MSFTNGP04.phx.gbl...

Hi Egbert
Problem is not displayng UTF-8, all pages are using UTF-8
Problem is when i wanna make a query from MSSQL server, then i must
convert UTF-8 to UNICODE.

And <% codepage=65001%> does not work on IIS4 :)

Why didn't you say so.
IIS4 indeed does not support that. Or better said, Oleautomation does not
support, so ADO and others do not support that either.
I'd really work on asking your boss upgrading! Because, if you need to
convert it manually, it will be a hard job, you'll end up converting all
SQL data / user-input data etc!

And this is only possible when i use ActiveX DLL with MultiByteToWidechar
and WideCharToMultybite API's.

Meelis


"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:eY**************@TK2MSFTNGP03.phx.gbl...

"Anthony Jones" <An*@yadayadayada.com> wrote in message
news:es**************@TK2MSFTNGP04.phx.gbl...

"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:ud**************@TK2MSFTNGP05.phx.gbl...
>
> "Meelis Lilbok" <me***********@deltmar.ee> wrote in message
> news:%2****************@TK2MSFTNGP04.phx.gbl...
> > Hi
> >
> > My ASP pages uses UTF-8 encoding.
> >
> > How to convert UTF-8 text from Request.Form("text") to UNICODE for
> > searching frm MSSQL Database?
>
> use at the first line of your ASP page
> <% codepage=65001%>
>

did you mean:-

<%@ codepage=65001 %>

I don't think that helps. The value of session.codepage doesn't seem
to
impact the assumptions made by server about the encoding of the request
data.

however you are wrong :)

This really is saying that all input Request.* and output
(response.write) processes UTF-8 format.

> --
> compatible web farm Session replacement for Asp and Asp.Net
> http://www.nieropwebconsult.nl/asp_session_manager.htm
>

[Anthony Jones]

"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:eY**************@TK2MSFTNGP03.phx.gbl...

"Anthony Jones" <An*@yadayadayada.com> wrote in message
news:es**************@TK2MSFTNGP04.phx.gbl...

"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:ud**************@TK2MSFTNGP05.phx.gbl...

"Meelis Lilbok" <me***********@deltmar.ee> wrote in message
news:%2****************@TK2MSFTNGP04.phx.gbl...
> Hi
>
> My ASP pages uses UTF-8 encoding.
>
> How to convert UTF-8 text from Request.Form("text") to UNICODE for
> searching frm MSSQL Database?

use at the first line of your ASP page
<% codepage=65001%>

did you mean:-

<%@ codepage=65001 %>

I don't think that helps. The value of session.codepage doesn't seem to
impact the assumptions made by server about the encoding of the request
data.

however you are wrong :)

I am. Don't how I managed it in my first round of tests. Did them again
and it works as you say.

The receiving page needs to be using a codepage that matches the character
set that the client browser thinks the source page is using.

In IIS 5.1/IIS 6 setting Response.codepage has the same effect which is a
bit counter intuative.

This really is saying that all input Request.* and output (response.write)
processes UTF-8 format.

--
compatible web farm Session replacement for Asp and Asp.Net
http://www.nieropwebconsult.nl/asp_session_manager.htm

[Egbert Nierop \(MVP for IIS\)]

"Anthony Jones" <An*@yadayadayada.com> wrote in message
news:uB**************@TK2MSFTNGP04.phx.gbl...

"Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
message news:eY**************@TK2MSFTNGP03.phx.gbl...

"Anthony Jones" <An*@yadayadayada.com> wrote in message
news:es**************@TK2MSFTNGP04.phx.gbl...

>
> "Egbert Nierop (MVP for IIS)" <eg***********@nospam.invalid> wrote in
> message news:ud**************@TK2MSFTNGP05.phx.gbl...
>>
>> "Meelis Lilbok" <me***********@deltmar.ee> wrote in message
>> news:%2****************@TK2MSFTNGP04.phx.gbl...
>> > Hi
>> >
>> > My ASP pages uses UTF-8 encoding.
>> >
>> > How to convert UTF-8 text from Request.Form("text") to UNICODE for
>> > searching frm MSSQL Database?
>>
>> use at the first line of your ASP page
>> <% codepage=65001%>
>>
>
> did you mean:-
>
> <%@ codepage=65001 %>
>
> I don't think that helps. The value of session.codepage doesn't seem
> to
> impact the assumptions made by server about the encoding of the request
> data.
however you are wrong :)

I am. Don't how I managed it in my first round of tests. Did them again
and it works as you say.

The receiving page needs to be using a codepage that matches the character
set that the client browser thinks the source page is using.

Right, and that is set by using

Response.CharSet = "utf-8"
In IIS 5.1/IIS 6 setting Response.codepage has the same effect which is a
bit counter intuative.

This really is saying that all input Request.* and output
(response.write)
processes UTF-8 format.