Unicode Character Issue

[Samuel]

Hi

I am trying to read text files that are saved in ANSI format with Unicode
characters such as French e German big S etc, and as I read the file these
characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the Stream
Reader the application will recognize it as Unicode characters.
Thank you,
Samuel

[Joe Duchtel]

Hello -

There is an overloaded constructor that takes an Encoding (e.g.
System.Text.UnicodeEncoding). Visual Basic treats all strings as
Unicode so if you do a ReadLine() to a String it should work.

Joe

On Jun 4, 12:04*pm, "Samuel" <[e-mail]:

Hi

I am trying to read text files that are saved in ANSI format with Unicode
characters such as French e German big S etc, and as I read the file these
characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the Stream
Reader the application will recognize it as Unicode characters.

Thank you,
Samuel

[Joe Duchtel]

Hello -

There is an overloaded constructor that takes an Encoding (e.g.
System.Text.UnicodeEncoding). Visual Basic treats its String as
Unicode so you should be find if you use ReadLine().

Joe

On Jun 4, 12:04*pm, "Samuel" <[e-mail]:

Hi

I am trying to read text files that are saved in ANSI format with Unicode
characters such as French e German big S etc, and as I read the file these
characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the Stream
Reader the application will recognize it as Unicode characters.

Thank you,
Samuel

[Joergen Bech]

Here is a small, working example. Disregard
the ReadBinary stuff. That was just something
from another project and could be written
smaller/cleaner/etc.

Regards,

Joergen Bech

---snip---

Option Explicit On
Option Strict On

Imports System.IO
Imports System.Text
Imports System

Public Class Form1

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles MyBase.Load
Dim ansi() As Byte = ReadBinary("d:\ansi.txt")
Dim s As String =
Encoding.GetEncoding("iso-8859-1").GetString(ansi)
TextBox1.Text = s

' Dim sb As New StringBuilder
' For Each b As Byte In ansi
' sb.Append(Chr(b))
' Next
' TextBox1.Text = sb.ToString
End Sub

Public Shared Function ReadBinary(ByVal file As String) As Byte()
Dim errorInformation As String = String.Empty
Dim result As Byte() = ReadBinary(file, errorInformation)
If String.IsNullOrEmpty(errorInformation) Then
Return result
Else
Throw New IOException(errorInformation)
End If
End Function

Public Shared Function ReadBinary(ByVal file As String, _
ByRef errorInformation As
String) As Byte()

Try
Dim fInfo As New FileInfo(file)
Dim numBytes As Long = fInfo.Length
Dim fStream As New FileStream(file, FileMode.Open,
FileAccess.Read)
Dim br As New BinaryReader(fStream)
Dim data() As Byte = br.ReadBytes(CInt(numBytes))

br.Close()
fStream.Close()

Return data

Catch ex As Exception
errorInformation = ex.Message
Return Nothing

End Try

End Function

On Wed, 4 Jun 2008 17:04:48 +0100, "Samuel"
<sa************@ntlworld.comwrote:

>Hi

I am trying to read text files that are saved in ANSI format with Unicode
characters such as French e German big S etc, and as I read the file these
characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the Stream
Reader the application will recognize it as Unicode characters.
Thank you,
Samuel

[Samuel]

Unicode didn't work but 'Default' did

Thank you,
Samuel
"Joe Duchtel" <du*****@gmail.comwrote in message
news:ae**********************************@79g2000h sk.googlegroups.com...
Hello -

There is an overloaded constructor that takes an Encoding (e.g.
System.Text.UnicodeEncoding). Visual Basic treats all strings as
Unicode so if you do a ReadLine() to a String it should work.

Joe

On Jun 4, 12:04 pm, "Samuel" <[e-mail]:

Hi

I am trying to read text files that are saved in ANSI format with Unicode
characters such as French e German big S etc, and as I read the file these
characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the Stream
Reader the application will recognize it as Unicode characters.

Thank you,
Samuel

[Cor Ligthert[MVP]]

Samuel,

Are you sure that the program that is showing you the characters is able to
read and show those characters.

You would not be the first to make the wrong conclussion because of this.

By instance Windows mail (Outlook Express) will not show these characters
as it is not send in HTML format.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:Og**************@TK2MSFTNGP02.phx.gbl...

Hi

I am trying to read text files that are saved in ANSI format with Unicode
characters such as French e German big S etc, and as I read the file these
characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the Stream
Reader the application will recognize it as Unicode characters.
Thank you,
Samuel

[Samuel]

Certainly, because when I save the same document in Unicode format I can
read it without any problem

Thank you,
Samuel
"Cor Ligthert[MVP]" <no************@planet.nlwrote in message
news:C3**********************************@microsof t.com...

Samuel,

Are you sure that the program that is showing you the characters is able
to read and show those characters.

You would not be the first to make the wrong conclussion because of this.

By instance Windows mail (Outlook Express) will not show these characters
as it is not send in HTML format.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:Og**************@TK2MSFTNGP02.phx.gbl...

>Hi

I am trying to read text files that are saved in ANSI format with Unicode
characters such as French e German big S etc, and as I read the file
these characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the Stream
Reader the application will recognize it as Unicode characters.
Thank you,
Samuel

[Cor Ligthert[MVP]]

Samuel,

By the way what do you mean with Ansi format.

Wikipedia does not know it.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:ej**************@TK2MSFTNGP06.phx.gbl...

Certainly, because when I save the same document in Unicode format I can
read it without any problem

Thank you,
Samuel
"Cor Ligthert[MVP]" <no************@planet.nlwrote in message
news:C3**********************************@microsof t.com...

>Samuel,

Are you sure that the program that is showing you the characters is able
to read and show those characters.

You would not be the first to make the wrong conclussion because of this.

By instance Windows mail (Outlook Express) will not show these
characters as it is not send in HTML format.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:Og**************@TK2MSFTNGP02.phx.gbl...

>>Hi

I am trying to read text files that are saved in ANSI format with
Unicode characters such as French e German big S etc, and as I read the
file these characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the Stream
Reader the application will recognize it as Unicode characters.
Thank you,
Samuel

[Samuel]

Open notepad and look in the list of the Encoding - did I have to say
encoding

Regards,
Samuel
"Cor Ligthert[MVP]" <no************@planet.nlwrote in message
news:5C**********************************@microsof t.com...

Samuel,

By the way what do you mean with Ansi format.

Wikipedia does not know it.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:ej**************@TK2MSFTNGP06.phx.gbl...

>Certainly, because when I save the same document in Unicode format I can
read it without any problem

Thank you,
Samuel
"Cor Ligthert[MVP]" <no************@planet.nlwrote in message
news:C3**********************************@microso ft.com...

>>Samuel,

Are you sure that the program that is showing you the characters is able
to read and show those characters.

You would not be the first to make the wrong conclussion because of
this.

By instance Windows mail (Outlook Express) will not show these
characters as it is not send in HTML format.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:Og**************@TK2MSFTNGP02.phx.gbl...
Hi

I am trying to read text files that are saved in ANSI format with
Unicode characters such as French e German big S etc, and as I read the
file these characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the
Stream Reader the application will recognize it as Unicode characters.
Thank you,
Samuel

[Joergen Bech]

In the context of the original post, Wikipedia does indeed have a few
articles describing the codes:
http://en.wikipedia.org/wiki/Windows_1252
http://en.wikipedia.org/wiki/ISO-8859-1

Regards,

Joergen Bech

On Thu, 5 Jun 2008 19:16:23 +0200, "Cor Ligthert[MVP]"
<no************@planet.nlwrote:

>Samuel,

By the way what do you mean with Ansi format.

Wikipedia does not know it.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:ej**************@TK2MSFTNGP06.phx.gbl...

>Certainly, because when I save the same document in Unicode format I can
read it without any problem

Thank you,
Samuel
"Cor Ligthert[MVP]" <no************@planet.nlwrote in message
news:C3**********************************@microso ft.com...

>>Samuel,

Are you sure that the program that is showing you the characters is able
to read and show those characters.

You would not be the first to make the wrong conclussion because of this.

By instance Windows mail (Outlook Express) will not show these
characters as it is not send in HTML format.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:Og**************@TK2MSFTNGP02.phx.gbl...
Hi

I am trying to read text files that are saved in ANSI format with
Unicode characters such as French e German big S etc, and as I read the
file these characters appear as squares etc.

I know that if the file would be saved as Unicode this wouldn't be a
problem.

The question is whether there is an option that when I create the Stream
Reader the application will recognize it as Unicode characters.
Thank you,
Samuel

[Cor Ligthert[MVP]]

Samuel,

There is often spoken about ANSI when is meant the code table for
characters, but ANSI does give as much information as when you ask for about
a problem with a car and tell that it is an European brand.

The used bit format, or tell that you are using a code table, or even ASCII
although most people are then in fact talking about extended ASCII which is
for English and Dutch code table 437 gives much more information. While
using true ASCII which only has 7 bits characters (in fact 26 alphabetic
ones in upper and lower case).

As you are encoding between those, it will always take the characters as are
in the resulting code.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:OH**************@TK2MSFTNGP03.phx.gbl...

Open notepad and look in the list of the Encoding - did I have to say
encoding

Regards,
Samuel
"Cor Ligthert[MVP]" <no************@planet.nlwrote in message
news:5C**********************************@microsof t.com...

>Samuel,

By the way what do you mean with Ansi format.

Wikipedia does not know it.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:ej**************@TK2MSFTNGP06.phx.gbl...

>>Certainly, because when I save the same document in Unicode format I can
read it without any problem

Thank you,
Samuel
"Cor Ligthert[MVP]" <no************@planet.nlwrote in message
news:C3**********************************@micros oft.com...
Samuel,

Are you sure that the program that is showing you the characters is
able to read and show those characters.

You would not be the first to make the wrong conclussion because of
this.

By instance Windows mail (Outlook Express) will not show these
characters as it is not send in HTML format.

Cor

"Samuel" <sa************@ntlworld.comschreef in bericht
news:Og**************@TK2MSFTNGP02.phx.gbl...
Hi
>
I am trying to read text files that are saved in ANSI format with
Unicode characters such as French e German big S etc, and as I read
the file these characters appear as squares etc.
>
I know that if the file would be saved as Unicode this wouldn't be a
problem.
>
The question is whether there is an option that when I create the
Stream Reader the application will recognize it as Unicode characters.
>
>
Thank you,
Samuel
>