Accentuated char

[Pascal Cloup]

Hello,

The methods Read() of the class StreamReader don't read the accentuated
characters. When an accentuated character is present in a file, Read() skip
it and read the following character. Missing something?

Thanks for help

P. Cloup

[Jon Skeet [C# MVP]]

Pascal Cloup <pc****@biogest a.fr> wrote:

The methods Read() of the class StreamReader don't read the accentuated
characters. When an accentuated character is present in a file, Read() skip
it and read the following character. Missing something?

Chances are you've got the wrong encoding. It certainly *does* read
accented characters when everything is correct. How are you
constructing your StreamReader, and what's your data source?

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

[Pascal Cloup]

Hello Jon
"Jon Skeet [C# MVP]" <sk***@pobox.co m> a écrit dans le message de
news:MP******** *************** *@msnews.micros oft.com...

Pascal Cloup <pc****@biogest a.fr> wrote:

The methods Read() of the class StreamReader don't read the accentuated
characters. When an accentuated character is present in a file, Read() skip it and read the following character. Missing something?
Chances are you've got the wrong encoding. It certainly *does* read
accented characters when everything is correct. How are you
constructing your StreamReader, and what's your data source?

In fact i create a file stream object and 2 stream objects:
itsFileStream = File.Open( itsPath , FileMode.Open );

itsBinaryReader = new BinaryReader( itsFileStream , Encoding.ASCII );
// Perhaps the problem is here but i also try UTF8

itsStreamReader = new StreamReader( itsFileStream );

Depending of the nature of the file (binary or text) i use one of the 2
streams , but the 2 remain open (??); i need the BinaryStream to determine
if the file is text or not. All works fine except for accentuated characters
(é à è) which are skipped.

Any idea?

Thanks in advance,

Pascal Cloup
--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

Pascal Cloup <pc****@biogest a.fr> wrote:

Chances are you've got the wrong encoding. It certainly *does* read
accented characters when everything is correct. How are you
constructing your StreamReader, and what's your data source?
In fact i create a file stream object and 2 stream objects:
itsFileStream = File.Open( itsPath , FileMode.Open );

itsBinaryReader = new BinaryReader( itsFileStream , Encoding.ASCII );
// Perhaps the problem is here but i also try UTF8

itsStreamReader = new StreamReader( itsFileStream );

That sounds like a bad idea to start with. Two readers on the same
stream is bound to cause problems.
Depending of the nature of the file (binary or text) i use one of the 2
streams , but the 2 remain open (??); i need the BinaryStream to determine
if the file is text or not. All works fine except for accentuated characters
(é à è) which are skipped.

I thought the point was that it was definitely a text file - otherwise
why are you trying to read it? *Any* file can be a text file, but it
depends on what encoding is being used as to what that file means.

You need to know what encoding the file is in, and specify that to the
StreamReader - ignore the BinaryReader.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

[Gary Chang]

Hi Pascal,

Thanks for posting in the community.

From your description, I understand that you want to read accented
characters(in a .txt file) with streamreader/binaryreader,
Please correct me if there is any misunderstand.

First you should have a valid data source as Jon mentioned, for example,
you can save the data with UTF-8/Unicode encoding in the Notepad.

Then, for the reason that the accented character is double character
encoding(byte), I suggest you to use the BinaryReader to read the accented
characters:
itsBinaryReader = new BinaryReader(it sFileStream, Encoding.UTF8); //or
Encoding.Unicod e

Now, you can use the Byte[] to read accented characters from the stream
object.
Please apply my suggestion above and let me know if it helps resolve your
problem.
Thanks!

Best regards,

Gary Chang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------

[Jon Skeet [C# MVP]]

Gary Chang <v-******@online.m icrosoft.com> wrote:

Hi Pascal,

Thanks for posting in the community.

From your description, I understand that you want to read accented
characters(in a .txt file) with streamreader/binaryreader,
Please correct me if there is any misunderstand.

First you should have a valid data source as Jon mentioned, for example,
you can save the data with UTF-8/Unicode encoding in the Notepad.

Then, for the reason that the accented character is double character
encoding(byte), I suggest you to use the BinaryReader to read the accented
characters:
itsBinaryReader = new BinaryReader(it sFileStream, Encoding.UTF8); //or
Encoding.Unicod e

Now, you can use the Byte[] to read accented characters from the stream
object.

Using BinaryReader is a bad idea - StreamReader is designed for exactly
the job required.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

[Pascal Cloup]

Hello Jon & Gary

You need to know what encoding the file is in, and specify that to the
StreamReader - ignore the BinaryReader.

Ok, now i create only a StreamReader object for text file.

When the file is created with Encoding.UTF8, the StreamReader object reads
correctly the accentueted char.
But if a file is created with Encoding.Defaul t (or ASCII), the StreamReader
object
skip the accentuated char.

I understand that my problem depends on the constrains of Encoding, but:

How to know the Encoding of a file before creating a reader?
How to specify the Encoding of a StreamReader?

Thanks for help,

Pascal Cloup
--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

Pascal Cloup <pc****@biogest a.fr> wrote:

You need to know what encoding the file is in, and specify that to the
StreamReader - ignore the BinaryReader.
Ok, now i create only a StreamReader object for text file.

When the file is created with Encoding.UTF8, the StreamReader object reads
correctly the accentueted char.
But if a file is created with Encoding.Defaul t (or ASCII), the StreamReader
object skip the accentuated char.

I understand that my problem depends on the constrains of Encoding, but:

How to know the Encoding of a file before creating a reader?

You should just know it - there's no absolutely accurate way of
determining an encoding from just the binary data. There are ways you
can guess it heuristically, but there's nothing in the framework which
will do this for you. (Some encodings will fix themselves in terms of
endianness, but that's not the kind of issue you're looking at here.)
How to specify the Encoding of a StreamReader?

StreamReader reader = new StreamReader(st ream, encoding)

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

[Pascal Cloup]

Hello,

Thank you both,

Finally, i use the Encoding.Defaul t (that corespond to ANSI character set)
encoding for both StreamWriter and StremReader. I can not use UTF8/7 or
unicode for some reasons of compatibility with files created with older
software or created on other platform (Mac).

Nevertheless, it seems that the StreamReader doesn't use the
Encoding.Defaul t by default.

with kind greetings

Pascal Cloup
"Jon Skeet [C# MVP]" <sk***@pobox.co m> a écrit dans le message de
news:MP******** *************** *@msnews.micros oft.com...

Pascal Cloup <pc****@biogest a.fr> wrote:

You need to know what encoding the file is in, and specify that to the
StreamReader - ignore the BinaryReader.

Ok, now i create only a StreamReader object for text file.

When the file is created with Encoding.UTF8, the StreamReader object reads correctly the accentueted char.
But if a file is created with Encoding.Defaul t (or ASCII), the StreamReader object skip the accentuated char.

I understand that my problem depends on the constrains of Encoding, but:

How to know the Encoding of a file before creating a reader?

You should just know it - there's no absolutely accurate way of
determining an encoding from just the binary data. There are ways you
can guess it heuristically, but there's nothing in the framework which
will do this for you. (Some encodings will fix themselves in terms of
endianness, but that's not the kind of issue you're looking at here.)

How to specify the Encoding of a StreamReader?

StreamReader reader = new StreamReader(st ream, encoding)

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

Pascal Cloup <pc****@biogest a.fr> wrote:

Nevertheless, it seems that the StreamReader doesn't use the
Encoding.Defaul t by default.

Indeed it does. As the docs for the constructor StreamReader(St ream)
say:

<quote>
This constructor initializes the encoding to UTF8Encoding
</quote>

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too