By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
455,892 Members | 1,343 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 455,892 IT Pros & Developers. It's quick & easy.

Accentuated char

P: n/a
Hello,

The methods Read() of the class StreamReader don't read the accentuated
characters. When an accentuated character is present in a file, Read() skip
it and read the following character. Missing something?

Thanks for help

P. Cloup
Nov 15 '05 #1
Share this Question
Share on Google+
13 Replies


P: n/a
Pascal Cloup <pc****@biogesta.fr> wrote:
The methods Read() of the class StreamReader don't read the accentuated
characters. When an accentuated character is present in a file, Read() skip
it and read the following character. Missing something?


Chances are you've got the wrong encoding. It certainly *does* read
accented characters when everything is correct. How are you
constructing your StreamReader, and what's your data source?

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 15 '05 #2

P: n/a
Hello Jon
"Jon Skeet [C# MVP]" <sk***@pobox.com> a crit dans le message de
news:MP************************@msnews.microsoft.c om...
Pascal Cloup <pc****@biogesta.fr> wrote:
The methods Read() of the class StreamReader don't read the accentuated
characters. When an accentuated character is present in a file, Read() skip it and read the following character. Missing something?
Chances are you've got the wrong encoding. It certainly *does* read
accented characters when everything is correct. How are you
constructing your StreamReader, and what's your data source?


In fact i create a file stream object and 2 stream objects:
itsFileStream = File.Open( itsPath , FileMode.Open );

itsBinaryReader = new BinaryReader( itsFileStream , Encoding.ASCII );
// Perhaps the problem is here but i also try UTF8

itsStreamReader = new StreamReader( itsFileStream );

Depending of the nature of the file (binary or text) i use one of the 2
streams , but the 2 remain open (??); i need the BinaryStream to determine
if the file is text or not. All works fine except for accentuated characters
( ) which are skipped.

Any idea?

Thanks in advance,

Pascal Cloup
--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Nov 15 '05 #3

P: n/a
Pascal Cloup <pc****@biogesta.fr> wrote:
Chances are you've got the wrong encoding. It certainly *does* read
accented characters when everything is correct. How are you
constructing your StreamReader, and what's your data source?
In fact i create a file stream object and 2 stream objects:
itsFileStream = File.Open( itsPath , FileMode.Open );

itsBinaryReader = new BinaryReader( itsFileStream , Encoding.ASCII );
// Perhaps the problem is here but i also try UTF8

itsStreamReader = new StreamReader( itsFileStream );


That sounds like a bad idea to start with. Two readers on the same
stream is bound to cause problems.
Depending of the nature of the file (binary or text) i use one of the 2
streams , but the 2 remain open (??); i need the BinaryStream to determine
if the file is text or not. All works fine except for accentuated characters
( ) which are skipped.


I thought the point was that it was definitely a text file - otherwise
why are you trying to read it? *Any* file can be a text file, but it
depends on what encoding is being used as to what that file means.

You need to know what encoding the file is in, and specify that to the
StreamReader - ignore the BinaryReader.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 15 '05 #4

P: n/a
Hi Pascal,

Thanks for posting in the community.

From your description, I understand that you want to read accented
characters(in a .txt file) with streamreader/binaryreader,
Please correct me if there is any misunderstand.

First you should have a valid data source as Jon mentioned, for example,
you can save the data with UTF-8/Unicode encoding in the Notepad.

Then, for the reason that the accented character is double character
encoding(byte), I suggest you to use the BinaryReader to read the accented
characters:
itsBinaryReader = new BinaryReader(itsFileStream, Encoding.UTF8); //or
Encoding.Unicode

Now, you can use the Byte[] to read accented characters from the stream
object.
Please apply my suggestion above and let me know if it helps resolve your
problem.
Thanks!

Best regards,

Gary Chang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------

Nov 15 '05 #5

P: n/a
Gary Chang <v-******@online.microsoft.com> wrote:
Hi Pascal,

Thanks for posting in the community.

From your description, I understand that you want to read accented
characters(in a .txt file) with streamreader/binaryreader,
Please correct me if there is any misunderstand.

First you should have a valid data source as Jon mentioned, for example,
you can save the data with UTF-8/Unicode encoding in the Notepad.

Then, for the reason that the accented character is double character
encoding(byte), I suggest you to use the BinaryReader to read the accented
characters:
itsBinaryReader = new BinaryReader(itsFileStream, Encoding.UTF8); //or
Encoding.Unicode

Now, you can use the Byte[] to read accented characters from the stream
object.


Using BinaryReader is a bad idea - StreamReader is designed for exactly
the job required.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 15 '05 #6

P: n/a
Hello Jon & Gary

You need to know what encoding the file is in, and specify that to the
StreamReader - ignore the BinaryReader.


Ok, now i create only a StreamReader object for text file.

When the file is created with Encoding.UTF8, the StreamReader object reads
correctly the accentueted char.
But if a file is created with Encoding.Default (or ASCII), the StreamReader
object
skip the accentuated char.

I understand that my problem depends on the constrains of Encoding, but:

How to know the Encoding of a file before creating a reader?
How to specify the Encoding of a StreamReader?

Thanks for help,

Pascal Cloup
--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Nov 15 '05 #7

P: n/a
Pascal Cloup <pc****@biogesta.fr> wrote:
You need to know what encoding the file is in, and specify that to the
StreamReader - ignore the BinaryReader.
Ok, now i create only a StreamReader object for text file.

When the file is created with Encoding.UTF8, the StreamReader object reads
correctly the accentueted char.
But if a file is created with Encoding.Default (or ASCII), the StreamReader
object skip the accentuated char.

I understand that my problem depends on the constrains of Encoding, but:

How to know the Encoding of a file before creating a reader?


You should just know it - there's no absolutely accurate way of
determining an encoding from just the binary data. There are ways you
can guess it heuristically, but there's nothing in the framework which
will do this for you. (Some encodings will fix themselves in terms of
endianness, but that's not the kind of issue you're looking at here.)
How to specify the Encoding of a StreamReader?


StreamReader reader = new StreamReader(stream, encoding)

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 15 '05 #8

P: n/a
Hello,

Thank you both,

Finally, i use the Encoding.Default (that corespond to ANSI character set)
encoding for both StreamWriter and StremReader. I can not use UTF8/7 or
unicode for some reasons of compatibility with files created with older
software or created on other platform (Mac).

Nevertheless, it seems that the StreamReader doesn't use the
Encoding.Default by default.

with kind greetings

Pascal Cloup
"Jon Skeet [C# MVP]" <sk***@pobox.com> a crit dans le message de
news:MP************************@msnews.microsoft.c om...
Pascal Cloup <pc****@biogesta.fr> wrote:
You need to know what encoding the file is in, and specify that to the
StreamReader - ignore the BinaryReader.


Ok, now i create only a StreamReader object for text file.

When the file is created with Encoding.UTF8, the StreamReader object reads correctly the accentueted char.
But if a file is created with Encoding.Default (or ASCII), the StreamReader object skip the accentuated char.

I understand that my problem depends on the constrains of Encoding, but:

How to know the Encoding of a file before creating a reader?


You should just know it - there's no absolutely accurate way of
determining an encoding from just the binary data. There are ways you
can guess it heuristically, but there's nothing in the framework which
will do this for you. (Some encodings will fix themselves in terms of
endianness, but that's not the kind of issue you're looking at here.)
How to specify the Encoding of a StreamReader?


StreamReader reader = new StreamReader(stream, encoding)

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Nov 15 '05 #9

P: n/a
Pascal Cloup <pc****@biogesta.fr> wrote:
Nevertheless, it seems that the StreamReader doesn't use the
Encoding.Default by default.


Indeed it does. As the docs for the constructor StreamReader(Stream)
say:

<quote>
This constructor initializes the encoding to UTF8Encoding
</quote>

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 15 '05 #10

P: n/a
GG
I had a similar problem while reading a text file with
unicode and UTF8. Both did not return chars that look weird example .
Had to use Encoding.ASCII which read all chars but getting ????????? for
unkown ascii chars.
I did not want to loose the bufferRead position so stayed with Ascii.

*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!
Nov 15 '05 #11

P: n/a
GG <gg@hotmail.com> wrote:
I had a similar problem while reading a text file with
unicode and UTF8. Both did not return chars that look weird example =3F.
Had to use Encoding.ASCII which read all chars but getting ????????? for
unkown ascii chars.
I did not want to loose the bufferRead position so stayed with Ascii.


That suggests you were using the wrong encoding then - perhaps you
should have used Encoding.Default instead? It's hard to know without
knowing what you were trying to read - but you should know what
encoding your file is in rather than guessing until something works.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 15 '05 #12

P: n/a
Hi Pascal,

Thanks for your response!

Skeet said:
Using BinaryReader is a bad idea,
StreamReader is designed for exactly the job required.


Yes, I agree with it, StreamReader is designed for the txt file.

However at first time I try to read the accented characters with
StreamReader, I got null for them, and the BinaryReader actually retrieve
the correct characters, so I think the BinaryReadermay be better.

Today, I tested that program again, and found the StreamReader.Read() works
fine this time, I think I have missed something.
Thanks!

Best regards,

Gary Chang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------

Nov 15 '05 #13

P: n/a
Gary Chang <v-******@online.microsoft.com> wrote:
Skeet said:
Using BinaryReader is a bad idea,
StreamReader is designed for exactly the job required.


Yes, I agree with it, StreamReader is designed for the txt file.

However at first time I try to read the accented characters with
StreamReader, I got null for them, and the BinaryReader actually retrieve
the correct characters, so I think the BinaryReadermay be better.


If BinaryReader was reading the correct characters, then you must have
been giving it the correct encoding, while giving StreamReader the
wrong encoding. BinaryReader isn't capable of guessing an encoding any
better than StreamReader is.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 15 '05 #14

This discussion thread is closed

Replies have been disabled for this discussion.