473,321 Members | 1,877 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,321 software developers and data experts.

StreamWriter Adds FFFE at the start of the file with Unicode Encoding!!

Hi , all
I'm trying to write unicode to a file for another app (not developed with
vs2003) to read it. I used StreamWriter with unicode encoding.but I was
surprised that the streamwriter adds FFFE to the start of the file,which
stopes the other app from reading it!!
any idea how to stope it frome doing that,do I have to use another class
#####writer that supports unicode?

help me Please!
Thanks
majed
I did some code with diferent encoding and here is the result:
UTF7 48656C6C6F2B4143454149512D0D0A
Hello+ACEAIQ-..
DEFAULT 48656C6C6F21210D0A
Hello!!..
ASCII 48656C6C6F21210D0A
Hello!!..
BIGENDIANUNICODE FEFF00480065006C006C006F00210021000D000A
....H.e.l.l.o.!.!....
UNICODE FFFE480065006C006C006F00210021000D000A00
...H.e.l.l.o.!.!.....
UTF8 EFBBBF48656C6C6F21210D0A
....Hello!!..

and here is the code

Imports System

Imports System.Text

Imports System.IO

Class filestream

Public Shared Sub main()

Dim ASCIIStream As New StreamWriter("ASCIIStream.txt", False,
System.Text.Encoding.ASCII)

Dim BigEndianUnicodeStream As New StreamWriter("BigEndianUnicodeStream.txt",
False, System.Text.Encoding.BigEndianUnicode)

Dim UnicodeStream As New StreamWriter("UnicodeStream.txt", False,
System.Text.Encoding.Unicode)

Dim DefaultStream As New StreamWriter("DefaultStream.txt", False,
System.Text.Encoding.Default)

Dim UTF7Stream As New StreamWriter("UTF7Stream.txt", False,
System.Text.Encoding.UTF7)

Dim UTF8Stream As New StreamWriter("UTF8Stream.txt", False,
System.Text.Encoding.UTF8)

ASCIIStream.WriteLine("Hello!!")

ASCIIStream.Close()

ASCIIStream = Nothing

BigEndianUnicodeStream.WriteLine("Hello!!")

BigEndianUnicodeStream.Close()

BigEndianUnicodeStream = Nothing

UnicodeStream.WriteLine("Hello!!")

UnicodeStream.Close()

UnicodeStream = Nothing

DefaultStream.WriteLine("Hello!!")

DefaultStream.Close()

DefaultStream = Nothing

UTF7Stream.WriteLine("Hello!!")

UTF7Stream.Close()

UTF7Stream = Nothing

UTF8Stream.WriteLine("Hello!!")

UTF8Stream.Close()

UTF8Stream = Nothing

Console.WriteLine ("Done!)

End Sub

End Class
Jul 21 '05 #1
4 2428
Majed,
Sounds like a bug in your other app!

FFFE is the byte order mark of Unicode, it signifies to Unicode that the
bytes are little endian. While FEFF signifies to Unicode that the bytes are
big endian!

Notice that in your samples, FEFF is Big Endian
BIGENDIANUNICODE FEFF00480065006C006C006F00210021000D000A
...H.e.l.l.o.!.!....
, while FFFE is little endian: UNICODE FFFE480065006C006C006F00210021000D000A00
..H.e.l.l.o.!.!.....
You can prevent the byte order mark from being written by creating a new
UnicodeEncoding object that you pass as the Encoding parameter of the
STreamWriter constructor.

Something like(untested):

' little endian, no byte order marks
Dim encoding As New UnicodeEncoding(False, False)

Dim stream as FileStream
Dim writer As New StreamWriter(stream, encoding)

Of course you will need to be certain that you other app does not have an
issue with little endian verses big endian Unicode characters.

Hope this helps
Jay

"Majed" <am******@hotmail.com> wrote in message
news:%2***************@TK2MSFTNGP11.phx.gbl... Hi , all
I'm trying to write unicode to a file for another app (not developed with
vs2003) to read it. I used StreamWriter with unicode encoding.but I was
surprised that the streamwriter adds FFFE to the start of the file,which
stopes the other app from reading it!!
any idea how to stope it frome doing that,do I have to use another class
#####writer that supports unicode?

help me Please!
Thanks
majed
I did some code with diferent encoding and here is the result:
UTF7 48656C6C6F2B4143454149512D0D0A
Hello+ACEAIQ-..
DEFAULT 48656C6C6F21210D0A
Hello!!..
ASCII 48656C6C6F21210D0A
Hello!!..
BIGENDIANUNICODE FEFF00480065006C006C006F00210021000D000A
...H.e.l.l.o.!.!....
UNICODE FFFE480065006C006C006F00210021000D000A00
..H.e.l.l.o.!.!.....
UTF8 EFBBBF48656C6C6F21210D0A
...Hello!!..

and here is the code

Imports System

Imports System.Text

Imports System.IO

Class filestream

Public Shared Sub main()

Dim ASCIIStream As New StreamWriter("ASCIIStream.txt", False,
System.Text.Encoding.ASCII)

Dim BigEndianUnicodeStream As New StreamWriter("BigEndianUnicodeStream.txt", False, System.Text.Encoding.BigEndianUnicode)

Dim UnicodeStream As New StreamWriter("UnicodeStream.txt", False,
System.Text.Encoding.Unicode)

Dim DefaultStream As New StreamWriter("DefaultStream.txt", False,
System.Text.Encoding.Default)

Dim UTF7Stream As New StreamWriter("UTF7Stream.txt", False,
System.Text.Encoding.UTF7)

Dim UTF8Stream As New StreamWriter("UTF8Stream.txt", False,
System.Text.Encoding.UTF8)

ASCIIStream.WriteLine("Hello!!")

ASCIIStream.Close()

ASCIIStream = Nothing

BigEndianUnicodeStream.WriteLine("Hello!!")

BigEndianUnicodeStream.Close()

BigEndianUnicodeStream = Nothing

UnicodeStream.WriteLine("Hello!!")

UnicodeStream.Close()

UnicodeStream = Nothing

DefaultStream.WriteLine("Hello!!")

DefaultStream.Close()

DefaultStream = Nothing

UTF7Stream.WriteLine("Hello!!")

UTF7Stream.Close()

UTF7Stream = Nothing

UTF8Stream.WriteLine("Hello!!")

UTF8Stream.Close()

UTF8Stream = Nothing

Console.WriteLine ("Done!)

End Sub

End Class

Jul 21 '05 #2
hi jay
the other app is Ntbackup which saves its backup files list in unicode
without FFFE at the start of the file. any idea how can I remove it from the
file.
TIA
"Jay B. Harlow [MVP - Outlook]" <Ja************@msn.com> wrote in message
news:ea*************@TK2MSFTNGP10.phx.gbl...
Majed,
Sounds like a bug in your other app!

FFFE is the byte order mark of Unicode, it signifies to Unicode that the
bytes are little endian. While FEFF signifies to Unicode that the bytes are big endian!

Notice that in your samples, FEFF is Big Endian
BIGENDIANUNICODE FEFF00480065006C006C006F00210021000D000A
...H.e.l.l.o.!.!....


, while FFFE is little endian:

Jul 21 '05 #3
Majed,
any idea how can I remove it from the
file. I take it you did not read my response! As I indicated in my response how
you remove it from the file!

Hope this helps
Jay
"Majed" <am******@hotmail.com> wrote in message
news:e1**************@TK2MSFTNGP12.phx.gbl... hi jay
the other app is Ntbackup which saves its backup files list in unicode
without FFFE at the start of the file. any idea how can I remove it from the file.
TIA
"Jay B. Harlow [MVP - Outlook]" <Ja************@msn.com> wrote in message
news:ea*************@TK2MSFTNGP10.phx.gbl...
Majed,
Sounds like a bug in your other app!

FFFE is the byte order mark of Unicode, it signifies to Unicode that the
bytes are little endian. While FEFF signifies to Unicode that the bytes

are
big endian!

Notice that in your samples, FEFF is Big Endian
BIGENDIANUNICODE FEFF00480065006C006C006F00210021000D000A
...H.e.l.l.o.!.!....


, while FFFE is little endian:


Jul 21 '05 #4
jay,
i'm sorry about that,I was late up because of this problem. yuor reply was
great and did solve it.
tahnks and sorry again.:)
majed
"Jay B. Harlow [MVP - Outlook]" <Ja************@msn.com> wrote in message
news:us**************@TK2MSFTNGP09.phx.gbl...
Majed,
any idea how can I remove it from the
file.

I take it you did not read my response! As I indicated in my response how
you remove it from the file!

Hope this helps
Jay
"Majed" <am******@hotmail.com> wrote in message
news:e1**************@TK2MSFTNGP12.phx.gbl...
hi jay
the other app is Ntbackup which saves its backup files list in unicode
without FFFE at the start of the file. any idea how can I remove it from

the
file.
TIA
"Jay B. Harlow [MVP - Outlook]" <Ja************@msn.com> wrote in message news:ea*************@TK2MSFTNGP10.phx.gbl...
Majed,
Sounds like a bug in your other app!

FFFE is the byte order mark of Unicode, it signifies to Unicode that the bytes are little endian. While FEFF signifies to Unicode that the
bytes are
big endian!

Notice that in your samples, FEFF is Big Endian
> BIGENDIANUNICODE FEFF00480065006C006C006F00210021000D000A
> ...H.e.l.l.o.!.!....

, while FFFE is little endian:



Jul 21 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

30
by: aurora | last post by:
I have long find the Python default encoding of strict ASCII frustrating. For one thing I prefer to get garbage character than an exception. But the biggest issue is Unicode exception often pop up...
2
by: garykpdx | last post by:
Every time I think I understand unicode, I prove I don't. I created a variable in interactive mode like this: s = u'ä' where this character is the a-umlaut that worked alright. Then I encoded...
6
by: Tom | last post by:
Hi I read http://dotgnu.org/pnetlib-doc/System/Text/UnicodeEncoding.html#UnicodeEncoding.GetBytes%28System.String%29%20Method and here is my code: <code string c = "Hello, How are you today?";...
1
by: Majed | last post by:
Hi , all I'm trying to write unicode to a file for another app (not developed with vs2003) to read it. I used StreamWriter with unicode encoding.but I was surprised that the streamwriter adds FFFE...
1
by: MaTianyi | last post by:
the encoding of file that is output by the file.write() function is always None. and the encoding attribute of the File Object is readonly. f = open('abc.txt','w') print f.encoding how can...
0
by: Iwan Petrow | last post by:
Hi, I want every file, which I add to my solution to be save with unicode encoding. How could I set this? Also is it possible team foundation server to check the encoding and to allow for...
6
by: Franz Steinhaeusler | last post by:
Hello NG, a little longer question, I'm working on our project DrPython and try fix bugs in Linux, (on windows, it works very good now with latin-1 encoding). On Windows, it works good now,...
0
by: UvT | last post by:
Can anyone tell me how to create a text file with Unicode Encoding. In am using FileStream Finalfile = new FileStream("finalfile.txt", FileMode.Append, FileAccess.Write); Now this creates...
1
by: ujjwaltrivedi | last post by:
Hey guys, Can anyone tell me how to create a text file with Unicode Encoding. In am using FileStream Finalfile = new FileStream("finalfile.txt", FileMode.Append, FileAccess.Write); ...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.