ASCII / ANSI encoding

[Mark Rae]

Hi,

I'm in the process if converting the data out of an old DOS-based
SunAccounts system (don't ask!) into SQL Server.

The data has been sent to me as a collection of hundreds of SunAccounts
backup files and it's a simple (yet extremely laborious!) process of opening
each backup file in turn, reading the file line by line, splitting it up
into its constituent parts, and then squirting it into SQL Server.

However, I'm experiencing a problem because Sun, being a DOS-based system,
uses ASCII encoding. Specifically, the old chestnut of the British pound
sign, which Sun considers to be hex 9c (i.e. Alt-156). However, Windows
using ANSI encoding sees this character as o instead of £ (Alt-163).

If I open the files into a StreamReader object without specifying an
encoding, C# appears to be using ASCII encoding. This has the effect of
ignoring the character completely which, in turn, throws the rest of the
import of of kilter because the file format is fixed-width, not delimted.

Therefore, can anyone please tell me what encoding I should use when reading
the file into a StreamReader object? I've tried several, but none appears to
work correctly so far. Some render the character as a question mark or a
blank square which, although not correct, at least doesn't stop the rest of
the import from working...

Any assistance gratefully received.

Mark

[tushar.n.patel]

hi Mark,
i've gone through such pain but my problem was little different. well
dude i would say you can not use StreamReader in your case. you have to
use BinaryReader to read whole file and than should use
"System.Text.ASCIIEncoding" to retrive the data into the WINDOWS ASCII
format.

this should work.

[Larry Lard]

Mark Rae wrote:

Hi,

I'm in the process if converting the data out of an old DOS-based
SunAccounts system (don't ask!) into SQL Server.

The data has been sent to me as a collection of hundreds of SunAccounts
backup files and it's a simple (yet extremely laborious!) process of opening
each backup file in turn, reading the file line by line, splitting it up
into its constituent parts, and then squirting it into SQL Server.

However, I'm experiencing a problem because Sun, being a DOS-based system,
uses ASCII encoding. Specifically, the old chestnut of the British pound
sign, which Sun considers to be hex 9c (i.e. Alt-156).
ASCII is a 7-bit character system, and does not define any character
code higher than 127 (hex 7F). A quick poke at Google suggests that 9C
is the pound sign in Microsoft's DOS codepage 437. Hmm, or maybe 850.
Take a look at <http://en.wikipedia.org/wiki/Category:DOS_code_pages>
and links therefrom to decide for yourself.
However, Windows
using ANSI encoding sees this character as o instead of £ (Alt-163).
I'm really no expert on these matters, but I'm fairly sure that
'Windows using ANSI encoding' isn't specific enough to be amgibuous.

If I open the files into a StreamReader object without specifying an
encoding, C# appears to be using ASCII encoding. This has the effect of
ignoring the character completely which, in turn, throws the rest of the
import of of kilter because the file format is fixed-width, not delimted.

Therefore, can anyone please tell me what encoding I should use when reading
the file into a StreamReader object?
If I'm right about the original encoding being DOS codepage 437, you
should use the Encoding created by
System.Text.Encoding.GetEncoding(437). Or try 850.
I've tried several, but none appears to
work correctly so far. Some render the character as a question mark or a
blank square which, although not correct, at least doesn't stop the rest of
the import from working...

Any assistance gratefully received.

Encodings are a nightmare. Best of luck!

--
Larry Lard
Replies to group please

[Mark Rae]

"Larry Lard" <la*******@hotmail.com> wrote in message
news:11********************@g49g2000cwa.googlegrou ps.com...

Hi Larry,

System.Text.Encoding.GetEncoding(437)

Worked perfectly! Thanks very much.

Mark

[Mark Rae]

<tu************@gmail.com> wrote in message
news:11*********************@o13g2000cwo.googlegro ups.com...

you can not use StreamReader in your case

Larry Lard's suggestion of System.Text.Encoding.GetEncoding(437) worked
perfectly.

[Jon Skeet [C# MVP]]

Mark Rae <ma**@markN-O-S-P-A-M.co.uk> wrote:

I'm in the process if converting the data out of an old DOS-based
SunAccounts system (don't ask!) into SQL Server.

The data has been sent to me as a collection of hundreds of SunAccounts
backup files and it's a simple (yet extremely laborious!) process of opening
each backup file in turn, reading the file line by line, splitting it up
into its constituent parts, and then squirting it into SQL Server.

However, I'm experiencing a problem because Sun, being a DOS-based system,
uses ASCII encoding. Specifically, the old chestnut of the British pound
sign, which Sun considers to be hex 9c (i.e. Alt-156). However, Windows
using ANSI encoding sees this character as o instead of £ (Alt-163).

Neither of those are ASCII characters. ASCII is 7-bit. See
http://www.pobox.com/~skeet/csharp/unicode.html for more information.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

<tu************@gmail.com> wrote:

i've gone through such pain but my problem was little different. well
dude i would say you can not use StreamReader in your case. you have to
use BinaryReader to read whole file and than should use
"System.Text.ASCIIEncoding" to retrive the data into the WINDOWS ASCII
format.

There's nothing wrong with using StreamReader here, but what's required
*isn't* ASCII. There's no such thing as "Windows ASCII" - there are
various 8-bit "extensions" to ASCII, in various "code pages"; they
aren't ASCII themselves, however.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Lucky]

what i mean by WINDOWS ASCII is windows compatiable string. that what
was your problem right? you were not able to get DOS ASCII stirng
exactly in WINDOWS string. am i right?

[Joerg Jooss]

Lucky wrote:

what i mean by WINDOWS ASCII is windows compatiable string. that what
was your problem right? you were not able to get DOS ASCII stirng
exactly in WINDOWS string. am i right?

As Jon said, there's no Windows ASCII, nor a DOS ASCII. There's only
ASCII. There's also no such thing as a Windows compatible string. It's
all just a matter of using the correct encoding.

Cheers,
--
http://www.joergjooss.de
mailto:ne********@joergjooss.de

[Jon Skeet [C# MVP]]

Lucky <tu************@gmail.com> wrote:

what i mean by WINDOWS ASCII is windows compatiable string. that what
was your problem right? you were not able to get DOS ASCII stirng
exactly in WINDOWS string. am i right?

"DOS ASCII" and "WINDOWS ASCII" aren't precisely defined terms though.
DOS used lots of different default code pages depending on your region,
and so does Windows.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Mihai N.]

> "DOS ASCII" and "WINDOWS ASCII" aren't precisely defined terms though.

DOS used lots of different default code pages depending on your region,
and so does Windows.

Joerg is right, there is no "DOS ASCII" and "WINDOWS ASCII", and ASCII is
very precisely defined.

ASCII = everything between 0-127, and is the same in all code pages (except
EBCDIC, but that is not ASCII, is EBCDIC :-)

ANSI code page = the Windows code pages. Same as "system code page". For Win
9x cannot be changed, for W2K, XP and newer it can be changed.
http://www.mihai-nita.net/20050611a.shtml

OEM code page = the DOS code pages. It is still used in Windows for console
applications.

See some info here:
http://blogs.msdn.com/michkap/archiv...08/369197.aspx
http://blogs.msdn.com/oldnewthing/ar...08/389527.aspx
http://www.mihai-nita.net/glossary.shtml

--
Mihai Nita [Microsoft MVP, Windows - SDK]
http://www.mihai-nita.net
------------------------------------------
Replace _year_ with _ to get the real email