CP500 EBCDIC encoding bug?

Tin Gherdanarra

This is probably a fringe programme...

In the past I used a C-program to convert from EBCDIC (CP500)
to ASCII/Unicode. As you might guess, this is simply a 256-byte
lookup table. I lifted it from Larry Wall's perl source, to
be found here:

http://www.nntp.perl.org/group/perl.mvs/735

This worked very well. For some reason, the C# codec for
CP500 does not convert CR/LF (0x15 in EBCDIC) to 0x0A in
ASCII. Instead, it converts to 0x85. The other character
translations look okay. I figured that I might have picked
the wrong encoding, but picking /500/ as code-page for
the ISO standard CP (codepage) 500 is pretty much a
no-brainer. Other EBCDIC encodings from C#'s library
give gross results. So... what gives?

Any experiences from the fringe? Post-processing the
CP500-decoded string by translating the 0x85s to 0x0As
is SLOW in C#.

Jul 3 '06 #1

Subscribe Reply

3729

Jon Skeet [C# MVP]

Tin Gherdanarra wrote:

This is probably a fringe programme...

In the past I used a C-program to convert from EBCDIC (CP500)
to ASCII/Unicode. As you might guess, this is simply a 256-byte
lookup table. I lifted it from Larry Wall's perl source, to
be found here:

http://www.nntp.perl.org/group/perl.mvs/735

This worked very well. For some reason, the C# codec for
CP500 does not convert CR/LF (0x15 in EBCDIC) to 0x0A in
ASCII. Instead, it converts to 0x85.

That's the correct encoding according to a source I've used for EBCDIC
before:

http://std.dkuug.dk/i18n/charmaps/500

Note that Unicode U+0085 is "next line" which fits with the description
on the above page.

Jon

Jul 3 '06 #2

Mark Rae

"Tin Gherdanarra" <ti*********@gm ail.comwrote in message
news:4g******** *****@individua l.net...

Any experiences from the fringe? Post-processing the
CP500-decoded string by translating the 0x85s to 0x0As
is SLOW in C#.

Have a look at this: http://www.yoda.arachsys.com/csharp/ebcdic/

When Mr Skeet logs in a little later, I'm sure he'll have the solution...

Jul 3 '06 #3

Jon Skeet [C# MVP]

Mark Rae <ma**@markN-O-S-P-A-M.co.ukwrote:

"Tin Gherdanarra" <ti*********@gm ail.comwrote in message
news:4g******** *****@individua l.net...

Any experiences from the fringe? Post-processing the
CP500-decoded string by translating the 0x85s to 0x0As
is SLOW in C#.

Have a look at this: http://www.yoda.arachsys.com/csharp/ebcdic/

When Mr Skeet logs in a little later, I'm sure he'll have the solution...

I posted a while ago on Google in fact, but it doesn't seem to have
come through yet.

Basically, according to the code charts I've got, the encoding is doing
the right thing - 0x15 is meant to translate to U+0085 (next line), not
U+000A.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

Jul 3 '06 #4

Mark Rae

"Jon Skeet [C# MVP]" <sk***@pobox.co mwrote in message
news:MP******** *************** *@msnews.micros oft.com...

Basically, according to the code charts I've got

Yes, but it's still a bit of a "black art", is it not...?

Jul 3 '06 #5

Jon Skeet [C# MVP]

Mark Rae <ma**@markN-O-S-P-A-M.co.ukwrote:

"Jon Skeet [C# MVP]" <sk***@pobox.co mwrote in message
news:MP******** *************** *@msnews.micros oft.com...

Basically, according to the code charts I've got

Yes, but it's still a bit of a "black art", is it not...?

Absolutely. I'd love to see some official specifications, but when I
looked at this a few years ago, the charts at dkuug.org were the only
good references I could find :)

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

Jul 3 '06 #6

Similar topics

4636

Write a string in EBCDIC

by: John Leslie | last post by:

I need to write a string to a file in EBCDIC. Do I need to do it character by character using a translation table, or is there a function to translate the whole string? (I am aware that I can convert a whole file using Unix utilities, but this file will have only a few header records in EBCDIC)

C / C++

925

EBCDIC file format to byte format

by: R.A. | last post by:

Hi Is there some support for this file conversion in c#? Thanks

C# / C Sharp

17952

EBCDIC to ASCII

by: Chris H. | last post by:

Is there an easy way to convert a string that is in EBCDIC to ASCII

C# / C Sharp

7120

EBCDIC Codepage file to Windows based file

by: Ram | last post by:

Dear All, Good Day I am trying to convert a file which is generated on AS400 with codepage 00420 (arabic & English data combination) with no success. But using the same code( and changing 20420 to 708) I am able to convert a file from codepage ASMO 708 to windows based file and it is perfect. The following is the code I tried. i used 20420 codepage which is the nearest match for 00420. but no luck. //Open file for reading and set...

C# / C Sharp

6757

Reading EBCDIC Files.

by: jeff M via .NET 247 | last post by:

I'm still having problems reading EBCDIC files. Currently itlooks like the lower range (0 to 127) is working. I have triedthe following code pages 20284, 20924, 1140, 37, 500 and 20127.By working I get the correct answer by taking the decimal valueand using that as an index to an array that will map to thecorrect EBCDIC value in hex. By larger values, an example would be "AA" in EBCDIC hex wouldgive me the value of 63 in decimal (ASCII) when...

Visual Basic .NET

3906

Overpunch (EBCDIC) Conversion Function

by: Ernie Otero | last post by:

Does anyone know where I can find a function to convert numeric data to overpunch format? I've looked around but I can't seem to find any. Ernie

Visual Basic .NET

2610

Reading raw data and writing out in EBCDIC format

by: Ven | last post by:

Hi, I am reading a kind of below mentioned raw data into my C++ program and writing it out but I want this to be converted into EBCDIC format, how can I accomplish this ? Right now I am using the following code ifstream Source("source_file"); ofstream Dest("dest_file");

C / C++

2906

Sorting UNICODE as EBCDIC

by: Santiago.Ciciliani | last post by:

Hi all, I'm trying to sort an alphanumerical column as EBCDIC. Reading on Internet I've found two things: a) Use CAST(column as CHAR(nn) CCSID EBCDIC). This gives me the following error: Message: An unexpected token "EBCDIC" was found following "c AS

DB2 Database

3902

ansi c compiler character encoding

by: Andreas Lundgren | last post by:

Hi! Is it determined that the C standard compiler always encode characters with the same character excoding? If for example the functions Foo and Bar are compiled by different compilers, is it unambiguous how to interpret the character string in Bar? Does string.h expect a specific string format? void Foo(void)

C / C++

8600

Changing the language in Windows 10

by: Hystou | last post by:

Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...

Windows Server

9156

Problem With Comparison Operator <=> in G++

by: Oralloy | last post by:

Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...

C / C++

8892

The easy way to turn off automatic updates for Windows 10/11

by: Hystou | last post by:

Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...

Windows Server

8860

Discussion: How does Zigbee compare with other wireless protocols in smart home applications?

by: tracyyun | last post by:

Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...

General

5860

Couldn’t get equations in html when convert word .docx file to html file in C#.

by: conductexam | last post by:

I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...

C# / C Sharp

4614

Windows Forms - .Net 8.0

by: adsilva | last post by:

A Windows Forms form does not have the event Unload, like VB6. What one acts like?

Visual Basic .NET

3038

transfer the data from one system to another through ip address

by: 6302768590 | last post by:

Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

C# / C Sharp

2323

How to add payments to a PHP MySQL app.

by: muto222 | last post by:

How can i add a mobile payment intergratation into php mysql website.

PHP

1998

Comprehensive Guide to Website Development in Toronto: Expert Insights from BSMN Consultancy

by: bsmnconsultancy | last post by:

In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

General