471,319 Members | 1,574 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,319 software developers and data experts.

unicode 3 digit decimal conversion

Hi,
I've got the string "Gratis l",or in english:"Free beer", I know there
is no such thing but...

Python 2.3 (#1, Aug 1 2003, 15:23:03)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
unicode("Gratis l","iso-8859-1") u'Gratis \xf8l'ord("\xf8")

248

What I need is the converted string to read u'Gratis \248l' (*
How do I do this without going through each and every character of the
string?
(not that I have figgured out how to do that right either)

regards

/rune
*) I need to communicate with a telnet interface that only accepts
accented characters as unicode decimals

Jul 18 '05 #1
9 3378
Rune Hansen wrote:
unicode("Gratis l","iso-8859-1") u'Gratis \xf8l'ord("\xf8") 248

What I need is the converted string to read u'Gratis \248l' (*
How do I do this without going through each and every character
of the string?
How about

#v+
print unicode('Gratis l', 'iso-8859-1').encode('utf-8') Gratis øl


#v-
// Klaus

--<> unselfish actions pay back better

Jul 18 '05 #2
Rune Hansen <ru*********@viventus.no> writes:
What I need is the converted string to read u'Gratis \248l' (*
How do I do this without going through each and every character of the
string?


You can register an error callback, like this:

import codecs

def decimal_escape(exc):
try:
data = exc.object
res = u""
for i in range(exc.start, exc.end):
char = ord(data[i])
if char < 1000:
res += u"\\%03d" % char
else:
# Unsupported character
raise exc

return res, exc.end
except:
raise exc

codecs.register_error("decimal-escape", decimal_escape)

print u"Gratis \xf8l".encode("us-ascii", "decimal-escape")

Notice That your specification is a bit unclear as to what to do with
characters > 1000; I assume they are not supported in your protocol.

Regards,
Martin

Jul 18 '05 #3
Hi , yes, of course *blush*.
Thanks

/regards

/rune

Klaus Alexander Seistrup wrote:
Rune Hansen wrote:

>unicode("Gratis l","iso-8859-1")


u'Gratis \xf8l'
>ord("\xf8")


248

What I need is the converted string to read u'Gratis \248l' (*
How do I do this without going through each and every character
of the string?

How about

#v+

print unicode('Gratis l', 'iso-8859-1').encode('utf-8')


Gratis øl
#v-
// Klaus


Jul 18 '05 #4
Hi,
The tip from Klaus "solved" my problem for the time beeing, but your
snipplet definitively goes into my "tool chest"

thanks

regards

/rune

Martin v. Lwis wrote:
Rune Hansen <ru*********@viventus.no> writes:

What I need is the converted string to read u'Gratis \248l' (*
How do I do this without going through each and every character of the
string?

You can register an error callback, like this:

import codecs

def decimal_escape(exc):
try:
data = exc.object
res = u""
for i in range(exc.start, exc.end):
char = ord(data[i])
if char < 1000:
res += u"\\%03d" % char
else:
# Unsupported character
raise exc

return res, exc.end
except:
raise exc

codecs.register_error("decimal-escape", decimal_escape)

print u"Gratis \xf8l".encode("us-ascii", "decimal-escape")

Notice That your specification is a bit unclear as to what to do with
characters > 1000; I assume they are not supported in your protocol.

Regards,
Martin


Jul 18 '05 #5
Rune Hansen wrote:
>>> unicode("Gratis l","iso-8859-1") u'Gratis \xf8l' >>>ord("\xf8") 248

What I need is the converted string to read u'Gratis \248l' (*
How do I do this without going through each and every character of the
string?
(not that I have figgured out how to do that right either)


I see your problem is already solved, just want to add that normally (read:
C and Python) the backslash notation is base 8 not base 10.
ord("\248") Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: ord() expected a character, but string of length 2 found oct(248) '0370' ord("\370") 248


Peter
Jul 18 '05 #6
Peter Otten <__*******@web.de> writes:
I see your problem is already solved


I'm quite uncertain as to what the solution is, though - or perhaps
what the problem is. The OP said that the telnet server expects
backslash characters in the data stream (atleast I interpreted his
message that way), but then he was happy with an approach that did not
send backslash characters.

Perhaps the telnet server really expects latin-1, in which case
encoding the Unicode string in that encoding, or in iso-8859-15,
would work fine most of the time.

Regards,
Martin
Jul 18 '05 #7
Hi Martin, you raise a very interesting question(at least for me it is :-).
The, otherwise, excelent support people at Stalker (it's a CommuniGate
pro server I'm speaking to) has me totally confused.

What I tried to do was to send a quoteattr(u'string') (using quoteattr
from sax). The server was very happy with this, showing the accented
characters, when viewed in a telnet session, as human readable text.
This, it became apparent, was obviously wrong. The data was garbled when
viewing it in the web interface.

Stalker told me to send the letter "" as \248 or as xf8 (notice the
missing "\"). At this point I'm sending
quoteattr(unicode('string',"iso-8859-1).encode("utf-8")) which is
neither of the above.(..?).
Anyway, the server is still happy, and the data views correctly in the
web interface.

Stalker provides a perl and java API for the telnet server. I don't
read perl code very well, and the java API is distributed as .class
files(nothing new there, it's java after all) so I really don't know how
Stalker is handling it.

regards

/rune

Martin v. Lwis wrote:
Peter Otten <__*******@web.de> writes:

I see your problem is already solved

I'm quite uncertain as to what the solution is, though - or perhaps
what the problem is. The OP said that the telnet server expects
backslash characters in the data stream (atleast I interpreted his
message that way), but then he was happy with an approach that did not
send backslash characters.

Perhaps the telnet server really expects latin-1, in which case
encoding the Unicode string in that encoding, or in iso-8859-15,
would work fine most of the time.

Regards,
Martin


Jul 18 '05 #8
Rune Hansen wrote:
Stalker told me to send the letter "" as \248 or as xf8 (notice the
missing "\"). At this point I'm sending
quoteattr(unicode('string',"iso-8859-1).encode("utf-8")) which is
neither of the above.(..?).
Correct: UTF-8 works differently. I find it surprising that anybody
actually proposes to send non-ASCII characters using xHH, as this
byte sequence my coincidently happen in ASCII text as well.
Anyway, the server is still happy, and the data views correctly in the
web interface.
It is relatively easy to recognize UTF-8 in the input; it is unlikely
that "real" data look like UTF-8 by coincidence (unlike \-escaping
or x-escaping). So it might be that the server studies the input to
guess the encoding. This is bad style, of course - the protocol should
be clear about encodings (this protocol couldn't be published in an
IETF RFC).
Stalker provides a perl and java API for the telnet server. I don't
read perl code very well, and the java API is distributed as .class
files(nothing new there, it's java after all) so I really don't know how
Stalker is handling it.


Even then, you could only find out what the perl and java clients do -
you couldn't tell, from that, what other options the server might support.

Regards,
Martin

Jul 18 '05 #9
Martin v. Lwis wrote:
Correct: UTF-8 works differently. I find it surprising that anybody
actually proposes to send non-ASCII characters using xHH, as this
byte sequence my coincidently happen in ASCII text as well.


unless they expect you to send "x" as "x78", of course.

</F>


Jul 18 '05 #10

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

5 posts views Thread by Bunyip Bluegum | last post: by
6 posts views Thread by S. | last post: by
3 posts views Thread by Andres A. | last post: by
3 posts views Thread by Scott David Daniels | last post: by
7 posts views Thread by Csaba Gabor | last post: by
6 posts views Thread by Bill Nguyen | last post: by
14 posts views Thread by abhi147 | last post: by
8 posts views Thread by Candace | last post: by
17 posts views Thread by =?Utf-8?B?R2Vvcmdl?= | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.