472,345 Members | 1,539 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,345 software developers and data experts.

does raw_input() return unicode?

In the announcement for Python-2.3
http://groups.google.com/group/comp....fe25388d?hl=en
it says "raw_input(): can now return Unicode objects".

But I didn't see anything about this in Andrew Kuchling's
"2.3 What's New", nor does the current python docs for
raw_input() say anything about this. A test on a MS
Windows system with a cp932 (japanese) default locale
shows the object returned by raw_input() is a str() object
containing cp932 encoded text. This remained true even
when I set Python's default encoding to cp932 (in
sitecustomize.py).

So, does raw_input() ever return unicode objects and if
so, under what conditions?

Oct 9 '06 #1
17 4063
"Stuart McGraw" <sm********@friizz.RimoovAllZZs.comwrote:
So, does raw_input() ever return unicode objects and if
so, under what conditions?
It returns unicode if reading from sys.stdin returns unicode.

Unfortunately, I can't tell you how to make sys.stdin return unicode for
use with raw_input. I tried what I thought should work and as you can see
it messed up the buffering on stdin. Does anyone else know how to wrap
sys.stdin so it returns unicode but is still unbuffered?

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>import sys, codecs
sys.stdin.encoding
'cp437'
>>sys.stdin = codecs.getreader(sys.stdin.encoding)(sys.stdin)
raw_input()
hello world
still going?
^Z
^Z
u'hello world'
>>>
Oct 10 '06 #2
Stuart McGraw schrieb:
So, does raw_input() ever return unicode objects and if
so, under what conditions?
At the moment, it only returns unicode objects when invoked
in the IDLE shell, and only if the character entered cannot
be represented in the locale's charset.

Regards,
Martin
Oct 10 '06 #3
On 10/10/06, "Martin v. Lwis" <ma****@v.loewis.dewrote:
Stuart McGraw schrieb:
So, does raw_input() ever return unicode objects and if
so, under what conditions?

At the moment, it only returns unicode objects when invoked
in the IDLE shell, and only if the character entered cannot
be represented in the locale's charset.
Why only IDLE? Does urwid or another console UI toolkit avoid this somehow?

-- Theerasak
Oct 10 '06 #4
Theerasak Photha wrote:
>>So, does raw_input() ever return unicode objects and if
so, under what conditions?

At the moment, it only returns unicode objects when invoked
in the IDLE shell, and only if the character entered cannot
be represented in the locale's charset.

Why only IDLE? Does urwid or another console UI toolkit avoid this somehow?
Martin was probably thinking of the standard distribution.

The 2.3 note says that "raw_input() *can* return Unicode", not that it
"should" or "must" do it.

</F>

Oct 10 '06 #5
On 10/10/06, Fredrik Lundh <fr*****@pythonware.comwrote:
Martin was probably thinking of the standard distribution.

The 2.3 note says that "raw_input() *can* return Unicode", not that it
"should" or "must" do it.
Practically speaking, at the heart of the matter: as of Python 2.5
final, does or can raw_input() return Unicode under the appropriate
circumstances, according to user wishes?

(Yes, I would test, but I am presently away from my Linux box with
Python, and can't install it here.)

-- Theerasak
Oct 10 '06 #6
Theerasak Photha wrote:
Practically speaking, at the heart of the matter: as of Python 2.5
final, does or can raw_input() return Unicode under the appropriate
circumstances, according to user wishes?
didn't Martin just answer that question?

</F>

Oct 10 '06 #7
On 10/10/06, Fredrik Lundh <fr*****@pythonware.comwrote:
Theerasak Photha wrote:
Practically speaking, at the heart of the matter: as of Python 2.5
final, does or can raw_input() return Unicode under the appropriate
circumstances, according to user wishes?

didn't Martin just answer that question?
*slaps forehead* D'oh!

-- Theerasak
Oct 10 '06 #8
Theerasak Photha schrieb:
>At the moment, it only returns unicode objects when invoked
in the IDLE shell, and only if the character entered cannot
be represented in the locale's charset.

Why only IDLE? Does urwid or another console UI toolkit avoid this somehow?
I admit I don't know what urwid is; from a shallow description I find
("a console user interface library") I can't see the connection to
raw_input(). How would raw_input() ever use urwid?

Regards,
Martin
Oct 10 '06 #9
Theerasak Photha schrieb:
>At the moment, it only returns unicode objects when invoked
in the IDLE shell, and only if the character entered cannot
be represented in the locale's charset.

Why only IDLE? Does urwid or another console UI toolkit avoid this somehow?
I admit I don't know what urwid is; from a shallow description I find
("a console user interface library") I can't see the connection to
raw_input(). How would raw_input() ever use urwid?

Regards,
Martin
Oct 10 '06 #10

"Martin v. Löwis" <ma****@v.loewis.dewrote in message news:45***********************@news.freenet.de...
Stuart McGraw schrieb:
So, does raw_input() ever return unicode objects and if
so, under what conditions?

At the moment, it only returns unicode objects when invoked
in the IDLE shell, and only if the character entered cannot
be represented in the locale's charset.
Thanks for the answer.

Also, if anyone has a solution for Duncan Booth's attempt
to wrap stdin, I would find it very useful too!

"Duncan Booth" <du**********@invalid.invalidwrote:
"Stuart McGraw" <sm********@friizz.RimoovAllZZs.comwrote:
So, does raw_input() ever return unicode objects and if
so, under what conditions?
It returns unicode if reading from sys.stdin returns unicode.

Unfortunately, I can't tell you how to make sys.stdin return unicode for
use with raw_input. I tried what I thought should work and as you can see
it messed up the buffering on stdin. Does anyone else know how to wrap
sys.stdin so it returns unicode but is still unbuffered?

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>import sys, codecs
sys.stdin.encoding
'cp437'
>sys.stdin = codecs.getreader(sys.stdin.encoding)(sys.stdin)
raw_input()
hello world
still going?
^Z
^Z
u'hello world'
>>
Oct 10 '06 #11
On 10/10/06, "Martin v. Lwis" <ma****@v.loewis.dewrote:
Theerasak Photha schrieb:
At the moment, it only returns unicode objects when invoked
in the IDLE shell, and only if the character entered cannot
be represented in the locale's charset.
Why only IDLE? Does urwid or another console UI toolkit avoid this somehow?

I admit I don't know what urwid is; from a shallow description I find
("a console user interface library") I can't see the connection to
raw_input(). How would raw_input() ever use urwid?
The other way around: would urwid use raw_input() or other Python
input functions anywhere?

And what causes Unicode input to work in IDLE alone?

-- Theerasak
Oct 11 '06 #12

Theerasak Photha wrote:
On 10/10/06, "Martin v. Lwis" <ma****@v.loewis.dewrote:
Theerasak Photha schrieb:
>At the moment, it only returns unicode objects when invoked
>in the IDLE shell, and only if the character entered cannot
>be represented in the locale's charset.
>
Why only IDLE? Does urwid or another console UI toolkit avoid this somehow?
I admit I don't know what urwid is; from a shallow description I find
("a console user interface library") I can't see the connection to
raw_input(). How would raw_input() ever use urwid?

The other way around: would urwid use raw_input() or other Python
input functions anywhere?

And what causes Unicode input to work in IDLE alone?
Other applications except python are actually free to implement unicode
stdin. python cannot do it because of backward compatibility. You can
argue that python interactive console could do it too, but think about
it this way: python interactive console deliberately behaves like a
running python program would.

Oct 11 '06 #13

Duncan Booth wrote:
"Stuart McGraw" <sm********@friizz.RimoovAllZZs.comwrote:
So, does raw_input() ever return unicode objects and if
so, under what conditions?
It returns unicode if reading from sys.stdin returns unicode.

Unfortunately, I can't tell you how to make sys.stdin return unicode for
use with raw_input. I tried what I thought should work and as you can see
it messed up the buffering on stdin. Does anyone else know how to wrap
sys.stdin so it returns unicode but is still unbuffered?
Considering that all consoles are ascii based, the following should
work where python was able to determine terminal encoding:

class ustdio(object):
def __init__(self, stream):
self.stream = stream
self.encoding = stream.encoding
def readline(self):
return self.stream.readline().decode(self.encoding)

sys.stdin = ustdio(sys.stdin)

answer = raw_input()
print type(answer)

Oct 11 '06 #14
Theerasak Photha schrieb:
The other way around: would urwid use raw_input() or other Python
input functions anywhere?
Since I still don't know what urwid is, I can't answer the question.
It should be easy enough to grep its source code to find out whether
it ever uses raw_input.
And what causes Unicode input to work in IDLE alone?
Because in IDLE, it is possible to enter characters that are not
in the user's charset. For example, if the user's charset is
cp-1252 (western european), you can still enter cyrillic characters
into IDLE. This is not possible in a regular terminal.

Regards,
Martin
Oct 11 '06 #15
On 2006-10-11, Leo Kislov <Le********@gmail.comwrote:
>Unfortunately, I can't tell you how to make sys.stdin return
unicode for use with raw_input. I tried what I thought should
work and as you can see it messed up the buffering on stdin.
Does anyone else know how to wrap sys.stdin so it returns
unicode but is still unbuffered?

Considering that all consoles are ascii based, the following
should work where python was able to determine terminal
encoding:

class ustdio(object):
def __init__(self, stream):
self.stream = stream
self.encoding = stream.encoding
def readline(self):
return self.stream.readline().decode(self.encoding)

sys.stdin = ustdio(sys.stdin)

answer = raw_input()
print type(answer)
This interesting discussion led me to a weird discovery:

PythonWin 2.4.3 (#69, Apr 11 2006, 15:32:42) [MSC v.1310 32 bit (Intel)] on win32.
Portions Copyright 1994-2004 Mark Hammond (mh******@skippinet.com.au) - see 'Help/About PythonWin' for further copyright information.
>>import sys
sys.stdout.encoding
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "C:\edconn32\Python24\Lib\site-packages\pythonwin\pywin\mfc\object.py", line 18, in __getattr__
return getattr(o, attr)
AttributeError: encoding
>>sys.stdin.encoding
I'm all mindboggley. Just when I thought I was starting to
understand how this character encoding stuff works. Are
PythonWin's stdout and stdin implementations is incomplete?

--
Neil Cerutti
A song fest was hell at the Methodist church Wednesday. --Church
Bulletin Blooper
Oct 11 '06 #16
Neil Cerutti schrieb:
I'm all mindboggley. Just when I thought I was starting to
understand how this character encoding stuff works. Are
PythonWin's stdout and stdin implementations is incomplete?
Simple and easy: yes, they are.

Regards,
Martin
Oct 11 '06 #17
On 2006-10-11, Martin v. Lwis <ma****@v.loewis.dewrote:
Neil Cerutti schrieb:
>I'm all mindboggley. Just when I thought I was starting to
understand how this character encoding stuff works. Are
PythonWin's stdout and stdin implementations is incomplete?

Simple and easy: yes, they are.
Oh, phew!

--
Neil Cerutti
Any time I've taken the mound, it's always been the old
Samson-and-Goliath story written about me. --Randy Johnson
Oct 12 '06 #18

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

12
by: Helmut Jarausch | last post by:
Hi, what does Python do if two objects aren't comparable (to my opinion) If I've understood "Python in a Nutschell" correctly it should raise an...
12
by: Fred Pacquier | last post by:
First off, sorry for this message-in-a-bottle-like post... I haven't been able to phrase my questions well enough to get a meaningful answer from...
4
by: foo | last post by:
"An unhandled exception of type 'System.NullReferenceException' occurred in Project.exe Additional information: Object reference not set to an...
21
by: planetthoughtful | last post by:
Hi All, As always, my posts come with a 'Warning: Newbie lies ahead!' disclaimer... I'm wondering if it's possible, using raw_input(), to...
2
by: tim | last post by:
I want to write a program that looks into a given folder, groups files that have a certain part of the filename in common and then copy those...
2
by: =?Utf-8?B?QWxleCBLLg==?= | last post by:
Hi all My TreeView has unicode and english labels. The treeview shows OK on the screen. When I am trying to get an item's label using...
8
by: Dox33 | last post by:
I ran into a very strange behaviour of raw_input(). I hope somebody can tell me how to fix this. (Or is this a problem in the python source?) I...
6
by: Nathan Pinno | last post by:
Why does my compiler say invalid syntax and then highlight the quotation marks in the following code: # This program is to find primes. primes =...
4
by: TP | last post by:
Hi everybody, When using raw_input(), the input of the user ends when he types Return on his keyboard. How can I change this behavior, so that...
0
by: concettolabs | last post by:
In today's business world, businesses are increasingly turning to PowerApps to develop custom business applications. PowerApps is a powerful tool...
0
better678
by: better678 | last post by:
Question: Discuss your understanding of the Java platform. Is the statement "Java is interpreted" correct? Answer: Java is an object-oriented...
0
by: teenabhardwaj | last post by:
How would one discover a valid source for learning news, comfort, and help for engineering designs? Covering through piles of books takes a lot of...
0
by: Kemmylinns12 | last post by:
Blockchain technology has emerged as a transformative force in the business world, offering unprecedented opportunities for innovation and...
0
jalbright99669
by: jalbright99669 | last post by:
Am having a bit of a time with URL Rewrite. I need to incorporate http to https redirect with a reverse proxy. I have the URL Rewrite rules made...
0
by: Matthew3360 | last post by:
Hi there. I have been struggling to find out how to use a variable as my location in my header redirect function. Here is my code. ...
0
by: AndyPSV | last post by:
HOW CAN I CREATE AN AI with an .executable file that would suck all files in the folder and on my computerHOW CAN I CREATE AN AI with an .executable...
0
by: Arjunsri | last post by:
I have a Redshift database that I need to use as an import data source. I have configured the DSN connection using the server, port, database, and...
0
hi
by: WisdomUfot | last post by:
It's an interesting question you've got about how Gmail hides the HTTP referrer when a link in an email is clicked. While I don't have the specific...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.