473,378 Members | 1,426 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,378 software developers and data experts.

Test for a unicode string

Hi,

I have a regular expression test in a script. When a unicode character
get tested in the regex it gives an error:

UnicodeError: ASCII decoding error: ordinal not in range(128)

Question: Is there a way to test a string for unicode chars (ie. test
if a string will throw the error cited above).

Like:
if unicode string:
print 'string's line #'
else:
process the string

How do I do "if unicode string" cited in pseudo-code above?

Thanks, still using Python 2.1

Oct 24 '07 #1
5 2300
2007/10/24, goldtech <go******@worldpost.com>:
Question: Is there a way to test a string for unicode chars (ie. test
if a string will throw the error cited above).
yes there ist :)
>>isinstance(u"a", basestring)
True
>>isinstance(u"a", unicode)
True
>>isinstance("a", unicode)
False
>>isinstance("a", str)
True
>>isinstance(u"a", str)
False
--
http://noneisyours.marcher.name
http://feeds.feedburner.com/NoneIsYours
Oct 24 '07 #2
goldtech wrote:
I have a regular expression test in a script. When a unicode character
get tested in the regex it gives an error:

UnicodeError: ASCII decoding error: ordinal not in range(128)

Question: Is there a way to test a string for unicode chars (ie. test
if a string will throw the error cited above).
How about:

if ord(max(thestring)) >= 128:
print 'whatever you want'

-Scott David Daniels
Sc***********@Acm.Org
Oct 25 '07 #3
I have a regular expression test in a script. When a unicode character
get tested in the regex it gives an error:

UnicodeError: ASCII decoding error: ordinal not in range(128)

Question: Is there a way to test a string for unicode chars (ie. test
if a string will throw the error cited above).

Like:
if unicode string:
print 'string's line #'
else:
process the string
Sure:

try:
s.decode("ascii")
except UnicodeError:
print 'string's line #'
else:
process the string

Notice that you get a *decoding* error, so you are not testing
Unicode characters, but byte strings.

HTH,
Martin
Oct 25 '07 #4
* goldtech (Wed, 24 Oct 2007 12:09:24 -0700)
I have a regular expression test in a script. When a unicode character
get tested in the regex it gives an error:
As Martin pointed out: you are *not* using unicode...
Question: Is there a way to test a string for unicode chars (ie. test
if a string will throw the error cited above).
There is but this is not your problem and if it was it wouldn't be the
right solution.

Read http://www.amk.ca/python/howto/unicode. You should at least have
a minimal understanding of unicode before trying to use unicode.
Thanks, still using Python 2.1
Best way to get yourself into trouble.
Thorsten
Oct 25 '07 #5
snip...
Like:
if unicode string:
print 'string's line #'
else:
process the string
If I use "re.UNICODE" like: m = re.match(r"\w+", s, re.UNICODE)

then it seems to fix my problem. Trying to read as much as I can on
unicode....

Oct 25 '07 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Michael Weir | last post by:
I'm sure this is a very simple thing to do, once you know how to do it, but I am having no fun at all trying to write utf-8 strings to a unicode file. Does anyone have a couple of lines of code...
18
by: alex | last post by:
Hi there, how can I check if a variable is a structure (i.e. a list)? For my special problem the variable is either a character string OR a list of character strings line So how can I test...
4
by: webdev | last post by:
lo all, some of the questions i'll ask below have most certainly been discussed already, i just hope someone's kind enough to answer them again to help me out.. so i started a python 2.3...
10
by: Nikolay Petrov | last post by:
How can I convert DOS cyrillic text to Unicode
18
by: Ger | last post by:
I have not been able to find a simple, straight forward Unicode to ASCII string conversion function in VB.Net. Is that because such a function does not exists or do I overlook it? I found...
5
by: wanghz | last post by:
Hello, everyone. I have a problem when I'm processing unicode strings. Is it possible to get the 8bit-string representation of any unicode string? Suppose I get a unicode string: a =...
24
by: ChaosKCW | last post by:
Hi I am reading from an oracle database using cx_Oracle. I am writing to a SQLite database using apsw. The oracle database is returning utf-8 characters for euopean item names, ie special...
14
by: ThazKool | last post by:
I want to see if this code works the way it should on a Big-Endian system. Also if anyone has any ideas on how determine this at compile-time so that I use the right decoding or encoding...
7
by: 7stud | last post by:
Based on this example and the error: ----- u_str = u"abc\u9999" print u_str UnicodeEncodeError: 'ascii' codec can't encode character u'\u9999' in position 3: ordinal not in range(128) ------
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.