469,934 Members | 2,055 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,934 developers. It's quick & easy.

python regex character group matches

hello python-list!

the other day, i was trying to match unicode character sequences that
looked like this:

\\uAD0X...

my issue, is that the pattern i used was returning:

[ '\\uAD0X', '\\u1BF3', ... ]

when i expected:

[ '\\uAD0X\\u1BF3', ]

the code looks something like this:

pat = re.compile("(\\\u[0-9A-F]{4})+", re.UNICODE|re.LOCALE)
#print pat.findall(txt_line)
results = pat.finditer(txt_line)

i ran the pattern through a couple of my colleagues and they were all
in agreement that my pattern should have matched correctly.

is this a simple case of a messed up regex or am i not using the regex
api correctly?

cheers,

ct
Sep 17 '08 #1
2 1646
On Wed, 17 Sep 2008 09:27:47 -0400, christopher taylor wrote:
the other day, i was trying to match unicode character sequences that
looked like this:

\\uAD0X...

my issue, is that the pattern i used was returning:

[ '\\uAD0X', '\\u1BF3', ... ]

when i expected:

[ '\\uAD0X\\u1BF3', ]

the code looks something like this:

pat = re.compile("(\\\u[0-9A-F]{4})+", re.UNICODE|re.LOCALE) #print
pat.findall(txt_line)
results = pat.finditer(txt_line)

i ran the pattern through a couple of my colleagues and they were all in
agreement that my pattern should have matched correctly.
Correctly for what input? And the examples above are not matching (no
pun intended) the regular expression. `pat` doesn't match '\\uAD0X'
because there's no 'X' in the character class. BTW: Are you sure you
need or want the `re.UNICODE` flag?

Ciao,
Marc 'BlackJack' Rintsch
Sep 17 '08 #2
On Wed, 17 Sep 2008 09:27:47 -0400, christopher taylor wrote:
hello python-list!

the other day, i was trying to match unicode character sequences that
looked like this:

\\uAD0X...
It is not clear what this is supposed to be. Is that matching a literal
pair of backslashes, or a single escaped backslash, or a single unicode
character with codepoint AD0X, or what?

If I read it *literally*, then you're trying to match:

backslash backslash lowercase-u uppercase-A uppercase-D zero uppercase-X

Is that what you intended to match?

my issue, is that the pattern i used was returning:

[ '\\uAD0X', '\\u1BF3', ... ]
Unless you are using Python 3, I see that you aren't actually dealing
with Unicode strings, you're using byte strings. Is that deliberate?

when i expected:

[ '\\uAD0X\\u1BF3', ]
I make that to be a string of length 12. Is that what you are expecting?
>>len('\\uAD0X\\u1BF3')
12

the code looks something like this:

pat = re.compile("(\\\u[0-9A-F]{4})+", re.UNICODE|re.LOCALE)
#print pat.findall(txt_line)
results = pat.finditer(txt_line)
First point: I don't think the UNICODE flag does what you think it does.
It redefines the meaning of special escape sequences \b etc. Since you
aren't using any special escape sequences, I'm going to guess that you
think it turns your search string into Unicode. It doesn't. (Apologies in
advance if I guessed wrong.) I don't think you need either the UNICODE or
LOCALE flag for this search: they don't seem to have any effect.

Secondly: you will generally save yourself a lot of trouble when writing
regexes to use raw strings, because backslashes in the regex engine clash
with backslashes in Python strings. But there's a gotcha: backslash
escapes behave differently in ordinary strings and the re engine.

In an ordinary string, the sequence backslash-char is treated as a
literal backslash-char if it isn't a special escape. So:
>>len('\t') # special escape
1
>>len('\u') # not a special escape
2

But that's not the case in the re engine! As the Fine Manual says:

The special sequences consist of "\" and a character from
the list below. If the ordinary character is not on the
list, then the resulting RE will match the second character.
For example, \$ matches the character "$".

http://docs.python.org/lib/re-syntax.html

So all of these match the same thing:
re.compile('\\u')
re.compile(r'\u')
re.compile('u')

To match a literal backslash-u, you need to escape the backslash before
the engine joins it to the u: r'\\u'.

Putting it all together again:

pat = re.compile(r"(\\u[0-9A-F]{4})+")

will probably do what you want, assuming I have guessed what you want
correctly!

--
Steven
Sep 17 '08 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

17 posts views Thread by clintonG | last post: by
2 posts views Thread by Andrew Robert | last post: by
17 posts views Thread by Mark | last post: by
7 posts views Thread by MrNobody | last post: by
4 posts views Thread by unexpected | last post: by
6 posts views Thread by Johny | last post: by
11 posts views Thread by proctor | last post: by
7 posts views Thread by Nightcrawler | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.