By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
424,846 Members | 2,082 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 424,846 IT Pros & Developers. It's quick & easy.

Splitting with Regular Expressions

P: n/a
Splitting with RE has (for me!) misterious behaviour!

I want to get the words from this string:
s= 'This+(that)= a.string!!!'

in a list like that ['This', 'that', 'a.string']
considering "a.string" as a word.

Python 2.3.4 (#2, Aug 19 2004, 15:49:40)
[GCC 3.4.1 (Mandrakelinux (Alpha 3.4.1-3mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
s= 'This+(that)= a.string!!!'
import re
p=re.compile('[\W].')
p.split(s) ['This', 'that', '', '', 'tring', '!']
p=re.compile('[\W.]')
p.split(s)

['This', '', 'that', '', '', 'a', 'string', '', '', '']

Help!
Jul 18 '05 #1
Share this Question
Share on Google+
6 Replies


P: n/a
Am Thu, 17 Mar 2005 06:51:19 -0800 schrieb qwweeeit:
Splitting with RE has (for me!) misterious behaviour!

I want to get the words from this string:
s= 'This+(that)= a.string!!!'

in a list like that ['This', 'that', 'a.string']
considering "a.string" as a word.


Hi,

try this:

re.findall(r'[\w\.]+', s)
['This', 'that', 'a.string']

If you use r'...' you don't need to
use \\ if you mean \

\w matches a-zA-Z0-9_
\W matches all except \w

Thomas

--
Thomas GŁttler, http://www.thomas-guettler.de/
Jul 18 '05 #2

P: n/a
A pyparsing example may be less mysterious. You can define words to be
any group of alphas, or you can define a word to be alphas concatenated
by '.'s. scanString is a generator that scans for matches in the input
string and returns the matching token list, and the start and end
location of the match within the input string. (Because it returns a
list, this is why we have to peel of element 0 of each match to get the
word.) See the sample code attached.

-- Paul
(get pyparsing at http://pyparsing.sourceforge.net)
from pyparsing import Word,alphas,delimitedList

test= 'This+(that)= a.string!!! This... is .just.a sentence.'

word = Word(alphas)

print [ wd[0] for wd,s,e in word.scanString(test) ]

# prints ['This', 'that', 'a', 'string', 'This', 'is', 'just', 'a',
'sentence']

word = delimitedList(Word(alphas), delim=".",combine=True)
print [ wd[0] for wd,s,e in word.scanString(test) ]

# prints ['This', 'that', 'a.string', 'This', 'is', 'just.a',
'sentence']

Jul 18 '05 #3

P: n/a
"qwweeeit" <qw******@yahoo.it> wrote:
Splitting with RE has (for me!) misterious behaviour!

I want to get the words from this string:
s= 'This+(that)= a.string!!!'

in a list like that ['This', 'that', 'a.string']
considering "a.string" as a word.


print re.findall("[\w.]+", s)

</F>

Jul 18 '05 #4

P: n/a
I thank you for your help.
The more flexible solution (Paul McGuire) is interesting but i don't
need such a flexibility. In fact I am implementing a cross-reference
tool and working on python sources, I don't need the '.' as separator
in order to capture variables and commands.
I thank nevertheless Paul for the advice on using pyparsing.

The other two solutions (Thomas Guettler and Fredrik Lundh) are quite
similar except for the use of "raw". I can use both.
Jul 18 '05 #5

P: n/a
"qwweeeit" <qw******@yahoo.it> wrote:
In fact I am implementing a cross-reference tool and working on
python sources, I don't need the '.' as separator in order to capture
variables and commands.


if you're parsing Python source code, consider using the tokenize module:

http://docs.python.org/lib/module-tokenize.html

if you don't have to support "broken" source code, the parse module may
be even more useful:

http://docs.python.org/lib/module-parser.html

for simpler cases, the "class browser" parser may be an even better choice:

http://docs.python.org/lib/module-pyclbr.html

</F>

Jul 18 '05 #6

P: n/a
It' s my faute that I have not read more deeply the Library
Reference...
In any case the time "wasted" in developping small applications to
number lines and remove comments, triple quoted strings, multiline
instructions etc. has been "useful" to learn the language...
Now I already have the single tokens of a source saved in a file with
the reference to the corresponding line number.
An example is:

052 PROGNAME
052 sys.argv
052 0
053 AUTHOR
053 .encode
054 VERSION
056 URL_BASE
057 OUTPUT_HTML
058 OUTPUT_RSS
060 CSS
071 urllib.URLopener.version
072 urllib.FancyURLopener.prompt_user_passwd
072 lambda
072 self
072 host
072 realm
072 None
072 None
074 categories

The corresponding source is
(from http://inigo.katxi.org/devel/misc/googlenews.py):

052 PROGNAME = sys.argv[0]
053 AUTHOR = u'IŮigo Serna'.encode('utf-8')
054 VERSION = '0.3'
055
056 URL_BASE = 'http://news.google.com'
057 OUTPUT_HTML = 'news-%s-%s.html'
058 OUTPUT_RSS = 'news-%s-%s.xml'
059
060 CSS = """<style type="text/css">
061 body { color: black; background: white; }
062 a { color : #003399; text-decoration : none; }
063 a:hover { color : #339900; text-decoration : none; }
064 a.main { font-size : 100%; }
065 span.text { font-size : 100%; }
066 span.data { color : #666666; font-size : 80%; }
067 a.other { color : #003366; font-size : 75%; }
068 a.other:hover { color : #339900; font-size : 75%; }
069 </style>"""
070
071 urllib.URLopener.version = 'Mozilla/4.0 (compatible; MSIE 5.5;
Windows NT 5.0; T312461)'
072 urllib.FancyURLopener.prompt_user_passwd = lambda self, host,
realm: (None, None)
073
074 categories = ['w', 'n', 'b', 't', 's', 'e', 'm']

As you can see there are no literal strings nor comments (they are
saved in an accompagning file).
Now I have "only" to sort them, display in a table or (if you prefer)
in an extended display in which all the references to lines are
expanded...
Of course many tokens (like for, in, if, not, etc.) will be eliminated
for space reasons and also because I already know how they are used.

I applied some years ago the same approach to understand an assembler
source...

In the case of python (and all others high level languages) it should
be very useful also the function tree or flow chart...
Jul 18 '05 #7

This discussion thread is closed

Replies have been disabled for this discussion.