472,800 Members | 1,226 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,800 software developers and data experts.

Splitting with Regular Expressions

Splitting with RE has (for me!) misterious behaviour!

I want to get the words from this string:
s= 'This+(that)= a.string!!!'

in a list like that ['This', 'that', 'a.string']
considering "a.string" as a word.

Python 2.3.4 (#2, Aug 19 2004, 15:49:40)
[GCC 3.4.1 (Mandrakelinux (Alpha 3.4.1-3mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
s= 'This+(that)= a.string!!!'
import re
p.split(s) ['This', 'that', '', '', 'tring', '!']

['This', '', 'that', '', '', 'a', 'string', '', '', '']

Jul 18 '05 #1
6 1860
Am Thu, 17 Mar 2005 06:51:19 -0800 schrieb qwweeeit:
Splitting with RE has (for me!) misterious behaviour!

I want to get the words from this string:
s= 'This+(that)= a.string!!!'

in a list like that ['This', 'that', 'a.string']
considering "a.string" as a word.


try this:

re.findall(r'[\w\.]+', s)
['This', 'that', 'a.string']

If you use r'...' you don't need to
use \\ if you mean \

\w matches a-zA-Z0-9_
\W matches all except \w


Thomas GŁttler, http://www.thomas-guettler.de/
Jul 18 '05 #2
A pyparsing example may be less mysterious. You can define words to be
any group of alphas, or you can define a word to be alphas concatenated
by '.'s. scanString is a generator that scans for matches in the input
string and returns the matching token list, and the start and end
location of the match within the input string. (Because it returns a
list, this is why we have to peel of element 0 of each match to get the
word.) See the sample code attached.

-- Paul
(get pyparsing at http://pyparsing.sourceforge.net)
from pyparsing import Word,alphas,delimitedList

test= 'This+(that)= a.string!!! This... is .just.a sentence.'

word = Word(alphas)

print [ wd[0] for wd,s,e in word.scanString(test) ]

# prints ['This', 'that', 'a', 'string', 'This', 'is', 'just', 'a',

word = delimitedList(Word(alphas), delim=".",combine=True)
print [ wd[0] for wd,s,e in word.scanString(test) ]

# prints ['This', 'that', 'a.string', 'This', 'is', 'just.a',

Jul 18 '05 #3
"qwweeeit" <qw******@yahoo.it> wrote:
Splitting with RE has (for me!) misterious behaviour!

I want to get the words from this string:
s= 'This+(that)= a.string!!!'

in a list like that ['This', 'that', 'a.string']
considering "a.string" as a word.

print re.findall("[\w.]+", s)


Jul 18 '05 #4
I thank you for your help.
The more flexible solution (Paul McGuire) is interesting but i don't
need such a flexibility. In fact I am implementing a cross-reference
tool and working on python sources, I don't need the '.' as separator
in order to capture variables and commands.
I thank nevertheless Paul for the advice on using pyparsing.

The other two solutions (Thomas Guettler and Fredrik Lundh) are quite
similar except for the use of "raw". I can use both.
Jul 18 '05 #5
"qwweeeit" <qw******@yahoo.it> wrote:
In fact I am implementing a cross-reference tool and working on
python sources, I don't need the '.' as separator in order to capture
variables and commands.

if you're parsing Python source code, consider using the tokenize module:


if you don't have to support "broken" source code, the parse module may
be even more useful:


for simpler cases, the "class browser" parser may be an even better choice:



Jul 18 '05 #6
It' s my faute that I have not read more deeply the Library
In any case the time "wasted" in developping small applications to
number lines and remove comments, triple quoted strings, multiline
instructions etc. has been "useful" to learn the language...
Now I already have the single tokens of a source saved in a file with
the reference to the corresponding line number.
An example is:

052 sys.argv
052 0
053 .encode
060 CSS
071 urllib.URLopener.version
072 urllib.FancyURLopener.prompt_user_passwd
072 lambda
072 self
072 host
072 realm
072 None
072 None
074 categories

The corresponding source is
(from http://inigo.katxi.org/devel/misc/googlenews.py):

052 PROGNAME = sys.argv[0]
053 AUTHOR = u'IŮigo Serna'.encode('utf-8')
054 VERSION = '0.3'
056 URL_BASE = 'http://news.google.com'
057 OUTPUT_HTML = 'news-%s-%s.html'
058 OUTPUT_RSS = 'news-%s-%s.xml'
060 CSS = """<style type="text/css">
061 body { color: black; background: white; }
062 a { color : #003399; text-decoration : none; }
063 a:hover { color : #339900; text-decoration : none; }
064 a.main { font-size : 100%; }
065 span.text { font-size : 100%; }
066 span.data { color : #666666; font-size : 80%; }
067 a.other { color : #003366; font-size : 75%; }
068 a.other:hover { color : #339900; font-size : 75%; }
069 </style>"""
071 urllib.URLopener.version = 'Mozilla/4.0 (compatible; MSIE 5.5;
Windows NT 5.0; T312461)'
072 urllib.FancyURLopener.prompt_user_passwd = lambda self, host,
realm: (None, None)
074 categories = ['w', 'n', 'b', 't', 's', 'e', 'm']

As you can see there are no literal strings nor comments (they are
saved in an accompagning file).
Now I have "only" to sort them, display in a table or (if you prefer)
in an extended display in which all the references to lines are
Of course many tokens (like for, in, if, not, etc.) will be eliminated
for space reasons and also because I already know how they are used.

I applied some years ago the same approach to understand an assembler

In the case of python (and all others high level languages) it should
be very useful also the function tree or flow chart...
Jul 18 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

by: Piet | last post by:
Hello, I have a very strange problem with regular expressions. The problem consists of analyzing the properties of columns of a MySQL database. When I request the column type, I get back a string...
by: Angelo Secchi | last post by:
Hi, I have string of numbers and words like ',,,,,,23,,,asd,,,,,"name,surname",,,,,,,\n' and I would like to split (I'm using string.split()) it using comma as separator but I do not want to...
by: qwweeeit | last post by:
Hi all, I am writing a script to visualize (and print) the web references hidden in the html files as: '<a href="web reference"> underlined reference</a>' Optimizing my code, I found that an...
by: Opettaja | last post by:
I am new to c# and I am currently trying to make a program to retrieve Battlefield 2 game stats from the gamespy servers. I have got it so I can retrieve the data but I do not know how to cut up...
by: Materialised | last post by:
Hi all, Just wondering if someone could help me with this little problem I'm having. I have a string value (it actually represents a barcode) which looks like this: 5021378002392 What I...
by: Anat | last post by:
Hi, What regex do I need to split a string, using javascript's split method, into words-array? Splitting accroding to whitespaces only is not enough, I need to split according to whitespace,...
by: Fabian Steiner | last post by:
I often have to deal with strings like "PCI:2:3.0" or "PCI:3.4:0" and need the single numbers as tuple (2, 3, 0) or (3, 4, 0). Is there any simple way to achieve this? So far I am using regular...
by: John Pye | last post by:
Hi all I have a file with a bunch of perl regular expressions like so: /(^|)\*(.*?)\*(|$)/$1'''$2'''$3/ # bold /(^|)\_\_(.*?)\_\_(|$)/$1''<b>$2<\/ b>''$3/ # italic bold...
by: xyz | last post by:
I have a string 16:23:18.659343 tcp 168 for example lets say for the above string 16:23:18.659343 -- time -- srcaddress 22 ...
by: David Jackson | last post by:
Hello, The company I'm working for has taken over a smaller company with a fairly large customer base. We want to send an email to that customer base informing them of the takeover but the...
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 2 August 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
by: linyimin | last post by:
Spring Startup Analyzer generates an interactive Spring application startup report that lets you understand what contributes to the application startup time and helps to optimize it. Support for...
by: erikbower65 | last post by:
Here's a concise step-by-step guide for manually installing IntelliJ IDEA: 1. Download: Visit the official JetBrains website and download the IntelliJ IDEA Community or Ultimate edition based on...
by: Taofi | last post by:
I try to insert a new record but the error message says the number of query names and destination fields are not the same This are my field names ID, Budgeted, Actual, Status and Differences ...
by: Rina0 | last post by:
I am looking for a Python code to find the longest common subsequence of two strings. I found this blog post that describes the length of longest common subsequence problem and provides a solution in...
by: lllomh | last post by:
Define the method first this.state = { buttonBackgroundColor: 'green', isBlinking: false, // A new status is added to identify whether the button is blinking or not } autoStart=()=>{
by: lllomh | last post by:
How does React native implement an English player?
by: Mushico | last post by:
How to calculate date of retirement from date of birth
by: DJRhino | last post by:
Was curious if anyone else was having this same issue or not.... I was just Up/Down graded to windows 11 and now my access combo boxes are not acting right. With win 10 I could start typing...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.