split a string of space separated substrings - elegant solution?

[Helmut Jarausch]

Hi,

I'm looking for an elegant solution to the following (quite common)
problem:

Given a string of substrings separated by white space,
split this into tuple/list of elements.
The problem are quoted substrings like

abc "xy z" "1 2 3" "a \" x"

should be split into ('abc','xy z','1 2 3','a " x')

For that, one probably has to protect white space between
quotes, then split by white space and finally converted the
'protected white space' to normal white space again.
Is there an elegant solution - perhaps without using a lexer
and something else. With regular expressions alone it seems
clumsy.

Many thanks for a hint,

Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[Wojciech =?iso-8859-2?Q?Mu=B3a?=]

Helmut Jarausch wrote:

Hi,

I'm looking for an elegant solution to the following (quite common)
problem:

Given a string of substrings separated by white space,
split this into tuple/list of elements.
The problem are quoted substrings like

abc "xy z" "1 2 3" "a \" x"

should be split into ('abc','xy z','1 2 3','a " x')

import csv

s = 'abc "xy z" "1 2 3" "a \\" x"'
r = iter(csv.reader([s], delimiter=" ", escapechar="\\"))
print r.next()

w.

[Carsten Haese]

On Tue, 2007-07-31 at 22:30 +0200, Helmut Jarausch wrote:

Hi,

I'm looking for an elegant solution to the following (quite common)
problem:

Given a string of substrings separated by white space,
split this into tuple/list of elements.
The problem are quoted substrings like

abc "xy z" "1 2 3" "a \" x"

should be split into ('abc','xy z','1 2 3','a " x')

>>import shlex
shlex.split('abc "xy z" "1 2 3" "a \\" x"')

['abc', 'xy z', '1 2 3', 'a " x']

I hope that's elegant enough ;)

--
Carsten Haese
http://informixdb.sourceforge.net

[Jerry Hill]

On 7/31/07, Helmut Jarausch <ja******@skynet.bewrote:

I'm looking for an elegant solution to the following (quite common)
problem:

Given a string of substrings separated by white space,
split this into tuple/list of elements.
The problem are quoted substrings like

abc "xy z" "1 2 3" "a \" x"

should be split into ('abc','xy z','1 2 3','a " x')

Using the csv module gets you most of the way there. For instance:

>>import csv
text = r'abc "xy z" "1 2 3" "a \" x"'
reader = csv.reader([text], delimiter=" ", escapechar='\\')
for row in reader:

print row

['abc', 'xy z', '', '1 2 3', '', 'a " x']

>>>

That does leave you with empty elements where you had double spaces
between items though. you could fix that with something like:

>>for row in reader:

row = [element for element in row if element != '']
print row

['abc', 'xy z', '1 2 3', 'a " x']

>>>

The CSV module can handle lots of delimited data other that quote and
comma delimited. See the docs at:
http://docs.python.org/lib/module-csv.html and PEP 305:
http://www.python.org/dev/peps/pep-0305/

--
Jerry

[Paul McGuire]

On Jul 31, 3:30 pm, Helmut Jarausch <[e-mail]:

I'm looking for an elegant solution to the following (quite common)
problem:

Given a string of substrings separated by white space,
split this into tuple/list of elements.
The problem are quoted substrings like

abc "xy z" "1 2 3" "a \" x"

should be split into ('abc','xy z','1 2 3','a " x')

Pyparsing has built-in support for special treatment of quoted
strings. Observe:

from pyparsing import *

data = r'abc "xy z" "1 2 3" "a \" x"'

quotedString.setParseAction(removeQuotes)
print OneOrMore(quotedString |
Word(printables) ).parseString(data)

prints:

['abc', 'xy z', '1 2 3', 'a \\" x']

Or perhaps a bit trickier, do the same while skipping items inside /*
*/ comments:

data = r'abc /* 456 "xy z" */ "1 2 3" "a \" x"'

quotedString.setParseAction(removeQuotes)
print OneOrMore(quotedString |
Word(printables) ) \
.ignore(cStyleComment).parseString(data)

prints:

['abc', '1 2 3', 'a \\" x']
-- Paul

[Helmut Jarausch]

Many thanks to all of you!
It's amazing how many elegant solutions there are in Python.
--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany