Parsing a search string

[Freddie]

Happy new year! Since I have run out of alcohol, I'll ask a question that I
haven't really worked out an answer for yet. Is there an elegant way to turn
something like:

moo cow "farmer john" -zug

into:

['moo', 'cow', 'farmer john'], ['zug']

I'm trying to parse a search string so I can use it for SQL WHERE constraints,
preferably without horrifying regular expressions. Uhh yeah.

From 2005,
Freddie

[M.E.Farmer]

How ,
I just posted on something similar earlier ;)
Ok first of all you might want to try shlex it is in the standard
library.
If you don't know what cStringIO is dont worry about it it is just to
give a file like object to pass to shlex.
If you have a file just pass it in opened.
example: a = shlex.shlex(ope n('mytxt.txt',' r'))

py>import cStringIO
py>d = cStringIO.Strin gIO()
py>d.write('moo cow "farmer john" -zug')
py>d.seek(0)
py>a = shlex.shlex(d)
py>a.get_token( )
'moo'
py>a.get_token( )
'cow'
py>a.get_token( )
'"farmer john"'
py>a.get_token( )
'-'
py>a.get_token( )
'zug'
py>a.get_token( )
''
# ok we try again this time we add - to valid chars so we can get it
grouped as a single token .
py>d.seek(0)
py>a = shlex.shlex(d)
py>a.wordchars += '-' # add the hyphen
py>a.get_token( )
'moo'
py>a.get_token( )
'cow'
py>a.get_token( )
'"farmer john"'
py>a.get_token( )
'-zug'
py>a.get_token( )
''

Hth,
M.E.Farmer

[Fuzzyman]

That's not bad going considering you've only run out of alcohol at 6 in
the morning and *then* ask python questions.

Anyway - you could write a charcter-by-character parser function that
would do that in a few minutes...

My 'listquote' module has one - but it splits on commas not whitespace.
Sounds like you're looking for a one-liner though.... regular
expressions *could* do it............. ..

Regards,

Fuzzy
http://www.voidspace.org.uk/atlantib...tml#llistquote

[Reinhold Birkenfeld]

Freddie wrote:

Happy new year! Since I have run out of alcohol, I'll ask a question that I
haven't really worked out an answer for yet. Is there an elegant way to turn
something like:

> moo cow "farmer john" -zug

into:

['moo', 'cow', 'farmer john'], ['zug']

I'm trying to parse a search string so I can use it for SQL WHERE constraints,
preferably without horrifying regular expressions. Uhh yeah.

The shlex approach, finished:

searchstring = 'moo cow "farmer john" -zug'
lexer = shlex.shlex(sea rchstring)
lexer.wordchars += '-'
poslist, neglist = [], []
while 1:
token = lexer.get_token ()
# token is '' on eof
if not token: break
# remove quotes
if token[0] in '"\'':
token = token[1:-1]
# select in which list to put it
if token[0] == '-':
neglist.append( token[1:])
else:
poslist.append( token)

regards,
Reinhold

[M.E.Farmer]

As I noted before shlex requires a file like object or a open file .
py> import shlex
py> a = shlex.shlex('fg fgfg dgfgfdgfdg')
py> a.get_token()
Traceback (most recent call last):
File "<input>", line 1, in ?
File ".\shlex.py ", line 74, in get_token
raw = self.read_token ()
File ".\shlex.py ", line 100, in read_token
nextchar = self.instream.r ead(1)
AttributeError: 'str' object has no attribute 'read'

M.E.Farmer

[Reinhold Birkenfeld]

M.E.Farmer wrote:

As I noted before shlex requires a file like object or a open file .
py> import shlex
py> a = shlex.shlex('fg fgfg dgfgfdgfdg')
py> a.get_token()
Traceback (most recent call last):
File "<input>", line 1, in ?
File ".\shlex.py ", line 74, in get_token
raw = self.read_token ()
File ".\shlex.py ", line 100, in read_token
nextchar = self.instream.r ead(1)
AttributeError: 'str' object has no attribute 'read'

Which Python version are you using?

The docs say that since Py2.3 strings are accepted.

regards,
Reinhold

[It's me]

I am right in the middle of doing text parsing so I used your example as a
mental exercise. :-)

Here's a NDFA for your text:

b 0 1-9 a-Z , . + - ' " \n
S0: S0 E E S1 E E E S3 E S2 E
S1: T1 E E S1 E E E E E E T1
S2: S2 E E S2 E E E E E T2 E
S3: T3 E E S3 E E E E E E T3

and the end-states are:

E: error in text
T1: You have the words: moo, cow
T2: You get "farmer john" (w quotes)
T3: You get zug

Can't gurantee that I did it right - I did it really quick - and it's
*specific* to your text string.

Now just need to hire a programmer to write some clean Python parsing code.
:-)

--
It's me

"Freddie" <li**********@z ebra-madcowdisease.g iraffe-org> wrote in message
news:kX******** ********@text.u senetserver.com ...

Happy new year! Since I have run out of alcohol, I'll ask a question that I haven't really worked out an answer for yet. Is there an elegant way to turn something like:

> moo cow "farmer john" -zug
into:

['moo', 'cow', 'farmer john'], ['zug']

I'm trying to parse a search string so I can use it for SQL WHERE

constraints, preferably without horrifying regular expressions. Uhh yeah.

From 2005,
Freddie

[M.E.Farmer]

Ah! that is what the __future__ brings I guess.........
Damn that progress making me outdated ;)
Python 2.2.3 ( a lot of extensions I use are stuck there , so I still
use it)
M.E.Farmer

[Reinhold Birkenfeld]

M.E.Farmer wrote:

Ah! that is what the __future__ brings I guess.........
Damn that progress making me outdated ;)
Python 2.2.3 ( a lot of extensions I use are stuck there , so I still
use it)

I'm also positively surprised how many cute little additions are there
every new Python version. Great thanks to the great devs!

Reinhold

[Andrew Dalke]

"It's me" wrote:

Here's a NDFA for your text:

b 0 1-9 a-Z , . + - ' " \n
S0: S0 E E S1 E E E S3 E S2 E
S1: T1 E E S1 E E E E E E T1
S2: S2 E E S2 E E E E E T2 E
S3: T3 E E S3 E E E E E E T3

Now if I only had an NDFA for parsing that syntax...

:)
Andrew
da***@dalkescie ntific.com