4  4858 
Ken Fettig wrote: Does Python have an equivelent to the Java StringTokenizer? If so,
what is it and how do you implement it?
Is this the kind of thing that you mean?
Python 2.3b1 (#40, Apr 25 2003, 19:06:24)
Type "help", "copyright", "credits" or "license" for more information. s = "This is a string to be tokenised"
s.split()
['This', 'is', 'a', 'string', 'to', 'be', 'tokenised']
s = "This:is:a:string:to:be:tokenised"
s.split(':')
['This', 'is', 'a', 'string', 'to', 'be', 'tokenised']
s.split(':', 2)
['This', 'is', 'a:string:to:be:tokenised']
Or maybe you have something more specific in mind?
--
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/mailto/alan
Ken Fetting wants a 'StringTokenizer'.
Alan Kennedy points out s = "This is a string to be tokenised"
s.split() ['This', 'is', 'a', 'string', 'to', 'be', 'tokenised']
... Or maybe you have something more specific in mind?
Another option is the little-known 'shlex' module, part of the standard
library.
import shlex, StringIO
infile = StringIO.StringIO("""ls -lart "have space.*" will travel""")
x = shlex.shlex(infile)
x.get_token()
'ls' x.get_token()
'-' x.get_token()
'lart' x.get_token()
'"have space.*"' x.get_token()
'will' x.get_token()
'travel' x.get_token()
''
As you can see, it treats '-' unexpectedly (compared to the shell).
Also, with __iter__ in newer Pythons, if these module were useful
then it would be nice if "for token in shlex..." worked.
Andrew da***@dalkescientific.com
"Ken Fettig" <kf*****@state.nd.us> writes: Does Python have an equivelent to the Java StringTokenizer? If so, what is
it and how do you implement it?
Thanks
Ken Fettig
ke*******@btinet.net
kf*****@state.nd.us
See shlex (in the main distribution) or see a variety of lexer/parser
tools such as Ply, Yapp.py,
shlex is about the level of complexity you want. See the Library
Reference Manual for instructions.
-- ha************@boeing.com
6-6M31 Knowledge Management
Phone: (425) 342-5601
Andrew Dalke schrieb: Another option is the little-known 'shlex' module, part of the standard
library.
.... As you can see, it treats '-' unexpectedly (compared to the shell).
This is why shellword was written (see
<http://www.crazy-compilers.com/py-lib/>)
Regards
Hartmut Goebel
--
| Hartmut Goebel | We build the crazy compilers |
| h.******@crazy-compilers.com | Compiler Manufacturer | This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Andr? Roberge |
last post by:
According to the Python documentation:
18.5 tokenize -- Tokenizer for Python source
....
The primary entry point is a generator:
generate_tokens(readline)
....
An older entry point is...
|
by: qwweeeit |
last post by:
In analysing a very big application (pysol) made of almost
100 sources, I had the need to remove comments.
Removing the comments which take all the line is straightforward...
Instead for the...
|
by: Kelvin |
last post by:
hi:
in C, we can use strtok() to tokenize a char*
but i can't find any similar member function of string that can tokenize a
string
so how so i tokenize a string in C++?
do it the C way?
...
|
by: Ram Laxman |
last post by:
Hi all,
How can I tokenize the integers using strtok. For example:
If I have some thing like:
"ram":"laxman":"deepak"
then I can safely use strtok.But if I have something like below:...
|
by: Nitin |
last post by:
Hi All,
Is there any ANSI library ( header file ) which contains functions to
extract or tokenize arguments of main( ) ?
We have strtok( ), but thats non re-entrant, as it maintains static...
|
by: bubunia2000 |
last post by:
Hi all,
I heard that strtok is not thread safe. So I want to write a
sample program which will tokenize string without using strtok.
Can I get a sample source code for the same.
For exp:...
|
by: glomde |
last post by:
Hi I would like to extend python so that you could create hiercical
tree structures (XML, HTML etc) easier and that resulting code would be
more readable.
The syntax i would like is something...
|
by: Tim |
last post by:
I ran into a problem with a script i was playing with to check code
indents and need some direction. It seems to depend on if tabsize is
set to 4 in editor and spaces and tabs indents are mixed on...
|
by: noobcprogrammer |
last post by:
#include "IndexADT.h"
int IndexInit(IndexADT* word)
{
word->head = NULL;
word->wordCount = 0;
return 1;
}
int IndexCreate(IndexADT* wordList,char* argv)
|
by: askalottaqs |
last post by:
there's in maya's scripting language mel, called tokenize, you simply tokenize("string i want to tokenize"," ",bufferArray)
which will fill the fufferArray wih the first string tokenized accorfing...
|
by: aa123db |
last post by:
Variable and constants
Use var or let for variables and const fror constants.
Var foo ='bar';
Let foo ='bar';const baz ='bar';
Functions
function $name$ ($parameters$) {
}
...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
| |
