split a text

[mickey0]

hello,
I have to read a text and I decided to split it with Scanner class.

    private Vector<String> splitWords(Scanner s) {
        Vector<String> v = new Vector<String>();        
        while ( s.hasNext() ) {
 
            String token = s.next();
            if ( token.length() == 1 && (token == "." || token == ";" || token == "," || token == ":") ) { 
                continue;
 
 
            }
            v.add( token );
        }
        return v;                                
    }    
 

and return a vector with all tokens. However as you can see I'd like to don't put in the vector the symbols ; ,.: ? )( etc. I can do as here above (with that ugly if), but I have a text like
"heloworls......................."
how can I delete the ........................... in agodd way? (the scanner doens't separate the "................................"

thanks

[Nepomuk]

Well, have a look at regular expressions. They should do the job. Especially make sure to read this.

Greetings,
Nepomuk

[samido]

string tokenisation my firend, use this class ...!

[JosAH]

@samido
Tokenizers are the old fashioned way of doing things; Scanners or a simple String.split() handle the job much better (by using regular expressions).

kind regards,

Jos

[mickey0]

@Nepomuk
Hi, I tri to do this to skip every non-word charachter but it dones't work at all.

        while ( s.hasNext() ) {
            try {
                s.skip("\\W");
            }
            catch (NoSuchElementException ex) {
 
            }
 
            String token = s.next();
 

What's wrong, please?

[JosAH]

Try this: Scanner.useDelimiter("\\W+");

kind regards,

Jos