best way to replace first word in string?

[hagai26]

I am looking for the best and efficient way to replace the first word
in a str, like this:
"aa to become" -> "/aa/ to become"
I know I can use spilt and than join them
but I can also use regular expressions
and I sure there is a lot ways, but I need realy efficient one

[Larry Bates]

There is a "gotcha" on this:

How do you define "word"? (e.g. can the
first word be followed by space, comma, period,
or other punctuation or is it always a space).

If it is always a space then this will be pretty
"efficient" .

string="aa to become"
firstword, restwords=s.spl it(' ',1)
newstring="/%s/ %s" % (firstword, restwords)

I'm sure the regular expression gurus here can come
up with something if it can be followed by other than
a space.

-Larry Bates
ha*****@gmail.c om wrote:

I am looking for the best and efficient way to replace the first word
in a str, like this:
"aa to become" -> "/aa/ to become"
I know I can use spilt and than join them
but I can also use regular expressions
and I sure there is a lot ways, but I need realy efficient one

[Mike Meyer]

"ha*****@gmail. com" <ha*****@gmail. com> writes:

I am looking for the best and efficient way to replace the first word
in a str, like this:
"aa to become" -> "/aa/ to become"
I know I can use spilt and than join them
but I can also use regular expressions
and I sure there is a lot ways, but I need realy efficient one

Assuming you know the whitespace will be spaces, I like find:

new = "/aa/" + old[old.find(' '):]

As for efficiency - I suggest you investigate the timeit module, and
do some tests on data representative of what you're actaully going to
be using.

<mike
--
Mike Meyer <mw*@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Micah Elliott]

On Oct 20, ha*****@gmail.c om wrote:

I am looking for the best and efficient way to replace the first word
in a str, like this:
"aa to become" -> "/aa/ to become"
I know I can use spilt and than join them
but I can also use regular expressions
and I sure there is a lot ways, but I need realy efficient one

Of course there are many ways to skin this cat; here are some trials.
The timeit module is useful for comparison (and I think I'm using it
correctly :-). I thought that string concatenation was rather
expensive, so its being faster than %-formatting surprised me a bit:

$ python -mtimeit '
res = "/%s/ %s"% tuple("a b c".split(" ", 1))'
100000 loops, best of 3: 3.87 usec per loop

$ python -mtimeit '
b,e = "a b c".split(" ", 1); res = "/"+b+"/ "+e'
100000 loops, best of 3: 2.78 usec per loop

$ python -mtimeit '
"/"+"a b c".replace(" ", "/ ", 1)'
100000 loops, best of 3: 2.32 usec per loop

$ python -mtimeit '
"/%s" % ("a b c".replace(" ", "/ ", 1))'
100000 loops, best of 3: 2.83 usec per loop

$ python -mtimeit '
"a b c".replace(" ", "/", 1).replace(" ", "/ ", 1)'
100000 loops, best of 3: 3.51 usec per loop

There are possibly better ways to do this with strings.

And the regex is comparatively slow, though I'm not confident this one
is optimally written:

$ python -mtimeit -s'import re' '
re.sub(r"^(\w*) ", r"/\1/", "a b c")'
10000 loops, best of 3: 44.1 usec per loop

You'll probably want to experiment with longer strings if a test like
"a b c" is not representative of your typical input.

--
_ _ ___
|V|icah |- lliott http://micah.elliott.name md*@micah.ellio tt.name
" " """

[Fredrik Lundh]

Micah Elliott wrote:

And the regex is comparatively slow, though I'm not confident this one
is optimally written:

$ python -mtimeit -s'import re' '
re.sub(r"^(\w*) ", r"/\1/", "a b c")'
10000 loops, best of 3: 44.1 usec per loop

the above has to look the pattern up in the compilation cache for each loop,
and it also has to parse the template string. precompiling the pattern and
using a callback instead of a template string can speed things up somewhat:

timeit -s"import re; sub = re.compile(r'^( \w*)').sub"
"sub(lambda x: '/%s/' % x.groups(), 'a b c')"

(but the replace solutions should be faster anyway; it's not free to prepare
for a RE match, and sub uses the same split/join implementation as replace...)

</F>

[Steven D'Aprano]

On Thu, 20 Oct 2005 08:26:43 -0700, ha*****@gmail.c om wrote:

I am looking for the best and efficient way to replace the first word
in a str, like this:
"aa to become" -> "/aa/ to become"
I know I can use spilt and than join them
but I can also use regular expressions
and I sure there is a lot ways, but I need realy efficient one

Efficient for what?

Efficient in disk-space used ("least source code")?

Efficient in RAM used ("smallest objects and compiled code")?

Efficient in execution time ("fastest")?

Efficient in program time ("quickest to write and debug")?

If you use regular expressions, does the time taken in loading the module
count, or can we assume you have already loaded it?

It will also help if you specify your problem a little better. Are you
replacing one word, and then you are done? Or at you repeating hundreds of
millions of times? What is the context of the problem?

Most importantly, have you actually tested your code to see if it is
efficient enough, or are you just wasting your -- and our -- time with
premature optimization?

def replace_word(so urce, newword):
"""Replace the first word of source with newword."""
return newword + " " + "".join(source. split(None, 1)[1:])

import time
def test():
t = time.time()
for i in range(10000):
s = replace_word("a a to become", "/aa/")
print ((time.time() - t)/10000), "s"

py> test()
3.6199092865e-06 s
Is that fast enough for you?
--
Steven.

[William Park]

ha*****@gmail.c om <ha*****@gmail. com> wrote:

I am looking for the best and efficient way to replace the first word
in a str, like this:
"aa to become" -> "/aa/ to become"
I know I can use spilt and than join them
but I can also use regular expressions
and I sure there is a lot ways, but I need realy efficient one

I doubt you'll find faster than Sed.

man sed

--
William Park <op**********@y ahoo.ca>, Toronto, Canada
ThinFlash: Linux thin-client on USB key (flash) drive
http://home.eol.ca/~parkw/thinflash.html
BashDiff: Super Bash shell
http://freshmeat.net/projects/bashdiff/

[Steven D'Aprano]

On Thu, 20 Oct 2005 10:25:27 -0700, Micah Elliott wrote:

I thought that string concatenation was rather
expensive, so its being faster than %-formatting surprised me a bit:

Think about what string concatenation actually does:

s = "hello " + "world"

In pseudo-code, it does something like this:

- Count chars in "hello" (six chars).
- Count chars in "world" (five chars).
- Allocate eleven bytes.
- Copy six chars from "hello " and five from "world" into the newly
allocated bit of memory.

(This should not be thought of as the exact process that Python uses, but
simply illustrating the general procedure.)

Now think of what str-formatting would do:

s = "hello %s" % "world"

In pseudo-code, it might do something like this:

- Allocate a chunk of bytes, hopefully not too big or too small.
- Repeat until done:
- Copy chars from the original string into the new string,
until it hits a %s placeholder.
- Grab the next string from the args, and copy chars from
that into the new string. If the new string is too small,
reallocate memory to make it bigger, potentially moving
chunks of bytes around.

The string formatting pseudo-code is a lot more complicated and has to do
more work than just blindly copying bytes. It has to analyse the bytes it
is copying, looking for placeholders.

So string concatenation is more efficient, right? No. The thing is, a
*single* string concatenation is almost certainly more efficient than a
single string concatenation. But now look what happens when you repeat it:

s = "h" + "e" + "l" + "l" + "o" + " " + "w" + "o" + "r" + "l" + "d"

This ends up doing something like this:

- Allocate two bytes, copying "h" and "e" into them.
- Allocate three bytes, copying "he" and "l" into them.
- Allocate four bytes, copying "hel" and "l" into them.
....
- Allocate eleven bytes, copying "hello worl" and "d" into them.

The problem is that string concatenation doesn't scale efficiently. String
formatting, on the other hand, does more work to get started, but scales
better.

See, for example, this test code:

py> def tester(n):
.... s1 = ""
.... s2 = "%s" * n
.... bytes = tuple([chr(i % 256) for i in range(n)])
.... t1 = time.time()
.... for i in range(n):
.... s1 = s1 + chr(i % 256)
.... t1 = time.time() - t1
.... t2 = time.time()
.... s2 = s2 % bytes
.... t2 = time.time() - t2
.... assert s1 == s2
.... print t1, t2
....
py> x = 100000
py> tester(x)
3.24212408066 0.01252317428
py> tester(x)
2.58376598358 0.01238489151
py> tester(x)
2.76262307167 0.01474809646

The string formatting is two orders of magnitude faster than the
concatenation. The speed difference becomes even more obvious when you
increase the number of strings being concatenated:

py> tester(x*10)
2888.56399703 0.13130998611

Almost fifty minutes, versus less than a quarter of a second.
--
Steven.

[Steven D'Aprano]

On Sat, 22 Oct 2005 21:05:43 +1000, Steven D'Aprano wrote:

The thing is, a
*single* string concatenation is almost certainly more efficient than a
single string concatenation.

Dagnabit, I meant a single string concatenation is more efficient than a
single string replacement using %.
--
Steven.

[Chris F.A. Johnson]

On 2005-10-22, William Park wrote:

ha*****@gmail.c om <ha*****@gmail. com> wrote:

I am looking for the best and efficient way to replace the first word
in a str, like this:
"aa to become" -> "/aa/ to become"
I know I can use spilt and than join them
but I can also use regular expressions
and I sure there is a lot ways, but I need realy efficient one

I doubt you'll find faster than Sed.

On the contrary; to change a string, almost anything will be faster
than sed (except another external program).

If you are in a POSIX shell, parameter expansion will be a lot
faster.

In a python program, one of the solutions already posted will be
much faster.

--
Chris F.A. Johnson <http://cfaj.freeshell. org>
=============== =============== =============== =============== ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>