Identifying the start of good data in a list

[tkpmep]

I have a list that starts with zeros, has sporadic data, and then has
good data. I define the point at which the data turns good to be the
first index with a non-zero entry that is followed by at least 4
consecutive non-zero data items (i.e. a week's worth of non-zero
data). For example, if my list is [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8,
9], I would define the point at which data turns good to be 4 (1
followed by 2, 3, 4, 5).

I have a simple algorithm to identify this changepoint, but it looks
crude: is there a cleaner, more elegant way to do this?

flag = True
i=-1
j=0
while flag and i < len(retHist)-1:
i += 1
if retHist[i] == 0:
j = 0
else:
j += 1
if j == 5:
flag = False

del retHist[:i-4]

Thanks in advance for your help

Thomas Philips

[bearophileHUGS]

First solutions I have found, not much tested beside the few doctests:

from itertools import islice

def start_good1(ali st, good_ones=4):
"""
Maybe more efficient for Python

>>start_good = start_good1
start_good([0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

4

>>start_good([])

-1

>>start_good([0, 0])

-1

>>start_good([0, 0, 0])

-1

>>start_good([0, 0, 0, 0, 1])

-1

>>start_good([0, 0, 1, 0, 1, 2, 3])

-1

>>start_good([0, 0, 1, 0, 1, 2, 3, 4])

4

>>start_good([0, 0, 1, 0, 1, 2, 3, 4, 5])

4

>>start_good([1, 2, 3, 4])

0

>>start_good([1, 2, 3])

-1

>>start_good([0, 0, 1, 0, 1, 2, 0, 4])

-1
"""
for i in xrange(len(alis t) - good_ones + 1):
if all(islice(alis t, i, i+good_ones)):
return i
return -1

def start_good2(ali st, good_ones=4):
"""
Maybe more efficient for Psyco

>>start_good = start_good2
start_good([0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

4

>>start_good([])

-1

>>start_good([0, 0])

-1

>>start_good([0, 0, 0])

-1

>>start_good([0, 0, 0, 0, 1])

-1

>>start_good([0, 0, 1, 0, 1, 2, 3])

-1

>>start_good([0, 0, 1, 0, 1, 2, 3, 4])

4

>>start_good([0, 0, 1, 0, 1, 2, 3, 4, 5])

4

>>start_good([1, 2, 3, 4])

0

>>start_good([1, 2, 3])

-1

>>start_good([0, 0, 1, 0, 1, 2, 0, 4])

-1
"""
n_good = 0
for i, el in enumerate(alist ):
if alist[i]:
if n_good == good_ones:
return i - good_ones
else:
n_good += 1
else:
n_good = 0
if n_good == good_ones:
return len(alist) - good_ones
else:
return -1
if __name__ == "__main__":
import doctest
doctest.testmod ()
print "Doctests done\n"

Bye,
bearophile

[bearophileHUGS]

Sorry, in the Psyco version replace this line:
for i, el in enumerate(alist ):

With:
for i in xrange(len(alis t)):

because Psyco doesn't digest enumerate well.

Bye,
bearophile

[Matthew Fitzgibbons]

tk****@hotmail. com wrote:

I have a list that starts with zeros, has sporadic data, and then has
good data. I define the point at which the data turns good to be the
first index with a non-zero entry that is followed by at least 4
consecutive non-zero data items (i.e. a week's worth of non-zero
data). For example, if my list is [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8,
9], I would define the point at which data turns good to be 4 (1
followed by 2, 3, 4, 5).

I have a simple algorithm to identify this changepoint, but it looks
crude: is there a cleaner, more elegant way to do this?

flag = True
i=-1
j=0
while flag and i < len(retHist)-1:
i += 1
if retHist[i] == 0:
j = 0
else:
j += 1
if j == 5:
flag = False

del retHist[:i-4]

Thanks in advance for your help

Thomas Philips
--
http://mail.python.org/mailman/listinfo/python-list

Maybe this will do?

reHist = [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
count = 0
for i, d in enumerate(reHis t):
if d == 0:
count = 0
else:
count += 1
if count == 5:
break
else:
raise Exception("No data found")
reHist = reHist[i-4:]
print reHist
-Matt

[Mensanator]

On Aug 26, 4:49*pm, tkp...@hotmail. com wrote:

I have a list that starts with zeros, has sporadic data, and then has
good data. I define the point at *which the data turns good to be the
first index with a non-zero entry that is followed by at least 4
consecutive non-zero data items (i.e. a week's worth of non-zero
data). For example, if my list is [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8,
9], I would define the point at which data turns good to be 4 (1
followed by 2, 3, 4, 5).

I have a simple algorithm to identify this changepoint, but it looks
crude: is there a cleaner, more elegant way to do this?

* * flag = True
* * i=-1
* * j=0
* * while flag and i < len(retHist)-1:
* * * * i += 1
* * * * if retHist[i] == 0:
* * * * * * j = 0
* * * * else:
* * * * * * j += 1
* * * * * * if j == 5:
* * * * * * * * flag = False

* * del retHist[:i-4]

Thanks in advance for your help

Thomas Philips

Here's my attempt:

LL = [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

i = 0

while (i<len(LL)) and (0 in LL[i:i+5]):
i += 1

print i, LL[i:i+5]

##
## 4 [1, 2, 3, 4, 5]
##

[Emile van Sebille]

tk****@hotmail. com wrote:

I have a list that starts with zeros, has sporadic data, and then has
good data. I define the point at which the data turns good to be the
first index with a non-zero entry that is followed by at least 4
consecutive non-zero data items (i.e. a week's worth of non-zero
data). For example, if my list is [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8,
9], I would define the point at which data turns good to be 4 (1
followed by 2, 3, 4, 5).

I have a simple algorithm to identify this changepoint, but it looks
crude: is there a cleaner, more elegant way to do this?

>>for ii,dummy in enumerate(retHi st):

.... if 0 not in retHist[ii:ii+5]:
.... break

>>del retHist[:ii]

Well, to the extent short and sweet is elegant...

Emile

[tdmj]

On Aug 26, 5:49 pm, tkp...@hotmail. com wrote:

I have a list that starts with zeros, has sporadic data, and then has
good data. I define the point at which the data turns good to be the
first index with a non-zero entry that is followed by at least 4
consecutive non-zero data items (i.e. a week's worth of non-zero
data). For example, if my list is [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8,
9], I would define the point at which data turns good to be 4 (1
followed by 2, 3, 4, 5).

I have a simple algorithm to identify this changepoint, but it looks
crude: is there a cleaner, more elegant way to do this?

flag = True
i=-1
j=0
while flag and i < len(retHist)-1:
i += 1
if retHist[i] == 0:
j = 0
else:
j += 1
if j == 5:
flag = False

del retHist[:i-4]

Thanks in advance for your help

Thomas Philips

With regular expressions:

import re

hist = [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
hist_str = ''.join(str(i) for i in hist)
match = re.search(r'[1-9]{5, }', hist_str)
hist = hist[-5:] if match is None else hist[match.start():]

Or slightly more concise:

import re

hist = [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
match = re.search(r'[1-9]{5, }', ''.join(str(i) for i in hist))
hist = hist[-5:] if match is None else hist[match.start():]

Tommy McDaniel

[tkpmep]

On Aug 26, 7:23*pm, Emile van Sebille <[e-mail] wrote:

tkp...@hotmail. com wrote:

I have a list that starts with zeros, has sporadic data, and then has
good data. I define the point at *which the data turns good to be the
first index with a non-zero entry that is followed by at least 4
consecutive non-zero data items (i.e. a week's worth of non-zero
data). For example, if my list is [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8,
9], I would define the point at which data turns good to be 4 (1
followed by 2, 3, 4, 5).

I have a simple algorithm to identify this changepoint, but it looks
crude: is there a cleaner, more elegant way to do this?

*>>for ii,dummy in enumerate(retHi st):
... * * if 0 not in retHist[ii:ii+5]:
... * * * * break

*>>del retHist[:ii]

Well, to the extent short and sweet is elegant...

Emile

This is just what the doctor ordered. Thank you, everyone, for the
help.

Sincerely

Thomas Philips

[Terry Reedy]

Matthew Fitzgibbons wrote:

tk****@hotmail. com wrote:

>
reHist = [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
count = 0
for i, d in enumerate(reHis t):
if d == 0:
count = 0
else:
count += 1
if count == 5:
break
else:
raise Exception("No data found")
reHist = reHist[i-4:]
print reHist

This is what I would have suggested, except that the 'if count' test
should be left under the else clause, as in the original, so I consider
it the best of the responses ;-)

I thought of the repeated slicing alternative, but it would be slightly
slower. However, for occasional runs, the difference would be trivial.

Worrying about what Psyco does for this problem is rather premature
optimization.

My quarter's worth....

tjr

[Scott David Daniels]

tk****@hotmail. com wrote:

I have a list that starts with zeros, has sporadic data, and then has
good data. I define the point at which the data turns good to be the
first index with a non-zero entry that is followed by at least 4
consecutive non-zero data items (i.e. a week's worth of non-zero
data). For example, if my list is [0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8,
9], I would define the point at which data turns good to be 4 (1
followed by 2, 3, 4, 5).

I have a simple algorithm to identify this changepoint, but it looks
crude: is there a cleaner, more elegant way to do this?

flag = True
i=-1
j=0
while flag and i < len(retHist)-1:
i += 1
if retHist[i] == 0:
j = 0
else:
j += 1
if j == 5:
flag = False

del retHist[:i-4]

Thanks in advance for your help

Thomas Philips

Here is one that can go iterator-to-iterator:

def started(source) :
src = iter(source)
lead = []
for x in src:
if x:
lead.append(x)
if len(lead) == 5:
return itertools.chain (lead, src)
else:
lead = []
print list(started([0, 0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]))
--Scott David Daniels
Sc***********@A cm.Org