groupby

[Bryan]

can some explain why in the 2nd example, m doesn't print the list [1, 1, 1]
which i had expected?

for k, g in groupby([1, 1, 1, 2, 2, 3]): .... print k, list(g)
....
1 [1, 1, 1]
2 [2, 2]
3 [3]

m = list(groupby([1, 1, 1, 2, 2, 3]))
m [(1, <itertools._gro uper object at 0x00AAC600>), (2, <itertools._gro uper object
at 0x00AAC5A0>), (3, <itertools._gro uper object at 0x00AAC5B0>)] list(m[0][1]) []

thanks,

bryan

[George Sakkis]

Bryan wrote:

can some explain why in the 2nd example, m doesn't print the list [1, 1, 1]
which i had expected?

>>> for k, g in groupby([1, 1, 1, 2, 2, 3]): ... print k, list(g)
...
1 [1, 1, 1]
2 [2, 2]
3 [3]

>>> m = list(groupby([1, 1, 1, 2, 2, 3]))
>>> m [(1, <itertools._gro uper object at 0x00AAC600>), (2, <itertools._gro uper object
at 0x00AAC5A0>), (3, <itertools._gro uper object at 0x00AAC5B0>)] >>> list(m[0][1]) [] >>>

thanks,

bryan

I've tripped on this more than once, but it's in the docs
(http://docs.python.org/lib/itertools-functions.html):

"The returned group is itself an iterator that shares the underlying
iterable with groupby(). Because the source is shared, when the groupby
object is advanced, the previous group is no longer visible. So, if
that data is needed later, it should be stored as a list"

George

[Bryan]

George Sakkis wrote:

Bryan wrote:

can some explain why in the 2nd example, m doesn't print the list [1, 1, 1]
which i had expected?

>>> for k, g in groupby([1, 1, 1, 2, 2, 3]):

... print k, list(g)
...
1 [1, 1, 1]
2 [2, 2]
3 [3]

>>> m = list(groupby([1, 1, 1, 2, 2, 3]))
>>> m

[(1, <itertools._gro uper object at 0x00AAC600>), (2, <itertools._gro uper object
at 0x00AAC5A0>), (3, <itertools._gro uper object at 0x00AAC5B0>)]

>>> list(m[0][1])

[]

>>>

thanks,

bryan

I've tripped on this more than once, but it's in the docs
(http://docs.python.org/lib/itertools-functions.html):

"The returned group is itself an iterator that shares the underlying
iterable with groupby(). Because the source is shared, when the groupby
object is advanced, the previous group is no longer visible. So, if
that data is needed later, it should be stored as a list"

George

i read that description in the docs so many times before i posted here. now that
i read it about 10 more times, i finally get it. there's just something about
the wording that kept tripping me up, but i can't explain why :)

thanks,

bryan

[Paul McGuire]

"Bryan" <be****@gmail.c om> wrote in message
news:ma******** *************** *************** *@python.org...

George Sakkis wrote: <snip>

"The returned group is itself an iterator that shares the underlying
iterable with groupby(). Because the source is shared, when the groupby
object is advanced, the previous group is no longer visible. So, if
that data is needed later, it should be stored as a list"

George

i read that description in the docs so many times before i posted here.

now that i read it about 10 more times, i finally get it. there's just something about the wording that kept tripping me up, but i can't explain why :)

thanks,

bryan

So here's how to save the values from the iterators while iterating over the
groupby:

m = [(x,list(y)) for x,y in groupby([1, 1, 1, 2, 2, 3])]
m

[(1, [1, 1, 1]), (2, [2, 2]), (3, [3])]

-- Paul

[Paul McGuire]

"Paul McGuire" <pt***@austin.r r._bogus_.com> wrote in message
news:bz******** **********@torn ado.texas.rr.co m...

So here's how to save the values from the iterators while iterating over the groupby:

m = [(x,list(y)) for x,y in groupby([1, 1, 1, 2, 2, 3])]
m

[(1, [1, 1, 1]), (2, [2, 2]), (3, [3])]

-- Paul

Playing some more with groupby. Here's a one-liner to tally a list of
integers into a histogram:

# create data set, random selection of numbers from 1-10
dataValueRange = range(1,11)
data = [random.choice(d ataValueRange) for i in xrange(10)]
print data

# tally values into histogram:
# (from the inside out:
# - sort data into ascending order, so groupby will see all like values
together
# - call groupby, return iterator of (value,valueIte mIterator) tuples
# - tally groupby results into a dict of (value, valueFrequency) tuples
# - expand dict into histogram list, filling in zeroes for any keys that
didn't get a value
hist = [ (k1,dict((k,len (list(g))) for k,g in
itertools.group by(sorted(data) )).get(k1,0)) for k1 in dataValueRange ]

print hist

Gives:
[9, 6, 8, 3, 2, 3, 10, 7, 6, 2]
[(1, 0), (2, 2), (3, 2), (4, 0), (5, 0), (6, 2), (7, 1), (8, 1), (9, 1),
(10, 1)]

Change the generation of the original data list to 10,000 values, and you
get something like:
[(1, 995), (2, 986), (3, 941), (4, 998), (5, 978), (6, 1007), (7, 997), (8,
1033), (9, 1038), (10, 1027)]

If you know there wont be any zero frequency values (or don't care about
them), you can skip the fill-in-the-zeros step, with one of these
expressions:
histAsList = [ (k,len(list(g)) ) for k,g in itertools.group by(sorted(data) ) ]
histAsDict = dict((k,len(lis t(g))) for k,g in
itertools.group by(sorted(data) ))

-- Paul