What is the fastest way to select N items at a time from a dictionary?
I'm iterating over a dictionary of many thousands of items.
I want to operate on only 100 items at a time.
I want to avoid copying items using any sort of slicing.
Does itertools copy items?
This works, but is ugly:
>>from itertools import * D = {'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7, 'h':8, 'i':9, 'j':10} N = 3 for G in izip(*[chain(D.items(), repeat(None, N-1))]*N):
.... print G
....
(('a', 1), ('c', 3), ('b', 2))
(('e', 5), ('d', 4), ('g', 7))
(('f', 6), ('i', 9), ('h', 8))
(('j', 10), None, None)
I'd prefer the last sequence not return None
elements and instead just return (('j',10)), but this isn't a huge
deal.
This works and is clear, but it makes copies of items:
>>ii = D.items() for i in range (0, len(ii), N):
.... print ii[i:i+N]
....
[('a', 1), ('c', 3), ('b', 2)]
[('e', 5), ('d', 4), ('g', 7)]
[('f', 6), ('i', 9), ('h', 8)]
[('j', 10)]
--
Noah 2 1377
Noah <no**@noah.orgwrites:
What is the fastest way to select N items at a time from a dictionary?
I'm iterating over a dictionary of many thousands of items.
I want to operate on only 100 items at a time.
I want to avoid copying items using any sort of slicing.
I'd do something like (untested):
def groups(seq, n):
while True:
s = list(itertools.islice(seq, n))
if not s: return
yield s
items = d.iteritems()
for g in groups(items, 100):
operate_on (g)
Does itertools copy items?
I don't understand this question.
On Mar 13, 6:34 pm, Noah <n...@noah.orgwrote:
What is the fastest way to select N items at a time from a dictionary?
I'm iterating over a dictionary of many thousands of items.
I want to operate on only 100 items at a time.
I want to avoid copying items using any sort of slicing.
Does itertools copy items?
This works, but is ugly:
>from itertools import * D = {'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7, 'h':8, 'i':9, 'j':10} N = 3 for G in izip(*[chain(D.items(), repeat(None, N-1))]*N):
... print G
...
(('a', 1), ('c', 3), ('b', 2))
(('e', 5), ('d', 4), ('g', 7))
(('f', 6), ('i', 9), ('h', 8))
(('j', 10), None, None)
I'd prefer the last sequence not return None
elements and instead just return (('j',10)), but this isn't a huge
deal.
This works and is clear, but it makes copies of items:
>ii = D.items() for i in range (0, len(ii), N):
... print ii[i:i+N]
...
[('a', 1), ('c', 3), ('b', 2)]
[('e', 5), ('d', 4), ('g', 7)]
[('f', 6), ('i', 9), ('h', 8)]
[('j', 10)]
groupby?
import itertools
D = {'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7, 'h':8, 'i':9,
'j':10}
N = 3
it = itertools.groupby(enumerate(D.items()), lambda t: int(t[0]/N))
for each in it:
print tuple(t[1] for t in each[1])
--
Hope this helps,
Steven This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: kevin parks |
last post by:
hi. I've been banging my head against this one a while and have asked
around, and i
am throwing this one out there in the hopes that some one can shed
some light on
what has turned out to be a...
|
by: SnuSnu |
last post by:
Okay - here's a (probably) really easy question:
I can't do the following, so what's the best way to handle this
case of wanting to delete within a loop?
x =
deletion_list =
for i in...
|
by: Donnal Walter |
last post by:
The following method is defined in one of my classes:
def setup(self, items={}):
"""perform setup based on a dictionary of items"""
if 'something' in items:
value = items
# now do something...
|
by: It's me |
last post by:
Okay, I give up.
What's the best way to count number of items in a list?
For instance,
a=,4,5,]
I want to know how many items are there in a (answer should be 7 - I don't
want it to be 4)
|
by: Steven Bethard |
last post by:
So, I have a list of lists, where the items in each sublist are of
basically the same form. It looks something like:
py> data = ,
....
.... ,
....
.... ]
Now, I'd like to...
|
by: Ric Deez |
last post by:
Hi there,
I have a list:
L1 =
How can I easily turn this into a list of tuples where the first element
is the list element and the second is the number of times it occurs in
the list (I...
|
by: Dave Dean |
last post by:
Hi all,
I'm looking for a way to iterate through a list, two (or more) items at a
time. Basically...
myList =
I'd like to be able to pull out two items at a time - simple examples would...
|
by: Drew |
last post by:
When is it appropriate to use dict.items() vs dict.iteritems. Both
seem to work for something like:
for key,val in mydict.items():
print key,val
for key,val in mydict.iteritems():
print...
|
by: =?Utf-8?B?YW1pcg==?= |
last post by:
Hi,
I have a Generic Object that has a private List field item.
I populate the List in a different function
I use a FOREACH LOOP with a FindAll function to get all items that have a
certain...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: aa123db |
last post by:
Variable and constants
Use var or let for variables and const fror constants.
Var foo ='bar';
Let foo ='bar';const baz ='bar';
Functions
function $name$ ($parameters$) {
}
...
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
| |