Batches of N

Sending a million rows to a server in one go is too much for most servers, so programs send them in batches. Write a generator batched(iterable, n, fill=NO_FILL) that yields the items of iterable in tuples of n, in order.

• The last batch may be shorter, when the items run out.
• If a fill value is given, a short last batch is padded with it up to the full length n. Any value can be the fill, including None.
• If n is less than 1, raise ValueError.

It must be lazy: to make a batch it reads at most n items, and only when that batch is asked for, so it also works on endless iterables.

Examples

list(batched('abcdefg', 3))                 # returns [('a', 'b', 'c'), ('d', 'e', 'f'), ('g',)]
list(batched([1, 2, 3, 4, 5], 2, fill=0))   # returns [(1, 2), (3, 4), (5, 0)]
list(batched([1, 2, 3], 2, fill=None))      # returns [(1, 2), (3, None)]

from itertools import count, islice
list(islice(batched(count(), 3), 2))        # returns [(0, 1, 2), (3, 4, 5)]

Constraints

• Up to 100,000 items, or endless

Note

None cannot be the default for fill, because None is a fill someone might want. The starter provides a unique object, NO_FILL, as the default instead: fill is NO_FILL is true only when no fill was given.