Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I see many answers suggesting itertools.tee , but that's ignoring one crucial warning in the docs for it:

This itertool may require significant auxiliary storage (depending on how much temporary data needs to be stored). In general, if one iterator uses most or all of the data before another iterator starts, it is faster to use list() instead of tee() .

Basically, tee is designed for those situation where two (or more) clones of one iterator, while "getting out of sync" with each other, don't do so by much -- rather, they say in the same "vicinity" (a few items behind or ahead of each other). Not suitable for the OP's problem of "redo from the start".

L = list(DictReader(...)) on the other hand is perfectly suitable, as long as the list of dicts can fit comfortably in memory. A new "iterator from the start" (very lightweight and low-overhead) can be made at any time with iter(L) , and used in part or in whole without affecting new or existing ones; other access patterns are also easily available.

As several answers rightly remarked, in the specific case of csv you can also .seek(0) the underlying file object (a rather special case). I'm not sure that's documented and guaranteed, though it does currently work; it would probably be worth considering only for truly huge csv files, in which the list I recommmend as the general approach would have too large a memory footprint.

you know that you can open the file for reading, and create a DictReader with

blah = open('blah.csv', 'r')
reader= csv.DictReader(blah)
Then, you will be able to get the next line with reader.next(), which should output
{'a':1,'b':2,'c':3,'d':4}
using it again will produce
{'a':2,'b':3,'c':4,'d':5}
However, at this point if you use blah.seek(0), the next time you call reader.next() you will get
{'a':1,'b':2,'c':3,'d':4}
again.
This seems to be the functionality you're looking for. I'm sure there are some tricks associated with this approach that I'm not aware of however. @Brian suggested simply creating another DictReader. This won't work if you're first reader is half way through reading the file, as your new reader will have unexpected keys and values from wherever you are in the file.
                @Wilduck: the behavior you're describing with another instance of DictReader won't happen if you make a new file handle and pass that to the second DictReader, right?
– user248237
                Oct 24, 2012 at 17:52
No. Python's iterator protocol is very simple, and only provides one single method (.next() or __next__()), and no method to reset an iterator in general.
The common pattern is to instead create a new iterator using the same procedure again.
If you want to "save off" an iterator so that you can go back to its beginning, you may also fork the iterator by using itertools.tee
                While you're analysis of the .next() method is probably correct, there is a fairly simple way to get what the op is asking for.
– Wilduck
                Jul 16, 2010 at 15:27
                @Wilduck: I see that your answer. I just answered the iterator question, and I have no idea about the csv module. Hopefully both answers are useful to the original poster.
– u0b34a0f6ae
                Jul 16, 2010 at 15:33
                Strictly, the iterator protocol also requires __iter__. That is, iterators are required also to be iterables.
– Steve Jessop
                Jan 22, 2014 at 13:39
                @LWZ: I don't think so, but you can try: the next() and on a StopIteration exception do a reset().
– Dennis Williamson
                Jul 28, 2016 at 22:07
There's a bug in using .seek(0) as advocated by Alex Martelli and Wilduck above, namely that the next call to .next() will give you a dictionary of your header row in the form of {key1:key1, key2:key2, ...}.  The work around is to follow file.seek(0) with a call to reader.next() to get rid of the header row.
So your code would look something like this:
f_in = open('myfile.csv','r')
reader = csv.DictReader(f_in)
for record in reader:
    if some_condition:
        # reset reader to first row of data on 2nd line of file
        f_in.seek(0)
        reader.next()
        continue
    do_something(record)
This is perhaps orthogonal to the original question, but one could wrap the iterator in a function that returns the iterator.
def get_iter():
    return iterator
To reset  the iterator just call the function again. 
This is of course trivial if the function when the said function takes no arguments.
In the case that the function requires some arguments, use functools.partial to create a closure that can be passed instead of the original iterator.
def get_iter(arg1, arg2):
   return iterator
from functools import partial
iter_clos = partial(get_iter, a1, a2)
This seems to avoid the caching that tee (n copies) or list (1 copy) would need to do
For small files, you may consider using more_itertools.seekable - a third-party tool that offers resetting iterables.
import csv
import more_itertools as mit
filename = "data/iris.csv"
with open(filename, "r") as f:
    reader = csv.DictReader(f)
    iterable = mit.seekable(reader)                    # 1
    print(next(iterable))                              # 2
    print(next(iterable))
    print(next(iterable))
    print("\nReset iterable\n--------------")
    iterable.seek(0)                                   # 3
    print(next(iterable))
    print(next(iterable))
    print(next(iterable))
Output
{'Sepal width': '3.5', 'Petal width': '0.2', 'Petal length': '1.4', 'Sepal length': '5.1', 'Species': 'Iris-setosa'}
{'Sepal width': '3', 'Petal width': '0.2', 'Petal length': '1.4', 'Sepal length': '4.9', 'Species': 'Iris-setosa'}
{'Sepal width': '3.2', 'Petal width': '0.2', 'Petal length': '1.3', 'Sepal length': '4.7', 'Species': 'Iris-setosa'}
Reset iterable
--------------
{'Sepal width': '3.5', 'Petal width': '0.2', 'Petal length': '1.4', 'Sepal length': '5.1', 'Species': 'Iris-setosa'}
{'Sepal width': '3', 'Petal width': '0.2', 'Petal length': '1.4', 'Sepal length': '4.9', 'Species': 'Iris-setosa'}
{'Sepal width': '3.2', 'Petal width': '0.2', 'Petal length': '1.3', 'Sepal length': '4.7', 'Species': 'Iris-setosa'}
Here a DictReader is wrapped in a seekable object (1) and advanced (2).  The seek() method is used to reset/rewind the iterator to the 0th position (3).
Note: memory consumption grows with iteration, so be wary applying this tool to  large files, as indicated in the docs.
While there is no iterator reset, the "itertools" module from python 2.6 (and later) has some utilities that can help there.
One of then is the "tee" which can make multiple copies of an iterator, and cache the results of the one running ahead, so that these results are used on the copies. I will seve your purposes:
>>> def printiter(n):
...   for i in xrange(n):
...     print "iterating value %d" % i
...     yield i
>>> from itertools import tee
>>> a, b = tee(printiter(5), 2)
>>> list(a)
iterating value 0
iterating value 1
iterating value 2
iterating value 3
iterating value 4
[0, 1, 2, 3, 4]
>>> list(b)
[0, 1, 2, 3, 4]
One possible option is to use itertools.cycle(), which will allow you to iterate indefinitely without any trick like .seek(0).
iterDic = itertools.cycle(csv.DictReader(open('file.csv')))
Return a newly created iterator at the last iteration during the 'iter()' call
class ResetIter: 
  def __init__(self, num):
    self.num = num
    self.i = -1
  def __iter__(self):
    if self.i == self.num-1: # here, return the new object
      return self.__class__(self.num) 
    return self
  def __next__(self):
    if self.i == self.num-1:
      raise StopIteration
    if self.i <= self.num-1:
      self.i += 1
      return self.i
reset_iter = ResetRange(10)
for i in reset_iter:
  print(i, end=' ')
print()
for i in reset_iter:
  print(i, end=' ')
print()
for i in reset_iter:
  print(i, end=' ')
Output:
0 1 2 3 4 5 6 7 8 9 
0 1 2 3 4 5 6 7 8 9 
0 1 2 3 4 5 6 7 8 9 
Problem
I've had the same issue before. After analyzing my code, I realized that attempting to reset the iterator inside of loops slightly increases the time complexity and it also makes the code a bit ugly.
Solution
Open the file and save the rows to a variable in memory.
# initialize list of rows
rows = []
# open the file and temporarily name it as 'my_file'
with open('myfile.csv', 'rb') as my_file:
    # set up the reader using the opened file
    myfilereader = csv.DictReader(my_file)
    # loop through each row of the reader
    for row in myfilereader:
        # add the row to the list of rows
        rows.append(row)
Now you can loop through rows anywhere in your scope without dealing with an iterator.
I'm arriving at this same issue - while I like the tee() solution, I don't know how big my files are going to be and the memory warnings about consuming one first before the other are putting me off adopting that method. 
Instead, I'm creating a pair of iterators using iter() statements, and using the first for my initial run-through, before switching to the second one for the final run. 
So, in the case of a dict-reader, if the reader is defined using:
d = csv.DictReader(f, delimiter=",")
I can create a pair of iterators from this "specification" - using:
d1, d2 = iter(d), iter(d)
I can then run my 1st-pass code against d1, safe in the knowledge that the second iterator d2 has been defined from the same root specification. 
I've not tested this exhaustively, but it appears to work with dummy data. 
The simplest solution possible: use deepcopy
from copy import deepcopy
iterator = your_iterator
# Start iteration
iterator_altered = deepcopy(iterator)
for _ in range(2):
    a = next(iter(iterator_altered))
# Your iterator is still unaltered. 
I think this is the simples approach.