How do python iterators work memory-wise? What are they designed for?

295 Views Asked by At

I came across everything related to iterators today and there are still some things I don't understand. I'm still a Python beginner but I'll try to explain the best I can.

  1. I understand the difference between an iterable and an iterator. However, couldn't we just simply implement the __next__ method on a list and somehow make it to go back to list[0] when StopIteration was raised? Wouldn't that free up some memory as iterators also allocate space in memory? What's really the reason for iterators to exist?
  2. I also understand the difference between a generator and a list, for example. Also, there is a type of iterator for each object (e.g range_iterator). How are generator_iterators different from other iterator objects? Are they added values on the fly?
1

There are 1 best solutions below

0
WhiteFox On
  1. Iterators (objects representing streams of data) are created because iterables (which iterators can be created from) couldn't keep the index of the last element returned, because otherwise, you would only be able to use it one at a time. So for that reason, each time you iterate over the same object, a new iterator is created. From the Python Wiki:

Iterators will typically need to maintain some kind of position state information (e.g., the index of the last element returned). If the iterable maintained that state itself, it would become inherently non-reentrant (meaning you could use it only one loop at a time).

  1. Generator iterators (which in turn are returned by generator functions) are also streams of data but generated on the fly. From the Python Documentation:

generator iterator

An object created by a generator function. Each yield temporarily suspends processing, remembering the location execution state (including local variables and pending try-statements). When the generator iterator resumes, it picks up where it left off (in contrast to functions which start fresh on every invocation).