The loops in collections work with lists, dictionaries, and sets. The same loop syntax can also process values produced on demand. This lets a program read a large stream without first building an equally large list. To use it safely, separate the object supplying values from the progress through those values.
An iterable supplies an iterator
An iterable is something you can ask for an iterator using iter(). An iterator remembers its current position and supplies its next item through next(). A list can create a fresh iterator for each loop. An iterator itself is generally consumed as you advance it.
topics = ["loops", "files"]
cursor = iter(topics)
print(next(cursor))
print(next(cursor))
print(next(cursor, "finished"))
print(list(cursor))
print(list(topics))
The output is loops, files, finished, [], and ['loops', 'files']. Without a fallback argument, advancing an exhausted iterator raises StopIteration. A for loop handles that signal for you. The list still exists unchanged; only this iterator has reached its end. See the iterator tutorial and the iter and next reference.
Do not assume a function can loop over every input twice. An iterator supplied by a caller might have no values left on the second pass. Either document a single pass, request a reusable collection, or deliberately materialize a small input into a list.
Yield one value at a time
A function containing yield creates a generator, a convenient kind of iterator. Calling it creates the generator object; the body starts when somebody requests a value. Each yield pauses the function while preserving its local state.
def positive_minutes(values):
for raw in values:
minutes = int(raw)
if minutes > 0:
yield minutes
stream = positive_minutes(["0", "15", "30"])
print(next(stream))
print(sum(stream))
print(list(stream))
total = sum(value * 2 for value in range(4))
print(total)
Expect 15, 30, [], and 12. The first next consumed 15 before sum saw the remaining 30. Calling positive_minutes again would create a fresh generator. Parentheses around the expression passed to sum create a generator expression; sum can consume it without a list of intermediate results.
Laziness changes when failures appear. If the input includes "oops", integer conversion fails when iteration reaches that item, not necessarily when the generator is created. Put error handling around consumption when that is the boundary responsible for reporting bad data. The generator tutorial describes this pause-and-resume behavior.
Combine iterators carefully
enumerate adds a counter to values. zip pairs corresponding values; by default it stops at the shortest input. Use strict=True when unequal lengths would indicate missing data. These are practical alternatives to manually managing indexes.
from itertools import batched, chain, islice
topics = ["loops", "files", "tests"]
minutes = [20, 30, 40]
for number, (topic, duration) in enumerate(
zip(topics, minutes, strict=True), start=1
):
print(number, topic, duration)
print(list(batched(range(5), 2)))
print(list(islice(chain([10, 20], [30, 40]), 3)))
The numbered rows begin with 1 loops 20 and end with 3 tests 40. The final outputs are [(0, 1), (2, 3), (4,)] and [10, 20, 30]. batched, available from Python 3.12, groups a stream into tuples; the last tuple may be shorter. chain joins iterable inputs, and islice takes a bounded section without ordinary indexing. These helpers are documented in itertools.
Pitfalls and practice
Lazy does not mean free. A generator can retain a large input, an open file, or other state. Converting its entire output with list() uses memory for all those results. Infinite iterators need a stopping rule. If a generator reads an open file, consume it while that file's with block is still active; returning it after closing the file will not work.
Exercise: write a generator that yields cumulative study totals. For [10, 20, 5], expect 10, 30, 35. What happens if you call sum on it twice?
Solution guidance: initialize a local total, update it for each item, and yield the updated total. The first sum is 75; the second is zero because the same generator is exhausted. Create a new generator for a second pass. itertools.accumulate provides this operation once you understand the loop.