Generators

A typical function has the following properties:

• execution starts at its first line and ends when it hits an exception, a return statement or reaches the end of the function (which is an implicit return None)
• it is forced to return all results at once
• once the function returns, it has no memory of the state of its local variables.

What then happens if we need a function to generate a list of values that is too large to store in memories? Python addresses this through its yield keyword.

Simply, generators are functions that return an object on which we can call next(). For each call, it returns some value until all values have been generated, after which it will raise a StopIteration exception.

For a normal function, execution stops once it hits the first return statement. Therefore, the function will never return ‘second’.

def normal_function():
    return 'first'
    return 'second'


print('Printing value from first iteration.')
func = normal_function()
print(func)
print()

print('Printing value from second iteration.')
same_func = normal_function()
print(same_func)

Printing value from first iteration.
first

Printing value from second iteration.
first

The first difference that you will notice for a generator object is when you print the variable it has been assigned to - it will not be printing the return value.

def generator_function():
    yield 'first'
    yield 'second'

gen = generator_function()
print(gen)

<generator object generator_function at 0x000002126EF6FBA0>

The first time that we call next on the generator, it returns the value tied to the first yield statement.

print(next(gen))

first

Notice that the function remembers where it last left off and it will return the value from the next yield statement.

print(next(gen))

second

Once the function has run out of values to return, it will throw a StopIteration exception.

try:
    print(next(gen))
except StopIteration:
    print('We have reached the end of the function.')

We have reached the end of the function.