Pitfall of Lambda in Python Mar 13th 2021 Words: 495

Problem

Here is a Python snippet:

1
2
3
4
5
6
fruits = ["apple", "banana", "tangerine", "coconut", "cherry"]

my_funcs = [lambda: print(x) for x in fruits]

for func in my_funcs:
  func()

The snippet first stores five different strings in fruits. Then it creates five lambda functions for each string in fruits. Finally, it execute all the lambda functions.

One could intuitively guess the output of the script be like:

1
2
3
4
5
apple
banana
tangerine
coconut
cherry

However, this script instead produces:

1
2
3
4
5
cherry
cherry
cherry
cherry
cherry

So after editing the script to print some info for debugging:

1
2
3
4
5
6
7
8
9
10
11
12
fruits = ["apple", "banana", "tangerine", "coconut", "cherry"]

my_funcs = list()

for x in fruits:
  print(x)
  func = lambda: print(x)
  print(func)
  my_funcs.append(func)

for func in my_funcs:
  func()

It prints:

1
2
3
4
5
6
7
8
9
10
apple
<function <lambda> at 0x7faa99aef940>
banana
<function <lambda> at 0x7faa99aef9d0>
tangerine
<function <lambda> at 0x7faa99aefa60>
coconut
<function <lambda> at 0x7faa99aefaf0>
cherry
<function <lambda> at 0x7faa99aefb80>

It seems both the lambda functions and their argument varies in each loop, but it prints the same string all the time.

The official explaination is:

This happens because x is not local to the lambdas, but is defined in the outer scope, and it is accessed when the lambda is called — not when it is defined.

In the above example, the scope of x is the for loop. When lambda is defined, the x is a reference to the variable in the loop, not a constant value, so when the lambda is called later, the x will always be cherry – it is the value in the last iteration of the loop.

So the script is effectively equal to:

1
2
3
4
5
6
7
8
9
10
fruits = ["apple", "banana", "tangerine", "coconut", "cherry"]

my_funcs = list()

for x in fruits:
  var = x   # var will change each iteration
  my_funcs.append(lambda: print(var))

for func in my_funcs:
  func()    # var is "cherry" now since the loop is done

Workaround

lambda factory

1
2
3
4
5
6
7
8
9
fruits = ["apple", "banana", "tangerine", "coconut", "cherry"]

def lambda_factory(*args):  # x becomes local here
  return lambda: print(*args)

my_funcs = [lambda_factory(x) for x in fruits]

for func in my_funcs:
  func()

functools.partial

1
2
3
4
5
6
7
8
from functools import partial

fruits = ["apple", "banana", "tangerine", "coconut", "cherry"]

my_funcs = [partial(print, x) for x in fruits]

for func in my_funcs:
  func()

lambda with argument

1
2
3
4
5
6
fruits = ["apple", "banana", "tangerine", "coconut", "cherry"]

my_funcs = [lambda x=x : print(x) for x in fruits]  # localize x

for func in my_funcs:
  func()

Note: x may be overwriten by the invoker, lambda *args, x=x : print(x) is safer.

See also