48

Where X is any programming language (C#, Javascript, Lisp, Perl, Ruby, Scheme, etc) which supports some flavour of closures.

Some limitations are mentioned in the Closures in Python (compared to Ruby's closures), but the article is old and many limitations do not exist in modern Python any more.

Seeing a code example for a concrete limitation would be great.

Related questions:

7 Answers 7

45

The most important limitation, currently, is that you cannot assign to an outer-scope variable. In other words, closures are read-only:

>>> def outer(x): 
...     def inner_reads():
...         # Will return outer's 'x'.
...         return x
...     def inner_writes(y):
...         # Will assign to a local 'x', not the outer 'x'
...         x = y
...     def inner_error(y):
...         # Will produce an error: 'x' is local because of the assignment,
...         # but we use it before it is assigned to.
...         tmp = x
...         x = y
...         return tmp
...     return inner_reads, inner_writes, inner_error
... 
>>> inner_reads, inner_writes, inner_error = outer(5)
>>> inner_reads()
5
>>> inner_writes(10)
>>> inner_reads()
5
>>> inner_error(10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 11, in inner_error
UnboundLocalError: local variable 'x' referenced before assignment

A name that gets assigned to in a local scope (a function) is always local, unless declared otherwise. While there is the 'global' declaration to declare a variable global even when it is assigned to, there is no such declaration for enclosed variables -- yet. In Python 3.0, there is (will be) the 'nonlocal' declaration that does just that.

You can work around this limitation in the mean time by using a mutable container type:

>>> def outer(x):
...     x = [x]
...     def inner_reads():
...         # Will return outer's x's first (and only) element.
...         return x[0]
...     def inner_writes(y):
...         # Will look up outer's x, then mutate it.      
...         x[0] = y
...     def inner_error(y):
...         # Will now work, because 'x' is not assigned to, just referenced.
...         tmp = x[0]
...         x[0] = y
...         return tmp
...     return inner_reads, inner_writes, inner_error
... 
>>> inner_reads, inner_writes, inner_error = outer(5)
>>> inner_reads()
5
>>> inner_writes(10)
>>> inner_reads()
10
>>> inner_error(15)
10
>>> inner_reads()
15
1
  • 8
    nonlocal as you mentined solves this problem. An inner class with __call__ can solve it too (but version with a list is more succinct).
    – jfs
    Sep 26, 2008 at 20:31
6

The only difficulty I've seen people encounter with Python's in particular is when they try to mix non-functional features like variable reassignment with closures, and are surprised when this doesn't work:

def outer ():
    x = 1
    def inner ():
        print x
        x = 2
    return inner
outer () ()

Usually just pointing out that a function has its own local variables is enough to deter such silliness.

2
  • 2
    @Moe thanks ; @JF exactly. Closures work just like any other function, but for some reason people think they should perform magic when assigning variables. Sep 26, 2008 at 20:23
  • My comment is too big. I've posted it as an answer.
    – jfs
    Sep 26, 2008 at 20:52
6

A limitation (or "limitation") of Python closures, comparing to Javascript closures, is that it cannot be used for effective data hiding

Javascript

var mksecretmaker = function(){
    var secrets = [];
    var mksecret = function() {
        secrets.push(Math.random())
    }
    return mksecret
}
var secretmaker = mksecretmaker();
secretmaker(); secretmaker()
// privately generated secret number list
// is practically inaccessible

Python

import random
def mksecretmaker():
    secrets = []
    def mksecret():
        secrets.append(random.random())
    return mksecret

secretmaker = mksecretmaker()
secretmaker(); secretmaker()
# "secrets" are easily accessible,
# it's difficult to hide something in Python:
secretmaker.__closure__[0].cell_contents # -> e.g. [0.680752847190161, 0.9068475951742101]
2
  • 3
    Even when using standard OO data-hiding, you can get to a Python Class' __private_variable using obj._Class__private_variable. The point of data hiding is to provide abstractions, not security, and if your client code is bent on breaking poking around. Jul 4, 2013 at 7:13
  • 1
    "easily accessible" - if you need truly obscure syntax then it's not easily accessible. Private members in C++ are equally "easily accessible" via pointer manipulation and knowledge of object layout in the target ABI. Information hiding isn't about making things inaccessible to someone determined. It's about making the internals hidden from casual inadvertent use. That's all. Dec 14, 2021 at 20:36
5

Fixed in Python 3 via the nonlocal statement:

The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.

2
  • My comment is too big. I've posted it as an answer.
    – jfs
    Sep 26, 2008 at 20:50
  • Hmmm ... presumably the nonlocal variable is shared between all instances of the closure? Making it rather like a static / class variable?
    – interstar
    Oct 12, 2008 at 3:34
2

@John Millikin

def outer():
    x = 1 # local to `outer()`

    def inner():
        x = 2     # local to `inner()`
        print(x)
        x = 3
        return x

    def inner2():
        nonlocal x
        print(x)  # local to `outer()`
        x = 4     # change `x`, it is not local to `inner2()`
        return x

    x = 5         # local to `outer()`
    return (inner, inner2)

for inner in outer():
    print(inner()) 

# -> 2 3 5 4
2

comment for @Kevin Little's answer to include the code example

nonlocal does not solve completely this problem on python3.0:

x = 0 # global x
def outer():
    x = 1 # local to `outer`
    def inner():
        global x
        x = 2 # change global
        print(x) 
        x = 3 # change global
        return x
    def inner2():
##        nonlocal x # can't use `nonlocal` here
        print(x)     # prints global
##        x = 4      # can't change `x` here
        return x
    x = 5
    return (inner, inner2)

for inner in outer():
    print(inner())
# -> 2 3 3 3

On the other hand:

x = 0
def outer():
    x = 1 # local to `outer`
    def inner():
##        global x
        x = 2
        print(x) # local to `inner` 
        x = 3 
        return x
    def inner2():
        nonlocal x
        print(x)
        x = 4  # local to `outer`
        return x
    x = 5
    return (inner, inner2)

for inner in outer():
    print(inner())
# -> 2 3 5 4

it works on python3.1-3.3

3
  • I know this is a really old answer, but it is plain wrong for python3.2 - you can totally use nonlocal in the first example and get the expected results. I don't have a python3.0 build to check if it was different back then, but if it was I would consider it a bug, as the PEP doesn't state it should be different. The only (obvious) limitation is that you can't use global x and nonlocal x in the same scope.
    – l4mpi
    Sep 28, 2012 at 8:53
  • @l4mpi: It produces SyntaxError: no binding for nonlocal 'x' found on python3.0 if you uncomment nonlocal. But it works on python3.1-3.3
    – jfs
    Sep 28, 2012 at 9:31
  • Ah, thanks for the answer. This seems even more like a bug now, although I wasn't able to find a bug report or a note in the 3.1 release notes...
    – l4mpi
    Sep 28, 2012 at 9:48
-2

The better workaround until 3.0 is to include the variable as a defaulted parameter in the enclosed function definition:

def f()
    x = 5
    def g(y, z, x=x):
        x = x + 1
1
  • 4
    The value of outer x will always be 5.
    – jfs
    Jun 12, 2009 at 23:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.