Why does decorating a class break the descriptor protocol, thus preventing staticmethod objects from behaving as expected?
- by Robru
I need a little bit of help understanding the subtleties of the descriptor protocol in Python, as it relates specifically to the behavior of staticmethod objects. I'll start with a trivial example, and then iteratively expand it, examining it's behavior at each step:
class Stub:
@staticmethod
def do_things():
"""Call this like Stub.do_things(), with no arguments or instance."""
print "Doing things!"
At this point, this behaves as expected, but what's going on here is a bit subtle: When you call Stub.do_things(), you are not invoking do_things directly. Instead, Stub.do_things refers to a staticmethod instance, which has wrapped the function we want up inside it's own descriptor protocol such that you are actually invoking staticmethod.__get__, which first returns the function that we want, and then gets called afterwards.
>>> Stub
<class __main__.Stub at 0x...>
>>> Stub.do_things
<function do_things at 0x...>
>>> Stub.__dict__['do_things']
<staticmethod object at 0x...>
>>> Stub.do_things()
Doing things!
So far so good. Next, I need to wrap the class in a decorator that will be used to customize class instantiation -- the decorator will determine whether to allow new instantiations or provide cached instances:
def deco(cls):
def factory(*args, **kwargs):
# pretend there is some logic here determining
# whether to make a new instance or not
return cls(*args, **kwargs)
return factory
@deco
class Stub:
@staticmethod
def do_things():
"""Call this like Stub.do_things(), with no arguments or instance."""
print "Doing things!"
Now, naturally this part as-is would be expected to break staticmethods, because the class is now hidden behind it's decorator, ie, Stub not a class at all, but an instance of factory that is able to produce instances of Stub when you call it. Indeed:
>>> Stub
<function factory at 0x...>
>>> Stub.do_things
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute 'do_things'
>>> Stub()
<__main__.Stub instance at 0x...>
>>> Stub().do_things
<function do_things at 0x...>
>>> Stub().do_things()
Doing things!
So far I understand what's happening here. My goal is to restore the ability for staticmethods to function as you would expect them to, even though the class is wrapped. As luck would have it, the Python stdlib includes something called functools, which provides some tools just for this purpose, ie, making functions behave more like other functions that they wrap. So I change my decorator to look like this:
def deco(cls):
@functools.wraps(cls)
def factory(*args, **kwargs):
# pretend there is some logic here determining
# whether to make a new instance or not
return cls(*args, **kwargs)
return factory
Now, things start to get interesting:
>>> Stub
<function Stub at 0x...>
>>> Stub.do_things
<staticmethod object at 0x...>
>>> Stub.do_things()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'staticmethod' object is not callable
>>> Stub()
<__main__.Stub instance at 0x...>
>>> Stub().do_things
<function do_things at 0x...>
>>> Stub().do_things()
Doing things!
Wait.... what? functools copies the staticmethod over to the wrapping function, but it's not callable? Why not? What did I miss here?
I was playing around with this for a bit and I actually came up with my own reimplementation of staticmethod that allows it to function in this situation, but I don't really understand why it was necessary or if this is even the best solution to this problem. Here's the complete example:
class staticmethod(object):
"""Make @staticmethods play nice with decorated classes."""
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
"""Provide the expected behavior inside decorated classes."""
return self.func(*args, **kwargs)
def __get__(self, obj, objtype=None):
"""Re-implement the standard behavior for undecorated classes."""
return self.func
def deco(cls):
@functools.wraps(cls)
def factory(*args, **kwargs):
# pretend there is some logic here determining
# whether to make a new instance or not
return cls(*args, **kwargs)
return factory
@deco
class Stub:
@staticmethod
def do_things():
"""Call this like Stub.do_things(), with no arguments or instance."""
print "Doing things!"
Indeed it works exactly as expected:
>>> Stub
<function Stub at 0x...>
>>> Stub.do_things
<__main__.staticmethod object at 0x...>
>>> Stub.do_things()
Doing things!
>>> Stub()
<__main__.Stub instance at 0x...>
>>> Stub().do_things
<function do_things at 0x...>
>>> Stub().do_things()
Doing things!
What approach would you take to make a staticmethod behave as expected inside a decorated class? Is this the best way? Why doesn't the builtin staticmethod implement __call__ on it's own in order for this to just work without any fuss?
Thanks.
