Followup: Python 2.6, 3 abstract base class misunderstanding
- by Aaron
I asked a question at Python 2.6, 3 abstract base class misunderstanding. My problem was that python abstract base classes didn't work quite the way I expected them to.
There was some discussion in the comments about why I would want to use ABCs at all, and Alex Martelli provided an excellent answer on why my use didn't work and how to accomplish what I wanted.
Here I'd like to address why one might want to use ABCs, and show my test code implementation based on Alex's answer.
tl;dr: Code after the 16th paragraph.
In the discussion on the original post, statements were made along the lines that you don't need ABCs in Python, and that ABCs don't do anything and are therefore not real classes; they're merely interface definitions.
An abstract base class is just a tool in your tool box. It's a design tool that's been around for many years, and a programming tool that is explicitly available in many programming languages. It can be implemented manually in languages that don't provide it.
An ABC is always a real class, even when it doesn't do anything but define an interface, because specifying the interface is what an ABC does. If that was all an ABC could do, that would be enough reason to have it in your toolbox, but in Python and some other languages they can do more.
The basic reason to use an ABC is when you have a number of classes that all do the same thing (have the same interface) but do it differently, and you want to guarantee that that complete interface is implemented in all objects. A user of your classes can rely on the interface being completely implemented in all classes.
You can maintain this guarantee manually. Over time you may succeed. Or you might forget something. Before Python had ABCs you could guarantee it semi-manually, by throwing NotImplementedError in all the base class's interface methods; you must implement these methods in derived classes. This is only a partial solution, because you can still instantiate such a base class. A more complete solution is to use ABCs as provided in Python 2.6 and above.
Template methods and other wrinkles and patterns are ideas whose implementation can be made easier with full-citizen ABCs.
Another idea in the comments was that Python doesn't need ABCs (understood as a class that only defines an interface) because it has multiple inheritance. The implied reference there seems to be Java and its single inheritance. In Java you "get around" single inheritance by inheriting from one or more interfaces.
Java uses the word "interface" in two ways. A "Java interface" is a class with method signatures but no implementations. The methods are the interface's "interface" in the more general, non-Java sense of the word.
Yes, Python has multiple inheritance, so you don't need Java-like "interfaces" (ABCs) merely to provide sets of interface methods to a class. But that's not the only reason in software development to use ABCs. Most generally, you use an ABC to specify an interface (set of methods) that will likely be implemented differently in different derived classes, yet that all derived classes must have. Additionally, there may be no sensible default implementation for the base class to provide.
Finally, even an ABC with almost no interface is still useful. We use something like it when we have multiple except clauses for a try. Many exceptions have exactly the same interface, with only two differences: the exception's string value, and the actual class of the exception. In many exception clauses we use nothing about the exception except its class to decide what to do; catching one type of exception we do one thing, and another except clause catching a different exception does another thing.
According to the exception module's doc page, BaseException is not intended to be derived by any user defined exceptions. If ABCs had been a first class Python concept from the beginning, it's easy to imagine BaseException being specified as an ABC.
But enough of that. Here's some 2.6 code that demonstrates how to use ABCs, and how to specify a list-like ABC. Examples are run in ipython, which I like much better than the python shell for day to day work; I only wish it was available for python3.
Your basic 2.6 ABC:
from abc import ABCMeta, abstractmethod
class Super():
__metaclass__ = ABCMeta
@abstractmethod
def method1(self): pass
Test it (in ipython, python shell would be similar):
In [2]: a = Super()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/aaron/projects/test/<ipython console> in <module>()
TypeError: Can't instantiate abstract class Super with abstract methods method1
Notice the end of the last line, where the TypeError exception tells us that method1 has not been implemented ("abstract methods method1"). That was the method designated as @abstractmethod in the preceding code. Create a subclass that inherits Super, implement method1 in the subclass and you're done.
My problem, which caused me to ask the original question, was how to specify an ABC that itself defines a list interface. My naive solution was to make an ABC as above, and in the inheritance parentheses say (list). My assumption was that the class would still be abstract (can't instantiate it), and would be a list. That was wrong; inheriting from list made the class concrete, despite the abstract bits in the class definition.
Alex suggested inheriting from collections.MutableSequence, which is abstract (and so doesn't make the class concrete) and list-like. I used collections.Sequence, which is also abstract but has a shorter interface and so was quicker to implement.
First, Super derived from Sequence, with nothing extra:
from abc import abstractmethod
from collections import Sequence
class Super(Sequence): pass
Test it:
In [6]: a = Super()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/aaron/projects/test/<ipython console> in <module>()
TypeError: Can't instantiate abstract class Super with abstract methods __getitem__, __len__
We can't instantiate it. A list-like full-citizen ABC; yea!
Again, notice in the last line that TypeError tells us why we can't instantiate it: __getitem__ and __len__ are abstract methods. They come from collections.Sequence.
But, I want a bunch of subclasses that all act like immutable lists (which collections.Sequence essentially is), and that have their own implementations of my added interface methods. In particular, I don't want to implement my own list code, Python already did that for me.
So first, let's implement the missing Sequence methods, in terms of Python's list type, so that all subclasses act as lists (Sequences).
First let's see the signatures of the missing abstract methods:
In [12]: help(Sequence.__getitem__)
Help on method __getitem__ in module _abcoll:
__getitem__(self, index) unbound _abcoll.Sequence method
(END)
In [14]: help(Sequence.__len__)
Help on method __len__ in module _abcoll:
__len__(self) unbound _abcoll.Sequence method
(END)
__getitem__ takes an index, and __len__ takes nothing.
And the implementation (so far) is:
from abc import abstractmethod
from collections import Sequence
class Super(Sequence):
# Gives us a list member for ABC methods to use.
def __init__(self):
self._list = []
# Abstract method in Sequence, implemented in terms of list.
def __getitem__(self, index):
return self._list.__getitem__(index)
# Abstract method in Sequence, implemented in terms of list.
def __len__(self):
return self._list.__len__()
# Not required. Makes printing behave like a list.
def __repr__(self):
return self._list.__repr__()
Test it:
In [34]: a = Super()
In [35]: a
Out[35]: []
In [36]: print a
[]
In [37]: len(a)
Out[37]: 0
In [38]: a[0]
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
/home/aaron/projects/test/<ipython console> in <module>()
/home/aaron/projects/test/test.py in __getitem__(self, index)
10 # Abstract method in Sequence, implemented in terms of list.
11 def __getitem__(self, index):
---> 12 return self._list.__getitem__(index)
13
14 # Abstract method in Sequence, implemented in terms of list.
IndexError: list index out of range
Just like a list. It's not abstract (for the moment) because we implemented both of Sequence's abstract methods.
Now I want to add my bit of interface, which will be abstract in Super and therefore required to implement in any subclasses. And we'll cut to the chase and add subclasses that inherit from our ABC Super.
from abc import abstractmethod
from collections import Sequence
class Super(Sequence):
# Gives us a list member for ABC methods to use.
def __init__(self):
self._list = []
# Abstract method in Sequence, implemented in terms of list.
def __getitem__(self, index):
return self._list.__getitem__(index)
# Abstract method in Sequence, implemented in terms of list.
def __len__(self):
return self._list.__len__()
# Not required. Makes printing behave like a list.
def __repr__(self):
return self._list.__repr__()
@abstractmethod
def method1(): pass
class Sub0(Super): pass
class Sub1(Super):
def __init__(self):
self._list = [1, 2, 3]
def method1(self):
return [x**2 for x in self._list]
def method2(self):
return [x/2.0 for x in self._list]
class Sub2(Super):
def __init__(self):
self._list = [10, 20, 30, 40]
def method1(self):
return [x+2 for x in self._list]
We've added a new abstract method to Super, method1. This makes Super abstract again.
A new class Sub0 which inherits from Super but does not implement method1, so it's also an ABC.
Two new classes Sub1 and Sub2, which both inherit from Super. They both implement method1 from Super, so they're not abstract. Both implementations of method1 are different. Sub1 and Sub2 also both initialize themselves differently; in real life they might initialize themselves wildly differently. So you have two subclasses which both "is a" Super (they both implement Super's required interface) although their implementations are different.
Also remember that Super, although an ABC, provides four non-abstract methods. So Super provides two things to subclasses: an implementation of collections.Sequence, and an additional abstract interface (the one abstract method) that subclasses must implement.
Also, class Sub1 implements an additional method, method2, which is not part of Super's interface. Sub1 "is a" Super, but it also has additional capabilities.
Test it:
In [52]: a = Super()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/aaron/projects/test/<ipython console> in <module>()
TypeError: Can't instantiate abstract class Super with abstract methods method1
In [53]: a = Sub0()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/aaron/projects/test/<ipython console> in <module>()
TypeError: Can't instantiate abstract class Sub0 with abstract methods method1
In [54]: a = Sub1()
In [55]: a
Out[55]: [1, 2, 3]
In [56]: b = Sub2()
In [57]: b
Out[57]: [10, 20, 30, 40]
In [58]: print a, b
[1, 2, 3] [10, 20, 30, 40]
In [59]: a, b
Out[59]: ([1, 2, 3], [10, 20, 30, 40])
In [60]: a.method1()
Out[60]: [1, 4, 9]
In [61]: b.method1()
Out[61]: [12, 22, 32, 42]
In [62]: a.method2()
Out[62]: [0.5, 1.0, 1.5]
[63]: a[:2]
Out[63]: [1, 2]
In [64]: a[0] = 5
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/aaron/projects/test/<ipython console> in <module>()
TypeError: 'Sub1' object does not support item assignment
Super and Sub0 are abstract and can't be instantiated (lines 52 and 53).
Sub1 and Sub2 are concrete and have an immutable Sequence interface (54 through 59).
Sub1 and Sub2 are instantiated differently, and their method1 implementations are different (60, 61).
Sub1 includes an additional method2, beyond what's required by Super (62).
Any concrete Super acts like a list/Sequence (63).
A collections.Sequence is immutable (64).
Finally, a wart:
In [65]: a._list
Out[65]: [1, 2, 3]
In [66]: a._list = []
In [67]: a
Out[67]: []
Super._list is spelled with a single underscore. Double underscore would have protected it from this last bit, but would have broken the implementation of methods in subclasses. Not sure why; I think because double underscore is private, and private means private.
So ultimately this whole scheme relies on a gentleman's agreement not to reach in and muck with Super._list directly, as in line 65 above. Would love to know if there's a safer way to do that.
