使委派在Python中工作

委托问题

让我们看一下所有编码人员都面临的问题;我叫的东西 委托问题。为了说明,我将使用一个示例。这是您可能在内容管理系统中看到的示例类:

class WebPage:
    def __init__(self, title, category="General", date=None, author="Jeremy"):
        self.title,self.category,self.author = title,category,author
        self.date = date 要么  datetime.now()
        ...

Then, you want to add a subclass for certain types of page, such as a product page. It should have all the details of WebPage, plus some extra stuff. One way to do it would be with inheritance, like this:

class ProductPage(WebPage):
    def __init__(self, title, price, cost, category="General", date=None, author="Jeremy"):
        super().__init__(title, category=category, date=date, author=author)
        ...

但是现在我们违反了 不要重复自己 (DRY) principal. We’ve duplicated both our list of parameter names, and the defaults. So later on, we might decide to change the default author to “Rachel”, so we change the definition in WebPage.__init__. But we forget to do the same in ProductPage, and now we have a bug 🐛! (When writing the 法泰 深度学习库我以这种方式多次创建了bug,有时很难找到它们,因为深度学习超参数中的差异可能非常微妙且难以测试或检测。)

为避免这种情况,也许我们可以这样写:

class ProductPage(WebPage):
    def __init__(self, title, price, cost, **kwargs):
        super().__init__(title, **kwargs)
        ...

The key to this approach is the use of **kwargs. In python **kwargs in a parameter like means “put any additional keyword arguments into a dict called kwarg. And **kwargs in an argument list means “insert all key/value pairs in the kwargs dict as named arguments here”. This approach is used in many popular libraries, such as matplotlib, in 哪一个 the main plot function simply has the signature plot(*args, **kwargs). The 绘图文件 说“The kwargs are Line2D properties”,然后列出这些属性。

使用这种方法的不仅是python。例如,在 R语言 the equivalent to **args is simply written ... (an 省略)。 R文档 解释:“另一个常见的要求是允许一个函数将参数设置传递给另一个。例如,许多图形函数使用函数par(),而诸如plot()之类的函数允许用户将图形参数传递给par()来控制图形输出。这可以通过添加一个额外的参数来实现,即“ ...”,的功能,然后可以将其传递”。

For more details on using **kwargs in python, Google will find you many nice tutorials, such as 这个. The **kwargs solution appears to work quite nicely:

p = ProductPage('Soap', 15.0, 10.50, category='Bathroom', author="Sylvain")
p.author
'Sylvain'

但是,这使我们的API很难使用,因为现在我们用于编辑Python代码的环境(本文中的示例假定我们正在使用Jupyter Notebook)不知道可用的参数,因此类似参数名称的制表符完成和签名的弹出列表不起作用-。此外,如果我们使用的是自动工具来生成API文档(例如fastai的 show_doc 要么 狮身人面像),我们的文档将不包含完整的参数列表,我们需要手动添加有关这些参数的信息 委托参数 (i.e. category, date, and author, in this case). In fact, we’ve seen this already, in matplotlib’s documentation for plot.

另一种选择是避免继承,而是使用组合,如下所示:

class ProductPage:
    def __init__(self, page, price, cost):
        self.page,self.price,self.cost = page,price,cost
        ...

p = ProductPage(WebPage('Soap', category='Bathroom', author="Sylvain"), 15.0, 10.50)
p.page.author
'Sylvain'

This has a new problem, however, 哪一个 is that the most basic attributes are now hidden underneath p.page, 哪一个 is not a great experience for our class users (and the constructor is now rather clunky compared to our inheritance version).

解决问题 委托继承

我最近想出的解决方案是创建一个装饰器,其用法如下:

@delegates()
class ProductPage(WebPage):
    def __init__(self, title, price, cost, **kwargs):
        super().__init__(title, **kwargs)
        ...

…which looks exactly like what we had before for our inheritance version with kwargs, but has this key difference:

print(inspect.signature(ProductPage))
(title, price, cost, category='General', date=None, author='Jeremy')

事实证明,这种方法,我称之为 委托继承,解决了我们所有的问题;在Jupyter中,如果我按标准的“显示参数”键 转移-标签 while instantiating a ProductPage, I see the full list of parameters, including those from WebPage. And hitting 标签 will show me a completion list including the WebPage parameters. In addition, documentation tools see the full, correct signature, including the WebPage parameter details.

To decorate delegating functions instead of class __init__ we use much the same syntax. The only difference is that we now need to pass the function we’re delegating to:

def create_web_page(title, category="General", date=None, author="Jeremy"):
    ...

@delegates(create_web_page)
def create_product_page(title, price, cost, **kwargs):
    ...

print(inspect.signature(create_product_page))
(title, price, cost, category='General', date=None, author='Jeremy')

I really can’t overstate how significant this little decorator is to my coding practice. In early versions of 法泰 we used kwargs frequently for delegation, because we wanted to ensure my code was as simple as possible to write (otherwise I tend to make a lot of mistakes!) We used it not just for delegating __init__ to the parent, but also for standard functions, similar to how it’s used in matplotlib’s plot function. However, as 法泰 got more popular, I heard more and more feedback along the lines of “I love everything about 法泰, except I hate dealing with kwargs”! And I totally empathized; indeed, dealing with ... in R APIs and kwargs in Python APIs has been a regular pain-point for me too. But here I was, inflicting it on my users! 😯

我当然不是第一个处理此问题的人。 Python中关键字参数的使用和滥用 是一篇深思熟虑的文章,其结论是“So it’s readability vs extensibility. I tend to argue for readability over extensibility, and that’s what I’ll do here: for the love of whatever deity/ies you believe in, use **kwargs sparingly and document their use when you do”。 This is what we ended up doing in 法泰 too. Last year Sylvain spent a pleasant (!) afternoon removing every kwargs he could and replaced it with explicit parameter lists. And of course now we get the occassional bug resulting from one of us failing to update parameter defaults in all functions…

But now that’s all in the past. We can use **kwargs again, and have the simpler and more reliable code thanks to 干燥, and also a great experience for developers. 🎈 And the basic functionality of delegates() is just a few lines of code (source at bottom of article).

解决问题 委托组成

作为替代解决方案,让我们再次看一下合成方法:

class ProductPage:
    def __init__(self, page, price, cost): self.page,self.price,self.cost = page,price,cost

page = WebPage('Soap', category='Bathroom', author="Sylvain")
p = ProductPage(page, 15.0, 10.50)
p.page.author
'Sylvain'

How do we make it so we can just write p.author, instead of p.page.author. It turns out that Python has a great solution to this: just override __getattr__, 哪一个 is called automatically any time an unknown attribute is requested:

class ProductPage:
    def __init__(self, page, price, cost): self.page,self.price,self.cost = page,price,cost
    def __getattr__(self, k): return getattr(self.page,k)

p = ProductPage(page, 15.0, 10.50)
p.author
'Sylvain'

That’s a good start. But we have a couple of problems. The first is that we’ve lost our tab-completion again… But we can fix it! Python calls __dir__ to figure out what attributes are provided by an object, so we can override it and list the attributes in self.page as well.

第二个问题是我们经常想控制 哪一个 attributes are forwarded to the composed object. Having anything and everything forwarded could lead to unexpected bugs. So we should consider providing a list of forwarded attributes, and use that in both __getattr__ and __dir__.

I’ve created a simple base class called GetAttr that fixes both of these issues. You just have to set the default property in your object to the attribute you wish to delegate to, and everything else is automatic! You can also optionally set the _xtra attribute to a string list containing the names of attributes you wish to forward (it defaults to every attribute in default, except those whose name starts with _).

class ProductPage(GetAttr):
    def __init__(self, page, price, cost):
        self.page,self.price,self.cost = page,price,cost
        self.default = page

p = ProductPage(page, 15.0, 10.50)
p.author
'Sylvain'

以下是您在标签页补全中看到的属性:

[o for o in dir(p) if not o.startswith('_')]
['author', 'category', 'cost', 'date', 'default', 'page', 'price', 'title']

因此,现在我们有两种非常好的委派方式:选择哪种方式取决于您要解决的问题的详细信息。如果您将在几个不同的地方使用合成对象,那么合成方法可能是最好的方法。如果您要向现有的类中添加某些功能,则委托继承可能会为您的类用户提供更简洁的API。

See the end of this post for the source of GetAttr and delegates.

使代表团为您服务

现在您已经在工具箱中使用了该工具,您将如何使用它?

I’ve recently started using it in many of my classes and functions. Most of my classes are building on the functionality of other classes, either my own, 要么 from another module, so I often use composition 要么 inheritance. When I do so, I normally like to make available the full functionality of the 要么 iginal class available too. By using GetAttr and delegates I don’t need to make any compromises between maintainability, readability, and usability!

无论您是否觉得有帮助,我都想听听您是否尝试过。我也想听听人们解决委派问题的其他方式。与我联系的最佳方式是在Twitter上提及我 @jeremyphoward.

编码风格的简要说明

PEP 8 显示“包含主要Python发行版中的标准库的Python代码的编码约定”。它们还广泛用于许多其他Python项目中。我不会将PEP 8用于数据科学工作或进行更一般的教学,因为目标和上下文与Python标准库的目标和上下文非常不同(PEP 8的第一点是“愚蠢的一致性是小头脑的妖精”。通常,我的代码倾向于遵循 法式风格指南,是专为数据科学和教学而设计的。所以,请:

源代码

Here’s the delegates() function; just copy it somewhere and use it… I don’t know that it’s worth creating a pip package for:

def delegates(to=None, keep=False):
    "Decorator: replace `**kwargs` in signature with params from `to`"
    def _f(f):
        if to is None: to_f,from_f = f.__base__.__init__,f.__init__
        else:          to_f,from_f = to,f
        sig = inspect.signature(from_f)
        sigd = dict(sig.parameters)
        k = sigd.pop('kwargs')
        s2 = {k:v for k,v in inspect.signature(to_f).parameters.items()
              if v.default != inspect.Parameter.empty and k not in sigd}
        sigd.update(s2)
        if keep: sigd['kwargs'] = k
        from_f.__signature__ = sig.replace(parameters=sigd.values())
        return f
    return _f

And here’s GetAttr. As you can see, there’s not much to it!

def custom_dir(c, add): return dir(type(c)) + list(c.__dict__.keys()) + add

class GetAttr:
    "Base class for attr accesses in `self._xtra` passed down to `self.default`"
    @property
    def _xtra(self): return [o for o in dir(self.default) if not o.startswith('_')]
    def __getattr__(self,k):
        if k in self._xtra: return getattr(self.default, k)
        raise AttributeError(k)
    def __dir__(self): return custom_dir(self, self._xtra)