Introduction

The weblog submit is a write up of my two talks from PyGotham and PyCon India titled, Win Plugins with Pluggy. The write-up covers a trivial employ-case, discusses why a plugin-essentially essentially based architecture is an efficient fit, what’s plugin-essentially essentially based architecture, the style to create plugin-essentially essentially based architecture the utilization of pluggy, and the plan pluggy works.

Trivial Use Case

For the scope of the weblog submit, have confidence in mind a present-line software program queries gutenberg provider, processes the guidelines, and shows the linked info. Let’s seek the style to manufacture such an software program the utilization of pluggy.

Here is the JSON output from the software program.

 $python host.py search -t  "My Bondage and My Freedom"
[
    {
        "bookshelves": [
            "African American Writers",
            "Slavery"
        ],
        "copyright":  faux,
        "download_count":  1538,
        "media_type":  "Textual remark",
        "name":  "Douglass, Frederick",
        "title":  "My Bondage and My Freedom",
        "xml":  "http://www.gutenberg.org/ebooks/202.rdf"
    }
]

Customary code

Build%20Plugins%20with%20Pluggy%203e282afb83124aa3a24625f192178932/Normal_Architecture.png

The software program has three substances – particular person input processor, necessary functions gatherer, and consequence renderer.

The beneath is the code

import click on
import requests
import json
from pygments import spotlight, lexers, formatters

def colorize(formatted_json): 
    return spotlight(
        formatted_json.encode("UTF-8"),
        lexers.JsonLexer(),
        formatters.TerminalFormatter(),
    )

def print_output(resp, kwargs): 
    data = resp.json()
    table = [
        {
            "name": result["authors"][0]["name"],
            "bookshelves":  consequence["bookshelves"],
            "copyright":  consequence["copyright"],
            "download_count":  consequence["download_count"],
            "title":  consequence["title"],
            "media_type":  consequence["media_type"],
            "xml":  consequence["formats"]["application/rdf+xml"],
        }
        for consequence in data["results"]
    ]
    if kwargs.rep('structure', '') == 'json': 
        indent = kwargs.rep("indent", 4)
        formatted_json = json.dumps(table, sort_keys=Factual, indent=indent)
        if kwargs.rep('colorize'): 
            print(colorize(formatted_json))
        else: 
            print(formatted_json)
    # TODO: Add YAML Format
    # TODO: Add Tabular Format

class Search: 
    def __init__(self, term, kwargs): 
        self.term = term
        self.kwargs = kwargs

    def make_request(self): 
        resp = requests.rep(f"http://gutendex.com/books/?search={self.term}")
        return resp

    def mosey(self): 
        resp = self.make_request()
        print_output(resp, self.kwargs)

@click on.community()
def cli(): 
    move

@cli.present()
@click on.possibility("--title", "-t", form=str, aid="Title to maneuver trying")
@click on.possibility("--creator", "-a", form=str, aid="Author to maneuver trying")
@click on.possibility("--structure", "-f", form=str, aid="Output structure", default='json')
def search(title, creator, kwargs): 
    if no longer (title or creator): 
        print("Pass either --title or --creator")
        exit(-1)
    else: 
        search = Search(title or creator, kwargs)
        search.mosey()

if __name__ == '__main__': 
    cli()

The print_output characteristic helps one output structure. It’s straightforward so as to add one extra structure. When the software program is a library, print_output suffers from a couple of concerns whereas supporting extra output renderers. It’s onerous for a developer to lend a hand all that you might even think of and requested codecs by stop-customers. It’s painful to extend the functionality to each and each structure. One ability to extend the functionality is to re-architect the code to follow plugin essentially essentially based architecture.

What are plugins?

noun: plug-in is a instrument factor that provides a explicit feature to an existing pc program.

A plugin is a instrument factor that enhances or modifies the behavior of the program at mosey-time. For instance, Google Chrome extension or Firefox addon trade the behavior or provides functionality to the browser. The browser extensions are honest appropriate instance for plugin essentially essentially based architecture.

Build%20Plugins%20with%20Pluggy%203e282afb83124aa3a24625f192178932/Plugin_Architecture_.png

In overall, plugin architecture has two indispensable substances – host/caller/core plan and plugin/hook. The host or core plan is accountable for calling the plugin or hook at registered functionality.

Pluggy introduction

Pluggy is a Python library that offers a structured ability to control, undercover agent plugins, and enable hooks to trade the host program’s behavior at runtime.

Here is the code structure of the software program.

$tree                                                                                                                                                                                               (pluggy_talk)
.
├── LICENSE
├── README.md
├── hookspecs.py
├── host.py
├── output.py
├── requirements.txt
└── assessments.py

Other than the take a look at file, there are three python data. Sooner than attending to hold what are these three data, let’s familiarize them with pluggy ideas.

  • Host Program/Core planhost.py is the core plan that orchestrates the program drift by discovering, registering, and calling them.

    class Search: 
        def __init__(self, term, hook, kwargs): 
            # initializes the attrs
    
        def make_request(self): 
            # makes the demand to gutenberg URL
    
        def mosey(self): 
            # co-ordinates the drift
    
    def get_plugin_manager(): 
        # plugin spec, implementation registration
    
    @click on.community()
    def cli(): 
        move
    
    @cli.present()
    # click on alternatives
    def search(title, creator, kwargs): 
        # validates the actual person input, manages search workflow
    
    def setup(): 
        pm = get_plugin_manager()
        pm.hook.get_click_group(community=cli)
    
    if __name__ == "__main__": 
        setup()
        cli()
    
  • Plugin – The file output.py implements the plugin[s] good judgment.

  • Plugin Manager (occasion in host.py) – Plugin supervisor is accountable for organising cases for plugin administration.

  • Hook Specification (hookspec.py) – Hook specification is the blueprint or contract for the plugin. The hook specification is a python characteristic or a manner with an empty body.

  • Hook Implementation (characteristic/manner in output.py) – Hook implementation carries hook good judgment.

Pluggy walkthrough

Build%20Plugins%20with%20Pluggy%203e282afb83124aa3a24625f192178932/Plugin_Flow.png

The plugin workflow occurs in a single machine. The registration, hooking calling occurs within the same process as of host program. The above image represents the logical drift of the plugin-essentially essentially based architecture. Every colored block represents different functionality, and the arrow represents the direction of the drift.

Hook Spec

# hookspec.py

import pluggy
hookspec = pluggy.HookspecMarker(project_name="gutenberg")

@hookspec
def print_output(resp, config): 
    """Print formatted output"""

A hook specification is a contract for the hook to place into effect. The first step in declaring the hook specification is to manufacture an occasion of HookspecMarker with the desired name. The 2nd step is to tag the python characteristic as hookspec the utilization of the marker as a decorator.

print_output hook name is print_output and defines two arguments within the characteristic signature – response object and configuration object.

Hook Implementation

# Establish must match hookspec marker (plugin.py)
hookimpl = pluggy.HookimplMarker(project_name="gutenberg")

@hookimpl
def print_output(resp, config): 
    """Print output"""
    data = resp.json()
    table = [
        {
            "name": result["authors"][0]["name"],
            "bookshelves":  consequence["bookshelves"],
            "copyright":  consequence["copyright"],
            "download_count":  consequence["download_count"],
            "title":  consequence["title"],
            "media_type":  consequence["media_type"],
            "xml":  consequence["formats"]["application/rdf+xml"],
        }
        for consequence in data["results"]
    ]
    indent = config.rep("indent", 4)
    if config.rep('structure', '') == 'json': 
        print(f"The employ of the indent dimension as {indent}")
        formatted_json = json.dumps(table, sort_keys=Factual,
                                    indent=indent)
        if config.rep('colorize'): 
            print(colorize(formatted_json))
        else: 
            print(formatted_json)

The characteristic print_output implements the hook implementation.
Hook spec and hook implementation functions must lift the same characteristic signature.

The first step in hook specification is to manufacture an occasion of HookimplMarker with the same name in HookspecMarker. The 2nd step is to tag the python characteristic as a hook implementation the utilization of the marker as a decorator.

print_output characteristic performs severe of operations – read JSON data from the response, earn the linked necessary functions from the JSON data, earn the configuration operation handed to the plugin, and at closing, print the necessary functions.

Plugin Manager

import hookspecs
import output

def get_plugin_manager(): 
    pm = pluggy.PluginManager(project_name="gutenberg")
    pm.add_hookspecs(hookspecs)
    # Add a Python file
    pm.register(output)
    # Or add a load from setuptools entrypoint
    pm.load_setuptools_entrypoints("gutenberg")
    return pm

The plugin supervisor is accountable for discovering, registering the hook specification, and hook implementation.

The first step is to manufacture the PluggyManager occasion with the general name. The 2nd step is so as to add the hook specification to plugin supervisor. The closing step is to register or undercover agent the python hook implementation. The implementation will also be in python data within the import plan course or registered the utilization of python setup instruments entry point. Within the instance, the output.py resides within the same itemizing.

Invoke the hook

# host.py
class Search: 
    def __init__(self, term, hook, kwargs): 
        self.term = term
        self.hook = hook
        self.kwargs = kwargs

    def make_request(self): 
        resp = requests.rep(f"http://gutendex.com/books/?search={self.term}")
        return resp

    def mosey(self): 
        resp = self.make_request()
        self.hook.print_output(resp=resp, config=self.kwargs)


@cli.present()
@click on.possibility("--title", "-t", form=str, aid="Title to maneuver trying")
@click on.possibility("--creator", "-a", form=str, aid="Author to maneuver trying")
def search(title, creator, kwargs): 
    if no longer (title or creator): 
        print("Pass either --title or --creator")
        exit(-1)
    else: 
        pm = get_plugin_manager()
        search = Search(title or creator, pm.hook, kwargs)
        search.mosey()

After organising the general substances for hook calling, the closing step within the workflow is to name the hook at the loyal time. The mosey manner after receiving the response, calls the print_output hook.

Output

Build%20Plugins%20with%20Pluggy%203e282afb83124aa3a24625f192178932/output_indent_2.png

Build%20Plugins%20with%20Pluggy%203e282afb83124aa3a24625f192178932/output_indent_4.png

The 2 screenshots are from two different inputs. The input terms are My freedom and My bondage and indent as 4 and 8.

Interior necessary functions

It’s that you might even think of to register loads of hook implementation for a single hook specification. In our case, there will also be two print_output implementations, one for JSON rendering and one other for YAML rendering. The pluggy will name each and each hook one after the opposite in Final In First Out uncover.

The hooks can return output. When the hooks return values, the caller will collect the return values as a checklist. In our case, self.hook.print_output(resp=resp, config=self.kwargs), hooks don’t return any cost because there might be most productive one plugin.

It’s sub-optimum to name other hooks when the outdated hook returns a cost. To brief circuit the drift, pluggy offers an possibility whereas declaring the specification.
@hookspec(firstresult=Factual) notifies the plugin supervisor
to entire calling the hooks once a return cost is equipped.

Testing the Plugin

Testing the hook implementation is same as trying out another python characteristic.

Here is how the unit take a look at looks to be esteem

def test_print_output(capsys): 
    resp = requests.rep("http://gutendex.com/books/?search=Kafka")
    print_output(resp, {})

    captured = capsys.readouterr()
    relate len(json.hundreds(captured.out)) >= 1

Here is how the mix take a look at looks to be esteem

def test_search(): 
    setup()
    runner = CliRunner()
    consequence = runner.invoke(
        search,
        ["-t", "My freedom and My bondage",
        "--indent", 8, "--colorize", "false"],
    )

    expected_output = """
[
        {
                "bookshelves": [
                        "African American Writers",
                        "Slavery"
                ],
                "copyright": faux,
                "download_count": 1201,
                "media_type": "Textual remark",
                "name": "Douglass, Frederick",
                "title": "My Bondage and My Freedom",
                "xml": "http://www.gutenberg.org/ebooks/202.rdf"
        }
]
    """
    relate consequence
    relate consequence.output.strip() == expected_output.strip()

Conclusion

  • Pytest take a look at runner makes employ of pluggy broadly. There are 100+ pytest plugins employ pluggy framework to create the trying out substances esteem take a look at protection .
  • Tox targets to automate and standardize trying out in Python. It’s allotment of a better imaginative and prescient of easing the packaging, trying out and delivery ability of Python instrument.
  • Datasette is a python instrument to post and stumble on the dataset.
  • The thought that of plugin is a tough thought, it has somewhat a couple of advantage whereas managing extremely configurable and extensible systems.

Crucial hyperlinks from the weblog submit:

Gaze also

Creative Commons License

This work is licensed beneath a Ingenious Commons Attribution-ShareAlike 4.0 Global License.