Xmas decorations, part 3
Summary
Today we'll see real-life examples of decorators so that you have a concrete idea on what to use them on:
Call interception
Function registration
Behavioral enrichment
From how to what
Now that we've seen how decorators work, and how to make your own, we can delve on more practical matters: what to write them for.
Decorators solve a very specific problem: syntactic sugar for code reuse of callback processing.
In other words, if you have logic meant to be applied to a function object, you can put this logic into a decorator for convenience and use it again and again, with a nice declarative syntax.
This is quite abstract, though.
At first glance it's not easy to picture what concrete use cases could be served with such features, so we are going to explore real life libraries examples.
While sky is the limit, there are 3 popular usages for decorators.
Case 1: intercepting calls
The idea is to add a check to the original function you decorate, and if this check is passed, you don't even execute it. You actually return a completely different function.
In the stdlib, this is what functools.cache does. The first time the decorated function runs, it is executed. But the second time, the check detect there is already a cached value for the given parameters, and instead of running the decorated function, it runs the cache lookup instead:
>>> from functools import cache
>>> @cache
... def expensive_function(a, b):
... print("The expensive function runs")
... return a + b
>>> expensive_function(1, 2) # decorated function runs
The expensive function runs
3
>>> expensive_function(1, 2) # other code runs
3This approach can be used with very different scenarios:
Django, the web framework, uses it to check if a user is authenticated. If it is, the original function for the web page runs. If it isn't, the code for the login page runs instead.
pydantic, the validation library, provides a decorator to check a function input. If the input matches the type hints, the original function runs. If it doesn't, pydantics raises an error instead.
call-throttle, a library to rate limit code, lets you cap a function to a number of call per seconds. If the limit is reached, the original function doesn't run at all.
Let's see how to implement a simplified version of this last example:
import time
import functools
def basic_throttle(calls_per_second):
def decorator(func):
last_called = 0.0
count = 0
@functools.wraps(func)
def wrapper(*args, **kwargs):
nonlocal last_called, count
current_time = time.time()
# Reset counter if new second
if current_time - last_called >= 1:
last_called = current_time
count = 0
# Enforce the limit
if count < calls_per_second:
count += 1
return func(*args, **kwargs)
return None
return wrapper
return decoratorLet's use it:
>>> @basic_throttle(5)
... def send_alert():
... print(f"Alert !")
...
... for i in range(10):
... send_alert()
... time.sleep(0.1)
...
Alert !
Alert !
Alert !
Alert !
Alert !Case 2: registering the function
Some decorators don't create a wrapper function, instead they return the original function!
What do they do then?
Well, they store a reference of the function to be reused later when something occurs. Typically for event systems, pattern matching, routing, etc.:
doit-api provides decorators to register doit tasks. Decorated functions are later called if you run a task from the command line that matches its name.
flask's routing associates URL paths with endpoints. When a user browses the URL, the associated function generates the web page.
pytest lets you define fixtures using decorators. If you write a test parameter that bears a fixture function name, it will be automatically called, and the result injected in the test.
Let's create a kind of registering decorator. For our example, we are going to leverage sys.excepthook (follow the link if you don’t know how it works) to replace any crash handling with a custom function:
import sys
import functools
# We don't create a wrapper, we just return func
# after registering it
def on_crash(func):
sys.excepthook = func
return func
@on_crash
def handle_exception(exc_type, exc_value, exc_traceback):
# For this example, we just swallow the exception
print(f"You crashed. Leaved this poor computer alone")
raise ValueError("This is never going to be seen")Running the script just gives us:
$
python script.py
You crashed. Leaved this poor computer aloneThis is a very primitive way of doing it, but it gives inspiration for fun shenanigans.
Case 3: enriching the call
Sometimes you don't want to register the function, or replace it' content. You really want the function to run, but with additional behavior:
tenacity's decorators set the function to be retried if it fails. You can specify exceptions, number of failures, delays before retrying, and various strategies. Useful for things that naturally will have transient errors, like network calls.
fabric uses decorators to configure deployment, such as telling which host the function should run on. The code will then run on a distant machine instead of your computer.
huey provides decorators to register tasks. If you try to call the function, it won't run, but will be put in a queue full of tasks that are executed one after the other asynchronously, in a different process.
For our example, we will use the classic timing decorator, that prints how long a function took execute:
import time
import functools
def timeit(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
start_time = time.time()
result = func(*args, **kwargs)
end_time = time.time()
print(f"{func.__name__} ran in {end_time - start_time} secs.")
return result
return wrapperWe use it this way:
>>> @timeit
... def very_dumb_function(n):
... sum = 0
... for i in range(n):
... sum += i
... return sum
...
... res = very_dumb_function(1000000)
very_dumb_function ran in 0.03560924530029297 secs.Classifying is not necessary
I made categories to give you perspectives on what you can do with the decorator design pattern. In reality there is no need for them. It's code, you can do whatever you want with them.
You can use all the tricks above at the same time if you want.
Click does exactly that. It registers functions to be run as commands, will intercept arguments to validate them and replace the original function by dynamically generated error handling, and will happily add behavior such as passing around context values if you ask it to.
Don't limit yourself to what you've seen so far.
But also don't overdo it. Like with all the goodies, there can be too much of a good thing.