Introduction to Python Programming. Section 18. Decorators

18 Decorators

The goal of this section is to introduce decorators – elegant wrappers which can dynamically alter a function, method or even a class without changing their source code.

18.1 Introduction

You already know how to write simple wrappers from Subsections 8.8 and 8.9. Decorators, however, take wrapping to an entirely new level. They are very powerful and simple at the same time. You will like them for sure.

Before we begin, let’s recall how a wrapper works. Say that we have a function myfun which does something. A wrapper is another function which calls myfun but it does some additional things before and/or after the call. At least that’s what you know from Section 8. Now let’s see what is possible with decorators.

For simplicity, our function myfun(n) will just add numbers 0, 1, ..., n and return the result. Here it is:

  def myfun(n):
      s = 0
      for i in range(n+1):
          s += i
      return s

Let’s make a sample call:

  res = myfun(10**7)
  print(~Result:~, res)

  Result: 50000005000000

So far so good? OK. Now let’s say that we want to time it. All we need to do is place one additional line, a decorator @report_time before the function definition:

  @report_time
  def myfun(n):
      s = 0
      for i in range(n+1):
          s += i
      return s

The call to the decorated function will now provide additional timing information:

  res = myfun(10**7)
  print(~Result:~, res)

  Time taken: 0.80092 sec.
  Result: 50000005000000

Don’t worry, in the next subsection we will tell you how the decorator was created. But before that let’s see another example.

Let’s say that we want to add debugging information to the function – whenever the function is called, we want to know about it, and also what value(s) it returns. This can be done by adding another decorator @debug to it:

  @debug
  def myfun(n):
      s = 0
      for i in range(n+1):
          s += i
      return s

Calling the decorated function now will provide debugging information:

  res = myfun(10**7)
  print(~Result:~, res)

  Entering function ’myfun’.
  The function returns 50000005000000
  Leaving function ’myfun’.
  Result: 50000005000000

And finally, what if we wanted to both time and debug the function? Easy - just add both decorators!

  @report_time
  @debug
  def myfun(n):
      s = 0
      for i in range(n+1):
          s += i
      return s

Here is the result of calling the twice decorated function:

  res = myfun(10**7)
  print(~Result:~, res)

  Entering function ’myfun’.
  The function returns 50000005000000
  Leaving function ’myfun’.
  Time taken: 0.8674440000000001 sec.
  Result: 50000005000000

The simplicity of use of decorators is amazing. Now let’s learn how to create them!

18.2 Building a timing decorator

In order to built decorators, we will need some things you learned in Section 8:

  • How to write functions which accept a variable number of arguments (Subsections 8.10 - 8.16).
  • How to define local functions within functions (Subsection 8.17).
  • How to return a function from another function (Subsection 8.23).
  • How to store functions in variables (Subsection 8.24).

This looks like a lot, but don’t worry. Even if you are not familiar with all these concepts, you may read on and chances are that you will be able to understand. We will build the timing decorator in several steps.

First, as you know from Subsection 11.4, the function process_time from the time library can be used to time functions. Let’s use it to time our function myfun:

  import time
  start = time.process_time()
  val = myfun(10**7)
  end = time.process_time()
  print(~Time taken:~, end - start, ~sec.~)

  Time taken: 0.8674440000000001 sec.

As the next step, let’s enclose the timing functionality in a new function named for example func_wrapper. This function is the extension of the original function myfun that matters. Notice that it returns the result val returned by myfun. In the main program, we will call the wrapper instead of the original function myfun to obtain the timing info:

  def myfun(n):
      s = 0
      for i in range(n+1):
          s += i
      return s
  
  import time
  def func_wrapper(n):
      start = time.process_time()
      val = myfun(n)
      end = time.process_time()
      print(~Time taken:~, end - start, ~sec.~)
      return val
  
  res = func_wrapper(10**7)
  print(~Result:~, res)

  Time taken: 0.804517757 sec.
  Result: 50000005000000

The next step is a technical trick: We define a new function report_time(func) which will accept an arbitrary function func, create the timing wrapper func_wrap for it, and return it:

  import time
  def report_time(func):
      def func_wrapper(n):
          start = time.clock()
          val = func(n)
          end = time.clock()
          print(~Time taken:~, end - start, ~sec.~)
          return val
      return func_wrapper

It is important to realize that report_time(myfun) is a function – the one that is returned from report_time. This is the wrapped myfun which prints timing information. To break it down, it can be called as follows:

  wrapped_myfun = report_time(myfun)
  res = wrapped_myfun(10**7)
  print(~Result:~, res)

  Time taken: 0.804517757 sec.
  Result: 50000005000000

And now the last step - we will replace wrapped_myfun with myfun which effectively redefines the original function with its timing wrapper. Now the function can be called as myfun, but actually executed is its timing wrapper:

  myfun = report_time(myfun)
  res = myfun(10**7)
  print(~Result:~, res)

  Time taken: 0.804517757 sec.
  Result: 50000005000000

That’s it, we are done! The line @report_time which is added in front of the definition of the function myfun is just syntactic sugar – a cosmetic replacement of the line myfun = report_time(myfun). Here is the code:

  import time
  def report_time(func):
      def func_wrapper(n):
          start = time.clock()
          val = func(n)
          end = time.clock()
          print(~Time taken:~, end - start, ~sec.~)
          return val
      return func_wrapper
  
  @report_time
  def myfun(n):
      s = 0
      for i in range(n+1):
          s += i
      return s
  
  res = myfun(10**7)
  print(~Result:~, res)

  Time taken: 0.804517757 sec.
  Result: 50000005000000

Oh, and one more thing: The decorator, as defined now, will only work for functions which accept one argument. It is easy to adjust it to work for functions which accept any number of arguments (or no arguments) by replacing the two occurrences of n with *args, **kwargs as follows:

  import time
  def report_time(func):
      def func_wrapper(*args, **kwargs):
          start = time.clock()
          val = func(*args, **kwargs)
          end = time.clock()
          print(~Time taken:~, end - start, ~sec.~)
          return val
      return func_wrapper
  
  @report_time
  def myfun(n):
      s = 0
      for i in range(n+1):
          s += i
      return s
  
  res = myfun(10**7)
  print(~Result:~, res)

  Time taken: 0.804517757 sec.
  Result: 50000005000000

In summary, the most important step is to define the wrapper func_wrap – and this is easy. The rest are only technical details.

18.3 Building a debugging decorator

In this subsection we will define the debugging decorator which we introduced in Subsection 18.1. The procedure is much the same as for the timing decorator, so let’s just show the result:

  def debug(func):
      def func_wrapper(*args, **kwargs):
          print(~Entering function ’~ + func.__name__ + ~’.~)
          val = func(*args, **kwargs)
          print(~The function returns~, val)
          print(~Leaving function ’~ + func.__name__ + ~’.~)
          return val
      return func_wrapper
  
  @debug
  def myfun(n):
      s = 0
      for i in range(n+1):
          s += i
      return s
  
  res = myfun(10**7)
  print(~Result:~, res)

  Entering function ’myfun’.
  The function returns 50000005000000
  Leaving function ’myfun’.
  Result: 50000005000000

Note that since function is a class, object introspection was used to obtain its name via func.__name__ (see Subsection 16.7 for more details).

18.4 Combining decorators

Let’s return for a moment to the function myfun and the decorator report_time from Subsection 18.2, and the debugging decorator debug from Subsection 18.3. As you know, the real line of code which stands behind

  @debug

is

  myfun = debug(myfun)

It redefines the original function to its debugging wrapper. There is nothing to stop us from redefining the wrapped function once more to its timing wrapper:

  myfun = debug(myfun)
  myfun = report_time(myfun)

At this moment, the function is wrapped twice. Using the decorator syntax, the same can be written more beautifully as

  @report_time
  @debug
  def myfun(n):
      ...

This can be done with as many wrappers as needed. Just notice the order – the decorator listed first is the most outer one – applied last.

18.5 Decorating class methods

Class methods can be decorated just like standard functions. While every class method has the reference self to the current object as its first parameter, this is taken care of by the variable parameter list *args, **kwargs. Hence, here are our timing and debugging wrappers again, with no changes required for wrapping class methods:

  import time
  def report_time(func):
      def func_wrapper(*args, **kwargs):
          start = time.clock()
          val = func(*args, **kwargs)
          end = time.clock()
          print(~Time taken:~, end - start, ~sec.~)
          return val
      return func_wrapper

  def debug(func):
      def func_wrapper(*args, **kwargs):
          print(~Entering function ’~ + func.__name__ + ~’.~)
          val = func(*args, **kwargs)
          print(~The function returns~, val)
          print(~Leaving function ’~ + func.__name__ + ~’.~)
          return val
      return func_wrapper

Application of the wrappers is very easy using the same decorator syntax as for functions. Do you still recall the class Circle which we defined In Subsection 14.8? In order to apply the debugging wrapper to its methods area and perimeter, just insert the line @debug in front of each. And to apply the timing wrapper to the method draw, use the line @report_time:

  class Circle:
      ~~~
      Circle with given radius R and center point (Cx, Cy).
      Default plotting subdivision: 100 linear edges.
      ~~~
      def __init__(self, r, cx, cy, n = 100):
          ~~~
          The initializer adds and initializes the radius R,
          and the center point coordinates Cx, Cy.
          It also creates the arrays of X and Y coordinates.
          ~~~
          self.R = r
          self.Cx = cx
          self.Cy = cy
          # Now define the arrays of X and Y coordinates:
          self.ptsx = []
          self.ptsy = []
          da = 2*np.pi/self.n
          for i in range(n):
              self.ptsx.append(self.Cx + self.R * np.cos(i * da))
              self.ptsy.append(self.Cy + self.R * np.sin(i * da))
          # Close the polyline by adding the 1st point again:
          self.ptsx.append(self.Cx + self.R)
          self.ptsy.append(self.Cy + 0)

      @debug
      def area(self):
          ~~~
          Calculates and returns the area.
          ~~~
          return np.pi * self.R**2
  
      @debug
      def perimeter(self):
          ~~~
          Calculates and returns the perimeter.
          ~~~
          return 2 * np.pi * self.R
  
      @report_time
      def draw(self, label):
          ~~~
          Plots the circle using Matplotlib.
          ~~~
          plt.axis(equal)
          plt.plot(self.ptsx, self.ptsy, label = label)
          plt.legend()

18.6 Passing arguments to decorators

It is possible to pass arguments to a decorator by creating a wrapper around it. The sole purpose of this extra wrapper is to take the arguments and pass them to the original decorator. Let’s illustrate this on our decorator report_time. This decorator measures process time by default. Imagine that we want to be able to pass an argument into it to choose whether wall time or process time will be used. The name of the extra wrapper will be timer(walltime=False):

  import time
  def timer(walltime=False):
      def report_time(func):
          def func_wrapper(*args, **kwargs):
              if walltime:
                  start = time.time()
                  print(~Measuring wall time.~)
              else:
                  start = time.process_time()
                  print(~Measuring process time.~)
              val = func(*args, **kwargs)
              if walltime: end = time.time()
              else: end = time.process_time()
              print(~Time taken:~, end - start, ~sec.~)
              return val
          return func_wrapper
      return report_time

When used without arguments or with walltime=False, timer will measure process time:

  @timer()
  def myfun(n):
      s = 0
      for i in range(n+1):
          s += i
      return s
  
  res = myfun(10**7)
  print(~Result:~, res)

  Measuring process time.
  Time taken: 0.666784343 sec.
  Result: 50000005000000

However, when used with walltime=True, it will measure wall time:

  @timer(walltime=True)
  def myfun(n):
      s = 0
      for i in range(n+1):
          s += i
      return s
  
  res = myfun(10**7)
  print(~Result:~, res)

  Measuring wall time.
  Time taken: 0.8023223876953125 sec.
  Result: 50000005000000

18.7 Debugging decorated functions

As you know from Subsection 8.27, useful information can be retrieved from functions via their attributes. However, the decorator redefines the original function with its wrapper. This causes a problem for debugging because the attributes such __name__, __doc__ (docstring) or __module__ are now those of the wrapper and not of the original function. Look at this:

  print(myfun.__name__)

  func_wrapper

Fortunately, this can be fixed using the function wraps from the functools module as follows:

  import functools as ft
  
  import time
  def timer(walltime=False):
      def report_time(func):
          @ft.wraps(func)
          def func_wrapper(*args, **kwargs):
              ...
          return func_wrapper
      return report_time

Now the attributes of the function myfun are correct:

  print(myfun.__name__)

  myfun

18.8 Python Decorator Library

The decorators are still in active development, so it is always a good idea to search for the newest information online. Besides a number of various online tutorials, we would like to bring to your attention a (uncesored) collection of wrappers at

https://wiki.python.org/moin/PythonDecoratorLibrary
This collection contains various interesting decorators, including (we just selected a few):
  • Counting function calls,
  • type enforcement (accepts/returns),
  • profiling / coverage analysis,
  • line tracing individual functions,
  • synchronization,
  • asynchronous calls,
  • etc.


Table of Contents

Created on August 6, 2018 in Python I,   Python II.
Add Comment
0 Comment(s)

Your Comment

By posting your comment, you agree to the privacy policy and terms of service.